Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

OLAP Connectors: Test the connection before exiting the "Add Data" modal #6892

Open
wants to merge 27 commits into
base: main
Choose a base branch
from

Conversation

ericpgreen2
Copy link
Contributor

@ericpgreen2 ericpgreen2 commented Mar 14, 2025

This PR ensures that a connection to the OLAP database is successfully established before finalizing the creation of a connector.yaml file and updating the .env file. Previously, submitting the ClickHouse, Druid, or Pinot connector forms would create these files regardless of the validity of the credentials.

Implementation details:

  • Optimistically create a connector.yaml file.
  • Optimistically create or update the .env file with the provided credentials.
  • Call the OLAPListTables API to verify the connection.
  • If the connection fails, remove the connector file, revert the changes made to the .env file, and display an error message.

Once #6671 lands, we can potentially switch the implementation to use a tmp directory and, in particular, a tmp/.env file to test credentials.

Checklist:

  • Covered by tests
  • Ran it and it works as intended
  • Reviewed the diff before requesting a review
  • Checked for unhandled edge cases
  • Linked the issues it closes
  • Checked if the docs need to be updated
  • Intend to cherry-pick into the release branch
  • I'm proud of this work!

@ericpgreen2 ericpgreen2 self-assigned this Mar 14, 2025
@ericpgreen2 ericpgreen2 requested a review from AdityaHegde March 18, 2025 13:30
@ericpgreen2 ericpgreen2 marked this pull request as ready for review March 18, 2025 13:30
@ericpgreen2 ericpgreen2 requested a review from AdityaHegde April 2, 2025 18:24

test.describe("ClickHouse connector", () => {
test.use({ project: "Blank" });

const clickhouse = new ClickHouseTestContainer();

test.beforeAll(async () => {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Would be good to move this to a static method on ClickHouseTestContainer that also returns the instance.

Copy link
Contributor Author

@ericpgreen2 ericpgreen2 Apr 11, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I can't quite tell exactly what you have in mind? Regarding another layer of abstraction– that might be valuable, but I'd prefer not to add extra abstractions here before we know how our usage of this class will evolve. For example, I could especially see the seed method being parameterized based on which dataset you want to use for a given test suite.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants