feature(connector)sdk): Adding probe to API, database and s3 example#518
Open
fivetran-sahilkhirwal wants to merge 6 commits intomainfrom
Open
feature(connector)sdk): Adding probe to API, database and s3 example#518fivetran-sahilkhirwal wants to merge 6 commits intomainfrom
fivetran-sahilkhirwal wants to merge 6 commits intomainfrom
Conversation
🧹 Python Code Quality Check✅ No issues found in Python Files. This comment is auto-updated with every commit. |
Contributor
There was a problem hiding this comment.
Pull request overview
Adds “probe” (connectivity validation) steps to several Connector SDK examples (API key auth, SQL Server, and S3 certificate retrieval) so the examples can be referenced from Concepts docs with clearer fail-fast behavior.
Changes:
- Added S3 connectivity probing and refactored S3 certificate retrieval helper usage in the AWS certificate example.
- Added an API probe + retry/backoff logic to the API key authentication example.
- Added a SQL Server connection probe and refactored the SQL Server connector to batch-fetch rows and checkpoint periodically.
Reviewed changes
Copilot reviewed 5 out of 5 changed files in this pull request and generated 17 comments.
Show a summary per file
| File | Description |
|---|---|
| examples/common_patterns_for_connectors/authentication/certificate/retrieve_from_aws/connector.py | Adds S3 probing, refactors S3 client/cert retrieval, introduces constants/checkpoint interval. |
| examples/common_patterns_for_connectors/authentication/certificate/retrieve_from_aws/aws_client.py | Adds an S3Client.probe() helper for testing S3 connectivity. |
| examples/common_patterns_for_connectors/authentication/api_key/connector.py | Adds API probe and retry/backoff behavior for API calls; updates main debug block comments. |
| examples/common_patterns_for_connectors/authentication/api_key/configuration.json | Updates api_key to use angle-bracket placeholder format. |
| connectors/sql_server/connector.py | Adds DB probe + batching/checkpointing refactor and improves docstrings/logging. |
...les/common_patterns_for_connectors/authentication/certificate/retrieve_from_aws/connector.py
Outdated
Show resolved
Hide resolved
examples/common_patterns_for_connectors/authentication/api_key/connector.py
Outdated
Show resolved
Hide resolved
...es/common_patterns_for_connectors/authentication/certificate/retrieve_from_aws/aws_client.py
Show resolved
Hide resolved
examples/common_patterns_for_connectors/authentication/api_key/connector.py
Show resolved
Hide resolved
...les/common_patterns_for_connectors/authentication/certificate/retrieve_from_aws/connector.py
Outdated
Show resolved
Hide resolved
...les/common_patterns_for_connectors/authentication/certificate/retrieve_from_aws/connector.py
Show resolved
Hide resolved
examples/common_patterns_for_connectors/authentication/api_key/connector.py
Outdated
Show resolved
Hide resolved
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Jira ticket
Closes https://fivetran.atlassian.net/browse/RD-1161619
Description of Change
For referencing the github example in the Concepts docs, Added probe to examples
Testing
authentication/api_keyauthentication/certificatesChecklist
Some tips and links to help validate your PR:
fivetran debugcommand.