fix: time based state refresh for each activity #796
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
abhishekagrawal-atlan
requested review from
AtMrun,
firecast and
inishchith
as code owners
Contributor
There was a problem hiding this comment.
Pull Request Overview
This PR addresses an issue where stale connection data was persisting across successive workflow runs in crossover 2.0. The fix ensures that state is properly cleaned before each activity execution.
Key Changes:
- Modified state management in
_get_stateto unconditionally clean state before setting new state
Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.
📜 Docstring Coverage ReportRESULT: PASSED (minimum: 30.0%, actual: 75.4%) Detailed Coverage Report |
📦 Trivy Vulnerability Scan Results
Report Summary
Scan Result Details✅ No vulnerabilities found during the scan for |
📦 Trivy Secret Scan Results
Report Summary
Scan Result Details✅ No secrets found during the scan for |
|
🛠 Docs available at: https://k.atlan.dev/application-sdk/activity-state-fix |
Collaborator
☂️ Python Coverage
Overall Coverage
New FilesNo new covered files... Modified FilesNo covered modified files...
|
|
🛠 Full Test Coverage Report: https://k.atlan.dev/coverage/application-sdk/pr/796 |
![cursor[bot]](https://avatars.githubusercontent.com/in/1210556?s=60&v=4){kind=small}
This comment was marked as outdated.
This comment was marked as outdated.
Sorry, something went wrong.
Bug: State Management Typo Causes Caching FailuresThe state management logic has two issues. A typo ( |
abhishekagrawal-atlan
changed the title
Bug: Timestamp Update Missing in Base ClassMissing timestamp update in base class |
Bug: Timestamp Handling Fails for Initial StatesThe timestamp refresh logic can fail if |
| if last_updated and current_timestamp - last_updated > timedelta( | ||
| minutes=15 | ||
| ): | ||
| await self._set_state(workflow_args) |
There was a problem hiding this comment.
Bug: Bug
The ActivitiesInterface._set_state method doesn't set the last_updated_timestamp field. This prevents the 15-minute state refresh logic in _get_state from ever triggering for activities using the base class, leading to inconsistent refresh behavior across different activity implementations.
Additional Locations (1)
inishchith
reviewed
Oct 31, 2025
| if workflow_id not in self._state: | ||
| await self._set_state(workflow_args) | ||
|
|
||
| if workflow_id in self._state: |
Member
There was a problem hiding this comment.
please put this under an else block
Member
|
@abhishekagrawal-atlan - has this been tested on production instances? is this ready to merge? |
📦 Example workflows test results
|
📦 Example workflows test results
|
📦 Example workflows test results
|
Collaborator
Author
This has been tested on internal instances - https://apps-framework-redshift-testing.atlan.com/. will create a ring of some production instances to test it out. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Changelog
Checklist
Copyleft License Compliance
Note
Refresh activity state if older than 15 minutes by tracking and updating last_updated_timestamp; initialize timestamp in SQL metadata extraction state.
ActivitiesInterface._get_state: ifcurrent_timestamp - last_updated_timestamp > 15m, reinitialize state via_set_state.ActivitiesStatewithlast_updated_timestamp.BaseSQLMetadataExtractionActivitiesStatewithlast_updated_timestamp.last_updated_timestampin_set_stateafter initializingsql_clientandhandler.Written by Cursor Bugbot for commit 54470ba. This will update automatically on new commits. Configure here.