-
Notifications
You must be signed in to change notification settings - Fork 0
Pull Request: JuniperData Service Extraction & Integration Infrastructure #38
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
pcalnon
merged 194 commits into
main
from
subproject.juniper_data.feature.spiral_gen.extract
Feb 18, 2026
Merged
Changes from 178 commits
Commits
Show all changes
194 commits
Select commit
Hold shift + click to select a range
5ad71cd
Add unit tests for dataset ID generation, spiral generator, and data …
pcalnon da088db
Add unit tests for dataset ID generation and spiral generator
pcalnon 866a66b
Update changelog and add Conda environment configuration for JuniperD…
pcalnon 1c9097a
Add legacy parity mode to spiral generator with new parameters and un…
pcalnon e995ae4
Add test reporting and execution scripts
pcalnon c863cc8
Add MCP setup documentation and log source file list in test script
pcalnon 21731fd
Add CLAUDE.md symlink to AGENTS.md
pcalnon 4df745c
Add .gitignore and project.yml for Serena configuration and installat…
pcalnon 7d9d971
Add Serena onboarding memory files
pcalnon c8e81b1
Add comprehensive test suite and CI/CD audit report
pcalnon 9342835
Add archived CI/CD audit report with timestamp
pcalnon 263e838
Add test suite and CI/CD enhancement development plans
pcalnon 10c18dc
Enhance test suite documentation and CI/CD configuration
pcalnon 2e99cce
Refactor code structure for improved readability and maintainability
pcalnon d88ea2c
Implement Test Suite and CI/CD enhancements (v0.3.0)
pcalnon dc3b25e
Refactor code structure for improved readability and maintainability
pcalnon 6727ede
Refactor code structure for improved readability and maintainability
pcalnon 34343bf
Update AGENTS.md and CHANGELOG.md for Integration Development Plan; a…
pcalnon d5df7f9
Refactor code structure for improved readability and maintainability
pcalnon 8341382
Fix type narrowing assertions and improve test coverage
pcalnon 883b9f3
Add parameter aliases for consumer compatibility and enhance API docu…
pcalnon 8c4ec8f
Refactor code structure for improved readability and maintainability
pcalnon 4bd37a6
Add juniper_data.bash script and remove obsolete run.bash script
pcalnon 84c1c27
Bump version to 0.4.0 and update related documentation
pcalnon aa27eb8
Add unit tests for dataset generators and API security features
pcalnon a57b77d
Refactor code structure for improved readability and maintainability
pcalnon 8fa3f95
Add Kaggle and PostgreSQL dataset storage implementations, along with…
pcalnon 192ce32
Fix formatting in integration development plan for improved readability
pcalnon 900fcc4
Update integration development plan and add missing files to egg-info
pcalnon a9187f7
Fix formatting in integration development plan for improved clarity a…
pcalnon 59e6c51
Potential fix for code scanning alert no. 11: An assert statement has…
pcalnon 574a54d
Potential fix for code scanning alert no. 6: An assert statement has …
pcalnon 42b5f99
Potential fix for code scanning alert no. 7: An assert statement has …
pcalnon 859a820
Potential fix for code scanning alert no. 8: An assert statement has …
pcalnon ffb037e
Potential fix for code scanning alert no. 9: An assert statement has …
pcalnon b3a4c33
Potential fix for code scanning alert no. 10: An assert statement has…
pcalnon bcbf730
Potential fix for code scanning alert no. 3: Unused import
pcalnon 8c37006
Potential fix for code scanning alert no. 5: Empty except
pcalnon c04adda
Update README.md
pcalnon dcbd418
Update Dockerfile
pcalnon 7cd0602
Update juniper_data.egg-info/PKG-INFO
pcalnon 6d319dc
Update juniper_data_client/juniper_data_client.egg-info/SOURCES.txt
pcalnon 220f731
Update juniper_data/core/artifacts.py
pcalnon 1953f65
Update juniper_data/__init__.py
pcalnon 9cb5e98
Update conf/source_tree.conf
pcalnon 4d2f6ad
Update juniper_data/storage/local_fs.py
pcalnon f39c203
Update docs/QUICK_START.md
pcalnon f954680
Update juniper_data/storage/local_fs.py
pcalnon c39f82d
Update juniper_data.egg-info/PKG-INFO
pcalnon 3dc015a
Update juniper_data/tests/fixtures/golden_datasets/README.md
pcalnon 24c5c20
Update juniper_data/generators/spiral/params.py
pcalnon 37ef677
Update Dockerfile
pcalnon 616b5c4
Update juniper_data/api/routes/health.py
pcalnon 81326e1
Potential fix for code scanning alert no. 12: Empty except
pcalnon 4be7531
Update juniper_data.egg-info/PKG-INFO
pcalnon e5fae24
Update juniper_data_client/juniper_data_client.egg-info/requires.txt
pcalnon 4c05ae3
Update juniper_data/generators/spiral/defaults.py
pcalnon bc91a2e
Update juniper_data_client/pyproject.toml
pcalnon 912eaf3
Update juniper_data/generators/__init__.py
pcalnon c9b35ad
Update juniper_data/api/routes/generators.py
pcalnon 67c5e9d
Update juniper_data/generators/spiral/__init__.py
pcalnon b5109d0
Update juniper_data/__init__.py
pcalnon 2061145
Update juniper_data/tests/fixtures/generate_golden_datasets.py
pcalnon 2d24bfe
Refactor code structure for improved readability and maintainability
pcalnon d8717e3
chore: clean up empty code change sections in the changes log
pcalnon cc5daf2
feat: add conda and requirements configuration files for Juniper Data…
pcalnon 768b916
fixing local branch to match remote
pcalnon d7b4fa9
Merge branch 'subproject.juniper_data.feature.spiral_gen.extract' of …
pcalnon 07f5296
updating precommit task projects' versions to resolve deprecations.
pcalnon bb084cb
Refactor code structure for improved readability and maintainability
pcalnon 5a2043e
feat: register additional generator types and update integration plan
pcalnon 2591678
Update juniper_data/__init__.py
pcalnon 56d89b3
Update juniper_data/storage/memory.py
pcalnon b51626a
Update conf/init.conf
pcalnon c2fa6de
Update docs/QUICK_START.md
pcalnon 2b30b4e
Update conf/logging.conf
pcalnon c105092
Update README.md
pcalnon fb295ec
Update juniper_data/tests/fixtures/golden_datasets/README.md
pcalnon 2fc8f61
Update juniper_data/__init__.py
pcalnon bf87edb
Update docs/QUICK_START.md
pcalnon b9e3b65
Update conf/run_all_tests.conf
pcalnon 399665f
Update juniper_data/storage/cached.py
pcalnon e2983e1
Update juniper_data/storage/__init__.py
pcalnon 54664f1
Update juniper_data/storage/__init__.py
pcalnon c2a2c2c
Update juniper_data/storage/__init__.py
pcalnon 7d5876f
Update conf/get_code_stats.conf
pcalnon e2f9804
Update juniper_data/storage/cached.py
pcalnon 32eaf72
Update .flake8
pcalnon f32001b
adding additional dataset sources to juniper Data
pcalnon 542be31
Merge branch 'subproject.juniper_data.feature.standalone_object.enhan…
pcalnon c066f9c
Refactor import order and clean up whitespace in storage module
pcalnon a0c5b65
Update settings and improve error handling in cached dataset storage
pcalnon 3250f46
Update juniper_data/storage/__init__.py
pcalnon 5c663a5
Update juniper_data/api/app.py
pcalnon 2b65b70
Update .flake8
pcalnon 48e7e02
Potential fix for code scanning alert no. 50: Use of 'return' or 'yie…
pcalnon 01cea83
Update juniper_data/storage/memory.py
pcalnon 45cc79a
Update conf/logging_config.yaml
pcalnon 2d434dd
Update docs/QUICK_START.md
pcalnon adaf36b
Update juniper_data/generators/csv_import/generator.py
pcalnon a672650
Update juniper_data/tests/unit/test_api_settings.py
pcalnon ee43e70
Update juniper_data/storage/__init__.py
pcalnon 273d4f6
Update juniper_data/__main__.py
pcalnon 98a8db4
Update juniper_data/api/settings.py
pcalnon 565848c
Update juniper_data/__main__.py
pcalnon b4ea26b
Update juniper_data/generators/csv_import/generator.py
pcalnon d10156e
Update juniper_data/generators/__init__.py
pcalnon 61e21f1
Update juniper_data/tests/fixtures/golden_datasets/README.md
pcalnon aff33ef
Update juniper_data/generators/spiral/params.py
pcalnon 3fc8dc3
Refactor storage module: add InMemoryDatasetStore and remove unused i…
pcalnon 55dd848
Merge branch 'subproject.juniper_data.feature.spiral_gen.extract' of …
pcalnon 0caebbb
Update juniper_data/generators/spiral/params.py
pcalnon 3864d41
Update conf/logging_config.yaml
pcalnon d7a992a
fixing ci issues with github actions
pcalnon f3f9b53
Merge branch 'subproject.juniper_data.feature.spiral_gen.extract' of …
pcalnon 9ae60ce
Refine .gitignore and remove cached Python files; update type hints i…
pcalnon 131b444
Add logging for failed dataset caching in CachedDatasetStore
pcalnon aefd2e8
Add unit tests for dataset generators, caching, and security middleware
pcalnon 83a1256
Potential fix for code scanning alert no. 51: Module is imported with…
pcalnon 551f6ce
Refactor .gitignore to enhance clarity and organization of ignored fi…
pcalnon e687b6a
Add .env to .gitignore and enhance environment variable handling in _…
pcalnon c4bf2a8
Add arc-agi dependency to requirements.txt
pcalnon 193f39d
Remove ARC_AGI_API environment variable from juniper_data.bash
pcalnon 63b1cd9
Update juniper_data/__init__.py
pcalnon a5bf7fa
Enhance environment variable handling and add tests for dataset gener…
pcalnon 12cc209
Merge branch 'subproject.juniper_data.feature.spiral_gen.extract' of …
pcalnon c831cc8
Update juniper_data/storage/cached.py
pcalnon c357090
Potential fix for code scanning alert no. 53: Commented-out code
pcalnon a91921b
Potential fix for code scanning alert no. 52: Commented-out code
pcalnon 43a42a8
Update conf/run_all_tests.conf
pcalnon 3e348f7
Update juniper_data/api/settings.py
pcalnon 36a45ed
Update juniper_data/generators/xor/params.py
pcalnon 93c0946
Refactor environment variable retrieval functions for clarity and con…
pcalnon 6f1b0fa
Remove commented-out code for environment variable retrieval in __ini…
pcalnon 27dde1a
Update conf/logging_config.yaml
pcalnon a99b41c
Update juniper_data/storage/__init__.py
pcalnon c3ad5c1
Update juniper_data/generators/xor/params.py
pcalnon aba31ca
Refactor tests to use default host and streamline mock assertions
pcalnon 0581103
Update coverage report file to reflect recent test changes
pcalnon 3d5286d
Add default configuration settings for API in settings module
pcalnon d38d0ab
Add security comment to API host global setting
pcalnon 9820ff9
Fix formatting of nosec comment for API host global setting
pcalnon 60e526c
Update juniper_data/__init__.py
pcalnon 3bb9fbd
Update conf/get_code_stats.conf
pcalnon 0de6998
Update conf/run_all_tests.conf
pcalnon 92e4257
Update juniper_data/tests/unit/test_api_settings.py
pcalnon 3942234
Update juniper_data/__init__.py
pcalnon 7721883
Update juniper_data/__init__.py
pcalnon 47f1c26
Potential fix for code scanning alert no. 70: Variable defined multip…
pcalnon dacee1a
removed redundant assignment statement
pcalnon f3f4222
Merge branch 'subproject.juniper_data.feature.spiral_gen.extract' of …
pcalnon 9aa8c3b
cleaned up formatting
pcalnon c02d31b
update test_get_arc_agi_env_calls_load_dotenv_when_unset to return Fa…
pcalnon b965d78
Update conf/logging_config.yaml
pcalnon a950406
Update juniper_data/storage/memory.py
pcalnon 1525ae1
Update juniper_data/generators/xor/generator.py
pcalnon 7ebcc3b
Update juniper_data/core/models.py
pcalnon c505b88
Update juniper_data/api/app.py
pcalnon 9907b45
Update juniper_data/generators/xor/generator.py
pcalnon 16ada3f
Update conf/common.conf
pcalnon 1a51f50
Update conf/get_code_stats.conf
pcalnon 44c5236
Update conf/run_all_tests.conf
pcalnon 7b5dc84
Remove unnecessary blank lines in models.py and memory.py
pcalnon 2c66df8
Enhance logging options and update API key format in settings
pcalnon 55e88d8
Refactor API key type annotation in settings and comment out unused i…
pcalnon ef2b80a
Add Gitleaks configuration and update dependencies in pyproject.toml
pcalnon 78d79d4
Update Python version requirements in CI configuration and pyproject.…
pcalnon 2b778a1
Refactor Gitleaks configuration: update allowlist format for historic…
pcalnon 2400f87
Update Bandit security scan output format to JSON and comment out SAR…
pcalnon 7b8cb3e
Update Bandit security scan output format to JSON and adjust upload step
pcalnon 7c46d8c
Refactor Bandit security scan to use SARIF format and update dataset …
pcalnon e01a306
adding fix to github actions ci pipeline to load bandit[sarif] instea…
pcalnon eecef41
updating ci pipeline github actions
pcalnon 162264e
fixed approach for pip-audit task to skip local juniper_data module
pcalnon d2c413e
Fix pip-audit grep pattern to handle underscore-normalized package names
pcalnon 4e039de
Update PR description with CI-001 pip-audit fix
pcalnon a888b87
Update CHANGELOG and PR description with all v0.4.0 changes
pcalnon 7a97eae
Update conf/common.conf
pcalnon 5118c45
Update conf/common_functions.conf
pcalnon 57842aa
Update conf/logging_config-CASCOR.yaml
pcalnon 169c822
Update juniper_data/storage/__init__.py
pcalnon 20bdef0
Refactor dataset loading assertions and streamline image processing i…
pcalnon 5b472bb
minor syntax changes to match repo
pcalnon b4a0419
Update juniper_data/generators/mnist/generator.py
pcalnon e8c4652
Update conf/run_all_tests.conf
pcalnon 52f05bc
Update juniper_data/generators/xor/params.py
pcalnon 568b06f
Update README.md
pcalnon 5499db9
Fix 12 failing MNIST generator tests by adding n_samples support and …
pcalnon 6648ed6
file containing coverage report
pcalnon 96d1863
Update juniper_data/generators/mnist/generator.py
pcalnon 45cef9f
Move nosec B615 inline so bandit recognizes the suppression
pcalnon 33d1a14
Make arc-agi an optional dependency to fix JuniperCascor import failures
pcalnon bcfb9ac
Update PR description notes file with MNIST and bandit fixes
pcalnon 31d2eae
Update conf/logging_config.yaml
pcalnon File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
The table of contents is too big for display.
Diff view
Diff view
There are no files selected for viewing
This file was deleted.
Oops, something went wrong.
Binary file not shown.
This file was deleted.
Oops, something went wrong.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,85 @@ | ||
| # ============================================================================= | ||
| # JuniperData Docker Build Exclusions | ||
| # ============================================================================= | ||
| # Excludes files not needed in the Docker build context to reduce image size | ||
| # and build time. Keep in sync with .gitignore where applicable. | ||
| # ============================================================================= | ||
|
|
||
| # Git and version control | ||
| .git/ | ||
| .gitignore | ||
| .github/ | ||
|
|
||
| # IDE and editor files | ||
| .vscode/ | ||
| .idea/ | ||
| .trunk/ | ||
| *.swp | ||
| *.swo | ||
|
|
||
| # Python artifacts | ||
| __pycache__/ | ||
| *.py[cod] | ||
| *.pyo | ||
| *.pyd | ||
| .Python | ||
| *.egg-info/ | ||
| *.egg | ||
| dist/ | ||
| build/ | ||
| wheels/ | ||
| *.whl | ||
|
|
||
| # Test and coverage | ||
| .pytest_cache/ | ||
| .mypy_cache/ | ||
| .coverage | ||
| htmlcov/ | ||
| reports/ | ||
| *.prof | ||
| TEST*.xml | ||
| TEST*.html | ||
|
|
||
| # Development tools | ||
| .pre-commit-config.yaml | ||
| .serena/ | ||
| .claude/ | ||
|
|
||
| # Documentation (not needed at runtime) | ||
| docs/ | ||
| notes/ | ||
| *.md | ||
| !README.md | ||
|
|
||
| # Local data and logs (will be mounted as volumes) | ||
| data/ | ||
| logs/ | ||
| images/ | ||
|
|
||
| # Configuration files not needed in container | ||
| conf/ | ||
| util/ | ||
| src/ | ||
|
|
||
| # Temporary files | ||
| *.tmp | ||
| *.temp | ||
| *.log | ||
| *.out | ||
|
|
||
| # Large media files | ||
| *.mp4 | ||
| *.avi | ||
| *.mov | ||
| *.jpg | ||
| *.jpeg | ||
| *.gif | ||
| *.png | ||
| *.xcf | ||
|
|
||
| # Archives | ||
| *.gz | ||
| *.tgz | ||
| *.tar | ||
| *.zip | ||
| *.bz |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,26 @@ | ||
| [flake8] | ||
| max-line-length = 512 | ||
| # E203: whitespace before ':' (conflicts with black) | ||
| # E266: too many leading '#' for block comment | ||
| # E501: line too long (handled by black) | ||
| # W503: line break before binary operator | ||
| # SIM102: nested if-statements (style preference) | ||
| # SIM105: contextlib.suppress suggestion (style preference) | ||
| # SIM117: multiple with statements (style preference, keeps code clearer) | ||
| extend-ignore = E203,E266,E501,W503,SIM102,SIM105,SIM117 | ||
| exclude = | ||
| .git, | ||
| __pycache__, | ||
| .pytest_cache, | ||
| .mypy_cache, | ||
| *.egg-info, | ||
| data, | ||
| logs, | ||
| reports | ||
|
|
||
| # B008: Function calls in argument defaults - intentional FastAPI pattern | ||
| # F841: Unused variables in tests are common (trigger side effects) | ||
| per-file-ignores = | ||
| juniper_data/api/routes/*.py:B008 | ||
| juniper_data/tests/*.py:F841 | ||
|
pcalnon marked this conversation as resolved.
|
||
| juniper_data/tests/*/*.py:F841 | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,65 @@ | ||
| ##################################################################################################################################################################################################### | ||
| # Project: Juniper | ||
| # Sub-Project: JuniperData | ||
| # Application: juniper_data | ||
| # File Name: dependabot.yml | ||
| # Author: Paul Calnon | ||
| # Version: 1.0.0 | ||
| # | ||
| # Date Created: 2026-02-04 | ||
| # Last Modified: 2026-02-04 | ||
| # | ||
| # License: MIT License | ||
| # Copyright: Copyright (c) 2024-2026 Paul Calnon | ||
| # | ||
| # Description: | ||
| # Dependabot configuration for JuniperData Dataset Generation Service | ||
| # Automatically creates PRs for dependency updates to maintain security | ||
| # | ||
| # References: | ||
| # - https://docs.github.com/en/code-security/dependabot/dependabot-version-updates | ||
| # - TEST_SUITE_CICD_ENHANCEMENT_DEVELOPMENT_PLAN.md (SEC-003) | ||
| ##################################################################################################################################################################################################### | ||
|
|
||
| version: 2 | ||
|
|
||
| updates: | ||
| # ═══════════════════════════════════════════════════════════════════════════ | ||
| # Python Dependencies | ||
| # ═══════════════════════════════════════════════════════════════════════════ | ||
| - package-ecosystem: "pip" | ||
| directory: "/" | ||
| schedule: | ||
| interval: "weekly" | ||
| day: "monday" | ||
| time: "09:00" | ||
| timezone: "America/New_York" | ||
| open-pull-requests-limit: 5 | ||
| labels: | ||
| - "dependencies" | ||
| - "security" | ||
| commit-message: | ||
| prefix: "deps" | ||
| groups: | ||
| # Group minor/patch updates together to reduce PR noise | ||
| python-minor: | ||
| patterns: | ||
| - "*" | ||
| update-types: | ||
| - "minor" | ||
| - "patch" | ||
|
|
||
| # ═══════════════════════════════════════════════════════════════════════════ | ||
| # GitHub Actions | ||
| # ═══════════════════════════════════════════════════════════════════════════ | ||
| - package-ecosystem: "github-actions" | ||
| directory: "/" | ||
| schedule: | ||
| interval: "weekly" | ||
| day: "monday" | ||
| open-pull-requests-limit: 3 | ||
| labels: | ||
| - "dependencies" | ||
| - "ci" | ||
| commit-message: | ||
| prefix: "ci" |
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.