Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Docs] Update flytectl_update_workflow-execution-config.rst #5865

Merged
merged 3 commits into from
Jan 22, 2025

Conversation

pranshustuff
Copy link
Contributor

@pranshustuff pranshustuff commented Oct 19, 2024

Added the usage of raw_output_data_config in flytectl update workflow-execution-config page of docs,

Tracking issue

Closes #4615

Why are the changes needed?

To show the usage of flytectl to update raw output data config.

What changes were proposed in this pull request?

Updated the examples to showcase the same.

Check all the applicable boxes

  • I updated the documentation accordingly.
  • All new and existing tests passed.
  • All commits are signed-off.

Docs link

The 2 example Code blocks in the synopsis part were changed

Summary by Bito

Comprehensive documentation updates covering multiple system components, including flytectl update workflow-execution-config command with raw_output_data_config feature, Kubernetes configuration, logging providers (CloudWatch, Kubernetes, Stackdriver), and resource management. Updates include WebAPI plugin configuration, GPU resource management, pod templates, authentication settings, and output_location_prefix configuration details. These changes enhance overall clarity and provide detailed guidance for system configuration.

Unit tests added: False

Estimated effort to review (1-5, lower is better): 5

Added the usage of raw_output_data_config in flytectl update workflow-execution-config page of docs,

Signed-off-by: Pranshu <[email protected]>
@pranshustuff pranshustuff changed the title Update flytectl_update_workflow-execution-config.rst [Docs] Update flytectl_update_workflow-execution-config.rst Oct 19, 2024
eapolinario
eapolinario previously approved these changes Dec 12, 2024
@eapolinario eapolinario enabled auto-merge (squash) December 12, 2024 18:53
pingsutw
pingsutw previously approved these changes Jan 22, 2025
Copy link

codecov bot commented Jan 22, 2025

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 37.06%. Comparing base (7c84707) to head (3a25012).
Report is 1 commits behind head on master.

Additional details and impacted files
@@           Coverage Diff           @@
##           master    #5865   +/-   ##
=======================================
  Coverage   37.06%   37.06%           
=======================================
  Files        1318     1318           
  Lines      132644   132644           
=======================================
+ Hits        49160    49161    +1     
+ Misses      79235    79233    -2     
- Partials     4249     4250    +1     
Flag Coverage Δ
unittests-datacatalog 51.58% <ø> (ø)
unittests-flyteadmin 54.33% <ø> (+0.02%) ⬆️
unittests-flytecopilot 30.99% <ø> (ø)
unittests-flytectl 62.29% <ø> (ø)
unittests-flyteidl 7.23% <ø> (ø)
unittests-flyteplugins 53.87% <ø> (ø)
unittests-flytepropeller 42.68% <ø> (-0.02%) ⬇️
unittests-flytestdlib 55.29% <100.00%> (+0.01%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@flyte-bot
Copy link
Collaborator

flyte-bot commented Jan 22, 2025

Code Review Agent Run #46cb36

Actionable Suggestions - 0
Review Details
  • Files reviewed - 1 · Commit Range: 40309f8..6254bfd
    • flytectl/docs/source/gen/flytectl_update_workflow-execution-config.rst
  • Files skipped - 0
  • Tools
    • Golangci-lint (Linter) - ✖︎ Failed
    • Whispers (Secret Scanner) - ✔︎ Successful
    • Detect-secrets (Secret Scanner) - ✔︎ Successful

AI Code Review powered by Bito Logo

@flyte-bot
Copy link
Collaborator

flyte-bot commented Jan 22, 2025

Changelist by Bito

This pull request implements the following key changes.

Key Change Files Impacted
Documentation - Configuration Documentation Updates

datacatalog_config.rst - Fixed typo in SQLite DB file path description (re-used -> reused)

flyteadmin_config.rst - Added new sections for catalog-cache, event configuration and updated plugin configurations

Documentation - Configuration Documentation Updates

flyteadmin_config.rst - Updated configuration documentation with detailed settings for resource constraints, rate limiters, caching, and K8s plugin configurations

datacatalog_config.rst - Fixed typo in SQLite DB file path description (re-used -> reused)

Documentation - Configuration Documentation Updates

flyteadmin_config.rst - Added extensive documentation for GPU configuration, co-pilot settings, and K8s array job parameters

datacatalog_config.rst - Fixed typo in SQLite DB file path description (re-used -> reused)

Documentation - Configuration Documentation Restructuring

flyteadmin_config.rst - Major restructuring of configuration documentation - removed co-pilot config section and added new sections for K8s cluster config, logging config, and resource management

Documentation - Configuration Documentation Updates

flyteadmin_config.rst - Added new sections for array node config, literal offloading, admin launcher, and resource manager configurations

flytepropeller_config.rst - Updated default service config for agent service with round-robin load balancing

scheduler_config.rst - Added new sections for catalog-cache, event configuration and updated plugin configurations

Documentation - Scheduler Configuration Documentation Updates

scheduler_config.rst - Major restructuring of scheduler configuration documentation - updated K8s array config, logging settings, and workqueue configurations

Documentation - Scheduler Configuration Documentation Enhancement

scheduler_config.rst - Added comprehensive documentation for scheduler configuration including default settings, co-pilot configuration, resource management, and GPU handling

Documentation - Scheduler Configuration Documentation Enhancement

scheduler_config.rst - Major update to scheduler configuration documentation - replaced co-pilot and resource management sections with new array job settings, logging configuration, and cluster management options

Documentation - Configuration Documentation Updates

scheduler_config.rst - Added new sections for task plugins, backoff config, log templates and resource management

workflows.rst - Fixed typos and improved wording around workflow task reuse

flytectl_update_workflow-execution-config.rst - Added documentation and examples for raw_output_data_config usage

Bug Fix - Minor Text Fixes

migration_models.go - Fixed typo in comment (re-using -> reusing)

application_configuration.go - Fixed typo in SQLite config description

root.go - Fixed typo in kubernets -> kubernetes

config.go - Fixed typo in SQLite config description

dbconfig_flags.go - Fixed typo in SQLite file path description

Signed-off-by: Eduardo Apolinario <[email protected]>
@eapolinario eapolinario dismissed stale reviews from pingsutw and themself via 3a25012 January 22, 2025 23:01
@eapolinario eapolinario merged commit 34d124d into flyteorg:master Jan 22, 2025
56 checks passed
Copy link

welcome bot commented Jan 22, 2025

Congrats on merging your first pull request! 🎉

@flyte-bot
Copy link
Collaborator

flyte-bot commented Jan 23, 2025

Code Review Agent Run #a4f945

Actionable Suggestions - 10
  • docs/deployment/configuration/generated/scheduler_config.rst - 4
  • docs/deployment/configuration/generated/flyteadmin_config.rst - 6
Additional Suggestions - 10
  • docs/deployment/configuration/generated/scheduler_config.rst - 4
  • docs/deployment/configuration/generated/flyteadmin_config.rst - 6
Review Details
  • Files reviewed - 10 · Commit Range: 6254bfd..3a25012
    • docs/deployment/configuration/generated/datacatalog_config.rst
    • docs/deployment/configuration/generated/flyteadmin_config.rst
    • docs/deployment/configuration/generated/flytepropeller_config.rst
    • docs/deployment/configuration/generated/scheduler_config.rst
    • docs/user_guide/concepts/main_concepts/workflows.rst
    • flyteadmin/pkg/repositories/config/migration_models.go
    • flyteadmin/pkg/runtime/interfaces/application_configuration.go
    • flytepropeller/cmd/manager/cmd/root.go
    • flytestdlib/database/config.go
    • flytestdlib/database/dbconfig_flags.go
  • Files skipped - 1
    • rfc/system/2633-eviction-of-cached-task-outputs.md - Reason: Filter setting
  • Tools
    • Golangci-lint (Linter) - ✖︎ Failed
    • Whispers (Secret Scanner) - ✔︎ Successful
    • Detect-secrets (Secret Scanner) - ✔︎ Successful

AI Code Review powered by Bito Logo



default-env-vars-from-env (map[string]string)
agent.Deployment
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Invalid default value for host networking

The default value for enable-host-networking-pod appears to be an invalid reflect value. Consider providing a proper boolean default value like false for clarity.

Code suggestion
Check the AI-generated fix before applying
Suggested change
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
false

Code Review Run #a4f945


Is this a valid issue, or was it incorrectly flagged by the Agent?

  • it was incorrectly flagged


.. code-block:: yaml

""
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Consider meaningful default for cloudwatch region

Consider providing a meaningful default value for cloudwatch-region instead of an empty string, as AWS services require a valid region to function properly.

Code suggestion
Check the AI-generated fix before applying
Suggested change
""
"us-east-1"

Code Review Run #a4f945


Is this a valid issue, or was it incorrectly flagged by the Agent?

  • it was incorrectly flagged

Comment on lines +4908 to +4909
http://localhost:30082/#!/log/{{ .namespace }}/{{ .podName }}/pod?namespace={{ .namespace
}}
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Malformed Kubernetes log template URI

The Kubernetes template URI appears to be malformed. The template string is incomplete and has an unexpected closing brace. Consider reviewing the template format.

Code suggestion
Check the AI-generated fix before applying
Suggested change
http://localhost:30082/#!/log/{{ .namespace }}/{{ .podName }}/pod?namespace={{ .namespace
}}
http://localhost:30082/#!/log/{{ .namespace }}/{{ .podName }}/pod?namespace={{ .namespace }}

Code Review Run #a4f945


Is this a valid issue, or was it incorrectly flagged by the Agent?

  • it was incorrectly flagged

Comment on lines +4908 to +4909
http://localhost:30082/#!/log/{{ .namespace }}/{{ .podName }}/pod?namespace={{ .namespace
}}
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Consider fixing malformed Kubernetes template URI

The Kubernetes template URI appears to be malformed. The template variable interpolation is incomplete with a dangling {{.namespace at the end and an extra closing brace on the next line.

Code suggestion
Check the AI-generated fix before applying
Suggested change
http://localhost:30082/#!/log/{{ .namespace }}/{{ .podName }}/pod?namespace={{ .namespace
}}
http://localhost:30082/#!/log/{{ .namespace }}/{{ .podName }}/pod?namespace={{ .namespace }}

Code Review Run #a4f945


Is this a valid issue, or was it incorrectly flagged by the Agent?

  • it was incorrectly flagged

Comment on lines +4632 to +4633
kubernetes-template-uri: http://localhost:30082/#!/log/{{ .namespace }}/{{ .podName
}}/pod?namespace={{ .namespace }}
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Consider combining template URI into single line

The kubernetes-template-uri value appears to be split across two lines which could cause parsing issues. Consider combining the URL into a single line.

Code suggestion
Check the AI-generated fix before applying
Suggested change
kubernetes-template-uri: http://localhost:30082/#!/log/{{ .namespace }}/{{ .podName
}}/pod?namespace={{ .namespace }}
kubernetes-template-uri: http://localhost:30082/#!/log/{{ .namespace }}/{{ .podName }}/pod?namespace={{ .namespace }}

Code Review Run #a4f945


Is this a valid issue, or was it incorrectly flagged by the Agent?

  • it was incorrectly flagged



gpu-partition-size-node-label (string)
enabled (bool)
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Parameter name change may alter functionality

The configuration parameter name has been changed from gpu-partition-size-node-label to enabled. This change appears to significantly alter the functionality from GPU configuration to a generic enable/disable flag. Consider if this is the intended change.

Code suggestion
Check the AI-generated fix before applying
Suggested change
enabled (bool)
gpu-partition-size-node-label (string)

Code Review Run #a4f945


Is this a valid issue, or was it incorrectly flagged by the Agent?

  • it was incorrectly flagged

Comment on lines +4908 to +4909
http://localhost:30082/#!/log/{{ .namespace }}/{{ .podName }}/pod?namespace={{ .namespace
}}
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Consider fixing malformed template URI string

The Kubernetes template URI appears to be malformed. The template string has an unexpected closing brace }} on a new line which could cause parsing issues.

Code suggestion
Check the AI-generated fix before applying
Suggested change
http://localhost:30082/#!/log/{{ .namespace }}/{{ .podName }}/pod?namespace={{ .namespace
}}
http://localhost:30082/#!/log/{{ .namespace }}/{{ .podName }}/pod?namespace={{ .namespace }}

Code Review Run #a4f945


Is this a valid issue, or was it incorrectly flagged by the Agent?

  • it was incorrectly flagged


.. code-block:: yaml

""
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Consider meaningful default for cloudwatch region

Consider providing a meaningful default value for cloudwatch-region instead of an empty string. An empty string might cause issues when Cloudwatch logging is enabled.

Code suggestion
Check the AI-generated fix before applying
Suggested change
""
"us-east-1"

Code Review Run #a4f945


Is this a valid issue, or was it incorrectly flagged by the Agent?

  • it was incorrectly flagged


.. code-block:: yaml

""
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Consider meaningful default for CloudWatch log group

The default value for cloudwatch-log-group is being changed from 1m40s to an empty string. This could potentially affect CloudWatch logging functionality if the value is required. Consider providing a meaningful default value if CloudWatch logging is intended to be used.

Code suggestion
Check the AI-generated fix before applying
Suggested change
""
"flyte-logs"

Code Review Run #a4f945


Is this a valid issue, or was it incorrectly flagged by the Agent?

  • it was incorrectly flagged

Comment on lines +4908 to +4909
http://localhost:30082/#!/log/{{ .namespace }}/{{ .podName }}/pod?namespace={{ .namespace
}}
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Malformed kubernetes template URI string

The kubernetes template URI appears to be malformed. The template string has an unmatched closing brace at the end and the namespace parameter is repeated. Consider reviewing the template format.

Code suggestion
Check the AI-generated fix before applying
Suggested change
http://localhost:30082/#!/log/{{ .namespace }}/{{ .podName }}/pod?namespace={{ .namespace
}}
http://localhost:30082/#!/log/{{ .namespace }}/{{ .podName }}/pod

Code Review Run #a4f945


Is this a valid issue, or was it incorrectly flagged by the Agent?

  • it was incorrectly flagged

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
Status: Done
Development

Successfully merging this pull request may close these issues.

[Docs] document the process of adding raw output prefix using flytectl
4 participants