Matthewz Datapuller Button #1096
+3,651
−22
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,64 @@ | ||
| name: "Datapuller: Manual Trigger" | ||
| run-name: "Datapuller: ${{ github.event.inputs.job }} triggered by ${{ github.actor }}" | ||
|
|
||
| on: | ||
| workflow_dispatch: | ||
| inputs: | ||
| job: | ||
| description: "Select which datapuller job to run" | ||
| required: true | ||
| type: choice | ||
| options: | ||
| - terms-all | ||
| - terms-nearby | ||
| - courses | ||
| - sections-active | ||
| - sections-l5y | ||
| - classes-active | ||
| - classes-l5y | ||
| - grades-recent | ||
| - grades-l5y | ||
| - enrollments | ||
| - enrollment-timeframe | ||
|
|
||
| jobs: | ||
| trigger: | ||
| name: Trigger and Stream Logs | ||
| runs-on: ubuntu-latest | ||
|
|
||
| steps: | ||
| - name: SSH, Trigger Job, and Stream Logs | ||
| uses: appleboy/ssh-action@v1.2.0 | ||
| with: | ||
| host: ${{ secrets.SSH_HOST }} | ||
| username: ${{ secrets.SSH_USERNAME }} | ||
| key: ${{ secrets.SSH_KEY }} | ||
| command_timeout: 25m | ||
| script: | | ||
| set -e | ||
|
|
||
| # define variables | ||
| CRONJOB="bt-prod-app-datapuller-${{ github.event.inputs.job }}" | ||
| JOB_NAME="${CRONJOB}-manual-$(date +%s)" | ||
| NAMESPACE="bt" | ||
|
|
||
| # set timeout based on job type | ||
| case "${{ github.event.inputs.job }}" in grades-l5y|sections-l5y|classes-l5y) | ||
| TIMEOUT=1200s ;; # 20 min | ||
| *) | ||
| TIMEOUT=600s ;; # 10 min | ||
| esac | ||
|
|
||
| echo "=== Creating job ${JOB_NAME} from cronjob ${CRONJOB} ===" | ||
| kubectl create job "${JOB_NAME}" --from="cronjob/${CRONJOB}" -n "${NAMESPACE}" | ||
|
|
||
| echo "=== Waiting for pod to start ===" | ||
| kubectl wait --for=condition=ready pod -l job-name="${JOB_NAME}" -n "${NAMESPACE}" --timeout=120s || true | ||
|
|
||
| echo "=== Streaming logs ===" | ||
| kubectl logs -n "${NAMESPACE}" -l job-name="${JOB_NAME}" --follow || true | ||
|
|
||
| echo "=== Waiting for job to complete ===" | ||
| kubectl wait --for=condition=complete job/"${JOB_NAME}" -n "${NAMESPACE}" --timeout="${TIMEOUT}" | ||
|
|
||
| echo "=== Job completed successfully ===" |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,42 @@ | ||
| drop df5cf8460 # feat: Implement OpenTelemetry-based observability stack | ||
| pick 46aabe3c0 # datapuller button test | ||
| pick ea932f783 # added status for data puller | ||
| pick 6bef7b42f # fix: resolve circular data reference in useDatapullerJobStatus | ||
| pick 1343d049a # package updates | ||
| pick bf5aa310a # package changes | ||
| pick 27900b265 # changed to stable selectors | ||
| pick 8ca60d940 # revert: disable deploy_staff in dev workflow | ||
| pick ecb2c3d22 # staff dashboard test in dev | ||
| pick a5c2072bb # staff dashboard deployment in dev | ||
| pick 9d8a1e6d6 # don't deploy staff dashboard | ||
|
|
||
| # Rebase 57fb92c94..9d8a1e6d6 onto 57fb92c94 (11 commands) | ||
| # | ||
| # Commands: | ||
| # p, pick <commit> = use commit | ||
| # r, reword <commit> = use commit, but edit the commit message | ||
| # e, edit <commit> = use commit, but stop for amending | ||
| # s, squash <commit> = use commit, but meld into previous commit | ||
| # f, fixup [-C | -c] <commit> = like "squash" but keep only the previous | ||
| # commit's log message, unless -C is used, in which case | ||
| # keep only this commit's message; -c is same as -C but | ||
| # opens the editor | ||
| # x, exec <command> = run command (the rest of the line) using shell | ||
| # b, break = stop here (continue rebase later with 'git rebase --continue') | ||
| # d, drop <commit> = remove commit | ||
| # l, label <label> = label current HEAD with a name | ||
| # t, reset <label> = reset HEAD to a label | ||
| # m, merge [-C <commit> | -c <commit>] <label> [# <oneline>] | ||
| # create a merge commit using the original merge commit's | ||
| # message (or the oneline, if no original merge commit was | ||
| # specified); use -c <commit> to reword the commit message | ||
| # u, update-ref <ref> = track a placeholder for the <ref> to be updated | ||
| # to this position in the new commits. The <ref> is | ||
| # updated at the end of the rebase | ||
| # | ||
| # These lines can be re-ordered; they are executed from top to bottom. | ||
| # | ||
| # If you remove a line here THAT COMMIT WILL BE LOST. | ||
| # | ||
| # However, if you remove everything, the rebase will be aborted. | ||
| # |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
this file shouldnt be pushed