Skip to content
This repository has been archived by the owner on Jul 30, 2024. It is now read-only.

Add support for download reproducers by TestRun ID #33

Merged
merged 2 commits into from
Dec 11, 2023
Merged

Add support for download reproducers by TestRun ID #33

merged 2 commits into from
Dec 11, 2023

Conversation

katieworton
Copy link
Member

Add support in squadutilslib and squad-create-reproducer for downloading the build or test reproducer by TestRun ID as an alternative to searching for the latest test reproducer in a given group and project.

@katieworton katieworton force-pushed the add_testrun_reproducer_download branch from 2469e31 to 441e807 Compare November 15, 2023 13:58
@katieworton
Copy link
Member Author

@roxell if you are able to look over this PR when you have the opportunity that would be great.

I changed the input from the planned "job_url" input to the testrun ID since I discovered downloading the build reproducer requires the metadata file (which is accessible via the testrun).

@roxell
Copy link
Collaborator

roxell commented Nov 20, 2023

This should probably be in its own script with its own parameters and then just call the common parts in squadutilslib to make it easier since the imput variables are so different or what do you think?

@katieworton katieworton force-pushed the add_testrun_reproducer_download branch 4 times, most recently from 505d6fe to f279421 Compare November 20, 2023 17:40
@katieworton
Copy link
Member Author

@roxell I have created a new script squad-create-reproducer-from-testrun and reverted the old script.

I also updated get_reproducer in squadutilslib.py to use the get_reproducer_from_testrun utility to do its download step.

Let me know what you think.

Also - I noticed the flake8 check only checks the .py files of the repo. I am not sure if it was intentionally set up this way, but I had some unused imports that it missed as a result. If we do want to run flake8 on the other Python files perhaps something like flake8 --exclude=dist/ --ignore=E501,W503 . squad-* could be used.

roxell
roxell reviewed Nov 23, 2023
View reviewed changes
squad-create-reproducer Outdated Show resolved Hide resolved
squadutilslib.py Show resolved Hide resolved
@katieworton katieworton force-pushed the add_testrun_reproducer_download branch from f279421 to b7112f1 Compare November 23, 2023 13:57
@katieworton
Copy link
Member Author

@roxell I have made the suggested updates and created a PR for fixing flake8 checks #34

@katieworton katieworton force-pushed the add_testrun_reproducer_download branch 2 times, most recently from d75c649 to f704014 Compare November 28, 2023 20:35
@katieworton
Copy link
Member Author

Latest pushes add support for fetching tux plan reproducers. Currently, the "test" side does not work as the new TuxSuite patch does not appear to expose the download URL for the tux plan test reproducer in the same way as the other test reproducer URLs. I have followed up on this with the Tux team.

@katieworton
squadutilslib: Add get_reproducer_from_testrun
5497e77 
Add get_reproducer_from_testrun to allow downloading and returning of a
reproducer from a given testrun ID. This function downloads the
reproducer for the given testrun to a provided filename. The optional
`local` flag can be provided to download the tuxmake or tuxrun
reproducer command rather than the tuxsuite reproducer command.

This function will be used in a script to download the reproducer file
for a build or test from a TestRun, which is useful for reproducing
builds and tests.

Update get_reproducer to use the get_reproducer_from_testrun function to
download the reproducer from the testrun.

Signed-off-by: Katie Worton <[email protected]>
@katieworton katieworton force-pushed the add_testrun_reproducer_download branch from f704014 to 505e051 Compare November 29, 2023 10:19
@katieworton
Copy link
Member Author

katieworton commented Nov 29, 2023 •
edited
Loading

The Tux team followed up by added an endpoint to download the tux plan reproducers for tests. I have updated my patch to perform this download and have tested that it works.

@roxell I have added support for fetching tux plan reproducers - if you could have a look over this patch again that would be great.

@roxell
Copy link
Collaborator

roxell commented Dec 7, 2023 •
edited
Loading

When I ran the command "./squad-create-reproducer-from-testrun --testrun 21531365 --plan --filename 21531365.json" I didn't expect to see any output in the console.
I mean when --filename are passed no need to print the output in the console then.

@katieworton katieworton force-pushed the add_testrun_reproducer_download branch from 505e051 to 864ac92 Compare December 7, 2023 10:35
@katieworton
Copy link
Member Author

@roxell - I have now pushed an update which means the reproducer is only printed when --filename is not provided.

@katieworton katieworton force-pushed the add_testrun_reproducer_download branch from 864ac92 to 646ee80 Compare December 7, 2023 10:47
@roxell
Copy link
Collaborator

roxell commented Dec 7, 2023

we do not need the yaml file to be exeutable...
maybe we just need to provide the url to the plan_file since tuxsuite can handle a url too what's your thoughts?

@katieworton
Copy link
Member Author

This is true - I think in both cases my only concern would be the number of code paths to maintain.

@katieworton katieworton force-pushed the add_testrun_reproducer_download branch 2 times, most recently from da2be2d to 8458589 Compare December 7, 2023 16:06
@katieworton
Copy link
Member Author

katieworton commented Dec 7, 2023 •
edited
Loading

I have removed the feature of fetching TuxTest reproducers to improve maintainability (since it is unnecessary if we download TuxPlans) and updated things so plans don't get marked as executable. I have updated the commit message and README to reflect these changes.

@roxell let me know what you think of these changes 👍

roxell
roxell reviewed Dec 11, 2023
View reviewed changes
squad-create-reproducer-from-testrun Outdated
filename = args.filename

try:
reproducer, is_test_reproducer = get_reproducer_from_testrun(
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

when is is_test_reproducer used?

Can't that be dropped?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Have now dropped this 👍

@katieworton
squad-create-reproducer-from-testrun: Download reproducers by testrun
f0ff779 
Add support for downloading reproducers by testrun ID in the
squad-create-reproducers script. This allows the user to provide the
--testrun parameter to download the build or test reproducer from a
specific testrun, using the get_reproducer_from_testrun function from
squadutilslib.

The --local flag can be provided to fetch TuxRun reproducer, otherwise
the TuxPlan reproducer will be downloaded by default.

Signed-off-by: Katie Worton <[email protected]>
@katieworton katieworton force-pushed the add_testrun_reproducer_download branch from 8458589 to f0ff779 Compare December 11, 2023 11:28
@roxell roxell merged commit 09ef6cd into Linaro:master Dec 11, 2023
1 check passed
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Reviewers

@roxell roxell roxell left review comments

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@katieworton @roxell