Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Generate report script (urgent section) #62

Merged

Conversation

Crola1702
Copy link
Contributor

@Crola1702 Crola1702 commented Jun 25, 2024

Description

As discussed in #57 (comment), the new report must contain an urgent section.

The generate_report.rb uses the library from #60 to generate a JSON file containing information from all sections of the defined report. As an example:

{
  "urgent": {
    "build_regressions": [ ... ],
    "test_regressions_consecutive": [ ... ],
    "test_regressions_flaky": [ ... ]
  },
  "maintenance": {
    "jobs_failing": [ <empty> ],
    "gh_issues_reported": [ <empty> ],
    "tests_disabled": [ <empty> ]
  },
  "pending": {
    "build_regressions_known": [ <empty> ],
    "test_regressions_all": [ <empty> ],
    "test_regressions_known": [ <empty> ],
    "build_regressions_known": [ <empty> ]
  }
}

A complete example generated report: buildfarm-report_2024-06-25_12-10.json

@Crola1702 Crola1702 requested a review from a team June 25, 2024 15:36
@Crola1702 Crola1702 self-assigned this Jun 25, 2024
database/scripts/generate_report.rb Show resolved Hide resolved
database/scripts/generate_report.rb Outdated Show resolved Hide resolved
Signed-off-by: Crola1702 <[email protected]>
@Crola1702 Crola1702 merged commit 08b5a0a into Crola1702/improved-buildfarm-tools-lib Jun 28, 2024
@Crola1702 Crola1702 deleted the Crola1702/report-generation branch June 28, 2024 17:31
Crola1702 added a commit that referenced this pull request Jun 28, 2024
* Add BuildfarmToolsLib module with basic functions

Signed-off-by: Crola1702 <[email protected]>

* Add flaky test regressions keys

Signed-off-by: Crola1702 <[email protected]>

* Restore test_regression_history script

Signed-off-by: Crola1702 <[email protected]>

* Make check-buildfarm work with current library

Signed-off-by: Crola1702 <[email protected]>

* Code cleanup

Signed-off-by: Crola1702 <[email protected]>

* Use flaky builds threshold variable

Signed-off-by: Crola1702 <[email protected]>

* Add warning age constat

Signed-off-by: Crola1702 <[email protected]>

* Handle empty results from query better

Signed-off-by: Crola1702 <[email protected]>

* Fix age parsing errors

Signed-off-by: Crola1702 <[email protected]>

* Remove irrelevant comments

Signed-off-by: Crola1702 <[email protected]>

* Address PR comments

Signed-off-by: Crola1702 <[email protected]>

* Rename test regression history

Signed-off-by: Crola1702 <[email protected]>

* Generate report script (urgent section) (#62)

* Add generate report script for urgent section

Signed-off-by: Crola1702 <[email protected]>

* Change subsection names

Signed-off-by: Crola1702 <[email protected]>

---------

Signed-off-by: Crola1702 <[email protected]>

---------

Signed-off-by: Crola1702 <[email protected]>
Crola1702 added a commit that referenced this pull request Jun 28, 2024
* Add age column

Signed-off-by: Crola1702 <[email protected]>

* Add latest buildfarmer.db data

Signed-off-by: Crola1702 <[email protected]>

* Improve buildfarm tools lib (#60)

* Add BuildfarmToolsLib module with basic functions

Signed-off-by: Crola1702 <[email protected]>

* Add flaky test regressions keys

Signed-off-by: Crola1702 <[email protected]>

* Restore test_regression_history script

Signed-off-by: Crola1702 <[email protected]>

* Make check-buildfarm work with current library

Signed-off-by: Crola1702 <[email protected]>

* Code cleanup

Signed-off-by: Crola1702 <[email protected]>

* Use flaky builds threshold variable

Signed-off-by: Crola1702 <[email protected]>

* Add warning age constat

Signed-off-by: Crola1702 <[email protected]>

* Handle empty results from query better

Signed-off-by: Crola1702 <[email protected]>

* Fix age parsing errors

Signed-off-by: Crola1702 <[email protected]>

* Remove irrelevant comments

Signed-off-by: Crola1702 <[email protected]>

* Address PR comments

Signed-off-by: Crola1702 <[email protected]>

* Rename test regression history

Signed-off-by: Crola1702 <[email protected]>

* Generate report script (urgent section) (#62)

* Add generate report script for urgent section

Signed-off-by: Crola1702 <[email protected]>

* Change subsection names

Signed-off-by: Crola1702 <[email protected]>

---------

Signed-off-by: Crola1702 <[email protected]>

---------

Signed-off-by: Crola1702 <[email protected]>

---------

Signed-off-by: Crola1702 <[email protected]>
@Crola1702 Crola1702 mentioned this pull request Jul 1, 2024
@Crola1702
Copy link
Contributor Author

@Blast545 #64 is what I've been working on for formatting.

Ideas about formatting of next sections include:

  • Pretty titles: Some capitalization and spacing to make titles more readable
  • Format as tables with internal lists and summaries: There are really long outputs, and these can make the report pretty much unreadable.
  • Group test consistent regressions: Consistent test regressions can be grouped by job name and age, assuming that two regressions that are happening in the same job and have the same amount of consecutive failures are part of a same issue.
  • Group flaky test regressions: Flaky test regressions can be grouped by similar flakiness amount, that is, if two regressions are happening in the same set of jobs, they might be the same issue (as they're flaky we can't assure it, but it's the most probable statement)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants