CI Improvement Tasks

[lucasssvaz]

Some ideas for improving our CI and making it more reliable and capable:

• Wokwi support (ci(wokwi): Add Wokwi emulator to workflow #9540)
• QEMU support (ci(qemu): Add QEMU emulator to CI #9558)
• Run Wokwi or QEMU always with regular compilation tests
  • Separate test workflows in different files so it can be run concurrently (ci(refactor): Refactor workflows and skip files #9696)
• Performance Tests (ci(performance): Add performance tests to CI #9560)
  • Add performance report
• Add runtime tests report
• Optimize workflows for running only when specific file types and paths are changed (ci(refactor): Refactor workflows and skip files #9696)
  • Compile only sketches that were changed (ci(refactor): Refactor workflows and skip files #9696)
• Add dynamic runner label selection for running specific tests
  • Get HW tests runners fixed first (We can't do it ourselves as the runners are not managed by us)
  • Add specialized hardware tests (e.g. Wi-Fi)
• Improve caching of libs and tools (ci(refactor): Refactor workflows and skip files #9696)
• Test variant.cpp in boards test
• Refactor and rename cfg.json files (ci(refactor): Refactor workflows and skip files #9696)
• Add release installation test (ci(release): Add JSON install test #10104)
• Add concurrency to Lib Builder cron jobs (Split cron jobs and move libs to releases esp32-arduino-lib-builder#187)
• Add more tests
  • CPU test (Freq. change, clocks, light/deep sleep, etc.)
  • FS test
  • I2C Master
  • SPI
• Boards test - add VID/PID check
• Add P4 (Waiting for new release of pytest-embedded)
• Add Bash linter and formatting
• Add way to re-trigger the pre-commit bot
• Trigger PR workflow only if not in push branches
• Make CI compatible with act (needs investigation)
• Use esp-idf-size for calculating sketch sizes
• Publish package in chinese mirrors
• Add CodeQL checks