Incremental Builds #1304
Incremental Builds #1304
Conversation
| # Old Schedule key is deprecated, but still supported | ||
| $oldWorkflowScheduleKey = "$($baseName)Schedule" | ||
| if ($repoSettings.Keys -contains $oldWorkflowScheduleKey) { | ||
| # DEPRECATION: REPLACE WITH ERROR AFTER April 1st 2025 ---> |
There was a problem hiding this comment.
The release notes say October 1st.
| # <--- REPLACE WITH ERROR AFTER April 1st 2025 | ||
| } | ||
|
|
||
| # Any workflow (except for the PullRequestHandler and reusable workflows (_*)) can have concurrency and schedule defined |
There was a problem hiding this comment.
Technically, PullRequestHandler workflow already has concurrency. Not sure if we should allow changing it though. What do you think?
| $skippedProjects = @($allProjects | Where-Object { $_ -notin $projectsToBuild }) | ||
| if ($skippedProjects) { | ||
| # If we are to publish artifacts for skipped projects later, we include the full project list and in the build step, just avoid building the skipped projects | ||
| $allProjects, $projectsToBuild, $projectDependencies, $buildOrder = Get-ProjectsToBuild -baseFolder $baseFolder -buildAllProjects $true -modifiedFiles $modifiedFiles -maxBuildDepth $maxBuildDepth | ||
| } |
There was a problem hiding this comment.
These few lines should be above line 79.
| @@ -108,6 +115,7 @@ Write-Host "Downloading dependencies for project '$project'. BuildMode: $buildMo | |||
| $downloadedDependencies = @() | |||
|
|
|||
| Write-Host "::group::Downloading project dependencies from current build" | |||
| Write-Host $projectsDependenciesJson | |||
There was a problem hiding this comment.
Could this logging be too chatty?
| break | ||
| } | ||
| if ($CICDRun.conclusion -eq 'cancelled') { | ||
| continue |
There was a problem hiding this comment.
What if the CICD run was cancelled during the deploy/deliver jobs?
If the build jobs are are successful, why not consider the CICD run successful?
|
|
||
| $settings = $env:Settings | ConvertFrom-Json | ||
|
|
||
| if ($settings.alwaysBuildAllProjects) { |
There was a problem hiding this comment.
The setting is now deprecated, so this part should be removed at some point.
| - More than 250 files have been modified | ||
| - The modified files contain a file that matches one of the fullBuildPatterns | ||
| #> | ||
| function Get-BuildAllProjects { |
There was a problem hiding this comment.
This function doesn't seem like a good fit here. There's nothing about it that is related to GitHub.
| - Get-BuildAllProjects returns true | ||
| - The .AL-Go/settings.json file has been | ||
| #> | ||
| function Get-BuildAllApps { |
There was a problem hiding this comment.
Same as Get-BuildAllProjects: consider moving it out of GitHub Helper.
| @@ -174,7 +198,7 @@ jobs: | |||
|
|
|||
| - name: Publish artifacts - apps | |||
| uses: actions/upload-artifact@65c4c4a1ddee5b72f698fdd19549f0f0fb45cf08 # v4.6.0 | |||
| if: inputs.artifactsRetentionDays >= 0 | |||
| if: inputs.artifactsRetentionDays >= 0 && (hashFiles(format('{0}/.buildartifacts/Apps/*',inputs.project)) != '') | |||
There was a problem hiding this comment.
Why is the new condition needed?
There's if-no-files-found: ignore
Co-authored-by: Maria Zhelezova <[email protected]>
Support incremental builds
TODO
Use git diff instead of API
In v6.1 and earlier versions, we use the GitHub API to determine last known good build and determine modified files in the Pull Request. It is however possible to have done modifications to the main branch after the last known good build - the changes will not be included in the PR and this the PR builds will only include the modified files in the PR together with the last known good build - potentially missing files.
This version locates the SHA, which was used to build the last known good build and uses git diff to determine all changes from that build including the changes in the PR to build what's needed.
git diff also supports many more files than the API, is faster and it doesn't count as API calls against the GitHub limit.
Changed Schedule to workflowSchedule
Instead of using dynamic setting key name - use a workflow specific setting and deprecate the old keys.
Add concurrency protection to workflows
Add new workflow specific setting called - workflowConcurrency. It is used to control concurrency of workflows.
Like with the
WorkflowSchedulesetting, this setting should be applied in workflow specific settings files or conditional settings. By default, all workflows allows for concurrency, except for the Create Release workflow. If you are using incremental builds in CI/CD it is also recommented to set WorkflowConcurrency to:in order to cancel prior incremental builds on the same branch.
Read more about workflow concurrency here.
Remove warnings that tests doesn't exist if doNotBuildTests is true
No reason to warn about missing test apps if you don't want to build them anyway
Remove warnings and error when locating previous builds
Getting rid of warnings like this:
due to DownloadProjectDependencies downloads artifacts double
and errors like:
if there wasn't a successful build in the repo.
Check whether a package has been delivered to GitHubPackages or NuGet before delivering
If the package already exists in the current version it has already been delivered (might be from an earlier build)