Skip to content

RFC: Gem and Engine versioning strategy #354

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 1 commit into
base: main
Choose a base branch
from
Open

RFC: Gem and Engine versioning strategy #354

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
120 changes: 120 additions & 0 deletions rfcs/rfc-202508-versioning.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,120 @@
# Versioning Strategy for O3DE Engine and Gems

# Summary:

A versioning strategy is essential for maintaining the integrity and usability of software projects. This document outlines the versioning practices used in our repositories, highlights the problems with the current approach, and lists some ideas without giving a definitive solution.

## What is the relevance of this feature?

### Current Versioning Strategy

The current versioning strategy is described in details in [RFC #44](https://github.com/o3de/sig-core/issues/44). Most of the document focuses on the implementation. The summary of the procedures can be found below.

Versioning and dependency information is currently stored inside `engine.json`, `gem.json`, and `project.json` files. We use the [Semantic Versioning](https://semver.org/) scheme, which consists of three components: _MAJOR_, _MINOR_, and _PATCH_.
Copy link

@byrcolin byrcolin Aug 13, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This should be all O3DE object files have a version field.


- _MAJOR_ version changes indicate breaking changes.
- _MINOR_ version changes indicate new features that are backward-compatible.
- _PATCH_ version changes indicate backward-compatible bug fixes.

The versioning information is updated by the developers manually. It will be automated in the future. The _stabilization_ branch takes the version number from the HEAD of the _development_ branch at the branch cutoff, and the _development_ branch should get an immediate update of the _MINOR_ version.
Copy link

@nick-l-o3de nick-l-o3de Aug 7, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this is one of the cornerstone changes we have to make.

I don't think that dev needs to necessarily get an immediate update, (it will naturally move forward anyway), but I would not object to it.

Any other decisions we make or processes we create basically have to deal with the above reality, that is, that released versions and dev versions are on the "same version timeline" now.

Copy link

@byrcolin byrcolin Aug 26, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I agree, I'm not convinced this an immediate bump is necessary, but if there is an argument for it, I think that it is fine.

Copy link

@byrcolin byrcolin Aug 13, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not sure it can be automated in the future. This is a human process and I would like to hear more about ideas to automate it. The problem is the version is contained in a file which itself is under source control the serial nature of which is not controllable. A branches, makes a change and correctly determines the version should be bumped to 1.0.1 and its PR is ready to merge, but before they can merge B makes a change, correctly determines that the version should be changed to 1.1.0 and merges. Then A merges and the version is back to 1.0.1 If we automate the AR to fail if the version isn't greater than the current dev, this still has a gaping whole of time because the second got in first. We could have many in queue and this would be all over the place. Also the AR is a snapshot in time, A passes right now because B isn't merged yet. B also passes because A hasn't merged yet. C also passes... and so on. So you can have any number of unmerged PR's at any one time. And once they pass they dont get unpassed when someone else merges ahead of you.


It is important to note, that there is one more RFC that describes the versioning strategy for the Engine itself, which is [rfc-core-2022-05-31-engine-versioning](https://github.com/o3de/sig-core/blob/main/rfcs/rfc-core-2022-05-31-engine-versioning.md), which describes the versioning in the format of `YEAR.MONTH.RELEASE`, e.g. `25.05.1` for the first path (point-release) version of the Engine with the planned release date in May 2025.
Copy link

@nick-l-o3de nick-l-o3de Aug 7, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this is actually our saving grace, since this is what is shown to users, as the "name of the release" rather than the version, and it lets us make deep changes or jumps to the internal versioning, without looking like we're making a big jump (as in, even though the internal release version will go from 2 to 4 or the dev will, it won't represent a outward facing big jump from a PR perspective, as it will still be 25.x to 26.x or whatever...)

Copy link

@byrcolin byrcolin Aug 27, 2025 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

After reading the rfc-core-2022-05-31-engine-versioning document it starts off by talking about a "advertised/named version" which is what we now call a "release version" or "release name" which I prefer, because it is a string not a version, and rightly says this is inadequate to coordinate the matching of other objects and suggests adding an "internal version" which is what we call an "object version" to do that. I agree... and as this is an old document, we already have one. All o3de objects, engine.json, project.json, gem.json, template.json, repo.json, restricted.json have a "version" field, which is the object version, though not currently used/enforced except for some minor checks that can produce a warning, done by project manager. It says the object version will not use the same schema as the release version; This is correct, it follows a standard semver. (1.0.0, 1.2.1, 2.0.1 etc. major.minor.patch)

It is less clear what the major, minor patch mean. I would suggest we define it to mean something very clear like:

  • Major number increment (major+1.minor+0.patch+0) = "Completely new object. No expectation of any backward compatibility with previous versions whatsoever. Upgrade process MAY exist, but not guaranteed at all."
  • Minor number increment (major+0.minor+1.patch+0) = "A significant new feature has been added/api has changed, an upgrade process MAY be needed to get previously working source code/assets to work again."
  • Patch number increment (major+0.minor+0.patch+1) = "Compilation change, compatibility with previous version guaranteed/contains nothing that would break compatibility, upgrade process is not needed, no new features, bug fixes to current feature set. No upgrade process needed, source code using the previous api and or assets will work without changes."
  • No increment (major+0,minor+0,patch+0) = "No compilation change, possibly resource only change (like a icon change), comment change, documentation change, meta data change that doesn't effect compilation. Absolute backward compatibility guaranteed with previous version, no upgrade process needed, previous source code using the api and or assets will work without changes."

For specifying compatibility/dependability with other object(s), semantically, it more or less follows the python notation which is object_name[OPTIONAL(<|>|>=|<=|~=)major.minor.patch OPTIONAL[(<|<=)major.minor.patch)]]
As an example, an object is expressing compatibility/dependence on the Achievement gem)
Valid:

  • "Achievement" (with no version) meaning the highest version of Achievement Gem found that can satisfy all constraints, in cmake without EXACT and no range args.
  • "Achievement=1.0.0" meaning the Achievement Gem version 1.0.0 and only 1.0.0 can satisfy the compatibility, in cmake EXACT with no range args.
  • "Achievement<1.0.0" meaning the highest version found that can satisfy all constraints, that is less than but not including 1.0.0, in cmake [0.0.0...<1.0.0] or [...<1.0.0] (Very uncommon, may never see this, but supported)
  • "Achievement>1.0.0" meaning the highest version found that can satisfy all constraints, that is greater than but not including 1.0.0, in cmake [>1.0.0...] (Very uncommon, may never see this, but supported)
  • "Achievement<=1.0.0" meaning the Achievement Gem highest version found that can satisfy all constraints, less than or equal to 1.0.0, in cmake [...1.0.0] or [0.0.0...1.0.0] (Very uncommon, may never see this, but supported)
  • "Achievement>=1.0.0" meaning the Achievement Gem highest version found that can satisfy all constraints, greater than or equal to 1.0.0, in cmake [1.0.0...] (This is probably the most common compatibility expression we will see on the objects because it marks the start of compatibility without a known upper limit and will not change until a compatibility break is known, then this should be updated to having the upper bound BEFORE updating the version to the next open ended compatibility)
  • "Achievement~=1.0.0" meaning the Achievement Gem highest version found that can satisfy all constraints, bounded by next minor version, in cmake [1.0.0...<1.1.0] (This may be somewhat common... as it describes an implicit upper bound on next minor version change meaning compatible until next minor version change) NOTE: that we don't support other forms like "Achievement~=1" which would imply major version upper bound [1.0.0...<2.0.0] or "Achievement~=1.0" which would imply a next patch bound [1.0.0....<1.0.1]
  • "Achievement>=1.0.0<2.5.0" meaning Achievement Gem highest version found that can satisfy all constraints, greater than or equal to 1.0.0 but less then 2.5.0 (This should be another very common expression as it expresses a known upper bound that is NOT major bound, and is clearer to read frankly)
  • "Achievement<1.0.0<2.5.0" meaning Achievement Gem highest version found that can satisfy all constraints, less than 1.0.0, the <2.5.0 is superfluous as 1.0.0 is already an upper bound and though technically not invalid as both constraints are technically satisfied (This should be very uncommon expression, but technically supported)
  • "Achievement<=1.0.0<=2.5.0" meaning Achievement Gem highest version found that can satisfy all constraints, less than or equal to 1.0.0, again the <=2.5.0 is superfluous as 1.0.0 is already an upper bound and though technically not invalid as both constraints are technically satisfied (This should be very uncommon expression, but technically supported)
  • "Achievement>=1.0.0<=2.5.0" meaning Achievement Gem highest version found that can satisfy all constraints, greater than or equal to 1.0.0 and less then or equal to 2.5.0 (This should be far less common as upper bound is including the version, sometimes you see this as >=1.0.0<=1.9999.9999 when they should have really just put <2.0.0 hopefully uncommon, but technically supported)
    Invalid
  • "Achievement<0.0.0" version are positive numbers and cannot be negative and therefore can never be satisfied.
  • "Achievement>1.0.0<1.0.0" versions can not be simultaneously lower than and greater than a specific version. This includes >= <= variants.
  • "Achievement>1.0.0<0.5.0" versions can not be simultaneously greater than a lower bound and lower then a bound lower then the lower bound. This includes >= <= variants.
  • "Achievement~=1.0.0<1.0.0 versions with implicit upper bounds combined with upper bound less then the lower bound. This includes <= variants.

The document resumes: "It can be used to detect the need for upgrades to compatible gems, assets and other data schemas after new change to the engine is pulled or installed."
Technically yes, but only because the engine is an object itself, just as any other o3de object, it is not exclusive to an engine object, this applies to all objects.

Solving Dependencies
It is not enough that all dependencies EXIST, they have to all be compatible: Say object A1.0.0 requires object B=1.0.0 and C=1.0.0 then C requires B=2.0.0 which can never be satisfied, or in a longer chain of dependency A->B->C->D->F->G->A (oops).
If exact matching i.e. "o3de_object=" is used this may happen often, if we use ranges, preferably open ended ranges, this should happen far less often: A1.0.0 requires B>=1.0.0 and C>=1.0.0, which finds B1.0.0, B1.5.0, B1.6.0, C1.2.0 and C1.3.0. Each of which has their own dependencies: C1.2.0 requires B>=1.5.0 and C1.3.0 requires B<1.6.0. B1.5.0 requires D>3.0... B1.6 requires D>3.5 and so on...
These dependencies CAN be solved by choosing a specific version of, B=1.5.0 EVEN IF B=1.6 is present... This is why above it says "highest version found that can satisfy all constraints" Solving all these constraints is not trivial, but there is open source software, Resolvelib, that can do it for us in the same manner in which CMAKE does it internally. Project Manager already uses it to determine compatibility of a set of objects.

The document continues: "All of the changes that need to be made for a project to support the latest version of the engine can be determined based on the version difference from the project's last upgrade. This system can also be used to predict whether a gem is compatible with the engine, as gems can provide a range of compatible engines versions."
Again technically yes, but this is not exclusive to the gem objects. And yes, technically CMAKE will only care about versioning of potential source code objects like engine, project, gem and restricted objects, as other objects compatibility such as repo, template objects do not matter for compilation.

This document goes on to state: "A new field will be added to engine.json at the root the o3de directory called 'EngineVersion'. The field 'O3DEVersion' already exists but is used for the advertised/named version number. This field will be used to track the current version of engine. To ensure this field is only updated when it should it will be it has been added to the .codeowners under sig-core to request a review from the group."
This is incorrect. "EngineVersion" should not be a thing, it seems that this was supposed to be ultimate parent version for all children objects... We should NOT do that. The engine object has it own "version" field, and do all objects and the children DO NOT inherit the ultimate parents "EngineVersion" or "version" field. I think either the author was unaware that all objects had a "version" field, because this document was created 5/32/2022 and the "version" field I believe was already present by then, I could be wrong, but I'm pretty sure it was. In any case each object has its own version and does not share the ultimate parents version.

Object Version Releases
Currently we do not support gem binary releases, but we should. We could currently support gem source releases without too much trouble, just a GHA script could be added to each gem and executed object that would create a zip of the source files which would be available on github releases. Currently in order to create a binary of a gem, you have to include that gem from an engine object or project object or from a gem the project or engine object already uses, as engines and projects are currently the only roots for CMAKE to configure/build. Now, we could theoretically grab the artifacts from one of these builds and create a binary release of sorts, but the goal here should be the ability to configure and build gems directly and independently. And to be clear, NOT ALL GEMS SHOULD NEED A BINARY or SOURCE RELEASE. I think only root level gems, like gems that describe a system, should. For instance Atom is a tree of gems. I'm not sure it really makes a whole lot of sense to have a release of the Atom RHI gem, i.e. do this for every gem inside Atom. Though technically you could, you could separate them all and it would work, but I think keeping them all in one "Atom release" might make more sense? Also projects currently have a game "Gem" inside them and though I think this is probably unnecessary and not useful, it would not make sense to have a release of the project and a separate release of the game gem inside the project. (Personally, I think a project having a permanent game "Gem" this is kind of an abuse of the object system, as the projects game "Gem" will never be reused in other projects and I would like to see the game "Gem" removed from projects in the future as I think it muddies the water for what we are actually trying to achieve here, but some might feel otherwise.)

Proposal: Specifying what CMAKE should do/which release to use:
Schema 2.0.0 will have the ability to specify releases, so CMAKE will know about them. Once we have this ability to independently configure and build a source and binary release of a gem, we need a way to tell CMAKE which to use. Project Manager would be able to detect/warn and even satisfy all such dependencies by downloading either the git repo at a certain branch/commit OR the source release zip OR the binary release zip OR any and all combination of them, so how do we encode which one we want CMAKE to use? We would have to come up with a way to tell CMAKE what combination we wanted or what would be acceptable and in what order. Like if we want to use binary release of Achievement gem at a certain version or range of versions, we would could WART the specification and imply preferred order maybe:

  • "Achievement>=1.5.0<2.10|BINARY" means must use binary release, must be found/local or error. In practice this might never be used...
  • "Achievement>=1.5.0<2.10|SOURCE" means must use source release, must be found/local or error. In practice this might never be used...
  • "Achievement>=1.5.0<2.10|GIT" means must use git at a certain branch/tag, must be found/local or error. In practice this might never be used...
  • "Achievement>=1.5.0<2.10|SOURCE|FETCH" means must use source release, CMAKE should fetch/download it if not found/local, if not available error...
  • "Achievement>=1.5.0<2.10|BINARY|SOURCE" means must use either binary or source release, prefer binary release, if neither found/local, error...
  • "Achievement>=1.5.0<2.10|SOURCE|BINARY|FETCH" meaning prefer source release, fallback to binary release, fallback to CMAKE should fetch/download source release, fallback to CMAKE should fetch/download binary release, if none are available error... So order is important...
  • "Achievement>=1.5.0<2.10|BINARY|SOURCE|GIT|FETCH" meaning prefer binary release if available, fallback to prefer source release if available, fallback to git at a certain branch/tag if available, fallback to CMAKE should try to fetch/download binary release, fallback to CMAKE should try to fetch/download source release, fallback to CMAKE should try to fetch/download/clone/shallow clone? git repo at a certain branch/tag, if none are available error... I imagine this might be the most common...
    I think this would cover use with a fairly fine grain control and be open ended enough to succeed most of the time if we defaulted to |BINARY|SOURCE|GIT|FETCH if no specifier present? Seems reasonable...


### What went wrong

The proposed versioning strategy has not been fully implemented, with the following points failing:
- **Manual Updates**: the developers did not update the version numbers at all (only Engine version was updated prior to the release) or did not update them correctly, leading to inconsistencies in versioning across Gems.
- **Incorrect Versioning**: the version on the _stabilization_ branch was created based on the previous _main_ release instead of the latest _development_ version (see [PR #17903](https://github.com/o3de/o3de/pull/17903)) in which there was a regression in the versioning logic
- **No Update on the _development_ branch**: the version numbers were not updated on the _development_ branch right after the _stabilization_ branch cutoff.

### Problems with Current Versioning

The current versioning strategy has several issues:
- **No Automation**: The manual process is error-prone and does not scale well with the high number of Gems.
- **No Information**: The current versioning strategy is not well documented and not spread across the community.
- **No Clear Update Path**: There is no clear information who and when should update the version numbers; there is no clear information how we should update the version numbers. Do we bump a _PATCH_ version for every change in the Gem when working on the _development_ branch? Do we bump a _PATCH_ or _MINOR_ version if the _MAJOR_ version is changed on the _development_ branch already (compared with the _main_ branch)?
- **No Definition of Engine Version**: The Engine version meaning is not clearly defined. If API changes occur in one of the core Gems (e.g. `Atom`), should the _MAJOR_ Engine version be incremented?
- **No Changes Possibility**: It can happen that a particular Gem does not get any updates; hence changing the version number right after the _stabilization_ branch cutoff defined in the RFC makes no sense.

### Current status

The _development_ branch is currently in a state where the version numbers of the Gems are not updated. E.g., `Atom` Gem has the version `0.1.0`. It was updated only once since the initial commit. The engine is set to `4.2.0` on the _development_ branch and `2.4.0` on the _main_ branch. The previous two releases were `2.3.0` and `2.2.0`, with the _PATCH_ version used for point releases. This happened even though there were some breaking changes in the API of the core Gems (e.g. `Atom` Gem).

# Feature design description:

Although the versioning strategy was suggested in the RFC, it was not fully implemented in practice. The current versioning strategy is not well documented, and there is no clear information on how to update the version numbers. This leads to inconsistencies in versioning across Gems and makes it difficult to track changes in the Engine and Gems.

### Temporary Solution for the 2510 Release

Before we can implement a new versioning strategy, we need to ensure that the version number of the Engine is updated correctly for the next release. I suggest the following solution:
- Keep the Engine version `4.2.0` for the upcoming release (2510).
- Keep the versions of the core Gems (e.g. `Atom`) unchanged for the upcoming release (2510).
- Bump the Engine version to `4.3.0` on the _development_ branch.
- Apply the selected versioning strategy described in this document on the _development_ branch for the future release (in 2026).

Copy link

@byrcolin byrcolin Aug 27, 2025 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

engine.json

{
    "engine_name": "o3de",
    "O3DEVersion": "0.1.0.0",
    "O3DEBuildNumber": 0,
    "display_version": "00.00",
    "version": "4.2.0",
    "api_versions": {
        "editor": "1.0.0",
        "framework": "1.2.1",
        "launcher": "1.0.0",
        "tools": "1.1.0"
    },
    "file_version": 2,
    "copyright_year": 2023,
    "build": 0,
...

It appears that "EngineVersion" was converted to use the engine objects version, that is fine. (o3de/o3de#14082)
"O3DEVersion" I believe is the release version? but it does not seem to be set to 25.01 or whatever... Although I'm not sure why that would be in the engine object, it should be external to the engine object.
"display_version" I don't know what this is... if it has anything to do with a o3de release should be external.
Just a Note here: In schema 2.0.0 there will be a "releases" section for every object, just so it is clear, this is not o3de release, it a release of the object only, if it ever has one. A release for an object can be code and/or binary, which we currently do not individually do on a release, the same goes for projects and gems, they are currently not able to be built independently from the release, but probably will be eventually.
"file_version" I don't know what that is... Perhaps this is $schemaVersion? before $schemaVersion existed? If so it should be removed.
"O3DEBuildNumber" and "build" I don't know what these are... Maybe has to do with a release? in which case it should be external.
"api_versions" I think this is meant to relate the engine version, not object version, (or release version) to editor, framework, launcher and tools release versions? (not object versions... right?) So like this engine version is compatible with editor release x.... or something.... not sure why this is here. This was added by alexpete and the commit message just says:
"Convert engine.json 'O3DEVersion' to be 'version' and 'display_version' (o3de/o3de#14082)

  • Update engine.json version field and related code"
    So no explanation of why this was added or what it does and should probably be removed?

When the engine is branched for stability for a release the same incremental approach should be observed on both the dev and stability branches independently from each other, that is OK. When stability is merged back into dev ALL version differences should be compared, NOT just the engine version, and the highest wins and that version should be patch incremented for the merge. This way the dev always moves forward, dev is ALWAYS ahead of stability after a merge back and of course is ALWAYS ahead of main, even if only by a patch increment.

Copy link

@nick-l-o3de nick-l-o3de Aug 29, 2025 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yea, I don't think we should have all these multiple verisons for a single Object. I'd be happy if we had at most 2

Version Friendly Name (display_version) 25.10 or Shiny Sphere or Rolling Cube or whatever. For making users smile and making it easier to connect "what version are you on so we can help you troubleshoot" to a date/time. Not used for anything but display and marketing.

Actual version Number: (version) 4.1.232 semver. Used for actual code and dependency management.

Copy link

@byrcolin byrcolin Sep 5, 2025 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm pretty sure all object should have 1 version and 1 only. Anything object related to a release should be in a new "release" section of the json (incoming in schema 2.0.0) As to where we would move any such needed "build"/"release" info should be external to all these objects. Unfortunately, currently the root object of the o3de/o3de repo is the main o3de engine object... so any "root" level object would be technically part of the engine object, which is unfortunate, but for now I would recommend a "release.json" or "build.json" in the root to contain this information and not be put in the engine.json. Hopefully in the future we can refactor this repo into a standard o3de repo format like o3de/extras. Where the "root" is an o3de "repo" object, which is just a container with no code meant to gather related objects and would be more appropriate place for such info. i.e. If we used the standard format that would open a bunch of possibilities like multiple engine objects... would look like maybe something like this: (I know this is a bit of a tangent here... just spit balling ideas, this has been on my mind and I want to write them down and put them out there and see what people think...)
https://github.com/o3de/o3de.git

/repo.json
/Engines/o3de/engine.json   <-maybe this one is what we now call the engine object. good for compiling/testing (maybe most if not all the dependencies in this engine are GIT/FETCH? could be interesting...) 

(just thinking out loud here... not sure of this yet... but interesting)-------------
/Engines/fps/engine.json   <-maybe this one is a good starting place for example FPS game engines?
/Engines/flight-sim/engine.json  <-maybe this one is a good starting place for an example flight sims engines?
-------------------------------------------------------------------------------------
                                 
(Some other ideas)----------------------------------------------------------------
/Engines/source/engine.json   <-maybe this ALL dependencies are SOURCE|FETCH
/Engines/binary/engine.json   <-maybe this is ALL dependencies are BINARY|FETCH
/Engines/git/engine.json <- maybe this is ALL dependencies are GIT|FETCH
...
--------------------------------------------------------------------------------------
/Projects/AutomatedTesting/project.json <-project meant for AR and uses the Engines/o3de engine object...
/Projects/Multiplayer/project.json   <-maybe this an example use of the FPS engine
/Projects/FlightSim/project.json    <-maybe this an example use of the flight-sim engine
/Projects/...
/Gems/Achievement/gem.json
/Gems/....
/Templates/DefaultGem/template.json
/Templates/...

I think if we do this reorg, we should consider moving to a master repo with sub-modules implementation. Then the o3de/o3de repository would be a master repository with NO CODE AT ALL, only a list of sub-modules. Using the same format as above:

/Engines/o3de             ---> https://github.com/o3de/engine.git
/Engines/source           ---> https://github.com/o3de/engine-source.git
/Engines/binary            ---> https://github.com/o3de/engine-binary.git
/Engines/fps                 ---> https://github.com/o3de/engine-fps.git
/Engines/flight-sim      ---> https://github.com/o3de/engine-flight-sim.git
...
/Gems/Achievement    ---> https://github.com/o3de/gem-achievement.git
/Gems/Atom                ---> https://github.com/o3de/gem-atom.git

(OR better...)
/Repos/Atom               ---> https://github.com/o3de/repo-atom.git
...

If we did the same reorg for Atom, since Atom is a large system and not just a single gem, and reorged it into standard o3de "repo" object like the above, then the atom repository would look like:

repo.json
/Projects/atom-test/project.json
/Projects/atom-viewer/project.json
/Projects/atom-experimental-lighting/project.json
...
/Gems/atom/gem.json           <We don't have to flatten the tree either, could have nested gems, just fine>
/Gems/atom/RHI/gem.json
/Gems/atom/RHI/whatever/gem.json
/Gems/atom/RPI/gem.json
/Gems/atom/RPI/whatever/gem.json

I would prefer this because this would keep all o3de objects like projects and gems related to Atom in one repo away from the the engine.
This repo itself is of a large system and I would even recommend this be a master repository as well, and maybe not all sub-modules... maybe only really containing the atom code and all the rest are just sub-modules...

repo.json     <--- This file is in the master repo
/Projects/atom-test              ---> https://github.com/o3de/atom-test.git
/Projects/atom-viewer             ---> https://github.com/o3de/atom-viewer.git
/Projects/atom-experimental-lighting             ---> https://github.com/o3de/atom-experimental-lighting.git
...
/Gems/atom/gem.json           <--- Maybe these file ARE in the master repo... or another sub-module... Whatever sig/graphics-audio wants...
/Gems/atom/RHI/gem.json 
/Gems/atom/whatever/gem.json 
/Gems/atom/RPI/gem.json

Nice and neat... I like that better. Perhaps most of the <...shutters... I caught myself saying "engine gems"> :) should be in o3de repos like this... even if the repo, at present, would only contain:

/repo.json
/Gems/Achievement/gem.json

It would give each system to room to grow a bit more organically...

A project like atom-test would have a dependency on an engine, maybe the "source" engine object:

atom-test/project.json
{
...
"dependencies": {
   "engines": [
        "source>=1.1<2.0|SOURCE|FETCH"
   ],
    "gems":[
    ... <any gems it depends on>...
   ]
}

And the "source" engine object would just have a gem dependency on atom, and let cmake resolve the best version for us.

source/engine.json
{
...
"dependencies": {
   "engines": [
   ],
    "gems": [
       "atom>=1.5<2.0|SOURCE|FETCH"
   ]
}

#### Benefits
- It ensures the Engine version on the _development_ branch and on the _stabilization_ branch is consistent and that the _development_ branch has newer version than release (2510).
- It allows us to continue working on the _development_ branch without breaking the versioning scheme.
- It provides a clear path for the next release without introducing additional complexity.

#### Drawbacks
- It means a bump from `2.4.0` to `4.2.0` for the Engine version between the releases.
Copy link

@byrcolin byrcolin Aug 27, 2025 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have no problem just jumping from 2.4.0 to 4.2.0 for the initial, lets get this going commit... Its a a new starting point, we will make a note of it in the release notes of why we did what we did. Should be fine.

I have thought it over and I can not find any problems with the inc on release strategy I talked about above... I'm pretty convinced at this point that the determination only of bump on commit, with DCO-like AR looking for tag in the commit comment, combined with the RELEASE only actual bump in version is the way to go, until someone can point out a flaw in the idea and can articulate a better easier strategy.

- It does not address the issues with the current versioning strategy for Gems.

### Remove the version number from the core Gems
Use the Engine version as the version number for all core Gems instead of a standalone versioning. This way much of the complexity of versioning is removed, making it easier to manage for the community. This could be automated, with a script that updates the version number of all core Gems to match the Engine version whenever the Engine version is updated.
Copy link

@byrcolin byrcolin Aug 27, 2025 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The more I think about this... NUCLEAR NO.. DON'T DO THIS... this will only make adding or removing versioned objects to or from the core near impossible. PLEASE NO.
They are NOT all the same version. We NEED to think about these objects as just coincidentally living as child inside a parent object. Parent/Child relationships HAVE NO bearing on object version. NONE. It is just a convenience.


#### Benefits
- It simplifies the versioning scheme by having a single version number for all core Gems.
Copy link

@byrcolin byrcolin Aug 27, 2025 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I cannot stress enough that there should be no concept as "Core Gems" or "Engine Gems" conferring on them ANY other significance. Treat them as if they are git submodule (because that is what they should probably be in a repo object like o3de-extras) and the reason they are included in the engine object at all is ONLY for convenience of the user.

- It reduces the complexity of managing version numbers for core Gems, as they will always follow the Engine version.

#### Drawbacks
- It removes the ability to track changes in core Gems between Engine releases.
- It is not clear to all developers which Gems are core Gems and which are not (there used to be a plan to move all Gems that are not core to the `o3de-extras` repository, but it was not implemented).
Copy link

@byrcolin byrcolin Aug 27, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

moving non core gems to the extras is an ongoing effort.


### Bump the version number of the Gem only when doing a release
For all Gems that are **NOT** core Gems, I propose to bump the version number only when doing a release. This means that the version number is updated only when creating a new _stabilization_ branch from the _development_ branch; only for Gems that have changes since the last release. The version number is updated based on the changes made in the Gem, following the Semantic Versioning scheme.
Copy link

@byrcolin byrcolin Aug 27, 2025 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There should be no difference between so called "Core Gems" and any other external gem.

We cannot control what external gems do for there versioning except suggest to them the same best practices that we ourselves follow in o3de objects.

(My recommendation)
The frequency of when we bump a version is theoretically independent of a release, HOWEVER, lets consider that we ONLY bump version on RELEASE, BUT we require all commits to determine what bump they THINK is necessary for THEIR commit:
We add a test like the DCO test we currently have that checks the commit for either "MAJOR VERSION INCREMENT" or "MINOR VERSION INCREMENT" or "PATCH VERSION INCREMENT" or "NO VERSION INCREMENT" and AR will fail just like DCO is not present. This way everyone determines this for themselves and can be PR reviewed by others. We DO NOT ACTUALLY increment on ANY commit. We would then NEVER have commit conflicts due to version. That would mean anyone can make any change they want in any order.
Then when stability branch is made, we could run a script or just look at all the commit messages for those tags and choose which one prevails, because ultimately version ONLY means anything to the RELEASE. The script or person could look through and find let say 5 commits for this object. 4 [PATCH VERSION INCREMENT], one [MINOR VERSION INCREMENT], the script or person can easily determine that the highest of these is [MINOR VERSION INCREMENT] and looks at the previous RELEASE version and performs a minor version bump and updates all instances of canonical object references to that. i.e. Lets say we are evaluating the Achievement gem previously at 1.4.4 becomes 1.5.0. Then any reference in the engine of Achievement can be updated from Achievement>=1.4.0 to Achievement>=1.5.0
If the highest was only patch version increment, then the new version would be 1.4.5, and we wouldn't even really need to update anything because we will use open ended >= so Achievement>=1.4.0 would still work fine as 1.4.5 and 1.4.6 are >= 1.4.0 and it would pick the highest compatible version it finds.
Best of all perhaps, we would cleanly merge to main as all versions are only increasing. And we could cleanly merge main back to dev because we aren't bumping dev either and would require no automation.... except for the DCO-Like addition to the AR, and that's pretty easy.

Copy link

@nick-l-o3de nick-l-o3de Sep 23, 2025 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That'd possibly work, although I'm not sure about all the github scripting for this. I'll put it on my todo to see if there's another github doing similar things, or a similar script...

Another option would be to have it so that the first one to change a gem bumps the revision if necessary - and if the revision has already been bumped since last release, we don't bump it again, until next release... with major taking precidence over minor.

Then though, it becomes a tracking issue... we'd have to know for a given PR whether it needs to bump the rev, and whether it already has bumped since last release, and thats also going to require a script or something (and may incur minor conflicts). Unless there's an easy way to tell. I guess github tools make it really easy to see the last change made to a speciifc file at least (ie, the gem.json) so it would be fairly trivial to check (git blame the json file, immediatley it will show when last the version was bumped and what it was before).

Not sure the pros and cons of either though. It does mean the version bumps would be in development as soon as they were relevant though, longer time for them to settle in there and reveal any other issues. Always worried about making a big bump during stab

Copy link

@nick-l-o3de nick-l-o3de Sep 23, 2025 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

TBH once we decide that versioning is only really necessary to push for release purposes (like once per cycle instead of every single commit), it becomes a lot less conflicty and opens up more options to script or update or work around it.


The developers can still bump the version of the Gem internally to any number while testing a new feature or fixing a bug, but this will not be reflected in the version number of the Gem on the _development_ branch.

#### Benefits
- This approach reduces the frequency of version number changes, leading to a more stable versioning scheme on the _main_ branch.
- It clearly defines the changes made in each release by incrementing the _MAJOR_, _MINOR_, or _PATCH_ version numbers based on the changes made in the Gem.

#### Drawbacks
- No information about the changes made in the Gem when working on the _development_ branch.
- It requires a large effort to ensure that the version number of each Gem is updated correctly when creating a new _stabilization_ branch (in particular, it requires a decision for each Gem whether to bump the _MAJOR_, _MINOR_, or _PATCH_ version number).
- It requires porting back the version number changed to the _development_ branch after the _stabilization_ branch is created.

> Note: this approach is used for all Gems and Templates in the `o3de-extras` repository, as it was the most intuitive to me, while the number of Gems is much smaller than in the Engine repository.

# Are there any alternatives to this feature?

### Keep the current versioning scheme for all Gems (including core Gems)
One alternative is to continue using the current versioning scheme for all Gems, including core Gems. This would mean that each Gem has its own version number, and changes to the Gem are reflected in its version number regardless of the Engine version. However, this approach would generate too much effort for the developers when changing the version numbers of the Gems for the release and would require another approach for changing the version numbers of the Gems on the _development_ branch. Two other approaches are described below.

### Always bump the version number of the Gem
One possible solution is to always bump the version number for every change in the Gem (when working on the _development_ branch). This means that the version number is updated by a developer working on a Gem or some automation when merging the Gem to the _development_ branch.

#### Benefits
- It enables tracking changes when working on the _development_ branch.
- It is easy to implement and understand (especially if some automation is implemented).
- No need to change the versions of the Gems on the _stabilization_ branch, as the _development_ branch will always have the latest version number.

#### Drawbacks
- It can lead to excessive version number changes, causing discontinuity in the versioning numbers on the _main_ branch (due to multiple updates on the _development_ branch between the releases).
- It can create conflicts when multiple developers are working on the same Gem.

### Update the version number of the Gem when doing a change in the Gem
Another alternative approach is a combination of the previous two solutions. The version number is updated when doing a change in the Gem on the _development_ branch, but the version number is not updated when creating a new _stabilization_ branch. The version number is updated based on the changes made in the Gem, following the Semantic Versioning scheme and taking into account the changes made in the Gem since the last release.

#### Benefits
- It enables tracking changes when working on the _development_ branch.
- It reduces the frequency of version number changes, leading to a more stable versioning scheme on the _main_ branch.
- It clearly defines the changes made in each release by incrementing the _MAJOR_, _MINOR_, or _PATCH_ version numbers based on the changes made in the Gem.

#### Drawbacks
- It requires a large effort to ensure that the version number of each Gem is updated correctly when creating/reviewing a pull request to the _development_ branch (it requires a check whether the version number was updated after the last release, as, e.g., the _PATCH_ version number should not be changed if the _MAJOR_ version number was already changed).

# How will users learn this feature?
We will update the documentation and announce it in the community channels. The most important part is to ensure that the maintainers are aware of the new versioning strategy and how to apply it. This way the maintainers can ensure that the version numbers are updated correctly in the pull requests to the _development_ or _stabilization_ branches.