[repositoryValidator] xi:include should be supported

[YuvalCohen]

There should be a simple way to support xi:include in the xsd repository files.

This is a basic requirement for Orchestra to serve any project which is slightly larger then FIX repository and requires to split the repository to multiple xsd files.

At the moment, repositoryValidator report an error: Attribute 'xml:base' is not allowed to appear in element '...' (name of the element where the xi:include was added)

Further information about this error can be found in:

[https://stackoverflow.com/questions/22774425/attempting-to-connect-xml-files-with-xinclude-attribute-xmlbase-error/22791471#22791471]
and way to supress the error can be found in: [http://xerces.apache.org/xerces2-j/features.html#xinclude.fixup-base-uris]

Thanks in advance.
The repositoryValidator is a very useful tool.

[kleihan]

Agree with the proposal for the next version of Orchestra. SBE V1.0 supports the import of encodings, see Message Schema - Data encodings - Encoding name - Importing encodings (Chapter 6.5.2.1 in PDF).

[donmendelson]

@YuvalCohen the Orchestra repository schema does both things recommended by that article. It imports xml.xsd and it declares xml:base in top-level elements under <repository>. Unfortunately, Xinclude requires that attribute in every element that you wish to include; you can't include at random locations. Are you trying to include at the top-level, or at lower levels of the schema?

Also, RepositoryValidator sets a switch to make the parser XIncludeAware, another requirement. I'll have to test validator with includes to see if it works as expected.

[YuvalCohen]

@donmendelson The requirement si to be able to split large and complicated repository to multiple files.
I believe that it is basic requirement for any future use of Orchestra (i.e. scalable).

At the moment, it seems that repositoryValidator does not like xi:include.
We are happy to follow any guidelines or methods that will enable us to have each message, component, and/or codeSet in separated files. So far we tried with xi:include which is not welcome by the repositoryValidator.

[donmendelson]

I altered a unit test for RepositoryValidator as follows and got it to run with an Xinclude:

Modified test resource OrchestraFIXLatest.xml and moved a portion to another file.

1. Added xmlns:xi="http://www.w3.org/2001/XInclude" to root element.
2. Replaced <fixr:datatypes> element with <xi:include href="src/test/resources/datatypes.xml"/> (path is tricky with xinclude)
3. Created datatypes.xml with contents of <fixr:datatypes>
4. Added namespace to the included file as <fixr:datatypes xmlns:fixr="http://fixprotocol.io/2020/orchestra/repository">

The output did produce some errors or warnings, but they seem to be correct, not a problem with include instructions.

[YuvalCohen]

@donmendelson Thanks.
Is there a possibility to write: a (single) message, (a single) component and (a single) codeset in dedicated files (i.e. each in its 'own file') and include all of them in the main repository file?

If so, can you please provide some description about what each of these files needs to contain?

[donmendelson]

@YuvalCohen since XIinclude requires explicit include points to be declared in the schema, what you ask requires a modification. We are about to promote Orchestra version 1.0 to Technical Standard, but we can take your request as a requirement for version 1.1.

[donmendelson]

Now that Orchestra version 1.0 is Technical Standard, we should reconsider all requests for schema changes to become version 1.1.

[kleihan]

Added the following to v1.1 RC1 section 3.1.4 Support for XInclude:

For example, <fixr:datatypes> element can be replaced with the path to the XML file, e.g. <xi:include href="src/test/resources/datatypes.xml"/>. The datatypes then need to be defined in a separate file datatypes.xml that needs to contain the same namespace as the root element of the repository, i.e. <fixr:datatypes xmlns:fixr="http://fixprotocol.io/2023/orchestra/repository">.

[kleihan]

This issue only updates the documentation of V1.1 RC1. There is currently no requirement to separate out individual elements of the top elements supporting XInclude (actors, concepts, sections, categories, messages, groups, components, fields, code sets, datatypes, scenarios). If the need arises, a separate issue should be entered and can be addressed in a later RC of v1.1.

The issue remains open to address the bug in the Repository Validator tool.