Library fails to process includeValidationRules in some cases

[EMaksymenko]

When I process following test file with --includeValidationRules true I receive an error "Failed to locate validate/validate.proto".

Example file: https://combat.vision/xsds/2019/04/11/MIP-IES.xsd

Console parameters: --includeValidationRules true --outputDirectory /home/user/MIP /home/user/MIP-IES.xsd
Result proto files: proto.zip
Source XSD file: MIP-IES.zip

[stevespringett]

I'm getting the same issue on a different xsd.

[andrew-otiv]

I get the same issue on schema2proto's own test file:

https://github.com/entur/schema2proto/blob/develop/schema2proto-lib/src/test/resources/xsd/test-min-max-occurs-range.xsd

Capturing the syscalls using strace:

strace -f -ofoo java -jar schema2proto --includeValidationRules true --outputDirectory . test-min-max-occurs-range.xsd
...
15384:194367 openat(AT_FDCWD, "/home/awagner/test-min-max-occurs-range.xsd", O_RDONLY) = 5
16371:194385 openat(AT_FDCWD, "/sys/fs/cgroup/user.slice/user-1000.slice/user@1000.service/app.slice/app-gnome-code-3262.scope/memory.max", O_RDONLY|O_CLOEXEC <unfinished ...>
16373:194385 <... openat resumed>) = 5
18023:194385 openat(AT_FDCWD, "/sys/fs/cgroup/user.slice/user-1000.slice/user@1000.service/app.slice/app-gnome-code-3262.scope/memory.max", O_RDONLY|O_CLOEXEC) = 5
20651:194381 openat(AT_FDCWD, "/sys/fs/cgroup/user.slice/user-1000.slice/user@1000.service/app.slice/app-gnome-code-3262.scope/memory.max", O_RDONLY|O_CLOEXEC) = 5
22356:194367 openat(AT_FDCWD, "./default/default.proto", O_WRONLY|O_CREAT|O_TRUNC, 0666) = 5
22534:194385 openat(AT_FDCWD, "/sys/fs/cgroup/user.slice/user-1000.slice/user@1000.service/app.slice/app-gnome-code-3262.scope/memory.max", O_RDONLY|O_CLOEXEC) = 6
22834:194367 openat(AT_FDCWD, "/sys/fs/cgroup/user.slice/user-1000.slice/user@1000.service/app.slice/app-gnome-code-3262.scope/cpu.max", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
22835:194367 openat(AT_FDCWD, "/sys/fs/cgroup/user.slice/user-1000.slice/user@1000.service/app.slice/app-gnome-code-3262.scope/cpu.max", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
23121:194385 openat(AT_FDCWD, "/sys/fs/cgroup/user.slice/user-1000.slice/user@1000.service/app.slice/app-gnome-code-3262.scope/memory.max", O_RDONLY|O_CLOEXEC <unfinished ...>
23125:194385 <... openat resumed>) = 5
23128:194381 openat(AT_FDCWD, "/sys/fs/cgroup/user.slice/user-1000.slice/user@1000.service/app.slice/app-gnome-code-3262.scope/memory.max", O_RDONLY|O_CLOEXEC <unfinished ...>
23133:194381 <... openat resumed>) = 6

default/default.proto was opened write only... and then there is "... openat resumed" for the same file descriptor... maybe the .proto output file needs to be re-opened with read permissions for the output validation? This is on linux.

[janmpo]

I have tried to get the validation with different XSDs and it is impossible for me also.

I built schema2proto via:

mvn install

I used this xsd https://www.ibm.com/docs/en/bpm/8.5.7?topic=files-example-xsd-file (as an example)

https://www.ibm.com/docs/en/bpm/8.5.7?topic=files-example-xsd-file

Which is:

<?xml version="1.0" encoding="UTF-8"?><xsd:schema targetNamespace="http://com.ibm.wbit.comptest.controller" 
		xmlns:Q1="http://com.ibm.wbit.comptest.controller" xmlns:xsd="http://www.w3.org/2001/XMLSchema">
<xsd:complexType name="TestResults">
		<xsd:sequence>
			<xsd:element maxOccurs="unbounded" minOccurs="0" name="testSuites" type="Q1:TestSuiteRun"/>
		</xsd:sequence>
		<xsd:attribute name="testProject" type="xsd:string"/>
	</xsd:complexType>

	<xsd:complexType name="TestSuiteRun">
		<xsd:complexContent>
			<xsd:extension base="Q1:TestRun">
				<xsd:sequence>
					<xsd:element maxOccurs="unbounded" minOccurs="0" name="testCases" type="Q1:TestCaseRun">
					</xsd:element>
				</xsd:sequence>

				<xsd:attribute name="tests" type="xsd:int"/>

			</xsd:extension>
		</xsd:complexContent>
	</xsd:complexType>

	<xsd:complexType name="TestCaseRun">
		<xsd:complexContent>
			<xsd:extension base="Q1:TestRun">
				<xsd:sequence>
     	<xsd:element name="result" type="Q1:Severity"/>
      <xsd:element maxOccurs="10" minOccurs="1" name="variations" type="Q1:VariationRun">
					</xsd:element>
				</xsd:sequence>
				<xsd:attribute name="variationCount" type="xsd:int"/>
			</xsd:extension>
		</xsd:complexContent>
	</xsd:complexType>

	<xsd:complexType name="VariationRun">
		<xsd:complexContent>
			<xsd:extension base="Q1:TestRun">
				<xsd:sequence>
     	<xsd:element name="result" type="Q1:Severity"/>
      <xsd:element name="exception" nillable="true" type="xsd:string">
						<xsd:annotation>
							<xsd:documentation>
								This element is used to display the exception of a failure or error.
							</xsd:documentation>
						</xsd:annotation>
					</xsd:element>
				</xsd:sequence>				
			</xsd:extension>
		</xsd:complexContent>
	</xsd:complexType>

	<xsd:simpleType name="Severity">
		<xsd:restriction base="xsd:string">
			<xsd:enumeration value="pass"/>
			<xsd:enumeration value="fail"/>
			<xsd:enumeration value="error"/>
		</xsd:restriction>
	</xsd:simpleType>

    <xsd:complexType name="TestRun">
      	<xsd:attribute name="name" type="xsd:string"/>
	  	<xsd:attribute name="startTime" type="xsd:dateTime"/>
	  	<xsd:attribute name="endTime" type="xsd:dateTime"/>
	  	<xsd:attribute name="result" type="xsd:string"/>
    </xsd:complexType>
</xsd:schema>

where I change minOccurs to 1 and maxOccurs to 10 inside TestCaseRun.

And I execute schema2proto with the following command:

java -jar /tmp/schema2proto/schema2proto-lib/target/schema2proto-lib-1.93-SNAPSHOT.jar --outputDirectory testdir --derivationBySubsumption true --includeFieldDocs true --includeGoPackageOptions true --includeMessageDocs true --includeSourceLocationInDoc true --includeValidationRules true --includeXsdOptions true  test.xsd

I got for this xsd a normal execution (the same happens with other xsds):

[main] INFO no.entur.schema2proto.generateproto.Schema2Proto - Starting to parse test.xsd
[main] INFO no.entur.schema2proto.generateproto.ProtoSerializer - Writing proto files to /tmp/testdir
[main] INFO no.entur.schema2proto.generateproto.Schema2Proto - Done

But the .proto file that I got is:

// controller_comptest_wbit_ibm_com.proto at 0:0
syntax = "proto3";
package controller.comptest.wbit.ibm.com;

import "validate/validate.proto";
import "xsd/xsd.proto";

option go_package = "controller/comptest/wbit/ibm/com";

// [test.xsd at 54:34]
enum Severity {
  // Default
  SEVERITY_UNSPECIFIED = 0;
  // [test.xsd at 56:35]
  SEVERITY_PASS = 1;
  // [test.xsd at 57:35]
  SEVERITY_FAIL = 2;
  // [test.xsd at 58:36]
  SEVERITY_ERROR = 3;
}
// [test.xsd at 24:38]
message TestCaseRun {
  option (xsd.base_type) = "TestRun";

  // [test.xsd at 63:54]
  string name = 1;
  // [test.xsd at 64:58]
  uint64 start_time = 2;
  // [test.xsd at 65:56]
  uint64 end_time = 3;
  // [test.xsd at 66:53]
  string attr_result = 4;
  // [test.xsd at 32:58]
  int32 variation_count = 5;
  // [test.xsd at 28:54]
  Severity result = 6;
  // [test.xsd at 29:97]
  repeated VariationRun variations = 7;
}
// [test.xsd at 3:37]
message TestResults {
  // [test.xsd at 7:56]
  string test_project = 1;
  // [test.xsd at 5:88]
  repeated TestSuiteRun test_suites = 2;
}
// [test.xsd at 62:37]
message TestRun {
  // [test.xsd at 63:54]
  string name = 1;
  // [test.xsd at 64:58]
  uint64 start_time = 2;
  // [test.xsd at 65:56]
  uint64 end_time = 3;
  // [test.xsd at 66:53]
  string result = 4;
}
// [test.xsd at 10:39]
message TestSuiteRun {
  option (xsd.base_type) = "TestRun";

  // [test.xsd at 63:54]
  string name = 1;
  // [test.xsd at 64:58]
  uint64 start_time = 2;
  // [test.xsd at 65:56]
  uint64 end_time = 3;
  // [test.xsd at 66:53]
  string result = 4;
  // [test.xsd at 18:49]
  int32 tests = 5;
  // [test.xsd at 14:94]
  repeated TestCaseRun test_cases = 6;
}
// [test.xsd at 37:39]
message VariationRun {
  option (xsd.base_type) = "TestRun";

  // [test.xsd at 63:54]
  string name = 1;
  // [test.xsd at 64:58]
  uint64 start_time = 2;
  // [test.xsd at 65:56]
  uint64 end_time = 3;
  // [test.xsd at 66:53]
  string attr_result = 4;
  // [test.xsd at 41:54]
  Severity result = 5;
  // This element is used to display the exception of a failure or error.   [test.xsd at 42:71]
  string exception = 6;
}

Where the only difference with switch --includeValidationRules false is: import "validate/validate.proto".

However I expect some annotations of protovalidate such

(buf.validate.field).repeated = {
   min_items: 1,
   max_items: 10,
}

Can anyone told me whether they are facing to something similar? In my case it didn't work neither with complex xml nor simple ones like this.

Thanks