From d26a582b62c5e26b4bcac895b9cb8960c3ce8522 Mon Sep 17 00:00:00 2001 From: Michael Beckerle Date: Wed, 8 Nov 2023 20:19:31 -0500 Subject: [PATCH] Refactoring to expose clean metadata and infoset walkers. This was designed to support integration of Daffodil directly (without intermediate XML or JSON or even EXI) to other data handling tools, specifically, Apache Drill. InfosetElement and related InfosetNode traits are now in runtime1.api package. InfosetOutputter now has methods which use InfosetElement and InfosetArray traits as the objects passed to the handler methods. This improves the API over making DIArray, DISimple, and DIComplex visible. (Though it is backward incompatible.) Also removed use of DIComplex, DISimple from most SAPI and JAPI tests. The SAX InfosetOutputter still downcasts somewhat to the DINode classes. Added Metadata, ElementMetadata, etc. (also runtime1.api package) which provide limited exposure to the RuntimeData and CompilerInfo information. Added MetadataHandler - which is walked by MetadataWalker which can be called from DataProcessor. Added unit test for metadata and data walking to core module Walking the runtime1 metadata is easier than walking the DSOM tree. And these data fabrics like Apache Drill are interfacing to the runtime1 specifically, not the schema compiler. It's natural for the runtime1 metadata structures and data structures to be the ones driving the interfacing. The InfosetNode types were always supposed to be the API, the DINodes the implementation. This solves the issue of what classes should show through to SAPI and JAPI about the infoset nodes. It should be the InfosetNode types, not the DINode types. Furthermore, the InfosetNode types can have methods to access the needed runtime metadata information needed by walkers, InfosetOutputters, etc. These hide our infoset implementation and runtime metdata (RuntimeData classes) implementions. Note: Nothing has changed with InfosetInputters, as those are not needed for Apache Drill integration - which is parse-only. BlobMethodMixin factored out of InfosetOutputter as a shared implementation trait for the basic blob implementation. Added features to SchemaUtils to avoid "tns" prefix definition (which is now officially frowned upon) Added isHidden to SequenceRuntimeData. Needed to avoid walking hidden elements that appear in metadata structures, but aren't relevant as they do not appear in InfosetOutputter events. Improved Infoset API access to simple type methods They now use the DFDL type names, not the underlying implementation type names. So a decimal is accessed via getDecimal not getBigDecimal. Improved java doc accordingly. They throw a predictable exception on conversion issues. DEPRECATION/COMPATIBILITY The InfosetOutputter trait methods have changed signatures. The types of the arguments have been replaced: DIArray -> InfosetArray DISimple -> InfosetSimpleElement DIComplex -> InfosetComplexElement This was done to hide the "DIxxx" types as they are internal and subject to change. Methods of DISimple that were named for implementation types (like getBigInt, getBigDecimal, etc.) have been replaced by methods named for the DFDL types. These methods are replaced: getBigDecimal -> getDecimal getBigInt -> getInteger These methods also changed names: dataValueAsString -> getText This method is new: getNonNegativeInteger Some methods have been removed (getStatus). DAFFODIL-2832 --- .../daffodil/core/dsom/ComplexTypes.scala | 2 +- .../daffodil/core/dsom/ModelGroup.scala | 3 +- .../daffodil/core/dsom/SchemaComponent.scala | 2 - .../daffodil/core/dsom/SequenceGroup.scala | 5 + .../runtime1/SequenceTermRuntime1Mixin.scala | 2 + .../core/api/TestMetadataWalking.scala | 223 ++++++++++++ .../dpath/TestDFDLExpressionEvaluation.scala | 2 +- .../core/general/TestRuntimeProperties.scala | 8 +- .../daffodil/core/infoset/TestInfoset.scala | 95 ++--- .../core/infoset/TestInfosetFree.scala | 2 +- .../apache/daffodil/core/util/TestUtils.scala | 3 + .../org/apache/daffodil/japi/Daffodil.scala | 17 +- .../daffodil/japi/infoset/Infoset.scala | 53 +-- .../example/TestInfosetOutputter.java | 49 +-- .../daffodil/example/TestJavaMetadataAPI.java | 131 +++++++ .../test/japi/metadataTestSchema1.dfdl.xsd | 55 +++ .../lib/exceptions/SchemaFileLocatable.scala | 18 +- .../apache/daffodil/lib/util/Numbers.scala | 40 ++- .../daffodil/lib/util/SchemaUtils.scala | 4 +- .../runtime1/api/DFDLParserUnparser.scala | 1 + .../daffodil/runtime1/api/Infoset.scala | 327 ++++++++++++++++++ .../daffodil/runtime1/api/Metadata.scala | 306 ++++++++++++++++ .../debugger/InteractiveDebugger.scala | 2 +- .../daffodil/runtime1/dpath/NodeInfo.scala | 25 +- .../daffodil/runtime1/dpath/UpDownMoves.scala | 6 +- .../runtime1/dsom/CompiledExpression1.scala | 2 +- .../daffodil/runtime1/infoset/DataValue.scala | 6 +- .../daffodil/runtime1/infoset/Infoset.scala | 102 ------ .../runtime1/infoset/InfosetImpl.scala | 251 ++++++++++---- .../runtime1/infoset/InfosetOutputter.scala | 67 ++-- .../runtime1/infoset/InfosetWalker.scala | 2 +- .../infoset/JDOMInfosetOutputter.scala | 46 +-- .../infoset/JsonInfosetOutputter.scala | 40 ++- .../infoset/NullInfosetOutputter.scala | 16 +- .../infoset/SAXInfosetOutputter.scala | 81 +++-- .../infoset/ScalaXMLInfosetOutputter.scala | 71 ++-- .../infoset/TeeInfosetOutputter.scala | 16 +- .../infoset/W3CDOMInfosetOutputter.scala | 44 ++- .../infoset/XMLInfosetOutputter.scala | 62 ---- .../infoset/XMLTextInfosetOutputter.scala | 28 +- .../runtime1/processors/DataProcessor.scala | 8 +- .../runtime1/processors/MetadataWalker.scala | 99 ++++++ .../processors/ProcessorStateBases.scala | 3 +- .../runtime1/processors/RuntimeData.scala | 67 +++- .../parsers/ExpressionEvaluatingParsers.scala | 3 +- .../runtime1/processors/parsers/PState.scala | 2 +- .../processors/unparsers/UState.scala | 2 +- .../org/apache/daffodil/sapi/Daffodil.scala | 8 + .../daffodil/sapi/infoset/Infoset.scala | 49 +-- .../TestInfosetInputterOutputter.scala | 37 +- .../udf/UserDefinedFunctionProvider.java | 16 +- 51 files changed, 1876 insertions(+), 633 deletions(-) create mode 100644 daffodil-core/src/test/scala/org/apache/daffodil/core/api/TestMetadataWalking.scala create mode 100644 daffodil-japi/src/test/java/org/apache/daffodil/example/TestJavaMetadataAPI.java create mode 100644 daffodil-japi/src/test/resources/test/japi/metadataTestSchema1.dfdl.xsd create mode 100644 daffodil-runtime1/src/main/scala/org/apache/daffodil/runtime1/api/Infoset.scala create mode 100644 daffodil-runtime1/src/main/scala/org/apache/daffodil/runtime1/api/Metadata.scala delete mode 100644 daffodil-runtime1/src/main/scala/org/apache/daffodil/runtime1/infoset/Infoset.scala delete mode 100644 daffodil-runtime1/src/main/scala/org/apache/daffodil/runtime1/infoset/XMLInfosetOutputter.scala create mode 100644 daffodil-runtime1/src/main/scala/org/apache/daffodil/runtime1/processors/MetadataWalker.scala diff --git a/daffodil-core/src/main/scala/org/apache/daffodil/core/dsom/ComplexTypes.scala b/daffodil-core/src/main/scala/org/apache/daffodil/core/dsom/ComplexTypes.scala index fd00287366..7f0a72b0b6 100644 --- a/daffodil-core/src/main/scala/org/apache/daffodil/core/dsom/ComplexTypes.scala +++ b/daffodil-core/src/main/scala/org/apache/daffodil/core/dsom/ComplexTypes.scala @@ -56,7 +56,7 @@ sealed abstract class ComplexTypeBase(xmlArg: Node, parentArg: SchemaComponent) private lazy val smg = { childrenForTerms.map { xmlChild => - ModelGroupFactory(xmlChild, this, 1, false) + ModelGroupFactory(xmlChild, this, 1, isHidden = false) } } diff --git a/daffodil-core/src/main/scala/org/apache/daffodil/core/dsom/ModelGroup.scala b/daffodil-core/src/main/scala/org/apache/daffodil/core/dsom/ModelGroup.scala index 039f68e4b6..729ce13886 100644 --- a/daffodil-core/src/main/scala/org/apache/daffodil/core/dsom/ModelGroup.scala +++ b/daffodil-core/src/main/scala/org/apache/daffodil/core/dsom/ModelGroup.scala @@ -147,7 +147,8 @@ object TermFactory { case Some(_) => ElementRef(child, lexicalParent, position) } } - case _ => ModelGroupFactory(child, lexicalParent, position, false, nodesAlreadyTrying) + case _ => + ModelGroupFactory(child, lexicalParent, position, isHidden = false, nodesAlreadyTrying) } childTerm } diff --git a/daffodil-core/src/main/scala/org/apache/daffodil/core/dsom/SchemaComponent.scala b/daffodil-core/src/main/scala/org/apache/daffodil/core/dsom/SchemaComponent.scala index e9eab707d5..4014a5cc96 100644 --- a/daffodil-core/src/main/scala/org/apache/daffodil/core/dsom/SchemaComponent.scala +++ b/daffodil-core/src/main/scala/org/apache/daffodil/core/dsom/SchemaComponent.scala @@ -202,8 +202,6 @@ trait SchemaComponent sscd } - final def sscd = shortSchemaComponentDesignator - /** * Elements only e.g., /foo/ex:bar */ diff --git a/daffodil-core/src/main/scala/org/apache/daffodil/core/dsom/SequenceGroup.scala b/daffodil-core/src/main/scala/org/apache/daffodil/core/dsom/SequenceGroup.scala index 90586193a2..32281a0b3d 100644 --- a/daffodil-core/src/main/scala/org/apache/daffodil/core/dsom/SequenceGroup.scala +++ b/daffodil-core/src/main/scala/org/apache/daffodil/core/dsom/SequenceGroup.scala @@ -70,6 +70,11 @@ abstract class SequenceTermBase( def isOrdered: Boolean + /** + * Overridden in sequence group ref + */ + def isHidden: Boolean = false + } /** diff --git a/daffodil-core/src/main/scala/org/apache/daffodil/core/runtime1/SequenceTermRuntime1Mixin.scala b/daffodil-core/src/main/scala/org/apache/daffodil/core/runtime1/SequenceTermRuntime1Mixin.scala index ea24a80f7a..fd074134a5 100644 --- a/daffodil-core/src/main/scala/org/apache/daffodil/core/runtime1/SequenceTermRuntime1Mixin.scala +++ b/daffodil-core/src/main/scala/org/apache/daffodil/core/runtime1/SequenceTermRuntime1Mixin.scala @@ -49,6 +49,7 @@ trait SequenceTermRuntime1Mixin { self: SequenceTermBase => fillByteEv, maybeCheckByteAndBitOrderEv, maybeCheckBitOrderAndCharsetEv, + isHidden, ) } @@ -82,6 +83,7 @@ trait ChoiceBranchImpliedSequenceRuntime1Mixin { self: ChoiceBranchImpliedSequen FillByteUseNotAllowedEv, Maybe.Nope, Maybe.Nope, + isHidden = false, ) } diff --git a/daffodil-core/src/test/scala/org/apache/daffodil/core/api/TestMetadataWalking.scala b/daffodil-core/src/test/scala/org/apache/daffodil/core/api/TestMetadataWalking.scala new file mode 100644 index 0000000000..83fc300884 --- /dev/null +++ b/daffodil-core/src/test/scala/org/apache/daffodil/core/api/TestMetadataWalking.scala @@ -0,0 +1,223 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.apache.daffodil.core.api + +import scala.collection.mutable.ArrayBuffer +import scala.xml.Elem + +import org.apache.daffodil.core.util.TestUtils +import org.apache.daffodil.io.InputSourceDataInputStream +import org.apache.daffodil.lib.util._ +import org.apache.daffodil.runtime1.api.ChoiceMetadata +import org.apache.daffodil.runtime1.api.ComplexElementMetadata +import org.apache.daffodil.runtime1.api.DFDL.ParseResult +import org.apache.daffodil.runtime1.api.ElementMetadata +import org.apache.daffodil.runtime1.api.InfosetArray +import org.apache.daffodil.runtime1.api.InfosetComplexElement +import org.apache.daffodil.runtime1.api.InfosetElement +import org.apache.daffodil.runtime1.api.InfosetItem +import org.apache.daffodil.runtime1.api.InfosetSimpleElement +import org.apache.daffodil.runtime1.api.Metadata +import org.apache.daffodil.runtime1.api.MetadataHandler +import org.apache.daffodil.runtime1.api.SequenceMetadata +import org.apache.daffodil.runtime1.api.SimpleElementMetadata +import org.apache.daffodil.runtime1.infoset.InfosetOutputter +import org.apache.daffodil.runtime1.processors.DataProcessor + +import org.junit.Assert.assertEquals +import org.junit.Assert.assertTrue +import org.junit.Test + +class TestMetadataWalking { + + def compileAndWalkMetadata(schema: Elem, mh: MetadataHandler): DataProcessor = { + val dp = TestUtils.compileSchema(schema) + assertTrue(!dp.isError) + dp.walkMetadata(mh) + dp + } + + def parseAndWalkData(dp: DataProcessor, infosetOutputter: InfosetOutputter)( + data: Array[Byte], + ): ParseResult = { + val isdis = InputSourceDataInputStream(data) + val res = dp.parse(isdis, infosetOutputter) + res + } + + class GatherMetadata extends MetadataHandler { + + private val buf = new ArrayBuffer[Metadata](); + + def getResult: Seq[Metadata] = { + val res: Seq[Metadata] = buf.toVector // makes a copy + buf.clear() + res + } + + override def simpleElementMetadata(m: SimpleElementMetadata): Unit = buf += m + + override def startComplexElementMetadata(m: ComplexElementMetadata): Unit = buf += m + + override def endComplexElementMetadata(m: ComplexElementMetadata): Unit = buf += m + + override def startSequenceMetadata(m: SequenceMetadata): Unit = buf += m + + override def endSequenceMetadata(m: SequenceMetadata): Unit = buf += m + + override def startChoiceMetadata(m: ChoiceMetadata): Unit = buf += m + + override def endChoiceMetadata(m: ChoiceMetadata): Unit = buf += m + } + + class GatherData extends InfosetOutputter { + + private val buf = new ArrayBuffer[InfosetItem] + + def getResult: Seq[InfosetItem] = { + val res = buf.toVector + reset() + res + } + + override def reset(): Unit = { buf.clear() } + + override def startDocument(): Unit = {} + + override def endDocument(): Unit = {} + + override def startSimple(diSimple: InfosetSimpleElement): Unit = { buf += diSimple } + + override def endSimple(diSimple: InfosetSimpleElement): Unit = {} + + override def startComplex(complex: InfosetComplexElement): Unit = { buf += complex } + + override def endComplex(complex: InfosetComplexElement): Unit = { buf += complex } + + override def startArray(array: InfosetArray): Unit = { buf += array } + + override def endArray(array: InfosetArray): Unit = { buf += array } + } + + @Test def testMetadataWalk_DataWalk_01(): Unit = { + val gatherData = new GatherData + val gatherMetadata = new GatherMetadata + val sch = SchemaUtils.dfdlTestSchema( + , + , + + + + + + + , + useTNS = false, + ) + val dp = compileAndWalkMetadata(sch, gatherMetadata) + val md = gatherMetadata.getResult + val mdQNames = md.map { + case e: ElementMetadata => e.toQName + case seq: SequenceMetadata => "seq" + case cho: ChoiceMetadata => "cho" + } + assertEquals("Vector(e1, seq, s1, seq, e1)", mdQNames.toString) + val parser: Array[Byte] => ParseResult = parseAndWalkData(dp, gatherData) + val inputData = "5;6;7;8;.".getBytes("utf-8") + val res = parser(inputData) + val infosetItems = gatherData.getResult + val itemQNames = infosetItems.flatMap { + case e: InfosetElement => Seq(e.metadata.toQName) + case e: InfosetArray => Seq(e.metadata.name + "_array") + case _ => Nil + } + assertEquals("Vector(e1, s1_array, s1, s1, s1, s1, s1_array, e1)", itemQNames.toString) + val itemValues = infosetItems.flatMap { + case e: InfosetSimpleElement => Seq(e.getText) + case _ => Nil + } + assertEquals("5678", itemValues.mkString) + } + + /** + * Shows that there are no hidden elements to deal with in + * the metadata walk nor the data walk. + */ + @Test def testMetadataWalk_DataWalk_NoHidden(): Unit = { + val gatherData = new GatherData + val gatherMetadata = new GatherMetadata + val sch = SchemaUtils.dfdlTestSchema( + , + , + Seq( + + + + + , + + + + + + + + + + + , + ), + useTNS = false, + useDefaultNamespace = false, + elementFormDefault = "unqualified", + ) + val dp = compileAndWalkMetadata(sch, gatherMetadata) + val md = gatherMetadata.getResult + val mdQNames = md.map { + case e: ElementMetadata => e.toQName + case seq: SequenceMetadata => "seq" + case cho: ChoiceMetadata => "cho" + } + assertEquals( + "Vector(ex:e1, cho, seq, seq, seq, s1, seq, cho, ex:e1)", + mdQNames.toString, + ) + val parser: Array[Byte] => ParseResult = parseAndWalkData(dp, gatherData) + val inputData = "1;5;6;7;8.".getBytes("utf-8") + val res = parser(inputData) + val infosetItems = gatherData.getResult + val itemQNames = infosetItems.flatMap { + case e: InfosetElement => Seq(e.metadata.toQName) + case e: InfosetArray => Seq(e.metadata.name + "_array") + case _ => Nil + } + assertEquals( + "Vector(ex:e1, s1_array, s1, s1, s1, s1, s1_array, ex:e1)", + itemQNames.toString, + ) + val itemValues = infosetItems.flatMap { + case e: InfosetSimpleElement => Seq(e.getText) + case _ => Nil + } + assertEquals("5678", itemValues.mkString) + } + +} diff --git a/daffodil-core/src/test/scala/org/apache/daffodil/core/dpath/TestDFDLExpressionEvaluation.scala b/daffodil-core/src/test/scala/org/apache/daffodil/core/dpath/TestDFDLExpressionEvaluation.scala index 722e30ffe6..1fd22f51f0 100644 --- a/daffodil-core/src/test/scala/org/apache/daffodil/core/dpath/TestDFDLExpressionEvaluation.scala +++ b/daffodil-core/src/test/scala/org/apache/daffodil/core/dpath/TestDFDLExpressionEvaluation.scala @@ -31,7 +31,7 @@ import org.junit.Test; object INoWarn2 { ImplicitsSuppressUnusedImportWarning() import org.apache.daffodil.core.infoset.TestInfoset import org.apache.daffodil.core.util.TestUtils import org.apache.daffodil.io.InputSourceDataInputStream -import org.apache.daffodil.runtime1.infoset.InfosetDocument +import org.apache.daffodil.runtime1.api.InfosetDocument import org.apache.daffodil.runtime1.infoset.NullInfosetOutputter import org.apache.daffodil.runtime1.processors.DataProcessor import org.apache.daffodil.runtime1.processors.parsers.PState diff --git a/daffodil-core/src/test/scala/org/apache/daffodil/core/general/TestRuntimeProperties.scala b/daffodil-core/src/test/scala/org/apache/daffodil/core/general/TestRuntimeProperties.scala index cd0083e14b..0a22377aeb 100644 --- a/daffodil-core/src/test/scala/org/apache/daffodil/core/general/TestRuntimeProperties.scala +++ b/daffodil-core/src/test/scala/org/apache/daffodil/core/general/TestRuntimeProperties.scala @@ -21,8 +21,8 @@ import org.apache.daffodil.core.util.TestUtils import org.apache.daffodil.io.InputSourceDataInputStream import org.apache.daffodil.lib.Implicits.intercept import org.apache.daffodil.lib.util.SchemaUtils +import org.apache.daffodil.runtime1.api.InfosetSimpleElement import org.apache.daffodil.runtime1.dpath.NodeInfo -import org.apache.daffodil.runtime1.infoset.DISimple import org.apache.daffodil.runtime1.infoset.ScalaXMLInfosetInputter import org.apache.daffodil.runtime1.infoset.ScalaXMLInfosetOutputter @@ -37,10 +37,10 @@ import org.junit.Test */ class RedactingScalaXMLInfosetOutputter extends ScalaXMLInfosetOutputter { - override def startSimple(diSimple: DISimple): Unit = { - super.startSimple(diSimple) + override def startSimple(se: InfosetSimpleElement): Unit = { + super.startSimple(se) - val runtimeProperties = diSimple.erd.runtimeProperties + val runtimeProperties = se.metadata.runtimeProperties val redactions = Option(runtimeProperties.get("redact")).map { value => value.split(",") } if (redactions.isDefined) { diff --git a/daffodil-core/src/test/scala/org/apache/daffodil/core/infoset/TestInfoset.scala b/daffodil-core/src/test/scala/org/apache/daffodil/core/infoset/TestInfoset.scala index 2c39443314..da42c06e89 100644 --- a/daffodil-core/src/test/scala/org/apache/daffodil/core/infoset/TestInfoset.scala +++ b/daffodil-core/src/test/scala/org/apache/daffodil/core/infoset/TestInfoset.scala @@ -17,12 +17,16 @@ package org.apache.daffodil.core.infoset +import java.math.BigInteger + import org.apache.daffodil.core.compiler._ import org.apache.daffodil.core.dsom.{ ElementBase, Root } import org.apache.daffodil.lib.api.DaffodilTunables import org.apache.daffodil.lib.exceptions.Assert import org.apache.daffodil.lib.util._ import org.apache.daffodil.lib.xml.XMLUtils +import org.apache.daffodil.runtime1.api.InfosetArray +import org.apache.daffodil.runtime1.api.InfosetSimpleElement import org.apache.daffodil.runtime1.infoset._ import org.apache.daffodil.runtime1.processors.DataProcessor import org.apache.daffodil.runtime1.processors.unparsers.UStateMain @@ -46,8 +50,6 @@ object TestInfoset { * classes. */ - private val tunableForTests = DaffodilTunables("allowExternalPathExpressions", "true") - def elem2Infoset(xmlElem: scala.xml.Node, dp: DataProcessor): DIElement = { // // A prior version of this code just pulled events to force the @@ -131,10 +133,16 @@ class TestInfoset1 { val list_erd = infoset.erd assertEquals(list_erd, infoset.runtimeData) val Seq(w_erd) = list_erd.childERDs - val wItem = infoset.getChild(w_erd, tunable).asInstanceOf[InfosetSimpleElement] + val wItem = infoset.getChild(w_erd, tunable).asSimple assertEquals(infoset, wItem.parent) - assertEquals(4, wItem.dataValue.getAnyRef) - + assertEquals(4.toLong, wItem.getLong) + assertEquals(4, wItem.getInt) + assertEquals(4.toShort, wItem.getShort) + assertEquals(4.toByte, wItem.getByte) + assertEquals(new BigInteger("4"), wItem.getUnsignedLong) + assertEquals(4.toLong, wItem.getUnsignedInt) + assertEquals(4.toInt, wItem.getUnsignedShort) + assertEquals(4.toShort, wItem.getUnsignedByte) } @Test def testXMLToInfoset2(): Unit = { @@ -160,10 +168,10 @@ class TestInfoset1 { val list_erd = infoset.erd val Seq(w_erd, _, _, c_erd) = list_erd.childERDs assertEquals(list_erd, infoset.runtimeData) - val wItem = infoset.asComplex.getChild(w_erd, tunable).asInstanceOf[InfosetSimpleElement] - assertEquals(4, wItem.dataValue.getAnyRef) - val cItem = infoset.asComplex.getChild(c_erd, tunable).asInstanceOf[InfosetSimpleElement] - assertEquals(7, cItem.dataValue.getAnyRef) + val wItem = infoset.asComplex.getChild(w_erd, tunable).asSimple + assertEquals(4, wItem.getShort.toInt) + val cItem = infoset.asComplex.getChild(c_erd, tunable).asSimple + assertEquals(7, cItem.getByte.toInt) assertEquals(infoset, cItem.parent) } @@ -183,16 +191,17 @@ class TestInfoset1 { val (infoset: DIComplex, _, tunable) = testInfoset(testSchema, xmlInfoset) val Seq(w_erd) = infoset.erd.childERDs infoset.getChildArray(w_erd, tunable) match { - case arr: DIArray => { + case arr: InfosetArray => { assertEquals(2, arr.length) - var a = arr(1).asInstanceOf[InfosetSimpleElement] - assertEquals(w_erd, a.runtimeData) - - assertEquals(4, a.dataValue.getAnyRef) - assertEquals(infoset, a.parent) + var a = arr(1).asInstanceOf[InfosetSimpleElement] // 1-based + assertEquals(w_erd, a.metadata) + assertEquals(4, a.getUnsignedLong.longValue().toInt) + var dia = a.asInstanceOf[DISimple] + assertEquals(infoset, dia.parent) a = arr(2).asInstanceOf[InfosetSimpleElement] // 1-based - assertEquals(5, a.dataValue.getAnyRef) - assertEquals(infoset, a.parent) + assertEquals(5, a.getObject) + dia = a.asInstanceOf[DISimple] + assertEquals(infoset, dia.parent) } } } @@ -220,19 +229,22 @@ class TestInfoset1 { val list_erd = infoset.erd val Seq(w_erd, _, _, c_erd) = list_erd.childERDs infoset.getChildArray(w_erd, tunable) match { - case arr: DIArray => { + case arr: InfosetArray => { var a = arr(1).asInstanceOf[InfosetSimpleElement] assertEquals(2, arr.length) - assertEquals(w_erd, a.runtimeData) - assertEquals(4, a.dataValue.getAnyRef) - assertEquals(infoset, a.parent) - a = arr(2).asInstanceOf[InfosetSimpleElement] // 1-based - assertEquals(5, a.dataValue.getAnyRef) - assertEquals(infoset, a.parent) + assertEquals(w_erd, a.metadata) + assertEquals(4, a.getUnsignedInt.toInt) + var dia = a.asInstanceOf[DISimple] + assertEquals(infoset, dia.parent) + a = arr(2).asInstanceOf[InfosetSimpleElement] + assertEquals(5, a.getUnsignedShort.toInt) + dia = a.asInstanceOf[DISimple] + assertEquals(infoset, dia.parent) } } infoset.getChild(c_erd, tunable) match { - case s: DISimple => assertEquals(7, s.dataValue.getAnyRef) + case s: DISimple => assertEquals(7, s.getUnsignedByte.toInt) + case _ => fail("children should be DISimple") } } @@ -259,10 +271,10 @@ class TestInfoset1 { assertTrue(infoset.isInstanceOf[DIComplex]) val xchild = infoset.getChildArray(x_erd, tunable) xchild match { - case arr: DIArray => { + case arr: InfosetArray => { assertEquals(1, arr.length) val xa = arr(1) - assertEquals(x_erd, xa.runtimeData) + assertEquals(x_erd, xa.metadata) assertTrue(xa.isNilled) } } @@ -294,9 +306,9 @@ class TestInfoset1 { val list_erd = infoset.erd val Seq(x_erd) = list_erd.childERDs infoset.getChildArray(x_erd, tunable) match { - case xa: DIArray => { + case xa: InfosetArray => { assertEquals(1, xa.length) - val e = xa.getOccurrence(1) + val e = xa(1) assertTrue(e.isNilled) } } @@ -336,15 +348,16 @@ class TestInfoset1 { val Seq(w_erd, x_erd) = list_erd.childERDs val Seq(a_erd, b_erd, c_erd) = x_erd.childERDs infoset.getChildArray(x_erd, tunable) match { - case arr: DIArray => { + case arr: InfosetArray => { assertEquals(2, arr.length) - var xa = arr(1).asInstanceOf[InfosetComplexElement] - assertEquals(x_erd, xa.runtimeData) + var xa = arr(1).asInstanceOf[DIComplex] + assertEquals(x_erd, xa.metadata) assertTrue(xa.isNilled) - xa = arr(2).asInstanceOf[InfosetComplexElement] // 1-based + xa = arr(2).asInstanceOf[DIComplex] // 1-based val c = xa.getChild(c_erd, tunable) c match { - case c: DISimple => assertEquals(7, c.dataValue.getAnyRef) + case c: DISimple => assertEquals(7, c.getAnyRef) + case _ => fail("should be DISimple") } } } @@ -385,21 +398,21 @@ class TestInfoset1 { case e: InfosetNoSuchChildElementException => /* w element is not in xmlInfoset */ } - assertTrue(infoset.isInstanceOf[DIComplex]) - infoset.getChildArray(x_erd, tunable) match { case arr: DIArray => { assertEquals(2, arr.length) - var xa = arr(1).asInstanceOf[InfosetComplexElement] - assertEquals(x_erd, xa.runtimeData) + var xa = arr(1).asInstanceOf[DIComplex] + assertEquals(x_erd, xa.metadata) val c = xa.getChild(c_erd, tunable) c match { - case c: DISimple => assertEquals(7, c.dataValue.getAnyRef) + case c: DISimple => assertEquals(7, c.getAnyRef) + case _ => fail("should be DISimple") } - xa = arr(2).asInstanceOf[InfosetComplexElement] + xa = arr(2).asInstanceOf[DIComplex] val b = xa.getChild(b_erd, tunable) b match { - case c: DISimple => assertEquals(8, c.dataValue.getAnyRef) + case c: DISimple => assertEquals(8, c.getAnyRef) + case _ => fail("should be DISimple") } } } diff --git a/daffodil-core/src/test/scala/org/apache/daffodil/core/infoset/TestInfosetFree.scala b/daffodil-core/src/test/scala/org/apache/daffodil/core/infoset/TestInfosetFree.scala index 2ef45f883a..1b372be1a0 100644 --- a/daffodil-core/src/test/scala/org/apache/daffodil/core/infoset/TestInfosetFree.scala +++ b/daffodil-core/src/test/scala/org/apache/daffodil/core/infoset/TestInfosetFree.scala @@ -87,7 +87,7 @@ object TestInfosetFree { def docToXML(doc: DIDocument): scala.xml.Node = { val detailedOutputter = - new ScalaXMLInfosetOutputter(showFormatInfo = false, showFreedInfo = true) + new ScalaXMLInfosetOutputter(showFreedInfo = true) val infosetWalker = InfosetWalker( doc, diff --git a/daffodil-core/src/test/scala/org/apache/daffodil/core/util/TestUtils.scala b/daffodil-core/src/test/scala/org/apache/daffodil/core/util/TestUtils.scala index b009e10b92..30dcc965f4 100644 --- a/daffodil-core/src/test/scala/org/apache/daffodil/core/util/TestUtils.scala +++ b/daffodil-core/src/test/scala/org/apache/daffodil/core/util/TestUtils.scala @@ -38,6 +38,7 @@ import org.apache.daffodil.lib.util._ import org.apache.daffodil.lib.xml.XMLUtils import org.apache.daffodil.lib.xml._ import org.apache.daffodil.runtime1.api.DFDL +import org.apache.daffodil.runtime1.api.MetadataHandler import org.apache.daffodil.runtime1.debugger._ import org.apache.daffodil.runtime1.infoset.InfosetInputter import org.apache.daffodil.runtime1.infoset.InfosetOutputter @@ -390,6 +391,7 @@ class Fakes private () { override def tunables: DaffodilTunables = DaffodilTunables() override def variableMap: VariableMap = VariableMap(Nil) override def validationMode: ValidationMode.Type = ValidationMode.Full + override def walkMetadata(handler: MetadataHandler): Unit = {} override def withExternalVariables(extVars: Seq[Binding]): DFDL.DataProcessor = this override def withExternalVariables(extVars: java.io.File): DFDL.DataProcessor = this @@ -404,6 +406,7 @@ class Fakes private () { override def newContentHandlerInstance( output: DFDL.Output, ): DFDL.DaffodilUnparseContentHandler = null + } lazy val fakeDP = new FakeDataProcessor diff --git a/daffodil-japi/src/main/scala/org/apache/daffodil/japi/Daffodil.scala b/daffodil-japi/src/main/scala/org/apache/daffodil/japi/Daffodil.scala index 8dff857a6a..0542badfbf 100644 --- a/daffodil-japi/src/main/scala/org/apache/daffodil/japi/Daffodil.scala +++ b/daffodil-japi/src/main/scala/org/apache/daffodil/japi/Daffodil.scala @@ -49,6 +49,7 @@ import org.apache.daffodil.runtime1.api.DFDL.{ import org.apache.daffodil.runtime1.api.DFDL.{ DaffodilUnparseErrorSAXException => SDaffodilUnparseErrorSAXException, } +import org.apache.daffodil.runtime1.api.MetadataHandler import org.apache.daffodil.runtime1.debugger.Debugger import org.apache.daffodil.runtime1.debugger.{ InteractiveDebugger => SInteractiveDebugger } import org.apache.daffodil.runtime1.debugger.{ TraceDebuggerRunner => STraceDebuggerRunner } @@ -167,7 +168,7 @@ class Compiler private[japi] (private var sCompiler: SCompiler) { def compileSource(uri: URI, rootName: String, rootNamespace: String): ProcessorFactory = { val source = URISchemaSource(uri) val pf = sCompiler.compileSource(source, Option(rootName), Option(rootNamespace)) - new ProcessorFactory(pf.asInstanceOf[SProcessorFactory]) + new ProcessorFactory(pf) } /** @@ -514,6 +515,20 @@ class DataProcessor private[japi] (private var dp: SDataProcessor) */ def save(output: WritableByteChannel): Unit = dp.save(output) + /** + * Walks the handler over the runtime [[org.apache.daffodil.runtime1.api.Metadata]] structures. + * These provide information about name, namespace, type, simple/complex, etc. + * + * This is used to interface Daffodil runtime1 metadata to the metadata structures + * of other software systems. + * + * See [[org.apache.daffodil.runtime1.api.MetadataHandler]] for more motivating materials about + * runtime1 metadata walking. + * + * @param handler - the handler is called-back during the walk as each metadata structure is encountered. + */ + def walkMetadata(handler: MetadataHandler): Unit = dp.walkMetadata(handler) + /** * Obtain a new [[DaffodilParseXMLReader]] from the current [[DataProcessor]]. */ diff --git a/daffodil-japi/src/main/scala/org/apache/daffodil/japi/infoset/Infoset.scala b/daffodil-japi/src/main/scala/org/apache/daffodil/japi/infoset/Infoset.scala index bcf5396c7a..5e33225d13 100644 --- a/daffodil-japi/src/main/scala/org/apache/daffodil/japi/infoset/Infoset.scala +++ b/daffodil-japi/src/main/scala/org/apache/daffodil/japi/infoset/Infoset.scala @@ -20,12 +20,10 @@ package org.apache.daffodil.japi.infoset import org.apache.daffodil.japi.packageprivate._ import org.apache.daffodil.lib.exceptions.Assert import org.apache.daffodil.lib.util.MaybeBoolean +import org.apache.daffodil.runtime1.api.InfosetArray +import org.apache.daffodil.runtime1.api.InfosetComplexElement +import org.apache.daffodil.runtime1.api.InfosetSimpleElement import org.apache.daffodil.runtime1.dpath.NodeInfo -import org.apache.daffodil.runtime1.infoset.DIArray -import org.apache.daffodil.runtime1.infoset.DIComplex -// TODO: Not sure about the access to internal infoset implementation details. -// Should API users have this deep access to our internal infoset? -import org.apache.daffodil.runtime1.infoset.DISimple import org.apache.daffodil.runtime1.infoset.InfosetInputterEventType import org.apache.daffodil.runtime1.infoset.{ InfosetInputter => SInfosetInputter } import org.apache.daffodil.runtime1.infoset.{ InfosetOutputter => SInfosetOutputter } @@ -154,69 +152,69 @@ abstract class InfosetOutputter extends SInfosetOutputter { /** * Called by Daffodil internals to signify the beginning of a simple element. * - * @param diSimple the simple element that is started. Various fields of + * @param simple the simple element that is started. Various fields of * DISimple can be accessed to determine things like the * value, nil, name, namespace, etc. * @throws Exception if there was an error and Daffodil should stop parsing */ @throws[Exception] - def startSimple(diSimple: DISimple): Unit + def startSimple(simple: InfosetSimpleElement): Unit /** * Called by Daffodil internals to signify the end of a simple element. * - * @param diSimple the simple element that is ended. Various fields of + * @param simple the simple element that is ended. Various fields of * DISimple can be accessed to determine things like the * value, nil, name, namespace, etc. * @throws Exception if there was an error and Daffodil should stop parsing */ @throws[Exception] - def endSimple(diSimple: DISimple): Unit + def endSimple(simple: InfosetSimpleElement): Unit /** * Called by Daffodil internals to signify the beginning of a complex element. * - * @param diComplex the complex element that is started. Various fields of + * @param complex the complex element that is started. Various fields of * DIComplex can be accessed to determine things like the * nil, name, namespace, etc. * @throws Exception if there was an error and Daffodil should stop parsing */ @throws[Exception] - def startComplex(diComplex: DIComplex): Unit + def startComplex(complex: InfosetComplexElement): Unit /** * Called by Daffodil internals to signify the end of a complex element. * - * @param diComplex the complex element that is ended. Various fields of + * @param complex the complex element that is ended. Various fields of * DIComplex can be accessed to determine things like the * nil, name, namespace, etc. * @throws Exception if there was an error and Daffodil should stop parsing */ @throws[Exception] - def endComplex(diComplex: DIComplex): Unit + def endComplex(complex: InfosetComplexElement): Unit /** * Called by Daffodil internals to signify the beginning of an array of elements. * - * @param diArray the array that is started. Various fields of + * @param array the array that is started. Various fields of * DIArray can be accessed to determine things like the * name, namespace, etc. * @throws Exception if there was an error and Daffodil should stop parsing */ @throws[Exception] - def startArray(diArray: DIArray): Unit + def startArray(array: InfosetArray): Unit /** * Called by Daffodil internals to signify the end of an array of elements. * - * @param diArray the array that is ended. Various fields of + * @param array the array that is ended. Various fields of * DIArray can be accessed to determine things like the * name, namespace, etc. * @throws Exception if there was an error and Daffodil should stop parsing */ @throws[Exception] - def endArray(diArray: DIArray): Unit + def endArray(array: InfosetArray): Unit } /** @@ -231,9 +229,9 @@ abstract class InfosetOutputter extends SInfosetOutputter { * * @param showFormatInfo add additional properties to each scala.xml.Node for debug purposes */ -class ScalaXMLInfosetOutputter(showFormatInfo: Boolean = false) extends InfosetOutputterProxy { +class ScalaXMLInfosetOutputter() extends InfosetOutputterProxy { - override val infosetOutputter = new SScalaXMLInfosetOutputter(showFormatInfo) + override val infosetOutputter = new SScalaXMLInfosetOutputter() /** * Get the scala.xml.Node representing the infoset created during a parse @@ -464,11 +462,14 @@ abstract class InfosetOutputterProxy extends InfosetOutputter { override def reset(): Unit = infosetOutputter.reset() override def startDocument(): Unit = infosetOutputter.startDocument() override def endDocument(): Unit = infosetOutputter.endDocument() - override def startSimple(diSimple: DISimple): Unit = infosetOutputter.startSimple(diSimple) - override def endSimple(diSimple: DISimple): Unit = infosetOutputter.endSimple(diSimple) - override def startComplex(diComplex: DIComplex): Unit = - infosetOutputter.startComplex(diComplex) - override def endComplex(diComplex: DIComplex): Unit = infosetOutputter.endComplex(diComplex) - override def startArray(diArray: DIArray): Unit = infosetOutputter.startArray(diArray) - override def endArray(diArray: DIArray): Unit = infosetOutputter.endArray(diArray) + override def startSimple(simple: InfosetSimpleElement): Unit = + infosetOutputter.startSimple(simple) + override def endSimple(simple: InfosetSimpleElement): Unit = + infosetOutputter.endSimple(simple) + override def startComplex(complex: InfosetComplexElement): Unit = + infosetOutputter.startComplex(complex) + override def endComplex(complex: InfosetComplexElement): Unit = + infosetOutputter.endComplex(complex) + override def startArray(array: InfosetArray): Unit = infosetOutputter.startArray(array) + override def endArray(array: InfosetArray): Unit = infosetOutputter.endArray(array) } diff --git a/daffodil-japi/src/test/java/org/apache/daffodil/example/TestInfosetOutputter.java b/daffodil-japi/src/test/java/org/apache/daffodil/example/TestInfosetOutputter.java index cf04486c26..864796ebd1 100644 --- a/daffodil-japi/src/test/java/org/apache/daffodil/example/TestInfosetOutputter.java +++ b/daffodil-japi/src/test/java/org/apache/daffodil/example/TestInfosetOutputter.java @@ -22,18 +22,18 @@ import org.apache.daffodil.japi.infoset.InfosetOutputter; // TODO: Shouldn't need to import things not in the japi package -import org.apache.daffodil.runtime1.infoset.DIArray; -import org.apache.daffodil.runtime1.infoset.DIComplex; -import org.apache.daffodil.runtime1.infoset.DISimple; +import org.apache.daffodil.runtime1.api.InfosetArray; +import org.apache.daffodil.runtime1.api.InfosetComplexElement; +import org.apache.daffodil.runtime1.api.InfosetSimpleElement; +import static org.junit.Assert.assertEquals; public class TestInfosetOutputter extends InfosetOutputter { - public ArrayList events; - TestInfosetOutputter() { - events = new ArrayList<>(); - } + public ArrayList events = new ArrayList<>(); + + TestInfosetOutputter() {} @Override public void reset() { @@ -51,45 +51,46 @@ public void endDocument() { } @Override - public void startSimple(DISimple diSimple) { + public void startSimple(InfosetSimpleElement simple) { events.add( TestInfosetEvent.startSimple( - diSimple.erd().name(), - diSimple.erd().namedQName().namespace().toString(), - diSimple.dataValueAsString(), - diSimple.erd().isNillable() ? diSimple.isNilled() : null)); + simple.metadata().name(), + simple.metadata().namespace(), + simple.getText(), + simple.metadata().isNillable() ? simple.isNilled() : null)); } @Override - public void endSimple(DISimple diSimple) { + public void endSimple(InfosetSimpleElement simple) { events.add( TestInfosetEvent.endSimple( - diSimple.erd().name(), - diSimple.erd().namedQName().namespace().toString())); + simple.metadata().name(), + simple.metadata().namespace())); } @Override - public void startComplex(DIComplex diComplex) { + public void startComplex(InfosetComplexElement complex) throws Exception { + events.add( TestInfosetEvent.startComplex( - diComplex.erd().name(), - diComplex.erd().namedQName().namespace().toString(), - diComplex.erd().isNillable() ? diComplex.isNilled() : null)); + complex.metadata().name(), + complex.metadata().namespace(), + complex.metadata().isNillable() ? complex.isNilled() : null)); } @Override - public void endComplex(DIComplex diComplex) { + public void endComplex(InfosetComplexElement complex) { events.add( TestInfosetEvent.endComplex( - diComplex.erd().name(), - diComplex.erd().namedQName().namespace().toString())); + complex.metadata().name(), + complex.metadata().namespace())); } @Override - public void startArray(DIArray diArray) { + public void startArray(InfosetArray array) { } @Override - public void endArray(DIArray diArray) { + public void endArray(InfosetArray array) { } } diff --git a/daffodil-japi/src/test/java/org/apache/daffodil/example/TestJavaMetadataAPI.java b/daffodil-japi/src/test/java/org/apache/daffodil/example/TestJavaMetadataAPI.java new file mode 100644 index 0000000000..941dd1b224 --- /dev/null +++ b/daffodil-japi/src/test/java/org/apache/daffodil/example/TestJavaMetadataAPI.java @@ -0,0 +1,131 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.apache.daffodil.example; + +import org.apache.daffodil.japi.Daffodil; +import org.apache.daffodil.japi.DataProcessor; +import org.apache.daffodil.japi.Diagnostic; +import org.apache.daffodil.japi.ProcessorFactory; +import org.apache.daffodil.runtime1.api.ChoiceMetadata; +import org.apache.daffodil.runtime1.api.ComplexElementMetadata; +import org.apache.daffodil.runtime1.api.ElementMetadata; +import org.apache.daffodil.runtime1.api.Metadata; +import org.apache.daffodil.runtime1.api.MetadataHandler; +import org.apache.daffodil.runtime1.api.SequenceMetadata; +import org.apache.daffodil.runtime1.api.SimpleElementMetadata; +import org.junit.Test; + +import java.io.File; +import java.io.IOException; +import java.util.ArrayList; +import java.util.List; +import java.util.Objects; +import java.util.stream.Collectors; + +import static org.junit.Assert.assertEquals; +import static org.junit.Assert.fail; + +public class TestJavaMetadataAPI { + + private java.io.File getResource(String resPath) { + try { + return new File(Objects.requireNonNull(getClass().getResource(resPath)).toURI()); + } catch (Exception e) { + return null; + } + } + + public static class GatherMetadata extends MetadataHandler { + + private final List buf = new ArrayList<>(); + + public List getResult() { + List res = new ArrayList<>(buf); // makes a copy + buf.clear(); + return res; + } + + @Override + public void simpleElementMetadata(SimpleElementMetadata m) { + buf.add(m); + } + + @Override + public void startComplexElementMetadata(ComplexElementMetadata m) { + buf.add(m); + } + + @Override + public void endComplexElementMetadata(ComplexElementMetadata m) { + buf.add(m); + } + + @Override + public void startSequenceMetadata(SequenceMetadata m) { + buf.add(m); + } + + @Override + public void endSequenceMetadata(SequenceMetadata m) { + buf.add(m); + } + + @Override + public void startChoiceMetadata(ChoiceMetadata m) { + buf.add(m); + } + + @Override + public void endChoiceMetadata(ChoiceMetadata m) { + buf.add(m); + } + } + + + @Test + public void testMetadataWalkDataWalk01() throws IOException { + GatherMetadata gatherMetadata = new GatherMetadata(); + org.apache.daffodil.japi.Compiler c = Daffodil.compiler(); + File schemaFile = getResource("/test/japi/metadataTestSchema1.dfdl.xsd"); + ProcessorFactory pf = c.compileFile(schemaFile); + if (pf.isError()) { + String diags = pf.getDiagnostics() + .stream() + .map(Diagnostic::getMessage) + .collect(Collectors.joining(System.lineSeparator())); + fail(diags); + } + DataProcessor dp = pf.onPath("/"); + dp.walkMetadata(gatherMetadata); + List md = gatherMetadata.getResult(); + List mdQNames = md.stream().map(item -> { + if (item instanceof ElementMetadata) { + ElementMetadata em = ((ElementMetadata) item); + String res = em.toQName() + ((em.isArray()) ? "_array" : ""); + return res; + } else if (item instanceof SequenceMetadata) { + return "seq"; + } else if (item instanceof ChoiceMetadata) { + return "cho"; + } else { + return ""; + } + }).collect(Collectors.toList()); + assertEquals("[ex:e1, cho, seq, seq, seq, s1_array, seq, cho, ex:e1]", mdQNames.toString()); + } +} diff --git a/daffodil-japi/src/test/resources/test/japi/metadataTestSchema1.dfdl.xsd b/daffodil-japi/src/test/resources/test/japi/metadataTestSchema1.dfdl.xsd new file mode 100644 index 0000000000..b27927ddac --- /dev/null +++ b/daffodil-japi/src/test/resources/test/japi/metadataTestSchema1.dfdl.xsd @@ -0,0 +1,55 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + diff --git a/daffodil-lib/src/main/scala/org/apache/daffodil/lib/exceptions/SchemaFileLocatable.scala b/daffodil-lib/src/main/scala/org/apache/daffodil/lib/exceptions/SchemaFileLocatable.scala index cd1f2fc87a..1c01cbd854 100644 --- a/daffodil-lib/src/main/scala/org/apache/daffodil/lib/exceptions/SchemaFileLocatable.scala +++ b/daffodil-lib/src/main/scala/org/apache/daffodil/lib/exceptions/SchemaFileLocatable.scala @@ -68,10 +68,13 @@ class SchemaFileLocation private ( override val toString = contextToString - override def fileDescription = { - val decodedString = URLDecoder.decode(uriString, "UTF-8") - " in " + limitMaxParentDirectories(decodedString, maxParentDirectoriesForDiagnostics) - } + lazy val fileURITrimmed = + limitMaxParentDirectories( + URLDecoder.decode(uriString, "UTF-8"), + maxParentDirectoriesForDiagnostics, + ) + + override def fileDescription = " in " + fileURITrimmed override def locationDescription = { val showInfo = lineDescription != "" || fileDescription != "" @@ -110,12 +113,11 @@ trait SchemaFileLocatable extends LocationInSchemaFile with HasSchemaFileLocatio case None => "" } + lazy val fileURITrimmed = schemaFileLocation.fileURITrimmed + // URLDecoder removes %20, etc from the file name. override lazy val fileDescription = { - val newUriString: String = limitMaxParentDirectories( - URLDecoder.decode(uriString, "UTF-8"), - tunables.maxParentDirectoriesForDiagnostics, - ) + val newUriString: String = fileURITrimmed " in " + newUriString } diff --git a/daffodil-lib/src/main/scala/org/apache/daffodil/lib/util/Numbers.scala b/daffodil-lib/src/main/scala/org/apache/daffodil/lib/util/Numbers.scala index e11fb7c5b4..b7fcf5bd81 100644 --- a/daffodil-lib/src/main/scala/org/apache/daffodil/lib/util/Numbers.scala +++ b/daffodil-lib/src/main/scala/org/apache/daffodil/lib/util/Numbers.scala @@ -30,7 +30,10 @@ import java.math.{ BigInteger => JBigInt } import org.apache.daffodil.lib.exceptions.Assert -object Numbers { +object Numbers extends Numbers { + + override protected def errorThrower(message: String): Nothing = + Assert.invariantFailed(message) def isValidInt(n: Number): Boolean = { val res = n match { @@ -90,6 +93,11 @@ object Numbers { d.equals(bd) } } +} + +trait Numbers { + + protected def errorThrower(message: String): Nothing def asInt(n: AnyRef): JInt = { val value = n match { @@ -102,7 +110,7 @@ object Numbers { case bi: JBigInt => bi.intValue() case bd: JBigDecimal => bd.intValue() case _ => - Assert.invariantFailed( + errorThrower( "Unsupported conversion to Int. %s of type %s".format(n, Misc.getNameFromClass(n)), ) } @@ -119,7 +127,7 @@ object Numbers { case bi: JBigInt => bi.byteValue() case bd: JBigDecimal => bd.byteValue() case _ => - Assert.invariantFailed( + errorThrower( "Unsupported conversion to Byte. %s of type %s".format(n, Misc.getNameFromClass(n)), ) } @@ -136,7 +144,7 @@ object Numbers { case bi: JBigInt => bi.shortValue() case bd: JBigDecimal => bd.shortValue() case _ => - Assert.invariantFailed( + errorThrower( "Unsupported conversion to Short. %s of type %s".format(n, Misc.getNameFromClass(n)), ) } @@ -154,7 +162,7 @@ object Numbers { case bi: JBigInt => bi.longValue() case bd: JBigDecimal => bd.longValue() case _ => - Assert.invariantFailed( + errorThrower( "Unsupported conversion to Long. %s of type %s".format(n, Misc.getNameFromClass(n)), ) } @@ -176,7 +184,7 @@ object Numbers { // the rest of the JNumbers are integers long or smaller. case jn: JNumber => new JBigInt(jn.toString()) case _ => - Assert.invariantFailed( + errorThrower( "Unsupported conversion to BigInt. %s of type %s".format(n, Misc.getNameFromClass(n)), ) } @@ -198,7 +206,7 @@ object Numbers { // the rest of the JNumbers are integers long or smaller. case jn: JNumber => JBigInt.valueOf(jn.longValue()) case _ => - Assert.invariantFailed( + errorThrower( "Unsupported conversion to BigInt. %s of type %s".format(n, Misc.getNameFromClass(n)), ) } @@ -222,7 +230,7 @@ object Numbers { case bi: JBigInt => bi.floatValue() case bd: JBigDecimal => bd.floatValue() case _ => - Assert.invariantFailed( + errorThrower( "Unsupported conversion to Float. %s of type %s".format(n, Misc.getNameFromClass(n)), ) } @@ -246,7 +254,7 @@ object Numbers { case bi: JBigInt => bi.doubleValue() case bd: JBigDecimal => bd.doubleValue() case _ => - Assert.invariantFailed( + errorThrower( "Unsupported conversion to Double. %s of type %s".format(n, Misc.getNameFromClass(n)), ) } @@ -270,9 +278,13 @@ object Numbers { case bi: JBigInt => new java.math.BigDecimal(bi.toString()) case bd: JBigDecimal => bd // The rest of the cases are integers long or smaller + case jl: JLong => java.math.BigDecimal.valueOf(jl) + case i: JInt => java.math.BigDecimal.valueOf(i.toLong) + case s: JShort => java.math.BigDecimal.valueOf(s.toLong) + case b: JByte => java.math.BigDecimal.valueOf(b.toLong) case jn: JNumber => new java.math.BigDecimal(jn.toString()) case _ => - Assert.invariantFailed( + errorThrower( "Unsupported conversion to BigDecimal. %s of type %s".format( n, Misc.getNameFromClass(n), @@ -301,7 +313,7 @@ object Numbers { // The rest of the cases are integers long or smaller case jn: JNumber => JBigDecimal.valueOf(jn.longValue()) case _ => - Assert.invariantFailed( + errorThrower( "Unsupported conversion to BigDecimal. %s of type %s".format( n, Misc.getNameFromClass(n), @@ -316,7 +328,7 @@ object Numbers { case bool: JBoolean => return bool case b: Boolean => JBoolean.valueOf(b) case _ => - Assert.invariantFailed( + errorThrower( "Unsupported conversion to Boolean. %s of type %s".format(n, Misc.getNameFromClass(n)), ) } @@ -332,7 +344,7 @@ object Numbers { case d: Double => JDouble.valueOf(d) case jn: JNumber => jn case _ => - Assert.invariantFailed( + errorThrower( "Unsupported conversion to Number. %s of type %s".format(n, Misc.getNameFromClass(n)), ) } @@ -355,7 +367,7 @@ object Numbers { case s: JShort => s.shortValue == 0 case b: JByte => b.byteValue == 0 // $COVERAGE-OFF$ - case _ => Assert.invariantFailed(s"Unknown JNumber type: ${n1.getClass.getName}") + case _ => errorThrower(s"Unknown JNumber type: ${n1.getClass.getName}") // $COVERAGE-ON$ } } diff --git a/daffodil-lib/src/main/scala/org/apache/daffodil/lib/util/SchemaUtils.scala b/daffodil-lib/src/main/scala/org/apache/daffodil/lib/util/SchemaUtils.scala index a7fc88008c..7445f26498 100644 --- a/daffodil-lib/src/main/scala/org/apache/daffodil/lib/util/SchemaUtils.scala +++ b/daffodil-lib/src/main/scala/org/apache/daffodil/lib/util/SchemaUtils.scala @@ -96,6 +96,7 @@ object SchemaUtils { defaultNamespace: NS = XMLUtils.targetNS, elementFormDefault: String = "qualified", useDefaultNamespace: Boolean = true, + useTNS: Boolean = true, ): Elem = { val fileAttrib = if (fileName == "") Null @@ -115,7 +116,8 @@ object SchemaUtils { if (useDefaultNamespace) { scope = XMLUtils.combineScopes(null, defaultNamespace, scope) } - scope = XMLUtils.combineScopes("tns", targetNamespace, scope) + if (useTNS) + scope = XMLUtils.combineScopes("tns", targetNamespace, scope) scope = XMLUtils.combineScopes("ex", targetNamespace, scope) scope = XMLUtils.combineScopes("dfdlx", XMLUtils.DFDLX_NAMESPACE, scope) diff --git a/daffodil-runtime1/src/main/scala/org/apache/daffodil/runtime1/api/DFDLParserUnparser.scala b/daffodil-runtime1/src/main/scala/org/apache/daffodil/runtime1/api/DFDLParserUnparser.scala index dcaf4dc926..a756556901 100644 --- a/daffodil-runtime1/src/main/scala/org/apache/daffodil/runtime1/api/DFDLParserUnparser.scala +++ b/daffodil-runtime1/src/main/scala/org/apache/daffodil/runtime1/api/DFDLParserUnparser.scala @@ -177,6 +177,7 @@ object DFDL { def save(output: DFDL.Output): Unit + def walkMetadata(handler: MetadataHandler): Unit def tunables: DaffodilTunables def variableMap: VariableMap def validationMode: ValidationMode.Type diff --git a/daffodil-runtime1/src/main/scala/org/apache/daffodil/runtime1/api/Infoset.scala b/daffodil-runtime1/src/main/scala/org/apache/daffodil/runtime1/api/Infoset.scala new file mode 100644 index 0000000000..b4df833fe5 --- /dev/null +++ b/daffodil-runtime1/src/main/scala/org/apache/daffodil/runtime1/api/Infoset.scala @@ -0,0 +1,327 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.apache.daffodil.runtime1.api + +import java.lang.{ Boolean => JBoolean } +import java.lang.{ Byte => JByte } +import java.lang.{ Double => JDouble } +import java.lang.{ Float => JFloat } +import java.lang.{ Integer => JInt } +import java.lang.{ Long => JLong } +import java.lang.{ Short => JShort } +import java.lang.{ String => JString } +import java.math.{ BigDecimal => JBigDecimal } +import java.math.{ BigInteger => JBigInt } +import java.net.URI + +import com.ibm.icu.util.Calendar + +/** + * API access to array objects in the DFDL Infoset + */ +trait InfosetArray extends InfosetItem { + + /** + * @return the metadata of the element that is an array + */ + override def metadata: ElementMetadata +} + +/** + * API access to elements of the DFDL Infoset of both + * complex and simple type. + */ +trait InfosetElement extends InfosetItem { + + /** + * In DFDL both simple and complex type elements can be + * nilled. + * + * @return true if the element is nilled, false otherwise. + */ + def isNilled: Boolean + + /* + * Access to the metadata information about this element. + * See [[ElementMetadata]] + */ + def metadata: ElementMetadata + +} + +/** + * Methods specific complex elements in the infoset + */ +trait InfosetComplexElement extends InfosetElement { + + /* + * Access to the metadata information about this element. + * See [[ComplexElementMetadata]] + */ + override def metadata: ComplexElementMetadata +} + +/** + * Methods specific to simple elements in the infoset + */ +trait InfosetSimpleElement extends InfosetElement { + + /* + * Access to the metadata information about this element. + * See [[SimpleElementMetadata]] + */ + override def metadata: SimpleElementMetadata + + /** + * Obtains the value, then converts it to a string. + * Caches the string so we're not allocating strings repeatedly + */ + def getText: String + + /* + * These are so that API users don't have to know about our + * very Scala-oriented DataValue type system. + */ + + /** + * @return the value of this simple element as a Scala AnyRef, which is + * equivalent to a Java Object. + */ + def getAnyRef: AnyRef + + /** + * @return the value of this simple element as an Object (java.lang.Object), + * which is equivalent to Scala AnyRef. + */ + final def getObject: java.lang.Object = getAnyRef + + // Note: I could not get @throws in scaladoc to work right. + // Complains "Could not find any member to link for ... and I tried various formulations of + // InfosetTypeException, with package, with and without [[..]]. + // So I've just converted it to plain text. + + /** + * @return Casts the value of this simple element as a java.math.BigDecimal + * or throws `InfosetTypeException` if the element is not of type decimal. + */ + def getDecimal: JBigDecimal + + /** + * @return the value of this simple element cast as a `com.ibm.icu.util.Calendar`. + * or throws `InfosetTypeException` if the element is not of type date. + */ + def getDate: Calendar + + /** + * @return the value of this simple element as a `com.ibm.icu.util.Calendar`. + * or throws `InfosetTypeException` if the element is not of type time. + */ + def getTime: Calendar + + /** + * @return the value of this simple element cast to `com.ibm.icu.util.Calendar`. + * or throws `InfosetTypeException` if the element is not of type dateTime. + */ + def getDateTime: Calendar + + /** + * @return the value of this simple element of HexBinary type cast to `Array[Byte]`. + * or throws `InfosetTypeException` if the element is not of type hexBinary. + */ + def getHexBinary: Array[Byte] + + /** + * @return the value of this simple element of Boolean type cast to java.lang.Boolean. + * or throws `InfosetTypeException` if the element is not of type boolean. + */ + def getBoolean: JBoolean + + /** + * Used to access simple element values of all integer types representable by + * a 64 bit signed java.lang.Long. + * + * The separate [[getUnsignedLong]] + * must be used for the DFDL unsignedLong type. + * @return the value of this simple element converted to java.lang.Long. + * or throws `InfosetTypeException` if the element value is not convertible to the result type. + */ + def getLong: JLong + + /** + * Used to access simple element values of all integer types representable by + * a 32 bit signed java.lang.Integer. + * + * The separate [[getUnsignedInt]] + * must be used for the DFDL unsignedInt type. + * + * Do not confuse DFDL integer type with java.lang.Integer, which is the object version of + * a java.lang.int, which is limited to only signed 32-bits of magnitude. The DFDL integer + * type is an unbounded magnitude integer (aka BigInteger). + * + * @return the value of this simple element converted to java.lang.Integer. + * or throws `InfosetTypeException` if the element value is not convertible to the result type. + */ + def getInt: JInt + + /** + * Used to access simple element values of all integer types representable by + * a 16 bit signed java.lang.Short. + * + * The separate [[getUnsignedShort]] + * must be used for the DFDL unsignedShorttype. + * @return the value of this simple element converted to java.lang.Short. + * or throws `InfosetTypeException` if the element value is not convertible to the result type. + */ + def getShort: JShort + + /** + * Used to access simple element values of all integer types representable by + * an 8-bit signed java.lang.Byte. + * + * The separate [[getUnsignedByte]] + * must be used for the DFDL unsignedBytetype. + * + * @return the value of this simple element converted to java.lang.Short. + * or throws `InfosetTypeException` if the element value is not convertible to the result type. + */ + def getByte: JByte + + /** + * Used to access simple element values of all non-negative integer types representable by + * a 32-bit unsigned integer. + * + * Note that the returned value is the larger signed type, java.lang.Long which is capable + * of representing unsigned integer values greater than java.lang.Integer.MAX_VALUE. + * + * @return the value of this simple element converted to java.lang.Long. + * or throws `InfosetTypeException` if the element value is not convertible to the result type. + */ + def getUnsignedInt: JLong + + /** + * Used to access simple element values of all non-negative integer types representable by + * a 16-bit unsigned integer. + * + * Note that the returned value is the larger signed type, java.lang.Int which is capable + * of representing unsigned integer values greater than java.lang.Short.MAX_VALUE. + * + * @return the value of this simple element converted to java.lang.Int. + * or throws `InfosetTypeException` if the element value is not convertible to the result type. + */ + def getUnsignedShort: JInt + + /** + * Used to access simple element values of all non-negative integer types representable by + * an 8-bit unsigned integer. + * + * Note that the returned value is the larger signed type, java.lang.Short which is capable + * of representing unsigned integer values greater than java.lang.Byte.MAX_VALUE. + * + * @return the value of this simple element converted to java.lang.Int. + * or throws `InfosetTypeException` if the element value is not convertible to the result type. + */ + def getUnsignedByte: JShort + + /** + * Used to access simple element values of all non-negative integer types representable by + * a 64-bit unsigned integer. + * + * Note that the returned value is the larger signed type, java.math.BigInteger which is capable + * of representing unsigned integer values greater than java.lang.Long.MAX_VALUE. + * + * @return the value of this simple element converted to java.lang.BigInteger. + * or throws `InfosetTypeException` if the element value is not convertible to the result type. + */ + def getUnsignedLong: JBigInt + + /** + * @return the value of this simple element of Double type cast to java.lang.Double. + * or throws `InfosetTypeException` if the element in not of type double. + */ + def getDouble: JDouble + + /** + * @return the value of this simple element of Float type cast to java.lang.Float. + * or throws `InfosetTypeException` if the element is not of type float. + */ + def getFloat: JFloat + + /** + * Used to get the value of DFDL `integer` type, which is an unbounded-magnitude integer. + * + * Do not confuse DFDL integer type with java.lang.Integer, which is the object version of + * a java.lang.int, which is limited to only signed 32-bits of magnitude. + * @return the value of this simple element of Integer type cast to java.math.BigInteger. + * or throws `InfosetTypeException` if the element is not of type integer. + */ + def getInteger: JBigInt + + /** + * @return the value of this simple element of NonNegativeInteger type cast to java.math.BigInteger. + * or throws `InfosetTypeException` if the element value is not convertible to the result type. + */ + def getNonNegativeInteger: JBigInt + + /** + * @return the value of this simple element of String type cast to java.lang.String. + * or throws `InfosetTypeException` if the element value is not of String type. + */ + def getString: JString + + /** + * @return the value of this simple element of URI type cast to java.net.URI. + * or throws `InfosetTypeException` if the element value is not of URI type. + */ + def getURI: URI +} + +// $COVERAGE-OFF$ +/** + * Thrown if you try to access a simple type but the value of the + * InfosetSimpleElement is not convertible to that type. + */ +class InfosetTypeException(msg: String, cause: Throwable) + extends Exception(msg: String, cause: Throwable) { + + def this(msg: String) = this(msg, null) + def this(cause: Throwable) = this(null, cause) +} +// $COVERAGE-ON$ + +/** + * Access to the infoset document element (also known as the root element). + */ +trait InfosetDocument extends InfosetItem { + + /** + * Access to the metadata information about this element. + * See [[ElementMetadata]] + */ + override def metadata: ElementMetadata +} + +/** + * Methods common to all infoset items + */ +trait InfosetItem { + + /** + * All infoset items have access to metadata. + */ + def metadata: Metadata +} diff --git a/daffodil-runtime1/src/main/scala/org/apache/daffodil/runtime1/api/Metadata.scala b/daffodil-runtime1/src/main/scala/org/apache/daffodil/runtime1/api/Metadata.scala new file mode 100644 index 0000000000..f63cbcaf9f --- /dev/null +++ b/daffodil-runtime1/src/main/scala/org/apache/daffodil/runtime1/api/Metadata.scala @@ -0,0 +1,306 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.daffodil.runtime1.api + +import java.lang.{ Long => JLong } +import scala.collection.JavaConverters._ +import scala.xml.NamespaceBinding + +import org.apache.daffodil.runtime1.dpath.NodeInfo +import org.apache.daffodil.runtime1.dpath.NodeInfo.PrimType + +/** + * This is the supportable API for access to the RuntimeData structures + * which provide access to static information about a given compiled schema + * metadata object. + * + * This is used to interface other data processing fabrics to Daffodil + * data and metadata, by mapping to/from these metadata objects. + */ +trait Metadata { + + /** + * Provides the file context of a metadata component. This refers to the specific + * DFDL schema file where the corresponding DFDL schema text resides corresponding + * to this metadata object. + *

+ * This is for use in diagnostic messaging. It is not the actual file URI, because + * those may contain personal-identifying information about the person/acccount and + * system that compiled the schema. It will provide enough content about the file URI that + * a user will be able to identify which file, but some prefix of the path + * components trimmed to make it of a manageable length. + *

+ * Used along with [[org.apache.daffodil.runtime1.api.Metadata.schemaFileLineNumber]] + * and [[org.apache.daffodil.runtime1.api.Metadata.schemaFileLineColumnNumber]], + * this can give a precise location in the DFDL schema file. + * @return a string containing the file information, or null if unknown. + */ + def schemaFileInfo: String + + /** + * Provides the line number to go with [[org.apache.daffodil.runtime1.api.Metadata.schemaFileInfo]]. + * @return the line number as a string, or null if unknown. + */ + def schemaFileLineNumber: JLong + + /** + * Provides the column number within the text line, to go with [[org.apache.daffodil.runtime1.api.Metadata.schemaFileLineNumber]]. + * @return the column number within the text line, as a string, or null if unknown. + */ + def schemaFileLineColumnNumber: JLong + + /** + * The name of the schema component, in a form suitable for diagnostic messages. + * Unnamed components like sequence or choice groups have a diagnosticDebugName, despite not having + * any actual name. + * @return the name of the component, suitable for use in diagnostic messages. + */ + def diagnosticDebugName: String +} + +/* + * Provides metadata access that is common to all Terms, which include + * Elements of simple or complex type, as well as the Sequence and Choice groups. + */ +trait TermMetadata extends Metadata { + // nothing here +} + +/** + * Common metadata access for all elements, of simple or complex type. + */ +trait ElementMetadata extends TermMetadata { + + /** + * @return the name of this element. In the case of a global/qualified name, this is only the local + * part of the QName. + */ + def name: String + + /** + * @return the namespace URI as a string, or null if no namespace. + */ + def namespace: String + + /** + * @return the namespace bindings needed to construct an XML element from a Daffodil infoset + * element of simple or complex type. + */ + def minimizedScope: NamespaceBinding + + /** + * @return the namespace prefix part of the XML QName of this component, or null if there + * is no prefix defined or no namespace. + */ + def prefix: String + + /** + * @return true if two or more occurrences are possible. + * Note that having only 0 or 1 occurrence is not considered an array, + * but rather an optional element. + */ + def isArray: Boolean + + /** + * @return true if only 0 or 1 occurrence are possible. + */ + def isOptional: Boolean + + /** + * @return the QName string for this element. + */ + def toQName: String + + /** + * @return true if the element is declared to be nillable. + */ + def isNillable: Boolean + + /** + * Provides access to the runtime properties. This is an extended collection of + * name-value pairs which are associated with a schema component. + *

+ * Runtime properties are intended to use for new ad-hoc property extensions to + * DFDL. These name-value pairs are visible to infoset outputters as well. + * + * @return a java-compatible map of name-value pairs. + */ + def runtimeProperties: java.util.Map[String, String] + +} + +/** + * Access to metadata values exclusive to elements of complex type. + */ +trait ComplexElementMetadata extends ElementMetadata { + // no specific methods +} + +/** + * Access to metadata values exclusive to elements of simple type. + */ +trait SimpleElementMetadata extends ElementMetadata { + + def primitiveType: PrimitiveType +} + +/** + * Instances are static objects that represent the DFDL primitive types. + */ +trait PrimitiveType { + def name: String +} + +/** + * Static methods related to PrimitiveType objects + */ +object PrimitiveType { + + private lazy val _list: java.util.List[PrimitiveType] = + NodeInfo.allDFDLTypes.asInstanceOf[Seq[PrimitiveType]].asJava + + /** + * Get a primitive type given a name string. + * + * @param name lookup key. Case insensitive. + * @return the PrimitiveType with that name, or null if there is no such primitive type. + */ + def fromName(name: String): PrimitiveType = + NodeInfo.primitiveTypeFromName(name) + + /** + * A list of all the primitive type objects. + */ + def list: java.util.List[PrimitiveType] = _list + + val String: PrimitiveType = PrimType.String + val Int: PrimitiveType = PrimType.Int + val Byte: PrimitiveType = PrimType.Byte + val Short: PrimitiveType = PrimType.Short + val Long: PrimitiveType = PrimType.Long + val Integer: PrimitiveType = PrimType.Integer + val Decimal: PrimitiveType = PrimType.Decimal + val UnsignedInt: PrimitiveType = PrimType.UnsignedInt + val UnsignedByte: PrimitiveType = PrimType.UnsignedByte + val UnsignedShort: PrimitiveType = PrimType.UnsignedShort + val UnsignedLong: PrimitiveType = PrimType.UnsignedLong + val NonNegativeInteger: PrimitiveType = PrimType.NonNegativeInteger + val Double: PrimitiveType = PrimType.Double + val Float: PrimitiveType = PrimType.Float + val HexBinary: PrimitiveType = PrimType.HexBinary + val AnyURI: PrimitiveType = PrimType.AnyURI + val Boolean: PrimitiveType = PrimType.Boolean + val DateTime: PrimitiveType = PrimType.DateTime + val Date: PrimitiveType = PrimType.Date + val Time: PrimitiveType = PrimType.Time +} + +/** + * Access to metadata values shared by both sequences and choices + * which are known collectively as Model Groups. + */ +trait ModelGroupMetadata extends TermMetadata {} + +/** + * Access to metadata values specific to sequences + */ +trait SequenceMetadata extends ModelGroupMetadata {} + +/** + * Access to metadata values specific to choices + */ +trait ChoiceMetadata extends ModelGroupMetadata {} + +/** + * Base class used by clients who want to walk the runtime1 metadata information. + * + * The runtime1 [[Metadata]] is the aspects of the DFDL schema information that are + * needed at runtime. + * + * Interfacing Daffodil to other data handling systems requires both a metadata bridge + * be built that takes Daffodil metadata into that system's metadata, and a data bridge + * that takes Daffodil data to that system's data. + * + * Bridging this runtime1 library to other data handling software is most easily done + * directly from runtime1's metadata and data structures. + * (The Daffodil Schema Compiler's walkers are an alternative, but are more + * for interfacing the Daffodil schema compiler data structures to other compiler backends.) + * + * This walker/handler works on the pre-compiled binary schema + * just as well as if the schema was just compiled. This bypasses the need for Daffodil's + * schema compiler to be involved at all in interfacing to say, Apache Drill or other + * data fabrics. A pre-compiled DFDL schema is all that is needed. + */ +abstract class MetadataHandler() { + + /** + * Called for simple type element metadata (for declarations or references) + */ + def simpleElementMetadata(m: SimpleElementMetadata): Unit + + /** + * Called for complex type element metadata (for declarations or references) + * + * Subsequent calls will be for the model group making up the content + * of the element. + */ + def startComplexElementMetadata(m: ComplexElementMetadata): Unit + + /** + * Called for complex type element metadata (for declarations or references) + * + * This is called after all the calls corresponding to the content of the + * complex type element. + * @param m + */ + def endComplexElementMetadata(m: ComplexElementMetadata): Unit + + /** + * Called for sequence groups. + * + * Subsequent calls will be for the content of the sequence. + * @param m + */ + def startSequenceMetadata(m: SequenceMetadata): Unit + + /** + * Called for sequence groups. + * + * This is called after all the calls corresponding to the content + * of the sequence group. + * @param m + */ + def endSequenceMetadata(m: SequenceMetadata): Unit + + /** + * Called for choice groups. + * + * Subsequent calls will be for the content of the choice. + * @param m + */ + def startChoiceMetadata(m: ChoiceMetadata): Unit + + /** + * Called for choice groups. + * + * This is called after all the calls corresponding to the content + * of the choice group. + * @param m + */ + def endChoiceMetadata(m: ChoiceMetadata): Unit + +} diff --git a/daffodil-runtime1/src/main/scala/org/apache/daffodil/runtime1/debugger/InteractiveDebugger.scala b/daffodil-runtime1/src/main/scala/org/apache/daffodil/runtime1/debugger/InteractiveDebugger.scala index bff7156b12..35f176f5c4 100644 --- a/daffodil-runtime1/src/main/scala/org/apache/daffodil/runtime1/debugger/InteractiveDebugger.scala +++ b/daffodil-runtime1/src/main/scala/org/apache/daffodil/runtime1/debugger/InteractiveDebugger.scala @@ -32,12 +32,12 @@ import org.apache.daffodil.lib.xml.GlobalQName import org.apache.daffodil.lib.xml.QName import org.apache.daffodil.lib.xml.XMLUtils import org.apache.daffodil.runtime1.BasicComponent +import org.apache.daffodil.runtime1.api.InfosetElement import org.apache.daffodil.runtime1.dpath.ExpressionEvaluationException import org.apache.daffodil.runtime1.dpath.NodeInfo import org.apache.daffodil.runtime1.dsom.ExpressionCompilerClass import org.apache.daffodil.runtime1.dsom.RelativePathPastRootError import org.apache.daffodil.runtime1.dsom.RuntimeSchemaDefinitionError -import org.apache.daffodil.runtime1.infoset.InfosetElement import org.apache.daffodil.runtime1.infoset.XMLTextInfosetOutputter import org.apache.daffodil.runtime1.infoset._ import org.apache.daffodil.runtime1.processors._ diff --git a/daffodil-runtime1/src/main/scala/org/apache/daffodil/runtime1/dpath/NodeInfo.scala b/daffodil-runtime1/src/main/scala/org/apache/daffodil/runtime1/dpath/NodeInfo.scala index fe669c21a1..3b2492a059 100644 --- a/daffodil-runtime1/src/main/scala/org/apache/daffodil/runtime1/dpath/NodeInfo.scala +++ b/daffodil-runtime1/src/main/scala/org/apache/daffodil/runtime1/dpath/NodeInfo.scala @@ -27,6 +27,7 @@ import java.math.{ BigDecimal => JBigDecimal } import java.math.{ BigInteger => JBigInt } import java.net.URI import java.net.URISyntaxException +import scala.collection.JavaConverters._ import org.apache.daffodil.lib.calendar.DFDLCalendar import org.apache.daffodil.lib.calendar.DFDLDateConversion @@ -44,6 +45,8 @@ import org.apache.daffodil.lib.xml.NoNamespace import org.apache.daffodil.lib.xml.QName import org.apache.daffodil.lib.xml.RefQName import org.apache.daffodil.lib.xml.XMLUtils +import org.apache.daffodil.runtime1.api +import org.apache.daffodil.runtime1.api.PrimitiveType import org.apache.daffodil.runtime1.dsom.walker._ import org.apache.daffodil.runtime1.infoset.DataValue.DataValueBigDecimal import org.apache.daffodil.runtime1.infoset.DataValue.DataValueBigInt @@ -122,7 +125,8 @@ sealed abstract class PrimTypeNode( parent: NodeInfo.Kind, childrenArg: => Seq[NodeInfo.Kind], ) extends TypeNode(sym, parent, childrenArg) - with NodeInfo.PrimType { + with NodeInfo.PrimType + with api.PrimitiveType { def this(sym: Symbol, parent: NodeInfo.Kind) = this(sym, parent, Seq(NodeInfo.Nothing)) } @@ -186,7 +190,7 @@ object NodeInfo extends Enum { /** * When class name is isomorphic to the type name, compute automatically. */ - override def name = { + override lazy val name: String = { val cname = super.name val first = cname(0).toLower val rest = cname.substring(1) @@ -206,6 +210,18 @@ object NodeInfo extends Enum { allTypes.find(stn => stn.lcaseName == namelc) } + /** + * For Java API use, we have a very restricted trait api.PrimitiveType + * mixed into PrimTypeNode, so that we can hand PrimTypeNode as result + * from methods callable from Java without exposing all of PrimType's + * implementation. + * @param name lookup key, case insensitive + * @return an api.PrimitiveType, or null if there is no type with that name. + */ + def primitiveTypeFromName(name: String): PrimitiveType = { + allDFDLTypesLookupTable.get(name) + } + def isXDerivedFromY(nameX: String, nameY: String): Boolean = { if (nameX == nameY) true else { @@ -973,7 +989,7 @@ object NodeInfo extends Enum { Opaque, AnyDateTime, ) - private lazy val allDFDLTypes = List( + lazy val allDFDLTypes = List( Float, Double, Decimal, @@ -996,6 +1012,9 @@ object NodeInfo extends Enum { DateTime, ) + private lazy val allDFDLTypesLookupTable: java.util.Map[String, PrimitiveType] = + allDFDLTypes.map { p => (p.name.toLowerCase, p.asInstanceOf[PrimitiveType]) }.toMap.asJava + lazy val allTypes = allDFDLTypes ++ List( Complex, diff --git a/daffodil-runtime1/src/main/scala/org/apache/daffodil/runtime1/dpath/UpDownMoves.scala b/daffodil-runtime1/src/main/scala/org/apache/daffodil/runtime1/dpath/UpDownMoves.scala index f1dd79bda5..5cb26a2789 100644 --- a/daffodil-runtime1/src/main/scala/org/apache/daffodil/runtime1/dpath/UpDownMoves.scala +++ b/daffodil-runtime1/src/main/scala/org/apache/daffodil/runtime1/dpath/UpDownMoves.scala @@ -53,8 +53,8 @@ case object UpMove extends RecipeOp { case object UpMoveArray extends RecipeOp { override def run(dstate: DState): Unit = { val now = dstate.currentElement - Assert.invariant(now.toParent.array.isDefined) - val n = now.toParent.array.get + Assert.invariant(now.toParent.maybeArray.isDefined) + val n = now.toParent.maybeArray.get dstate.setCurrentNode(n.asInstanceOf[DIArray]) } } @@ -102,7 +102,7 @@ case class DownArrayOccurrence(nqn: NamedQName, indexRecipe: CompiledDPath) savedCurrentElement.getChildArray(childArrayElementERD, dstate.tunable), ) val occurrence = - dstate.withRetryIfBlocking(arr.getOccurrence(index)) // will throw on out of bounds + dstate.withRetryIfBlocking(arr(index)) // will throw on out of bounds dstate.setCurrentNode(occurrence.asInstanceOf[DIElement]) } diff --git a/daffodil-runtime1/src/main/scala/org/apache/daffodil/runtime1/dsom/CompiledExpression1.scala b/daffodil-runtime1/src/main/scala/org/apache/daffodil/runtime1/dsom/CompiledExpression1.scala index 44c54a641a..b842733e65 100644 --- a/daffodil-runtime1/src/main/scala/org/apache/daffodil/runtime1/dsom/CompiledExpression1.scala +++ b/daffodil-runtime1/src/main/scala/org/apache/daffodil/runtime1/dsom/CompiledExpression1.scala @@ -350,7 +350,7 @@ class DPathElementCompileInfo( val optPrimType: Option[PrimType], sfl: SchemaFileLocation, override val unqualifiedPathStepPolicy: UnqualifiedPathStepPolicy, - val sscd: String, + val shortSchemaComponentDesignator: String, val isOutputValueCalc: Boolean, val isDistinguishedRoot: Boolean, ) extends DPathCompileInfo( diff --git a/daffodil-runtime1/src/main/scala/org/apache/daffodil/runtime1/infoset/DataValue.scala b/daffodil-runtime1/src/main/scala/org/apache/daffodil/runtime1/infoset/DataValue.scala index 8a6dbbfd1d..3871336878 100644 --- a/daffodil-runtime1/src/main/scala/org/apache/daffodil/runtime1/infoset/DataValue.scala +++ b/daffodil-runtime1/src/main/scala/org/apache/daffodil/runtime1/infoset/DataValue.scala @@ -119,8 +119,8 @@ object DataValue { /** All values which are legal for DPath and infoset data values. Note that this incudes * DINodes, which is legal for DPath, but not infoset data values. - * Also note that at any given time, the infoset may have no value, which is not directly - * representable by this type. + * Also note that at any given time, the infoset may have no value, which is not directly + * representable by this type. */ type DataValuePrimitive = DataValue[AnyRef, NonNullable with DataValuePrimitiveType] @@ -204,7 +204,7 @@ object DataValue { val NoValue: DataValueEmpty = new DataValue(null) /** Used as a sentinal value for Element's defaultValue, when said element - * is nillable and has dfdl:useNilForDefault set to true, + * is nillable and has dfdl:useNilForDefault set to true, */ val UseNilForDefault: DataValueUseNilForDefault = new DataValue(new UseNilForDefaultObj) diff --git a/daffodil-runtime1/src/main/scala/org/apache/daffodil/runtime1/infoset/Infoset.scala b/daffodil-runtime1/src/main/scala/org/apache/daffodil/runtime1/infoset/Infoset.scala deleted file mode 100644 index 85ff6ae6ad..0000000000 --- a/daffodil-runtime1/src/main/scala/org/apache/daffodil/runtime1/infoset/Infoset.scala +++ /dev/null @@ -1,102 +0,0 @@ -/* - * Licensed to the Apache Software Foundation (ASF) under one or more - * contributor license agreements. See the NOTICE file distributed with - * this work for additional information regarding copyright ownership. - * The ASF licenses this file to You under the Apache License, Version 2.0 - * (the "License"); you may not use this file except in compliance with - * the License. You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ - -package org.apache.daffodil.runtime1.infoset - -import org.apache.daffodil.lib.Implicits.ImplicitsSuppressUnusedImportWarning -import org.apache.daffodil.lib.api.DaffodilTunables -import org.apache.daffodil.lib.util.Maybe -import org.apache.daffodil.lib.util.MaybeBoolean -import org.apache.daffodil.lib.xml.NS -import org.apache.daffodil.runtime1.infoset.DataValue.DataValuePrimitiveNullable -import org.apache.daffodil.runtime1.processors.ElementRuntimeData - -object INoWarn2 { ImplicitsSuppressUnusedImportWarning() } - -trait InfosetArray { - def append(ie: InfosetElement): Unit - def getOccurrence(occursIndex: Long): InfosetElement - def length: Long -} - -trait InfosetElement extends InfosetItem { - - def parent: InfosetComplexElement - def setParent(p: InfosetComplexElement): Unit - - def array: Maybe[InfosetArray] - def setArray(a: InfosetArray): Unit - - def isNilled: Boolean - def setNilled(): Unit - - def isEmpty: Boolean - - def valid: MaybeBoolean - def setValid(validity: Boolean): Unit - - /** - * Retrieve the schema component that gave rise to this infoset - * item. - */ - def runtimeData: ElementRuntimeData - def namespace: NS - def name: String - def isHidden: Boolean - def setHidden(): Unit - -} - -trait InfosetComplexElement extends InfosetElement { - - def getChild(erd: ElementRuntimeData, tunable: DaffodilTunables): InfosetElement - def getChildArray(erd: ElementRuntimeData, tunable: DaffodilTunables): InfosetArray - - /** - * Determines slotInParent from the ERD of the infoset element arg. - * Hooks up the parent pointer of the new child to reference this. - * - * When slot contains an array, this appends to the end of the array. - */ - def addChild(e: InfosetElement, tunable: DaffodilTunables): Unit - -} - -trait InfosetSimpleElement extends InfosetElement { - - def dataValue: DataValuePrimitiveNullable - - /** - * Caches the string so we're not allocating strings just to do facet checks - */ - def dataValueAsString: String - def setDataValue(s: DataValuePrimitiveNullable): Unit - def isDefaulted: Boolean -} - -trait InfosetDocument extends InfosetItem {} - -trait InfosetItem { - - /** - * The totalElementCount is the total count of how many elements this InfosetItem contains. - * - * (Used to call this 'size', but size is often a length-like thing, so changed name - * to be more distinctive) - */ - def totalElementCount: Long -} diff --git a/daffodil-runtime1/src/main/scala/org/apache/daffodil/runtime1/infoset/InfosetImpl.scala b/daffodil-runtime1/src/main/scala/org/apache/daffodil/runtime1/infoset/InfosetImpl.scala index 486225df42..025acfc1ad 100644 --- a/daffodil-runtime1/src/main/scala/org/apache/daffodil/runtime1/infoset/InfosetImpl.scala +++ b/daffodil-runtime1/src/main/scala/org/apache/daffodil/runtime1/infoset/InfosetImpl.scala @@ -14,13 +14,20 @@ * See the License for the specific language governing permissions and * limitations under the License. */ - package org.apache.daffodil.runtime1.infoset import java.lang.{ Boolean => JBoolean } +import java.lang.{ Byte => JByte } +import java.lang.{ Double => JDouble } +import java.lang.{ Float => JFloat } +import java.lang.{ Integer => JInt } +import java.lang.{ Long => JLong } import java.lang.{ Number => JNumber } +import java.lang.{ Short => JShort } +import java.lang.{ String => JString } import java.math.{ BigDecimal => JBigDecimal } -import java.util.HashMap +import java.math.{ BigInteger => JBigInt } +import java.net.URI import java.util.concurrent.atomic.AtomicInteger import java.util.concurrent.atomic.AtomicLong import scala.collection.mutable.ArrayBuffer @@ -44,9 +51,17 @@ import org.apache.daffodil.lib.util.MaybeInt import org.apache.daffodil.lib.util.MaybeULong import org.apache.daffodil.lib.util.Misc import org.apache.daffodil.lib.util.Numbers -import org.apache.daffodil.lib.xml.NS import org.apache.daffodil.lib.xml.NamedQName import org.apache.daffodil.lib.xml.XMLUtils +import org.apache.daffodil.runtime1.api.ComplexElementMetadata +import org.apache.daffodil.runtime1.api.ElementMetadata +import org.apache.daffodil.runtime1.api.InfosetArray +import org.apache.daffodil.runtime1.api.InfosetComplexElement +import org.apache.daffodil.runtime1.api.InfosetDocument +import org.apache.daffodil.runtime1.api.InfosetElement +import org.apache.daffodil.runtime1.api.InfosetSimpleElement +import org.apache.daffodil.runtime1.api.InfosetTypeException +import org.apache.daffodil.runtime1.api.SimpleElementMetadata import org.apache.daffodil.runtime1.dpath.NodeInfo import org.apache.daffodil.runtime1.dsom.DPathCompileInfo import org.apache.daffodil.runtime1.dsom.DPathElementCompileInfo @@ -60,6 +75,7 @@ import org.apache.daffodil.runtime1.processors.SimpleTypeRuntimeData import org.apache.daffodil.runtime1.processors.TermRuntimeData import org.apache.daffodil.runtime1.processors.parsers.PState +import com.ibm.icu.util.Calendar import passera.unsigned.ULong sealed trait DINode { @@ -110,7 +126,6 @@ sealed trait DINode { def isHidden: Boolean def children: Stream[DINode] - def totalElementCount: Long def namedQName: NamedQName def erd: ElementRuntimeData @@ -413,28 +428,25 @@ case class InfosetMultipleScalarError(val erd: ElementRuntimeData) * they use for that purpose. */ final class FakeDINode extends DISimple(null) { + override def dataValue: DataValuePrimitiveNullable = _value + override def setDataValue(s: DataValuePrimitiveNullable): Unit = { _value = s } + override def dataValueAsString: String = _value.toString + // $COVERAGE-OFF$ private def die = throw new InfosetNoInfosetException(Nope) - override def parent = die override def diParent = die - override def setParent(p: InfosetComplexElement): Unit = die - + override def setParent(p: DIComplex): Unit = die override def isNilled: Boolean = die override def setNilled(): Unit = die - override def valid = die override def setValid(validity: Boolean): Unit = die - - override def dataValue: DataValuePrimitiveNullable = _value - override def setDataValue(s: DataValuePrimitiveNullable): Unit = { _value = s } - - override def dataValueAsString: String = _value.toString override def isDefaulted: Boolean = die - override def children = die - override def contentLength: ContentLengthState = die override def valueLength: ValueLengthState = die + override def primType: NodeInfo.PrimType = die + override def getNonNegativeInteger: JBigInt = die + // $COVERAGE-ON$ } /** @@ -999,8 +1011,8 @@ object DISimpleState { sealed trait DIElement extends DINode with DITerm - with InfosetElement - with DIElementSharedImplMixin { + with DIElementSharedImplMixin + with InfosetElement { final override protected def allocContentLength = new ContentLengthState(this) final override protected def allocValueLength = new ValueLengthState(this) @@ -1008,8 +1020,8 @@ sealed trait DIElement def isSimple: Boolean def isComplex: Boolean def isArray: Boolean - override final def name: String = erd.name - override final def namespace: NS = erd.targetNamespace + + final def name: String = erd.name override final def namedQName = erd.namedQName override final def trd = erd @@ -1050,7 +1062,7 @@ sealed trait DIElement def valueStringForDebug: String - def isRoot = toParent match { + def isRoot: Boolean = toParent match { case doc: DIDocument => !doc.isCompileExprFalseRoot case _ => false } @@ -1079,27 +1091,27 @@ sealed trait DIElement protected final var _isHidden = false final def isHidden: Boolean = _isHidden - override def setHidden(): Unit = { + def setHidden(): Unit = { _isHidden = true } final def runtimeData = erd - protected final var _parent: InfosetComplexElement = null + protected final var _parent: DIComplex = null protected final var _isNilledSet: Boolean = false - override def parent = _parent + def parent = _parent def diParent = _parent.asInstanceOf[DIComplex] - override def setParent(p: InfosetComplexElement): Unit = { + def setParent(p: DIComplex): Unit = { Assert.invariant(_parent eq null) _parent = p } - private var _array: Maybe[InfosetArray] = Nope - override def array = _array - override def setArray(a: InfosetArray) = { + private var _array: Maybe[DIArray] = Nope + def maybeArray: Maybe[DIArray] = _array + def setArray(a: DIArray): Unit = { _array = One(a) } @@ -1123,7 +1135,7 @@ sealed trait DIElement */ def isNilled: Boolean - override def setNilled(): Unit = { + def setNilled(): Unit = { Assert.invariant(erd.isNillable) Assert.invariant(!_isNilled) _isNilled = true @@ -1135,8 +1147,9 @@ sealed trait DIElement * valid = One(true) means valid * valid = One(false) means invalid */ - override def valid = _validity - override def setValid(validity: Boolean): Unit = { _validity = MaybeBoolean(validity) } + def valid: MaybeBoolean = _validity + def setValid(validity: Boolean): Unit = { _validity = MaybeBoolean(validity) } + } // This is not a mutable collection class on purpose. @@ -1157,6 +1170,8 @@ final class DIArray( ) extends DINode with InfosetArray { + override def metadata: ElementMetadata = erd + private lazy val nfe = new InfosetArrayNotFinalException(this) override def requireFinal(): Unit = { @@ -1207,7 +1222,7 @@ final class DIArray( protected final val _contents = new ArrayBuffer[DIElement](initialSize) - override def children = _contents.toStream.asInstanceOf[Stream[DINode]] + override def children: Stream[DINode] = _contents.toStream.asInstanceOf[Stream[DINode]] /** * Used to shorten array when backtracking out of having appended elements. @@ -1217,6 +1232,7 @@ final class DIArray( } override def contents: IndexedSeq[DINode] = _contents + def elementContents: IndexedSeq[DIElement] = _contents override def maybeLastChild: Maybe[DINode] = { val len = _contents.length @@ -1229,9 +1245,10 @@ final class DIArray( } /** + * Access an item of the array. * Note that occursIndex argument starts at position 1. */ - def getOccurrence(occursIndex1b: Long) = { + def apply(occursIndex1b: Long): DIElement = { if (occursIndex1b < 1) erd.toss( new InfosetFatalArrayIndexOutOfBoundsException(this, occursIndex1b, length), @@ -1243,29 +1260,23 @@ final class DIArray( _contents(occursIndex1b.toInt - 1) } - @inline final def apply(occursIndex1b: Long) = getOccurrence(occursIndex1b) - - def append(ie: InfosetElement): Unit = { + def append(ie: DIElement): Unit = { _contents += ie.asInstanceOf[DIElement] ie.setArray(this) } - def concat(array: DIArray) = { - val newContents = array.contents - newContents.foreach(ie => { - ie.asInstanceOf[InfosetElement].setArray(this) - append(ie.asInstanceOf[InfosetElement]) - }) + def concat(array: DIArray): Unit = { + val newContents = array.elementContents + newContents.foreach { ie => + { + ie.setArray(this) + append(ie) + } + } } final def length: Long = _contents.length - final def totalElementCount: Long = { - var a: Long = 0 - _contents.foreach { c => a += c.totalElementCount } - a - } - final def isDefaulted: Boolean = children.forall { _.isDefaulted } final def freeChildIfNoLongerNeeded(index: Int, doFree: Boolean): Unit = { @@ -1296,10 +1307,16 @@ sealed class DISimple(override val erd: ElementRuntimeData) with DISimpleSharedImplMixin with InfosetSimpleElement { + override def metadata: SimpleElementMetadata = erd + final override def isSimple = true + final override def isComplex = false + final override def isArray = false + def primType: NodeInfo.PrimType = erd.optPrimType.orNull + def contents: IndexedSeq[DINode] = IndexedSeq.empty private var _stringRep: String = null @@ -1315,6 +1332,7 @@ sealed class DISimple(override val erd: ElementRuntimeData) } def unionMemberRuntimeData = _unionMemberRuntimeData + def setUnionMemberRuntimeData(umrd: SimpleTypeRuntimeData): Unit = { _unionMemberRuntimeData = Maybe(umrd) this.setValid(true) @@ -1324,7 +1342,7 @@ sealed class DISimple(override val erd: ElementRuntimeData) * Parsing of a text number first does setDataValue to a string, then a conversion does overwrite data value * with a number. Unparsing does setDataValue to a value, then overwriteDataValue to a string. */ - override def setDataValue(x: DataValuePrimitiveNullable): Unit = { + def setDataValue(x: DataValuePrimitiveNullable): Unit = { Assert.invariant(!hasValue) overwriteDataValue(x) } @@ -1412,6 +1430,11 @@ sealed class DISimple(override val erd: ElementRuntimeData) _unionMemberRuntimeData = Nope } + /** + * @return true if the element is nilled, false otherwise. + * @throws InfosetNoDataException if neither data value nor setNull has happened yet + * so the nil status is undetermined. + */ override def isNilled: Boolean = { if (!erd.isNillable) false else if (_isNilledSet) { @@ -1451,7 +1474,7 @@ sealed class DISimple(override val erd: ElementRuntimeData) * Obtain the data value. Implements default * values, and outputValueCalc for unparsing. */ - override def dataValue: DataValuePrimitiveNullable = { + def dataValue: DataValuePrimitiveNullable = { if (_value.isEmpty) if (erd.optDefaultValue.isDefined) { val defaultVal = erd.optDefaultValue @@ -1486,7 +1509,7 @@ sealed class DISimple(override val erd: ElementRuntimeData) mv } - override def dataValueAsString = { + def dataValueAsString: JString = { if (_stringRep ne null) _stringRep else { dataValue.getAnyRef match { @@ -1528,7 +1551,7 @@ sealed class DISimple(override val erd: ElementRuntimeData) _isDefaulted } - final override def isEmpty: Boolean = { + final def isEmpty: Boolean = { if (isNilled) false else { val nodeKind = erd.optPrimType.getOrElse( @@ -1542,8 +1565,6 @@ sealed class DISimple(override val erd: ElementRuntimeData) } } - override def totalElementCount = 1L - /** * requireFinal is only ever used on unparse, and we never need to require a * simple type to be final during unparse. However, we do need to have an @@ -1558,6 +1579,72 @@ sealed class DISimple(override val erd: ElementRuntimeData) Assert.invariantFailed("Should not try to remove a child of a simple type") } + /** + * + * @return the value of this simple element as a Scala AnyRef, which is + * equivalent to a Java java.lang.Object. + */ + override def getAnyRef: AnyRef = this.primType match { + case NodeInfo.PrimType.Float => getFloat + case NodeInfo.PrimType.Double => getDouble + case NodeInfo.PrimType.Decimal => getDecimal + case NodeInfo.PrimType.Integer => getInteger + case NodeInfo.PrimType.Long => getLong + case NodeInfo.PrimType.Int => getInt + case NodeInfo.PrimType.Short => getShort + case NodeInfo.PrimType.Byte => getByte + case NodeInfo.PrimType.NonNegativeInteger => getNonNegativeInteger + case NodeInfo.PrimType.UnsignedLong => getUnsignedLong + case NodeInfo.PrimType.UnsignedInt => getUnsignedInt + case NodeInfo.PrimType.UnsignedShort => getUnsignedShort + case NodeInfo.PrimType.UnsignedByte => getUnsignedByte + case NodeInfo.PrimType.String => getString + case NodeInfo.PrimType.Boolean => getBoolean + case NodeInfo.PrimType.HexBinary => getHexBinary + case NodeInfo.PrimType.AnyURI => getURI + case NodeInfo.PrimType.Date => getDate + case NodeInfo.PrimType.Time => getTime + case NodeInfo.PrimType.DateTime => getDateTime + } + + override def getText: String = dataValueAsString + override def getDecimal: JBigDecimal = withTry(dataValue.getBigDecimal) + override def getDate: Calendar = withTry(dataValue.getDate.calendar) + override def getTime: Calendar = withTry(dataValue.getTime.calendar) + override def getDateTime: Calendar = withTry(dataValue.getDateTime.calendar) + override def getHexBinary: Array[Byte] = withTry(dataValue.getByteArray) + override def getBoolean: JBoolean = withTry(dataValue.getBoolean) + override def getLong: JLong = withTry(Converter.asLong(dataValue.getAnyRef)) + override def getInt: JInt = withTry(Converter.asInt(dataValue.getAnyRef)) + override def getShort: JShort = withTry(Converter.asShort(dataValue.getAnyRef)) + override def getByte: JByte = withTry(Converter.asByte(dataValue.getAnyRef)) + override def getUnsignedInt: JLong = withTry(Converter.asLong(dataValue.getAnyRef)) + override def getUnsignedShort: JInt = withTry(Converter.asInt(dataValue.getAnyRef)) + override def getUnsignedByte: JShort = withTry(Converter.asShort(dataValue.getAnyRef)) + override def getDouble: JDouble = withTry(dataValue.getDouble) + override def getFloat: JFloat = withTry(dataValue.getFloat) + override def getInteger: JBigInt = withTry(dataValue.getBigInt) + override def getNonNegativeInteger: JBigInt = withTry(dataValue.getBigInt) + override def getString: JString = withTry(dataValue.getString) + override def getURI: URI = withTry(dataValue.getURI) + override def getUnsignedLong: JBigInt = withTry(Converter.asBigInt(dataValue.getAnyRef)) + + private def withTry[A, B](f: => B): B = try { + f + } catch { + case ite: InfosetTypeException => throw ite + // + // Catch other exceptions like numbers out of range, + // and convert to InfosetTypeException + // + case e: Exception => + throw new InfosetTypeException(e) + } + + private object Converter extends Numbers { + override protected def errorThrower(message: JString): Nothing = + throw new InfosetTypeException(message) + } } /** @@ -1575,6 +1662,8 @@ sealed class DIComplex(override val erd: ElementRuntimeData) with DIComplexSharedImplMixin with InfosetComplexElement { diComplex => + override def metadata: ComplexElementMetadata = erd + final override def isSimple = false final override def isComplex = true final override def isArray = false @@ -1594,7 +1683,7 @@ sealed class DIComplex(override val erd: ElementRuntimeData) override def valueStringForDebug: String = "" - final override def isEmpty: Boolean = false + final def isEmpty: Boolean = false final override def isNilled: Boolean = { if (!erd.isNillable) false @@ -1613,9 +1702,9 @@ sealed class DIComplex(override val erd: ElementRuntimeData) } val childNodes = new ArrayBuffer[DINode] - // - // TODO: Cleanup - Change below to use NonAllocatingMap to improve code style. - lazy val nameToChildNodeLookup = new HashMap[NamedQName, ArrayBuffer[DINode]] + + private lazy val nameToChildNodeLookup = + new java.util.HashMap[NamedQName, ArrayBuffer[DINode]] override lazy val contents: IndexedSeq[DINode] = childNodes @@ -1631,11 +1720,11 @@ sealed class DIComplex(override val erd: ElementRuntimeData) } else Nope } - final def getChild(erd: ElementRuntimeData, tunable: DaffodilTunables): InfosetElement = { + final def getChild(erd: ElementRuntimeData, tunable: DaffodilTunables): DIElement = { getChild(erd.dpathElementCompileInfo.namedQName, tunable) } - private def noQuerySupportCheck(nodes: Seq[DINode], nqn: NamedQName) = { + private def noQuerySupportCheck(nodes: Seq[DINode], nqn: NamedQName): Unit = { if (nodes.length > 1) { // might be more than one result // but we have to rule out there being an empty DIArray @@ -1650,10 +1739,10 @@ sealed class DIComplex(override val erd: ElementRuntimeData) } } - final def getChild(nqn: NamedQName, tunable: DaffodilTunables): InfosetElement = { + final def getChild(nqn: NamedQName, tunable: DaffodilTunables): DIElement = { val maybeNode = findChild(nqn, tunable) if (maybeNode.isDefined) - maybeNode.get.asInstanceOf[InfosetElement] + maybeNode.get.asInstanceOf[DIElement] else erd.toss(new InfosetNoSuchChildElementException(this, nqn)) } @@ -1661,16 +1750,16 @@ sealed class DIComplex(override val erd: ElementRuntimeData) final def getChildArray( childERD: ElementRuntimeData, tunable: DaffodilTunables, - ): InfosetArray = { + ): DIArray = { Assert.usage(childERD.isArray) getChildArray(childERD.dpathElementCompileInfo.namedQName, tunable) } - final def getChildArray(nqn: NamedQName, tunable: DaffodilTunables): InfosetArray = { + final def getChildArray(nqn: NamedQName, tunable: DaffodilTunables): DIArray = { val maybeNode = findChild(nqn, tunable) if (maybeNode.isDefined) { - maybeNode.get.asInstanceOf[InfosetArray] + maybeNode.get.asInstanceOf[DIArray] } else { erd.toss(new InfosetNoSuchChildElementException(this, nqn)) } @@ -1749,11 +1838,17 @@ sealed class DIComplex(override val erd: ElementRuntimeData) childNodes ++= unordered.sortBy(_.erd.position) } - override def addChild(e: InfosetElement, tunable: DaffodilTunables): Unit = { + /** + * Determines slotInParent from the ERD of the infoset element arg. + * Hooks up the parent pointer of the new child to reference this. + * + * When slot contains an array, this appends to the end of the array. + */ + def addChild(e: DIElement, tunable: DaffodilTunables): Unit = { if (e.runtimeData.isArray) { val childERD = e.runtimeData val needsNewArray = - if (childNodes.length == 0) { + if (childNodes.isEmpty) { // This complex element has no children, so we must need to create a // new DIArray to add this array element true @@ -1809,13 +1904,24 @@ sealed class DIComplex(override val erd: ElementRuntimeData) } def findChild(qname: NamedQName, tunable: DaffodilTunables): Maybe[DINode] = { + findChild(qname, tunable.allowExternalPathExpressions) + } + + /** + * Find a child, using the preferred hash lookup, with an optional + * linear search through the children. + * @param qname + * @param enableLinearSearchIfNotFound + * @return + */ + def findChild(qname: NamedQName, enableLinearSearchIfNotFound: Boolean): Maybe[DINode] = { val fastSeq = nameToChildNodeLookup.get(qname) if (fastSeq != null) { // Daffodil does not support query expressions yet, so there should only // be one item in the list noQuerySupportCheck(fastSeq, qname) One(fastSeq(0)) - } else if (tunable.allowExternalPathExpressions) { + } else if (enableLinearSearchIfNotFound) { // Only DINodes used in expressions defined in the schema are added to // the nameToChildNodeLookup hashmap. If an expression defined outside of // the schema (like via the debugger) attempts to access an element that @@ -1881,13 +1987,6 @@ sealed class DIComplex(override val erd: ElementRuntimeData) } } - override def totalElementCount: Long = { - if (erd.isNillable && isNilled) return 1L - var a: Long = 1 - childNodes.foreach(node => a += node.totalElementCount) - a - } - } /* @@ -1906,7 +2005,7 @@ final class DIDocument(erd: ElementRuntimeData) extends DIComplex(erd) with Info object Infoset { - def newElement(erd: ElementRuntimeData): InfosetElement = { + def newElement(erd: ElementRuntimeData): DIElement = { if (erd.isSimpleType) new DISimple(erd) else new DIComplex(erd) } @@ -1926,7 +2025,7 @@ object Infoset { def newDetachedElement( state: ParseOrUnparseState, erd: ElementRuntimeData, - ): InfosetElement = { + ): DIElement = { val detachedDoc = Infoset.newDocument(erd).asInstanceOf[DIDocument] val detachedElem = Infoset.newElement(erd) detachedDoc.addChild(detachedElem, state.tunable) diff --git a/daffodil-runtime1/src/main/scala/org/apache/daffodil/runtime1/infoset/InfosetOutputter.scala b/daffodil-runtime1/src/main/scala/org/apache/daffodil/runtime1/infoset/InfosetOutputter.scala index 6222a010c9..736a52e28d 100644 --- a/daffodil-runtime1/src/main/scala/org/apache/daffodil/runtime1/infoset/InfosetOutputter.scala +++ b/daffodil-runtime1/src/main/scala/org/apache/daffodil/runtime1/infoset/InfosetOutputter.scala @@ -14,12 +14,15 @@ * See the License for the specific language governing permissions and * limitations under the License. */ - package org.apache.daffodil.runtime1.infoset import java.nio.file.Path import java.nio.file.Paths +import org.apache.daffodil.runtime1.api.InfosetArray +import org.apache.daffodil.runtime1.api.InfosetComplexElement +import org.apache.daffodil.runtime1.api.InfosetSimpleElement + /** * Defines the interface for InfosetOutputters. * @@ -34,15 +37,11 @@ import java.nio.file.Paths * by implementations. This does mean some exceptions that you might normally * expect to bubble up and will not, and will instead be turned into an SDE. */ -trait InfosetOutputter { - - import Status._ - - def status: Status = READY +trait InfosetOutputter extends BlobMethodsMixin { /** * Reset the internal state of this InfosetOutputter. This should be called - * inbetween calls to the parse method. + * in between calls to the parse method. */ def reset(): Unit @@ -72,7 +71,7 @@ trait InfosetOutputter { * value, nil, name, namespace, etc. */ @throws[Exception] - def startSimple(diSimple: DISimple): Unit + def startSimple(diSimple: InfosetSimpleElement): Unit /** * Called by Daffodil internals to signify the end of a simple element. @@ -84,75 +83,65 @@ trait InfosetOutputter { * value, nil, name, namespace, etc. */ @throws[Exception] - def endSimple(diSimple: DISimple): Unit + def endSimple(diSimple: InfosetSimpleElement): Unit /** * Called by Daffodil internals to signify the beginning of a complex element. * * Throws java.lang.Exception if there was an error and Daffodil should stop parsing * - * @param diComplex the complex element that is started. Various fields of + * @param complex the complex element that is started. Various fields of * DIComplex can be accessed to determine things like the * nil, name, namespace, etc. */ @throws[Exception] - def startComplex(diComplex: DIComplex): Unit + def startComplex(complex: InfosetComplexElement): Unit /** * Called by Daffodil internals to signify the end of a complex element. * * Throws java.lang.Exception if there was an error and Daffodil should stop parsing * - * @param diComplex the complex element that is ended. Various fields of + * @param complex the complex element that is ended. Various fields of * DIComplex can be accessed to determine things like the * nil, name, namespace, etc. */ @throws[Exception] - def endComplex(diComplex: DIComplex): Unit + def endComplex(complex: InfosetComplexElement): Unit /** * Called by Daffodil internals to signify the beginning of an array of elements. * * Throws java.lang.Exception if there was an error and Daffodil should stop parsing * - * @param diComplex the array that is started. Various fields of + * @param array the array that is started. Various fields of * DIArray can be accessed to determine things like the * name, namespace, etc. */ @throws[Exception] - def startArray(diArray: DIArray): Unit + def startArray(array: InfosetArray): Unit /** * Called by Daffodil internals to signify the end of an array of elements. * * Throws java.lang.Exception if there was an error and Daffodil should stop parsing * - * @param diComplex the array that is ended. Various fields of + * @param array the array that is ended. Various fields of * DIArray can be accessed to determine things like the * name, namespace, etc. */ @throws[Exception] - def endArray(diArray: DIArray): Unit - - def getStatus(): Status = { - // Done, Ready (Not started), Visiting (part way done - can retry to visit more)... - status - } + def endArray(array: InfosetArray): Unit +} - /** - * Helper function to determine if an element is nilled or not, taking into - * account whether or not the nilled state has been set yet. - * - * @param diElement the element to check the nilled state of - * - * @return true if the nilled state has been set and is true. false if the - * nilled state is false or if the nilled state has not been set yet - * (e.g. during debugging) - */ - final def isNilled(diElement: DIElement): Boolean = { - val maybeIsNilled = diElement.maybeIsNilled - maybeIsNilled.isDefined && maybeIsNilled.get == true - } +/** + * An available basic implementation of the BLOB methods. + * Stores blobs in files in directory identified by Java system property + * `java.io.tempdir`. + * + * FIXME: Scaladoc + */ +trait BlobMethodsMixin { /** * Set the attributes for how to create blob files. @@ -175,7 +164,6 @@ trait InfosetOutputter { * This is the same as what would be found by iterating over the infoset. */ final def getBlobPaths(): Seq[Path] = blobPaths - final def getBlobDirectory(): Path = blobDirectory final def getBlobPrefix(): String = blobPrefix final def getBlobSuffix(): String = blobSuffix @@ -185,8 +173,3 @@ trait InfosetOutputter { private var blobSuffix: String = ".blob" private var blobPaths: Seq[Path] = Seq.empty } - -object Status extends Enumeration { - type Status = Value - val DONE, READY, VISITING = Value -} diff --git a/daffodil-runtime1/src/main/scala/org/apache/daffodil/runtime1/infoset/InfosetWalker.scala b/daffodil-runtime1/src/main/scala/org/apache/daffodil/runtime1/infoset/InfosetWalker.scala index 4b9a845881..c091327b73 100644 --- a/daffodil-runtime1/src/main/scala/org/apache/daffodil/runtime1/infoset/InfosetWalker.scala +++ b/daffodil-runtime1/src/main/scala/org/apache/daffodil/runtime1/infoset/InfosetWalker.scala @@ -93,7 +93,7 @@ object InfosetWalker { // container of the root node to start at and finds the index in that // container val container: DINode = - if (root.array.isDefined) root.array.get.asInstanceOf[DINode] + if (root.maybeArray.isDefined) root.maybeArray.get.asInstanceOf[DINode] else root.parent.asInstanceOf[DINode] (container, container.contents.indexOf(root)) } diff --git a/daffodil-runtime1/src/main/scala/org/apache/daffodil/runtime1/infoset/JDOMInfosetOutputter.scala b/daffodil-runtime1/src/main/scala/org/apache/daffodil/runtime1/infoset/JDOMInfosetOutputter.scala index 47c66c8a81..9e4664cda2 100644 --- a/daffodil-runtime1/src/main/scala/org/apache/daffodil/runtime1/infoset/JDOMInfosetOutputter.scala +++ b/daffodil-runtime1/src/main/scala/org/apache/daffodil/runtime1/infoset/JDOMInfosetOutputter.scala @@ -21,9 +21,13 @@ import org.apache.daffodil.lib.exceptions.Assert import org.apache.daffodil.lib.util.MStackOf import org.apache.daffodil.lib.util.Maybe import org.apache.daffodil.lib.xml.XMLUtils -import org.apache.daffodil.runtime1.dpath.NodeInfo +import org.apache.daffodil.runtime1.api.InfosetArray +import org.apache.daffodil.runtime1.api.InfosetComplexElement +import org.apache.daffodil.runtime1.api.InfosetElement +import org.apache.daffodil.runtime1.api.InfosetSimpleElement +import org.apache.daffodil.runtime1.api.PrimitiveType -class JDOMInfosetOutputter extends InfosetOutputter with XMLInfosetOutputter { +class JDOMInfosetOutputter extends InfosetOutputter { private val stack = new MStackOf[org.jdom2.Parent] private var result: Maybe[org.jdom2.Document] = Maybe.Nope @@ -46,16 +50,16 @@ class JDOMInfosetOutputter extends InfosetOutputter with XMLInfosetOutputter { result = Maybe(root.asInstanceOf[org.jdom2.Document]) } - def startSimple(diSimple: DISimple): Unit = { + override def startSimple(simple: InfosetSimpleElement): Unit = { - val elem = createElement(diSimple) + val elem = createElement(simple) - if (diSimple.hasValue) { + if (!simple.isNilled) { val text = - if (diSimple.erd.optPrimType.get.isInstanceOf[NodeInfo.String.Kind]) { - remapped(diSimple.dataValueAsString) + if (simple.metadata.primitiveType == PrimitiveType.String) { + XMLUtils.remapXMLIllegalCharactersToPUA(simple.getText) } else { - diSimple.dataValueAsString + simple.getText } elem.addContent(text) } @@ -63,22 +67,22 @@ class JDOMInfosetOutputter extends InfosetOutputter with XMLInfosetOutputter { stack.top.addContent(elem) } - def endSimple(diSimple: DISimple): Unit = {} + override def endSimple(se: InfosetSimpleElement): Unit = {} - def startComplex(diComplex: DIComplex): Unit = { + override def startComplex(ce: InfosetComplexElement): Unit = { - val elem = createElement(diComplex) + val elem = createElement(ce) stack.top.addContent(elem) stack.push(elem) } - def endComplex(diComplex: DIComplex): Unit = { + override def endComplex(ce: InfosetComplexElement): Unit = { stack.pop } - def startArray(diArray: DIArray): Unit = {} - def endArray(diArray: DIArray): Unit = {} + override def startArray(ar: InfosetArray): Unit = {} + override def endArray(ar: InfosetArray): Unit = {} def getResult(): org.jdom2.Document = { Assert.usage( @@ -88,18 +92,18 @@ class JDOMInfosetOutputter extends InfosetOutputter with XMLInfosetOutputter { result.get } - private def createElement(diElement: DIElement): org.jdom2.Element = { + private def createElement(element: InfosetElement): org.jdom2.Element = { val elem: org.jdom2.Element = - if (diElement.erd.namedQName.namespace.isNoNamespace) - new org.jdom2.Element(diElement.erd.name) + if (element.metadata.namespace eq null) + new org.jdom2.Element(element.metadata.name) else new org.jdom2.Element( - diElement.erd.name, - diElement.erd.prefix, - diElement.erd.namedQName.namespace, + element.metadata.name, + element.metadata.prefix, + element.metadata.namespace, ) - if (isNilled(diElement)) { + if (element.isNilled) { elem.setAttribute("nil", "true", xsiNS) } elem diff --git a/daffodil-runtime1/src/main/scala/org/apache/daffodil/runtime1/infoset/JsonInfosetOutputter.scala b/daffodil-runtime1/src/main/scala/org/apache/daffodil/runtime1/infoset/JsonInfosetOutputter.scala index 56a6f25c31..07560d5d01 100644 --- a/daffodil-runtime1/src/main/scala/org/apache/daffodil/runtime1/infoset/JsonInfosetOutputter.scala +++ b/daffodil-runtime1/src/main/scala/org/apache/daffodil/runtime1/infoset/JsonInfosetOutputter.scala @@ -21,7 +21,11 @@ import java.nio.charset.StandardCharsets import org.apache.daffodil.lib.util.Indentable import org.apache.daffodil.lib.util.MStackOfBoolean -import org.apache.daffodil.runtime1.dpath.NodeInfo +import org.apache.daffodil.runtime1.api.InfosetArray +import org.apache.daffodil.runtime1.api.InfosetComplexElement +import org.apache.daffodil.runtime1.api.InfosetElement +import org.apache.daffodil.runtime1.api.InfosetSimpleElement +import org.apache.daffodil.runtime1.api.PrimitiveType import com.fasterxml.jackson.core.io.JsonStringEncoder @@ -52,7 +56,7 @@ class JsonInfosetOutputter private (writer: java.io.Writer, pretty: Boolean) // starting a newline, and adding indenting for whatever ends up coming after // it private def startNode(): Unit = { - if (isFirstChildStack.top == true) { + if (isFirstChildStack.top) { // the first child does not need a comma before it, but all following children will isFirstChildStack.pop() isFirstChildStack.push(false) @@ -64,12 +68,12 @@ class JsonInfosetOutputter private (writer: java.io.Writer, pretty: Boolean) } // handles logic for printing the name of a simple or complex element - private def startElement(element: DIElement): Unit = { - if (!element.erd.isArray) { + private def startElement(element: InfosetElement): Unit = { + if (!element.metadata.isArray) { // Only write the name if this is not an array of simple/complex types. // If it is an array, the name is written in startArray writer.write('"') - writer.write(element.erd.name) + writer.write(element.metadata.name) writer.write("\": ") } } @@ -92,17 +96,17 @@ class JsonInfosetOutputter private (writer: java.io.Writer, pretty: Boolean) if (pretty) outputIndentation(writer) } - override def startSimple(simple: DISimple): Unit = { + override def startSimple(simple: InfosetSimpleElement): Unit = { startNode() startElement(simple) - if (!isNilled(simple) && simple.hasValue) { + if (!simple.isNilled) { val text = - if (simple.erd.optPrimType.get.isInstanceOf[NodeInfo.String.Kind]) { + if (simple.metadata.primitiveType == PrimitiveType.String) { new String( - stringEncoder.quoteAsString(simple.dataValueAsString), + stringEncoder.quoteAsString(simple.getText), ) // escapes according to Json spec } else { - simple.dataValueAsString + simple.getText } writer.write('"') writer.write(text) @@ -112,14 +116,14 @@ class JsonInfosetOutputter private (writer: java.io.Writer, pretty: Boolean) } } - override def endSimple(simple: DISimple): Unit = { + override def endSimple(se: InfosetSimpleElement): Unit = { // nothing to do } - override def startComplex(complex: DIComplex): Unit = { + override def startComplex(complex: InfosetComplexElement): Unit = { startNode() startElement(complex) - if (!isNilled(complex)) { + if (!complex.isNilled) { writer.write('{') prepareForChildren() } else { @@ -127,8 +131,8 @@ class JsonInfosetOutputter private (writer: java.io.Writer, pretty: Boolean) } } - override def endComplex(complex: DIComplex): Unit = { - if (!isNilled(complex)) { + override def endComplex(complex: InfosetComplexElement): Unit = { + if (!complex.isNilled) { endNodeWithChildren() writer.write('}') } else { @@ -136,15 +140,15 @@ class JsonInfosetOutputter private (writer: java.io.Writer, pretty: Boolean) } } - override def startArray(array: DIArray): Unit = { + override def startArray(array: InfosetArray): Unit = { startNode() writer.write('"') - writer.write(array.erd.name) + writer.write(array.metadata.name) writer.write("\": [") prepareForChildren() } - override def endArray(array: DIArray): Unit = { + override def endArray(array: InfosetArray): Unit = { endNodeWithChildren() writer.write(']') } diff --git a/daffodil-runtime1/src/main/scala/org/apache/daffodil/runtime1/infoset/NullInfosetOutputter.scala b/daffodil-runtime1/src/main/scala/org/apache/daffodil/runtime1/infoset/NullInfosetOutputter.scala index 7b5c4984f9..883e32c25d 100644 --- a/daffodil-runtime1/src/main/scala/org/apache/daffodil/runtime1/infoset/NullInfosetOutputter.scala +++ b/daffodil-runtime1/src/main/scala/org/apache/daffodil/runtime1/infoset/NullInfosetOutputter.scala @@ -17,6 +17,10 @@ package org.apache.daffodil.runtime1.infoset +import org.apache.daffodil.runtime1.api.InfosetArray +import org.apache.daffodil.runtime1.api.InfosetComplexElement +import org.apache.daffodil.runtime1.api.InfosetSimpleElement + /** * Ignores all infoset events, outputting nothing */ @@ -24,14 +28,14 @@ class NullInfosetOutputter() extends InfosetOutputter { override def reset(): Unit = {} - override def startSimple(simple: DISimple): Unit = {} - override def endSimple(simple: DISimple): Unit = {} + override def startSimple(simple: InfosetSimpleElement): Unit = {} + override def endSimple(simple: InfosetSimpleElement): Unit = {} - override def startComplex(complex: DIComplex): Unit = {} - override def endComplex(complex: DIComplex): Unit = {} + override def startComplex(complex: InfosetComplexElement): Unit = {} + override def endComplex(complex: InfosetComplexElement): Unit = {} - override def startArray(array: DIArray): Unit = {} - override def endArray(array: DIArray): Unit = {} + override def startArray(array: InfosetArray): Unit = {} + override def endArray(array: InfosetArray): Unit = {} override def startDocument(): Unit = {} override def endDocument(): Unit = {} diff --git a/daffodil-runtime1/src/main/scala/org/apache/daffodil/runtime1/infoset/SAXInfosetOutputter.scala b/daffodil-runtime1/src/main/scala/org/apache/daffodil/runtime1/infoset/SAXInfosetOutputter.scala index cb2cbcf07d..f33886e9dd 100644 --- a/daffodil-runtime1/src/main/scala/org/apache/daffodil/runtime1/infoset/SAXInfosetOutputter.scala +++ b/daffodil-runtime1/src/main/scala/org/apache/daffodil/runtime1/infoset/SAXInfosetOutputter.scala @@ -21,7 +21,11 @@ import scala.xml.NamespaceBinding import org.apache.daffodil.lib.xml.XMLUtils import org.apache.daffodil.runtime1.api.DFDL -import org.apache.daffodil.runtime1.dpath.NodeInfo +import org.apache.daffodil.runtime1.api.InfosetArray +import org.apache.daffodil.runtime1.api.InfosetComplexElement +import org.apache.daffodil.runtime1.api.InfosetElement +import org.apache.daffodil.runtime1.api.InfosetSimpleElement +import org.apache.daffodil.runtime1.api.PrimitiveType import org.xml.sax.ContentHandler import org.xml.sax.helpers.AttributesImpl @@ -30,8 +34,7 @@ class SAXInfosetOutputter( xmlReader: DFDL.DaffodilParseXMLReader, val namespacesFeature: Boolean, val namespacePrefixesFeature: Boolean, -) extends InfosetOutputter - with XMLInfosetOutputter { +) extends InfosetOutputter { /** * Reset the internal state of this InfosetOutputter. This should be called @@ -58,16 +61,16 @@ class SAXInfosetOutputter( } } - override def startSimple(diSimple: DISimple): Unit = { + override def startSimple(simple: InfosetSimpleElement): Unit = { val contentHandler = xmlReader.getContentHandler if (contentHandler != null) { - doStartElement(diSimple, contentHandler) - if (diSimple.hasValue) { + doStartElement(simple, contentHandler) + if (!simple.isNilled) { val text = - if (diSimple.erd.optPrimType.get.isInstanceOf[NodeInfo.String.Kind]) { - remapped(diSimple.dataValueAsString) + if (simple.metadata.primitiveType == PrimitiveType.String) { + XMLUtils.remapXMLIllegalCharactersToPUA(simple.getText) } else { - diSimple.dataValueAsString + simple.getText } val arr = text.toCharArray contentHandler.characters(arr, 0, arr.length) @@ -75,33 +78,36 @@ class SAXInfosetOutputter( } } - override def endSimple(diSimple: DISimple): Unit = { + override def endSimple(simple: InfosetSimpleElement): Unit = { val contentHandler = xmlReader.getContentHandler if (contentHandler != null) { - doEndElement(diSimple, contentHandler) + doEndElement(simple, contentHandler) } } - override def startComplex(diComplex: DIComplex): Unit = { + override def startComplex(complex: InfosetComplexElement): Unit = { val contentHandler = xmlReader.getContentHandler if (contentHandler != null) { - doStartElement(diComplex, contentHandler) + doStartElement(complex, contentHandler) } } - override def endComplex(diComplex: DIComplex): Unit = { + override def endComplex(complex: InfosetComplexElement): Unit = { val contentHandler = xmlReader.getContentHandler if (contentHandler != null) { - doEndElement(diComplex, contentHandler) + doEndElement(complex, contentHandler) } } - override def startArray(diArray: DIArray): Unit = {} // not applicable + override def startArray(ar: InfosetArray): Unit = {} // not applicable - override def endArray(diArray: DIArray): Unit = {} // not applicable + override def endArray(ar: InfosetArray): Unit = {} // not applicable - private def doStartPrefixMapping(diElem: DIElement, contentHandler: ContentHandler): Unit = { - val (nsbStart: NamespaceBinding, nsbEnd: NamespaceBinding) = getNsbStartAndEnd(diElem) + private def doStartPrefixMapping( + elem: InfosetElement, + contentHandler: ContentHandler, + ): Unit = { + val (nsbStart: NamespaceBinding, nsbEnd: NamespaceBinding) = getNsbStartAndEnd(elem) var n = nsbStart while (n.ne(nsbEnd) && n.ne(null) && n.ne(scala.xml.TopScope)) { val prefix = if (n.prefix == null) "" else n.prefix @@ -111,8 +117,8 @@ class SAXInfosetOutputter( } } - private def doEndPrefixMapping(diElem: DIElement, contentHandler: ContentHandler): Unit = { - val (nsbStart: NamespaceBinding, nsbEnd: NamespaceBinding) = getNsbStartAndEnd(diElem) + private def doEndPrefixMapping(elem: InfosetElement, contentHandler: ContentHandler): Unit = { + val (nsbStart: NamespaceBinding, nsbEnd: NamespaceBinding) = getNsbStartAndEnd(elem) var n = nsbStart while (n.ne(nsbEnd) && n.ne(null) && n.ne(scala.xml.TopScope)) { val prefix = if (n.prefix == null) "" else n.prefix @@ -126,10 +132,10 @@ class SAXInfosetOutputter( * when namespacePrefixes feature is true */ private def doAttributesPrefixMapping( - diElem: DIElement, + elem: InfosetElement, attrs: AttributesImpl, ): AttributesImpl = { - val (nsbStart: NamespaceBinding, nsbEnd: NamespaceBinding) = getNsbStartAndEnd(diElem) + val (nsbStart: NamespaceBinding, nsbEnd: NamespaceBinding) = getNsbStartAndEnd(elem) var n = nsbStart while (n.ne(nsbEnd) && n.ne(null) && n.ne(scala.xml.TopScope)) { val prefix = if (n.prefix == null) "xmlns" else s"xmlns:${n.prefix}" @@ -181,7 +187,8 @@ class SAXInfosetOutputter( } } - private def getNsbStartAndEnd(diElem: DIElement) = { + private def getNsbStartAndEnd(elem: InfosetElement): (NamespaceBinding, NamespaceBinding) = { + val diElem = elem.asInstanceOf[DIElement] val nsbStart = diElem.erd.minimizedScope val nsbEnd = if (diElem.isRoot) { scala.xml.TopScope @@ -208,8 +215,8 @@ class SAXInfosetOutputter( } } - private def doStartElement(diElem: DIElement, contentHandler: ContentHandler): Unit = { - val (ns: String, localName: String, qName: String) = getNamespaceLocalNameAndQName(diElem) + private def doStartElement(elem: InfosetElement, contentHandler: ContentHandler): Unit = { + val (ns: String, localName: String, qName: String) = getNamespaceLocalNameAndQName(elem) val attrs = new AttributesImpl() val elemUri: String = if (namespacesFeature) ns else "" val elemLocalName: String = if (namespacesFeature) localName else "" @@ -217,16 +224,16 @@ class SAXInfosetOutputter( if (namespacesFeature) { // only when this feature is true do we use prefix mappings - doStartPrefixMapping(diElem, contentHandler) + doStartPrefixMapping(elem, contentHandler) } if (namespacePrefixesFeature) { // handle prefix attribute - doAttributesPrefixMapping(diElem, attrs) + doAttributesPrefixMapping(elem, attrs) } // handle xsi:nil attribute - if (isNilled(diElem)) { + if (elem.isNilled) { val isNilled = "true" val nType: String = "CDATA" val nValue: String = isNilled @@ -240,8 +247,8 @@ class SAXInfosetOutputter( contentHandler.startElement(elemUri, elemLocalName, elemQname, attrs) } - private def doEndElement(diElem: DIElement, contentHandler: ContentHandler): Unit = { - val (ns: String, localName: String, qName: String) = getNamespaceLocalNameAndQName(diElem) + private def doEndElement(elem: InfosetElement, contentHandler: ContentHandler): Unit = { + val (ns: String, localName: String, qName: String) = getNamespaceLocalNameAndQName(elem) val elemUri: String = if (namespacesFeature) ns else "" val elemLocalName = if (namespacesFeature) localName else "" val elemQname = if (namespacePrefixesFeature) qName else "" @@ -249,18 +256,18 @@ class SAXInfosetOutputter( contentHandler.endElement(elemUri, elemLocalName, elemQname) // only when this feature is true do we use prefix mappings - if (namespacesFeature) doEndPrefixMapping(diElem, contentHandler) + if (namespacesFeature) doEndPrefixMapping(elem, contentHandler) } - private def getNamespaceLocalNameAndQName(diElem: DIElement): (String, String, String) = { + private def getNamespaceLocalNameAndQName(elem: InfosetElement): (String, String, String) = { val ns: String = - if (diElem.erd.namedQName.namespace.isNoNamespace) { + if (elem.metadata.namespace eq null) { "" } else { - diElem.erd.namedQName.namespace.toString + elem.metadata.namespace } - val elemName = diElem.erd.namedQName.local - val qName = diElem.erd.prefixedName + val elemName = elem.metadata.name + val qName = elem.asInstanceOf[DIElement].erd.prefixedName (ns, elemName, qName) } diff --git a/daffodil-runtime1/src/main/scala/org/apache/daffodil/runtime1/infoset/ScalaXMLInfosetOutputter.scala b/daffodil-runtime1/src/main/scala/org/apache/daffodil/runtime1/infoset/ScalaXMLInfosetOutputter.scala index d0baa81ef5..65157fdc5d 100644 --- a/daffodil-runtime1/src/main/scala/org/apache/daffodil/runtime1/infoset/ScalaXMLInfosetOutputter.scala +++ b/daffodil-runtime1/src/main/scala/org/apache/daffodil/runtime1/infoset/ScalaXMLInfosetOutputter.scala @@ -26,11 +26,13 @@ import org.apache.daffodil.lib.exceptions.Assert import org.apache.daffodil.lib.util.MStackOf import org.apache.daffodil.lib.util.Maybe import org.apache.daffodil.lib.xml.XMLUtils -import org.apache.daffodil.runtime1.dpath.NodeInfo +import org.apache.daffodil.runtime1.api.InfosetArray +import org.apache.daffodil.runtime1.api.InfosetComplexElement +import org.apache.daffodil.runtime1.api.InfosetElement +import org.apache.daffodil.runtime1.api.InfosetSimpleElement +import org.apache.daffodil.runtime1.api.PrimitiveType -class ScalaXMLInfosetOutputter(showFormatInfo: Boolean = false, showFreedInfo: Boolean = false) - extends InfosetOutputter - with XMLInfosetOutputter { +class ScalaXMLInfosetOutputter(showFreedInfo: Boolean = false) extends InfosetOutputter { protected val stack = new MStackOf[ListBuffer[scala.xml.Node]] private var resultNode: Maybe[scala.xml.Node] = Maybe.Nope @@ -52,13 +54,18 @@ class ScalaXMLInfosetOutputter(showFormatInfo: Boolean = false, showFreedInfo: B } private def getAttributes(diElem: DIElement): MetaData = { - val nilAttr = if (isNilled(diElem)) XMLUtils.xmlNilAttribute else Null + val nilAttr = if (diElem.isNilled) XMLUtils.xmlNilAttribute else Null val freedAttr = if (showFreedInfo) { val selfFreed = diElem.wouldHaveBeenFreed val arrayFreed = if (diElem.erd.isArray) - diElem.diParent.children.find { _.erd eq diElem.erd }.get.wouldHaveBeenFreed + diElem.diParent.children + .find { + _.erd eq diElem.erd + } + .get + .wouldHaveBeenFreed else false if (selfFreed || arrayFreed) { val freedAttrVal = @@ -75,15 +82,15 @@ class ScalaXMLInfosetOutputter(showFormatInfo: Boolean = false, showFreedInfo: B freedAttr } - def startSimple(diSimple: DISimple): Unit = { - + override def startSimple(se: InfosetSimpleElement): Unit = { + val diSimple = se.asInstanceOf[DISimple] val attributes = getAttributes(diSimple) val children = if (!isNilled(diSimple) && diSimple.hasValue) { val text = - if (diSimple.erd.optPrimType.get.isInstanceOf[NodeInfo.String.Kind]) { - remapped(diSimple.dataValueAsString) + if (diSimple.metadata.primitiveType == PrimitiveType.String) { + XMLUtils.remapXMLIllegalCharactersToPUA(diSimple.dataValueAsString) } else { diSimple.dataValueAsString } @@ -94,47 +101,46 @@ class ScalaXMLInfosetOutputter(showFormatInfo: Boolean = false, showFreedInfo: B val elem = scala.xml.Elem( - diSimple.erd.prefix, - diSimple.erd.name, + diSimple.metadata.prefix, + diSimple.metadata.name, attributes, - diSimple.erd.minimizedScope, + diSimple.metadata.minimizedScope, minimizeEmpty = true, children: _*, ) - val elemWithFmt = addFmtInfo(diSimple, elem, showFormatInfo) - stack.top.append(elemWithFmt) + stack.top.append(elem) } - def endSimple(diSimple: DISimple): Unit = {} + override def endSimple(se: InfosetSimpleElement): Unit = {} - def startComplex(diComplex: DIComplex): Unit = { + override def startComplex(ce: InfosetComplexElement): Unit = { stack.push(new ListBuffer()) } - def endComplex(diComplex: DIComplex): Unit = { + override def endComplex(ce: InfosetComplexElement): Unit = { + val diComplex = ce.asInstanceOf[DIComplex] val attributes = getAttributes(diComplex) val children = stack.pop val elem = scala.xml.Elem( - diComplex.erd.prefix, - diComplex.erd.name, + diComplex.metadata.prefix, + diComplex.metadata.name, attributes, - diComplex.erd.minimizedScope, + diComplex.metadata.minimizedScope, minimizeEmpty = true, children: _*, ) - val elemWithFmt = addFmtInfo(diComplex, elem, showFormatInfo) - stack.top.append(elemWithFmt) + stack.top.append(elem) } - def startArray(diArray: DIArray): Unit = { + override def startArray(ar: InfosetArray): Unit = { // Array elements are started individually } - def endArray(diArray: DIArray): Unit = {} + def endArray(ar: InfosetArray): Unit = {} def getResult(): scala.xml.Node = { Assert.usage( @@ -143,4 +149,19 @@ class ScalaXMLInfosetOutputter(showFormatInfo: Boolean = false, showFreedInfo: B ) resultNode.get } + + /** + * Helper function to determine if an element is nilled or not, taking into + * account whether or not the nilled state has been set yet. + * + * @param elem the element to check the nilled state of + * @return true if the nilled state has been set and is true. false if the + * nilled state is false or if the nilled state has not been set yet + * (e.g. during debugging) + */ + private def isNilled(elem: InfosetElement): Boolean = { + val diElement = elem.asInstanceOf[DIElement] + val maybeIsNilled = diElement.maybeIsNilled + maybeIsNilled.isDefined && maybeIsNilled.get == true + } } diff --git a/daffodil-runtime1/src/main/scala/org/apache/daffodil/runtime1/infoset/TeeInfosetOutputter.scala b/daffodil-runtime1/src/main/scala/org/apache/daffodil/runtime1/infoset/TeeInfosetOutputter.scala index d78d1ef791..e2bfe76eeb 100644 --- a/daffodil-runtime1/src/main/scala/org/apache/daffodil/runtime1/infoset/TeeInfosetOutputter.scala +++ b/daffodil-runtime1/src/main/scala/org/apache/daffodil/runtime1/infoset/TeeInfosetOutputter.scala @@ -17,6 +17,10 @@ package org.apache.daffodil.runtime1.infoset +import org.apache.daffodil.runtime1.api.InfosetArray +import org.apache.daffodil.runtime1.api.InfosetComplexElement +import org.apache.daffodil.runtime1.api.InfosetSimpleElement + /** * Receive infoset events and forward them to one or more InfosetOutputters. A * thrown exception from any outputter is not caught and bubbles up resulting @@ -32,27 +36,27 @@ class TeeInfosetOutputter(outputters: InfosetOutputter*) extends InfosetOutputte outputters.foreach { _.reset() } } - override def startSimple(simple: DISimple): Unit = { + override def startSimple(simple: InfosetSimpleElement): Unit = { outputters.foreach { _.startSimple(simple) } } - override def endSimple(simple: DISimple): Unit = { + override def endSimple(simple: InfosetSimpleElement): Unit = { outputters.foreach { _.endSimple(simple) } } - override def startComplex(complex: DIComplex): Unit = { + override def startComplex(complex: InfosetComplexElement): Unit = { outputters.foreach { _.startComplex(complex) } } - override def endComplex(complex: DIComplex): Unit = { + override def endComplex(complex: InfosetComplexElement): Unit = { outputters.foreach { _.endComplex(complex) } } - override def startArray(array: DIArray): Unit = { + override def startArray(array: InfosetArray): Unit = { outputters.foreach { _.startArray(array) } } - override def endArray(array: DIArray): Unit = { + override def endArray(array: InfosetArray): Unit = { outputters.foreach { _.endArray(array) } } diff --git a/daffodil-runtime1/src/main/scala/org/apache/daffodil/runtime1/infoset/W3CDOMInfosetOutputter.scala b/daffodil-runtime1/src/main/scala/org/apache/daffodil/runtime1/infoset/W3CDOMInfosetOutputter.scala index 5825b83210..12e1049145 100644 --- a/daffodil-runtime1/src/main/scala/org/apache/daffodil/runtime1/infoset/W3CDOMInfosetOutputter.scala +++ b/daffodil-runtime1/src/main/scala/org/apache/daffodil/runtime1/infoset/W3CDOMInfosetOutputter.scala @@ -23,13 +23,17 @@ import org.apache.daffodil.lib.exceptions.Assert import org.apache.daffodil.lib.util.MStackOf import org.apache.daffodil.lib.util.Maybe import org.apache.daffodil.lib.xml.XMLUtils -import org.apache.daffodil.runtime1.dpath.NodeInfo +import org.apache.daffodil.runtime1.api.InfosetArray +import org.apache.daffodil.runtime1.api.InfosetComplexElement +import org.apache.daffodil.runtime1.api.InfosetElement +import org.apache.daffodil.runtime1.api.InfosetSimpleElement +import org.apache.daffodil.runtime1.api.PrimitiveType import org.w3c.dom.Document import org.w3c.dom.Element import org.w3c.dom.Node; -class W3CDOMInfosetOutputter extends InfosetOutputter with XMLInfosetOutputter { +class W3CDOMInfosetOutputter extends InfosetOutputter { private var document: Document = null private val stack = new MStackOf[Node] @@ -56,16 +60,16 @@ class W3CDOMInfosetOutputter extends InfosetOutputter with XMLInfosetOutputter { result = Maybe(root.asInstanceOf[Document]) } - def startSimple(diSimple: DISimple): Unit = { + override def startSimple(se: InfosetSimpleElement): Unit = { - val elem = createElement(diSimple) + val elem = createElement(se) - if (diSimple.hasValue) { + if (!se.isNilled) { val text = - if (diSimple.erd.optPrimType.get.isInstanceOf[NodeInfo.String.Kind]) { - remapped(diSimple.dataValueAsString) + if (se.metadata.primitiveType == PrimitiveType.String) { + XMLUtils.remapXMLIllegalCharactersToPUA(se.getText) } else { - diSimple.dataValueAsString + se.getText } elem.appendChild(document.createTextNode(text)) } @@ -73,21 +77,20 @@ class W3CDOMInfosetOutputter extends InfosetOutputter with XMLInfosetOutputter { stack.top.appendChild(elem) } - def endSimple(diSimple: DISimple): Unit = {} - - def startComplex(diComplex: DIComplex): Unit = { + override def endSimple(se: InfosetSimpleElement): Unit = {} + override def startComplex(diComplex: InfosetComplexElement): Unit = { val elem = createElement(diComplex) stack.top.appendChild(elem) stack.push(elem) } - def endComplex(diComplex: DIComplex): Unit = { + override def endComplex(ce: InfosetComplexElement): Unit = { stack.pop } - def startArray(diArray: DIArray): Unit = {} - def endArray(diArray: DIArray): Unit = {} + override def startArray(ar: InfosetArray): Unit = {} + def endArray(ar: InfosetArray): Unit = {} def getResult(): Document = { Assert.usage( @@ -97,18 +100,21 @@ class W3CDOMInfosetOutputter extends InfosetOutputter with XMLInfosetOutputter { result.get } - private def createElement(diElement: DIElement): Element = { + private def createElement(ie: InfosetElement): Element = { assert(document != null) val elem: Element = - if (diElement.erd.namedQName.namespace.isNoNamespace) { - document.createElementNS(null, diElement.erd.name) + if (ie.metadata.namespace eq null) { + document.createElementNS(null, ie.metadata.name) } else { - document.createElementNS(diElement.erd.namedQName.namespace, diElement.erd.prefixedName) + document.createElementNS( + ie.metadata.namespace, + ie.metadata.toQName, + ) } - if (isNilled(diElement)) { + if (ie.isNilled) { elem.setAttributeNS(XMLUtils.XSI_NAMESPACE.toString, "xsi:nil", "true") } diff --git a/daffodil-runtime1/src/main/scala/org/apache/daffodil/runtime1/infoset/XMLInfosetOutputter.scala b/daffodil-runtime1/src/main/scala/org/apache/daffodil/runtime1/infoset/XMLInfosetOutputter.scala deleted file mode 100644 index f18077a29b..0000000000 --- a/daffodil-runtime1/src/main/scala/org/apache/daffodil/runtime1/infoset/XMLInfosetOutputter.scala +++ /dev/null @@ -1,62 +0,0 @@ -/* - * Licensed to the Apache Software Foundation (ASF) under one or more - * contributor license agreements. See the NOTICE file distributed with - * this work for additional information regarding copyright ownership. - * The ASF licenses this file to You under the Apache License, Version 2.0 - * (the "License"); you may not use this file except in compliance with - * the License. You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ - -package org.apache.daffodil.runtime1.infoset - -import org.apache.daffodil.lib.equality._ -import org.apache.daffodil.lib.util.Maybe -import org.apache.daffodil.lib.xml.XMLUtils - -trait XMLInfosetOutputter { - - def remapped(dataValueAsString: String) = - XMLUtils.remapXMLIllegalCharactersToPUA(dataValueAsString) - - /** - * String suitable for use in the text of a Processing Instruction. - * - * The text is a pseudo-XML string. - */ - protected final def fmtInfo(diTerm: DITerm): Maybe[String] = { - val pecXML = diTerm.parserEvalCache.toPseudoXML() - val uecXML = diTerm.unparserEvalCache.toPseudoXML() - val puxml = { - (if (pecXML =:= "") "" else "\n" + pecXML) + - (if (uecXML =:= "") "" else "\n" + uecXML) - } - Maybe(if (puxml =:= "") null else puxml) - } - - final def addFmtInfo( - diTerm: DITerm, - elem: scala.xml.Elem, - showFormatInfo: Boolean, - ): scala.xml.Elem = { - if (!showFormatInfo) return elem - val maybeFI = fmtInfo(diTerm) - val res = - if (maybeFI.isEmpty) elem - else { - val fi = maybeFI.value - val pi = new scala.xml.ProcInstr("formatInfo", fi) - val res = elem.copy(child = elem.child :+ pi) - res - } - res - } - -} diff --git a/daffodil-runtime1/src/main/scala/org/apache/daffodil/runtime1/infoset/XMLTextInfosetOutputter.scala b/daffodil-runtime1/src/main/scala/org/apache/daffodil/runtime1/infoset/XMLTextInfosetOutputter.scala index e51caea73d..dd7b9f331b 100644 --- a/daffodil-runtime1/src/main/scala/org/apache/daffodil/runtime1/infoset/XMLTextInfosetOutputter.scala +++ b/daffodil-runtime1/src/main/scala/org/apache/daffodil/runtime1/infoset/XMLTextInfosetOutputter.scala @@ -23,6 +23,10 @@ import javax.xml.stream.XMLStreamConstants._ import org.apache.daffodil.lib.exceptions.Assert import org.apache.daffodil.lib.util.Indentable +import org.apache.daffodil.lib.xml.XMLUtils +import org.apache.daffodil.runtime1.api.InfosetArray +import org.apache.daffodil.runtime1.api.InfosetComplexElement +import org.apache.daffodil.runtime1.api.InfosetSimpleElement import org.apache.daffodil.runtime1.dpath.NodeInfo /** @@ -41,8 +45,7 @@ class XMLTextInfosetOutputter private ( xmlTextEscapeStyle: XMLTextEscapeStyle.Value, minimal: Boolean, ) extends InfosetOutputter - with Indentable - with XMLInfosetOutputter { + with Indentable { def this( os: java.io.OutputStream, @@ -105,7 +108,7 @@ class XMLTextInfosetOutputter private ( } } - if (isNilled(elem)) { + if (elem.isNilled) { writer.write(" xsi:nil=\"true\"") } @@ -171,20 +174,21 @@ class XMLTextInfosetOutputter private ( } } - override def startSimple(simple: DISimple): Unit = { + override def startSimple(se: InfosetSimpleElement): Unit = { + val simple = se.asInstanceOf[DISimple] if (pretty) { writer.write(System.lineSeparator()) outputIndentation(writer) } outputStartTag(simple) - if (!isNilled(simple) && simple.hasValue) { + if (simple.hasValue) { if (simple.erd.optPrimType.get == NodeInfo.String) { val simpleVal = simple.dataValueAsString if (simple.erd.runtimeProperties.get(XMLTextInfoset.stringAsXml) == "true") { writeStringAsXml(simpleVal) } else { - val xmlSafe = remapped(simpleVal) + val xmlSafe = XMLUtils.remapXMLIllegalCharactersToPUA(simpleVal) val escaped = xmlTextEscapeStyle match { case XMLTextEscapeStyle.CDATA => { val needsCDataEscape = xmlSafe.exists { c => @@ -209,11 +213,12 @@ class XMLTextInfosetOutputter private ( inScopeComplexElementHasChildren = true } - override def endSimple(simple: DISimple): Unit = { + override def endSimple(simple: InfosetSimpleElement): Unit = { // do nothing, everything is done in startSimple } - override def startComplex(complex: DIComplex): Unit = { + override def startComplex(ce: InfosetComplexElement): Unit = { + val complex = ce.asInstanceOf[DIComplex] if (pretty) { writer.write(System.lineSeparator()) outputIndentation(writer) @@ -223,7 +228,8 @@ class XMLTextInfosetOutputter private ( inScopeComplexElementHasChildren = false } - override def endComplex(complex: DIComplex): Unit = { + override def endComplex(ce: InfosetComplexElement): Unit = { + val complex = ce.asInstanceOf[DIComplex] decrementIndentation() if (pretty && inScopeComplexElementHasChildren) { // only output newline and indentation for non-empty complex types @@ -234,11 +240,11 @@ class XMLTextInfosetOutputter private ( inScopeComplexElementHasChildren = true } - override def startArray(array: DIArray): Unit = { + override def startArray(array: InfosetArray): Unit = { // do nothing } - override def endArray(array: DIArray): Unit = { + override def endArray(array: InfosetArray): Unit = { // do nothing } diff --git a/daffodil-runtime1/src/main/scala/org/apache/daffodil/runtime1/processors/DataProcessor.scala b/daffodil-runtime1/src/main/scala/org/apache/daffodil/runtime1/processors/DataProcessor.scala index 633b50054e..c8874475e5 100644 --- a/daffodil-runtime1/src/main/scala/org/apache/daffodil/runtime1/processors/DataProcessor.scala +++ b/daffodil-runtime1/src/main/scala/org/apache/daffodil/runtime1/processors/DataProcessor.scala @@ -29,6 +29,8 @@ import java.util.zip.GZIPOutputStream import org.apache.daffodil.lib.Implicits._ import org.apache.daffodil.lib.api.Diagnostic +import org.apache.daffodil.runtime1.api.MetadataHandler +import org.apache.daffodil.runtime1.infoset.InfosetOutputter import org.apache.daffodil.runtime1.layers.LayerExecutionException object INoWarn4 { @@ -65,7 +67,6 @@ import org.apache.daffodil.runtime1.externalvars.ExternalVariablesLoader import org.apache.daffodil.runtime1.infoset.DIElement import org.apache.daffodil.runtime1.infoset.InfosetException import org.apache.daffodil.runtime1.infoset.InfosetInputter -import org.apache.daffodil.runtime1.infoset.InfosetOutputter import org.apache.daffodil.runtime1.infoset.TeeInfosetOutputter import org.apache.daffodil.runtime1.infoset.XMLTextInfosetOutputter import org.apache.daffodil.runtime1.processors.parsers.PState @@ -335,6 +336,11 @@ class DataProcessor( oos.close() } + def walkMetadata(handler: MetadataHandler): Unit = { + val walker = new MetadataWalker(this) + walker.walk(handler) + } + /** * Here begins the parser runtime. Compiler-oriented mechanisms (OOLAG etc.) aren't used in the * runtime. Instead we deal with success and failure statuses. diff --git a/daffodil-runtime1/src/main/scala/org/apache/daffodil/runtime1/processors/MetadataWalker.scala b/daffodil-runtime1/src/main/scala/org/apache/daffodil/runtime1/processors/MetadataWalker.scala new file mode 100644 index 0000000000..60fcbb3812 --- /dev/null +++ b/daffodil-runtime1/src/main/scala/org/apache/daffodil/runtime1/processors/MetadataWalker.scala @@ -0,0 +1,99 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.apache.daffodil.runtime1.processors + +import org.apache.daffodil.lib.exceptions.Assert +import org.apache.daffodil.runtime1.api.MetadataHandler +import org.apache.daffodil.runtime1.api.SequenceMetadata + +/** + * Walks the schema, but not the DSOM schema, it walks the RuntimeData objects that + * represent the DFDL schema at runtime. + * + * @param dp + */ +class MetadataWalker(private val dp: DataProcessor) { + + private lazy val rootERD = dp.ssrd.elementRuntimeData + + def walk(handler: MetadataHandler): Unit = { + walkTerm(handler, rootERD) + } + + private def walkTerm(handler: MetadataHandler, trd: TermRuntimeData): Unit = { + trd match { + // $COVERAGE-OFF$ + case err: ErrorERD => Assert.invariantFailed("should not get ErrorERDs") + // $COVERAGE-ON$ + case erd: ElementRuntimeData => walkElement(handler, erd) + case srd: SequenceRuntimeData => walkSequence(handler, srd) + case crd: ChoiceRuntimeData => walkChoice(handler, crd) + // $COVERAGE-OFF$ + case _ => Assert.invariantFailed(s"unrecognized TermRuntimeData subtype: $trd") + // $COVERAGE-ON$ + } + } + + private def walkElement(handler: MetadataHandler, erd: ElementRuntimeData): Unit = { + if (erd.optComplexTypeModelGroupRuntimeData.isDefined) + walkComplexElement(handler, erd) + else + walkSimpleElement(handler, erd) + } + + private def walkComplexElement( + handler: MetadataHandler, + erd: ElementRuntimeData, + ): Unit = { + val mgrd = erd.optComplexTypeModelGroupRuntimeData.getOrElse { + // $COVERAGE-OFF$ + Assert.invariantFailed("not a complex type element") + // $COVERAGE-ON$ + } + handler.startComplexElementMetadata(erd) + walkTerm(handler, mgrd) + handler.endComplexElementMetadata(erd) + } + + private def walkSimpleElement( + handler: MetadataHandler, + erd: ElementRuntimeData, + ): Unit = { + handler.simpleElementMetadata(erd) + } + + private def walkSequence(handler: MetadataHandler, sm: SequenceMetadata): Unit = { + val srd = sm.asInstanceOf[SequenceRuntimeData] + if (!srd.isHidden) { + handler.startSequenceMetadata(srd) + srd.groupMembers.map { trd => + walkTerm(handler, trd) + } + handler.endSequenceMetadata(srd) + } + } + + private def walkChoice(handler: MetadataHandler, crd: ChoiceRuntimeData): Unit = { + handler.startChoiceMetadata(crd) + crd.groupMembers.map { trd => + walkTerm(handler, trd) + } + handler.endChoiceMetadata(crd) + } + +} diff --git a/daffodil-runtime1/src/main/scala/org/apache/daffodil/runtime1/processors/ProcessorStateBases.scala b/daffodil-runtime1/src/main/scala/org/apache/daffodil/runtime1/processors/ProcessorStateBases.scala index 8bea3bb1c2..4779122908 100644 --- a/daffodil-runtime1/src/main/scala/org/apache/daffodil/runtime1/processors/ProcessorStateBases.scala +++ b/daffodil-runtime1/src/main/scala/org/apache/daffodil/runtime1/processors/ProcessorStateBases.scala @@ -48,6 +48,7 @@ import org.apache.daffodil.lib.util.Maybe.One import org.apache.daffodil.lib.util.MaybeInt import org.apache.daffodil.lib.util.MaybeULong import org.apache.daffodil.runtime1.api.DFDL +import org.apache.daffodil.runtime1.api.InfosetElement import org.apache.daffodil.runtime1.dpath.DState import org.apache.daffodil.runtime1.dsom.DPathCompileInfo import org.apache.daffodil.runtime1.dsom.RuntimeSchemaDefinitionError @@ -526,7 +527,7 @@ abstract class ParseOrUnparseState protected ( final def getContext(): ElementRuntimeData = { // threadCheck() - val currentElement = infoset.asInstanceOf[InfosetElement] + val currentElement = infoset val res = currentElement.runtimeData res } diff --git a/daffodil-runtime1/src/main/scala/org/apache/daffodil/runtime1/processors/RuntimeData.scala b/daffodil-runtime1/src/main/scala/org/apache/daffodil/runtime1/processors/RuntimeData.scala index 5424638ed0..61eada0300 100644 --- a/daffodil-runtime1/src/main/scala/org/apache/daffodil/runtime1/processors/RuntimeData.scala +++ b/daffodil-runtime1/src/main/scala/org/apache/daffodil/runtime1/processors/RuntimeData.scala @@ -17,7 +17,10 @@ package org.apache.daffodil.runtime1.processors -import java.lang.{ Double => JDouble, Float => JFloat } +import java.lang.{ Double => JDouble } +import java.lang.{ Float => JFloat } +import java.lang.{ Long => JLong } +import scala.collection.JavaConverters._ import scala.util.matching.Regex import scala.xml.NamespaceBinding @@ -41,8 +44,18 @@ import org.apache.daffodil.lib.xml.QNameBase import org.apache.daffodil.lib.xml.RefQName import org.apache.daffodil.lib.xml.StepQName import org.apache.daffodil.lib.xml.XMLUtils +import org.apache.daffodil.runtime1.api.ChoiceMetadata +import org.apache.daffodil.runtime1.api.ComplexElementMetadata +import org.apache.daffodil.runtime1.api.ElementMetadata +import org.apache.daffodil.runtime1.api.Metadata +import org.apache.daffodil.runtime1.api.ModelGroupMetadata +import org.apache.daffodil.runtime1.api.PrimitiveType +import org.apache.daffodil.runtime1.api.SequenceMetadata +import org.apache.daffodil.runtime1.api.SimpleElementMetadata +import org.apache.daffodil.runtime1.api.TermMetadata import org.apache.daffodil.runtime1.dpath.NodeInfo import org.apache.daffodil.runtime1.dpath.NodeInfo.PrimType +import org.apache.daffodil.runtime1.dpath.PrimTypeNode import org.apache.daffodil.runtime1.dsom.CompiledExpression import org.apache.daffodil.runtime1.dsom.DPathCompileInfo import org.apache.daffodil.runtime1.dsom.DPathElementCompileInfo @@ -74,19 +87,26 @@ import org.apache.daffodil.runtime1.processors.unparsers.UnparseError */ sealed trait RuntimeData - extends ImplementsThrowsSDE + extends Metadata + with ImplementsThrowsSDE with HasSchemaFileLocation with Serializable { - def schemaFileLocation: SchemaFileLocation + + def variableMap: VariableMap + def unqualifiedPathStepPolicy: UnqualifiedPathStepPolicy + def namespaces: NamespaceBinding def diagnosticDebugName: String + override def toString = + diagnosticDebugName // diagnostic messages depend on toString doing this def path: String - def variableMap: VariableMap - override def toString = diagnosticDebugName + final override def schemaFileLineNumber: JLong = + schemaFileLocation.lineNumber.map { JLong.getLong(_) }.orNull - def unqualifiedPathStepPolicy: UnqualifiedPathStepPolicy + final override def schemaFileLineColumnNumber: JLong = + schemaFileLocation.columnNumber.map { JLong.getLong(_) }.orNull - def namespaces: NamespaceBinding + final override def schemaFileInfo: String = schemaFileLocation.fileURITrimmed } object TermRuntimeData { @@ -139,7 +159,8 @@ sealed abstract class TermRuntimeData( val fillByteEv: FillByteEv, val maybeCheckByteAndBitOrderEv: Maybe[CheckByteAndBitOrderEv], val maybeCheckBitOrderAndCharsetEv: Maybe[CheckBitOrderAndCharsetEv], -) extends RuntimeData { +) extends RuntimeData + with TermMetadata { /** * Cyclic structures require initialization @@ -163,7 +184,6 @@ sealed abstract class TermRuntimeData( } def isRequiredScalar: Boolean - def isArray: Boolean /** * At some point TermRuntimeData is a ResolvesQNames which requires tunables: @@ -193,7 +213,7 @@ sealed class NonTermRuntimeData( val path: String, override val namespaces: NamespaceBinding, val unqualifiedPathStepPolicy: UnqualifiedPathStepPolicy, -) extends RuntimeData +) extends RuntimeData {} /** * Singleton. If found as the default value, means to use nil as @@ -608,7 +628,7 @@ sealed class ElementRuntimeData( val schemaFileLocation: SchemaFileLocation, val diagnosticDebugName: String, val path: String, - val minimizedScope: NamespaceBinding, + override val minimizedScope: NamespaceBinding, defaultBitOrderArg: BitOrder, val optPrimType: Option[PrimType], val targetNamespace: NS, @@ -665,7 +685,12 @@ sealed class ElementRuntimeData( fillByteEvArg, maybeCheckByteAndBitOrderEvArg, maybeCheckBitOrderAndCharsetEvArg, - ) { + ) + with ElementMetadata + with SimpleElementMetadata + with ComplexElementMetadata { + + override def toQName: String = namedQName.toQNameString override def isRequiredScalar = !isArray && isRequiredInUnparseInfoset @@ -673,6 +698,11 @@ sealed class ElementRuntimeData( def isSimpleType = optPrimType.isDefined + def primType: PrimTypeNode = + optPrimType.asInstanceOf[Option[PrimTypeNode]].orNull + + override def primitiveType: PrimitiveType = primType.asInstanceOf[PrimitiveType] + lazy val schemaURIStringsForFullValidation: Seq[String] = schemaURIStringsForFullValidation1.distinct private def schemaURIStringsForFullValidation1: Seq[String] = (schemaFileLocation.uriString +: @@ -689,6 +719,11 @@ sealed class ElementRuntimeData( name } } + + override def namespace: String = + dpathElementCompileInfo.namedQName.namespace.toStringOrNullIfNoNS + def optNamespacePrefix: Option[String] = dpathElementCompileInfo.namedQName.prefix + } /** @@ -871,10 +906,11 @@ sealed abstract class ModelGroupRuntimeData( fillByteEvArg, maybeCheckByteAndBitOrderEvArg, maybeCheckBitOrderAndCharsetEvArg, - ) { + ) + with ModelGroupMetadata { final override def isRequiredScalar = true - final override def isArray = false + final def isArray = false } @@ -902,6 +938,7 @@ final class SequenceRuntimeData( fillByteEvArg: FillByteEv, maybeCheckByteAndBitOrderEvArg: Maybe[CheckByteAndBitOrderEv], maybeCheckBitOrderAndCharsetEvArg: Maybe[CheckBitOrderAndCharsetEv], + val isHidden: Boolean, ) extends ModelGroupRuntimeData( positionArg, partialNextElementResolverDelay, @@ -922,6 +959,7 @@ final class SequenceRuntimeData( maybeCheckByteAndBitOrderEvArg, maybeCheckBitOrderAndCharsetEvArg, ) + with SequenceMetadata /* * These Delay-type args are part of how we @@ -967,6 +1005,7 @@ final class ChoiceRuntimeData( maybeCheckByteAndBitOrderEvArg, maybeCheckBitOrderAndCharsetEvArg, ) + with ChoiceMetadata final class VariableRuntimeData( schemaFileLocationArg: SchemaFileLocation, diff --git a/daffodil-runtime1/src/main/scala/org/apache/daffodil/runtime1/processors/parsers/ExpressionEvaluatingParsers.scala b/daffodil-runtime1/src/main/scala/org/apache/daffodil/runtime1/processors/parsers/ExpressionEvaluatingParsers.scala index d6729cbcc3..ae6adccad8 100644 --- a/daffodil-runtime1/src/main/scala/org/apache/daffodil/runtime1/processors/parsers/ExpressionEvaluatingParsers.scala +++ b/daffodil-runtime1/src/main/scala/org/apache/daffodil/runtime1/processors/parsers/ExpressionEvaluatingParsers.scala @@ -25,7 +25,6 @@ import org.apache.daffodil.runtime1.dpath.ParserNonBlocking import org.apache.daffodil.runtime1.dsom.CompiledExpression import org.apache.daffodil.runtime1.infoset.DataValue import org.apache.daffodil.runtime1.infoset.DataValue.DataValuePrimitive -import org.apache.daffodil.runtime1.infoset.InfosetSimpleElement import org.apache.daffodil.runtime1.processors.ElementRuntimeData import org.apache.daffodil.runtime1.processors.Failure import org.apache.daffodil.runtime1.processors.RuntimeData @@ -61,7 +60,7 @@ class IVCParser(expr: CompiledExpression[AnyRef], e: ElementRuntimeData) def parse(start: PState): Unit = { Logger.log.debug(s"This is ${toString}") - val currentElement: InfosetSimpleElement = start.simpleElement + val currentElement = start.simpleElement val res = eval(start) currentElement.setDataValue(res) if (start.processorStatus ne Success) return diff --git a/daffodil-runtime1/src/main/scala/org/apache/daffodil/runtime1/processors/parsers/PState.scala b/daffodil-runtime1/src/main/scala/org/apache/daffodil/runtime1/processors/parsers/PState.scala index adf085ff1c..0ba0544b74 100644 --- a/daffodil-runtime1/src/main/scala/org/apache/daffodil/runtime1/processors/parsers/PState.scala +++ b/daffodil-runtime1/src/main/scala/org/apache/daffodil/runtime1/processors/parsers/PState.scala @@ -43,6 +43,7 @@ import org.apache.daffodil.lib.util.MaybeULong import org.apache.daffodil.lib.util.Pool import org.apache.daffodil.lib.util.Poolable import org.apache.daffodil.runtime1.api.DFDL +import org.apache.daffodil.runtime1.api.InfosetDocument import org.apache.daffodil.runtime1.infoset.DIComplex import org.apache.daffodil.runtime1.infoset.DIComplexState import org.apache.daffodil.runtime1.infoset.DIElement @@ -50,7 +51,6 @@ import org.apache.daffodil.runtime1.infoset.DISimple import org.apache.daffodil.runtime1.infoset.DISimpleState import org.apache.daffodil.runtime1.infoset.DataValue.DataValuePrimitive import org.apache.daffodil.runtime1.infoset.Infoset -import org.apache.daffodil.runtime1.infoset.InfosetDocument import org.apache.daffodil.runtime1.infoset.InfosetOutputter import org.apache.daffodil.runtime1.infoset.InfosetWalker import org.apache.daffodil.runtime1.processors.DataLoc diff --git a/daffodil-runtime1/src/main/scala/org/apache/daffodil/runtime1/processors/unparsers/UState.scala b/daffodil-runtime1/src/main/scala/org/apache/daffodil/runtime1/processors/unparsers/UState.scala index fe99f16598..c3f2b9c54a 100644 --- a/daffodil-runtime1/src/main/scala/org/apache/daffodil/runtime1/processors/unparsers/UState.scala +++ b/daffodil-runtime1/src/main/scala/org/apache/daffodil/runtime1/processors/unparsers/UState.scala @@ -168,7 +168,7 @@ abstract class UState( Assert.invariant(Maybe.WithNulls.isDefined(currentInfosetNode)) currentInfosetNode match { case a: DIArray => { - a.getOccurrence(arrayIterationPos) + a(arrayIterationPos) } case e: DIElement => thisElement } diff --git a/daffodil-sapi/src/main/scala/org/apache/daffodil/sapi/Daffodil.scala b/daffodil-sapi/src/main/scala/org/apache/daffodil/sapi/Daffodil.scala index 2ea330a2e0..805babdcc9 100644 --- a/daffodil-sapi/src/main/scala/org/apache/daffodil/sapi/Daffodil.scala +++ b/daffodil-sapi/src/main/scala/org/apache/daffodil/sapi/Daffodil.scala @@ -44,6 +44,7 @@ import org.apache.daffodil.runtime1.api.DFDL.{ import org.apache.daffodil.runtime1.api.DFDL.{ DaffodilUnparseErrorSAXException => SDaffodilUnparseErrorSAXException, } +import org.apache.daffodil.runtime1.api.MetadataHandler import org.apache.daffodil.runtime1.debugger.Debugger import org.apache.daffodil.runtime1.debugger.{ InteractiveDebugger => SInteractiveDebugger } import org.apache.daffodil.runtime1.debugger.{ TraceDebuggerRunner => STraceDebuggerRunner } @@ -480,6 +481,13 @@ class DataProcessor private[sapi] (private var dp: SDataProcessor) */ def save(output: WritableByteChannel): Unit = dp.save(output) + /** + * Walks the handler over the runtime metadata structures + * + * @param handler - the handler is called-back during the walk as each metadata structure is encountered. + */ + def walkMetadata(handler: MetadataHandler) = dp.walkMetadata(handler) + /** * Obtain a new [[DaffodilParseXMLReader]] from the current [[DataProcessor]]. */ diff --git a/daffodil-sapi/src/main/scala/org/apache/daffodil/sapi/infoset/Infoset.scala b/daffodil-sapi/src/main/scala/org/apache/daffodil/sapi/infoset/Infoset.scala index 62d1006805..640d1f7aef 100644 --- a/daffodil-sapi/src/main/scala/org/apache/daffodil/sapi/infoset/Infoset.scala +++ b/daffodil-sapi/src/main/scala/org/apache/daffodil/sapi/infoset/Infoset.scala @@ -19,12 +19,10 @@ package org.apache.daffodil.sapi.infoset import org.apache.daffodil.lib.exceptions.Assert import org.apache.daffodil.lib.util.MaybeBoolean +import org.apache.daffodil.runtime1.api.InfosetArray +import org.apache.daffodil.runtime1.api.InfosetComplexElement +import org.apache.daffodil.runtime1.api.InfosetSimpleElement import org.apache.daffodil.runtime1.dpath.NodeInfo -import org.apache.daffodil.runtime1.infoset.DIArray -import org.apache.daffodil.runtime1.infoset.DIComplex -// TODO: Not sure about the access to internal infoset implementation details. -// Should API users have this deep access to our internal infoset? -import org.apache.daffodil.runtime1.infoset.DISimple import org.apache.daffodil.runtime1.infoset.InfosetInputterEventType import org.apache.daffodil.runtime1.infoset.{ InfosetInputter => SInfosetInputter } import org.apache.daffodil.runtime1.infoset.{ InfosetOutputter => SInfosetOutputter } @@ -161,7 +159,7 @@ abstract class InfosetOutputter extends SInfosetOutputter { * value, nil, name, namespace, etc. */ @throws[Exception] - def startSimple(diSimple: DISimple): Unit + def startSimple(diSimple: InfosetSimpleElement): Unit /** * Called by Daffodil internals to signify the end of a simple element. @@ -173,55 +171,55 @@ abstract class InfosetOutputter extends SInfosetOutputter { * value, nil, name, namespace, etc. */ @throws[Exception] - def endSimple(diSimple: DISimple): Unit + def endSimple(diSimple: InfosetSimpleElement): Unit /** * Called by Daffodil internals to signify the beginning of a complex element. * * Throws java.lang.Exception if there was an error and Daffodil should stop parsing * - * @param diComplex the complex element that is started. Various fields of + * @param complex the complex element that is started. Various fields of * DIComplex can be accessed to determine things like the * nil, name, namespace, etc. */ @throws[Exception] - def startComplex(diComplex: DIComplex): Unit + def startComplex(complex: InfosetComplexElement): Unit /** * Called by Daffodil internals to signify the end of a complex element. * * Throws java.lang.Exception if there was an error and Daffodil should stop parsing * - * @param diComplex the complex element that is ended. Various fields of + * @param complex the complex element that is ended. Various fields of * DIComplex can be accessed to determine things like the * nil, name, namespace, etc. */ @throws[Exception] - def endComplex(diComplex: DIComplex): Unit + def endComplex(complex: InfosetComplexElement): Unit /** * Called by Daffodil internals to signify the beginning of an array of elements. * * Throws java.lang.Exception if there was an error and Daffodil should stop parsing * - * @param diArray the array that is started. Various fields of + * @param array the array that is started. Various fields of * DIArray can be accessed to determine things like the * name, namespace, etc. */ @throws[Exception] - def startArray(diArray: DIArray): Unit + def startArray(array: InfosetArray): Unit /** * Called by Daffodil internals to signify the end of an array of elements. * * Throws java.lang.Exception if there was an error and Daffodil should stop parsing * - * @param diArray the array that is ended. Various fields of + * @param array the array that is ended. Various fields of * DIArray can be accessed to determine things like the * name, namespace, etc. */ @throws[Exception] - def endArray(diArray: DIArray): Unit + def endArray(array: InfosetArray): Unit } /** @@ -236,9 +234,9 @@ abstract class InfosetOutputter extends SInfosetOutputter { * * @param showFormatInfo add additional properties to each scala.xml.Node for debug purposes */ -class ScalaXMLInfosetOutputter(showFormatInfo: Boolean = false) extends InfosetOutputterProxy { +class ScalaXMLInfosetOutputter() extends InfosetOutputterProxy { - override val infosetOutputter = new SScalaXMLInfosetOutputter(showFormatInfo) + override val infosetOutputter = new SScalaXMLInfosetOutputter() /** * Get the scala.xml.Node representing the infoset created during a parse @@ -453,11 +451,14 @@ abstract class InfosetOutputterProxy extends InfosetOutputter { override def reset(): Unit = infosetOutputter.reset() override def startDocument(): Unit = infosetOutputter.startDocument() override def endDocument(): Unit = infosetOutputter.endDocument() - override def startSimple(diSimple: DISimple): Unit = infosetOutputter.startSimple(diSimple) - override def endSimple(diSimple: DISimple): Unit = infosetOutputter.endSimple(diSimple) - override def startComplex(diComplex: DIComplex): Unit = - infosetOutputter.startComplex(diComplex) - override def endComplex(diComplex: DIComplex): Unit = infosetOutputter.endComplex(diComplex) - override def startArray(diArray: DIArray): Unit = infosetOutputter.startArray(diArray) - override def endArray(diArray: DIArray): Unit = infosetOutputter.endArray(diArray) + override def startSimple(diSimple: InfosetSimpleElement): Unit = + infosetOutputter.startSimple(diSimple) + override def endSimple(diSimple: InfosetSimpleElement): Unit = + infosetOutputter.endSimple(diSimple) + override def startComplex(complex: InfosetComplexElement): Unit = + infosetOutputter.startComplex(complex) + override def endComplex(complex: InfosetComplexElement): Unit = + infosetOutputter.endComplex(complex) + override def startArray(array: InfosetArray): Unit = infosetOutputter.startArray(array) + override def endArray(array: InfosetArray): Unit = infosetOutputter.endArray(array) } diff --git a/daffodil-sapi/src/test/scala/org/apache/daffodil/example/TestInfosetInputterOutputter.scala b/daffodil-sapi/src/test/scala/org/apache/daffodil/example/TestInfosetInputterOutputter.scala index e32af31ce1..058c3e59ac 100644 --- a/daffodil-sapi/src/test/scala/org/apache/daffodil/example/TestInfosetInputterOutputter.scala +++ b/daffodil-sapi/src/test/scala/org/apache/daffodil/example/TestInfosetInputterOutputter.scala @@ -20,11 +20,10 @@ package org.apache.daffodil.example import scala.collection.mutable.ArrayBuffer import org.apache.daffodil.lib.util.MaybeBoolean -// TODO: Shouldn't need to import things not in the sapi package +import org.apache.daffodil.runtime1.api.InfosetArray +import org.apache.daffodil.runtime1.api.InfosetComplexElement +import org.apache.daffodil.runtime1.api.InfosetSimpleElement import org.apache.daffodil.runtime1.dpath.NodeInfo -import org.apache.daffodil.runtime1.infoset.DIArray -import org.apache.daffodil.runtime1.infoset.DIComplex -import org.apache.daffodil.runtime1.infoset.DISimple import org.apache.daffodil.runtime1.infoset.InfosetInputterEventType import org.apache.daffodil.runtime1.infoset.InfosetInputterEventType.EndDocument import org.apache.daffodil.runtime1.infoset.InfosetInputterEventType.EndElement @@ -105,40 +104,40 @@ case class TestInfosetOutputter() extends InfosetOutputter { events.append(TestInfosetEvent.endDocument()) } - override def startSimple(diSimple: DISimple): Unit = { + override def startSimple(simple: InfosetSimpleElement): Unit = { events.append( TestInfosetEvent.startSimple( - diSimple.erd.name, - diSimple.erd.namedQName.namespace, - diSimple.dataValueAsString, - if (diSimple.erd.isNillable) MaybeBoolean(diSimple.isNilled) else MaybeBoolean.Nope, + simple.metadata.name, + simple.metadata.namespace, + simple.getText, + if (simple.metadata.isNillable) MaybeBoolean(simple.isNilled) else MaybeBoolean.Nope, ), ) } - override def endSimple(diSimple: DISimple): Unit = { + override def endSimple(simple: InfosetSimpleElement): Unit = { events.append( - TestInfosetEvent.endSimple(diSimple.erd.name, diSimple.erd.namedQName.namespace), + TestInfosetEvent.endSimple(simple.metadata.name, simple.metadata.namespace), ) } - override def startComplex(diComplex: DIComplex): Unit = { + override def startComplex(complex: InfosetComplexElement): Unit = { events.append( TestInfosetEvent.startComplex( - diComplex.erd.name, - diComplex.erd.namedQName.namespace, - if (diComplex.erd.isNillable) MaybeBoolean(diComplex.isNilled) else MaybeBoolean.Nope, + complex.metadata.name, + complex.metadata.namespace, + if (complex.metadata.isNillable) MaybeBoolean(complex.isNilled) else MaybeBoolean.Nope, ), ) } - override def endComplex(diComplex: DIComplex): Unit = { + override def endComplex(complex: InfosetComplexElement): Unit = { events.append( - TestInfosetEvent.endComplex(diComplex.erd.name, diComplex.erd.namedQName.namespace), + TestInfosetEvent.endComplex(complex.metadata.name, complex.metadata.namespace), ) } - override def startArray(diArray: DIArray): Unit = {} + override def startArray(array: InfosetArray): Unit = {} - override def endArray(diArray: DIArray): Unit = {} + override def endArray(array: InfosetArray): Unit = {} } diff --git a/daffodil-udf/src/main/java/org/apache/daffodil/udf/UserDefinedFunctionProvider.java b/daffodil-udf/src/main/java/org/apache/daffodil/udf/UserDefinedFunctionProvider.java index 801fad6349..213e54b05f 100644 --- a/daffodil-udf/src/main/java/org/apache/daffodil/udf/UserDefinedFunctionProvider.java +++ b/daffodil-udf/src/main/java/org/apache/daffodil/udf/UserDefinedFunctionProvider.java @@ -19,17 +19,17 @@ /** * Abstract class used by ServiceLoader to poll for UDF providers on classpath. - * + *

* Through this class, several User Defined Functions can be made available to * Daffodil via a single entry in the META-INF/services file. - * + *

* UDF Providers must subclass this, and must initialize the * userDefinedFunctionClasses array with all the UDF classes it is providing. - * + *

* If the UDFs being provided have constructors with arguments, the provider * subclass must also implement the createUserDefinedFunction to return an * initialized function class object based on the supplied namespace and name. - * + *

* Subclasses must also supply a * src/META-INF/services/org.apache.daffodil.udf.UserDefinedFunctionProvider * file in their JAVA project in order to be discoverable by Daffodil. @@ -61,13 +61,13 @@ public abstract class UserDefinedFunctionProvider { * @return initialized UserDefinedFunction object that must contain evaluate * function with desired functionality * - * @throws SecurityException + * @throws java.lang.SecurityException * if security manager exists and disallows access - * @throws IllegalArgumentException + * @throws java.lang.IllegalArgumentException * if the UDF doesn't have a no-argument constructor - * @throws ExceptionInInitializerError + * @throws java.lang.ExceptionInInitializerError * if there is an issue initializing the UDF object - * @throws ReflectiveOperationException + * @throws java.lang.ReflectiveOperationException * if the UDF doesn't have a no-argument constructor or if there is an * issue initializing the UDF object */