architecture interfaces and data types

.gitignore

@@ -0,0 +1,7 @@
+################################################################################
+# This .gitignore file was automatically created by Microsoft(R) Visual Studio.
+################################################################################
+
+.vs/
+/**/obj/**
+/**/bin/**

docs/architecture.md → docs/0_architecture.md

@@ -486,61 +486,3 @@ Clearly, there's still a lot of infrastructure to flesh out, and a lot of implem
 4. handling serialization of customer-defined types
 5. CSDL defined authz
 6. custom headers, query options, endpoints, etc.
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-

docs/1_interfaces+datatypes.md

@@ -0,0 +1,70 @@
+## pattern overview
+
+We should stick to the naming conventions laid out in the [architecture](./architecture.md). So, we should have "parser"s to create CSTs from strings, "converter"s to move between CSTs and ASTs, "translator"s to move between different types of ASTs, "transcriber"s to create strings from CSTs, "serializer"s to go from user defined types to ASTs, and "deserializer"s to go from ASTs to user defined types. 
+
+The CSTs will be a discriminated union that corresponds directly to the [OData ABNF](https://docs.oasis-open.org/odata/odata/v4.01/cs01/abnf/odata-abnf-construction-rules.txt).
+
+The ASTs will be a discriminated union that corresponds to the same ABNF but with the string literals removed, as well as "overhead" from the CST like aliases. 
+
+The transcribers will be implemented as visitors on the CST nodes to convert them to strings using an intermediate `StringBuilder`.
+
+The converters will be implemented as visitors on the AST or CST nodes to produce instances of the other.
+
+TODO parsers will be...
+
+The translators will be implemented as visitors on the AST nodes to produce a corresponding AST for the desired data store.
+
+Please see the [appendix](#appendix) for other modeling options that were explored.
+
+## odata resource path example
+
+Let's use the odata resource path as an example of the above patterns.
+
+### syntactic CST?
+
+TODO do you want to have a syntactic CST that is a context-free grammar (should the context free grammar be defined by the literals used in the ABNF? write this out and keep it somewhere if so)? then your parsers could be combinators; if you do this you will need a converter from syntactic to semantic CST
+TODO regardless of the above, you need to have a syntactic CST **anyway** to have parity with ODL
+
+### (semantic?) CST
+
+A sample implementation of the CST is [here](../odata/Root/OdataResourcePath/ConcreteSyntaxTreeNodes/OdataRelativeUri.cs).
+
+### AST
+
+A sample implementation of the AST is [here](../odata/Root/OdataResourcePath/AbstractSyntaxTreeNodes/OdataRelativeUri.cs).
+
+### CST transcriber
+
+A sample implementation of the transcriber is [here](../odata/Root/OdataResourcePath/Transcribers/OdataRelativeUriTranscriber.cs).
+
+### CST to AST converter
+
+A sample implementation of the CST to AST translator is [here](../odata/Root/OdataResourcePath/CstToAstTranslators/OdataRelativeUriTranslator.cs).
+
+### AST to CST converter
+
+A sample implementation of the CST to AST translator is [here](../odata/Root/OdataResourcePath/AstToCstTranslators/OdataRelativeUriTranslator.cs).
+
+### uri parser
+
+TODO
+
+### translator
+
+TODO
+
+## let's now attempt take some time to show mechanically how to implement a node in the AST and all of its associated utilities
+
+Let's try with `keyPredicate` using the [ABNF](https://docs.oasis-open.org/odata/odata/v4.01/cs01/abnf/odata-abnf-construction-rules.txt).
+
+## some takeaways
+
+Using this discriminated union pattern, it is clear from the code where the "feature gaps" are: any nodes that only have a private constructor have not yet been implemented. We should implement a node in its completeness so that we can maintain this status. Doing this will ensure that the "feature gaps" are never tribal knowledge, but something that any developer can discover just by looking at the code.
+
+These unions *also* allow us to easily scale across developers. Any number of developers can implement as many nodes as there are developers provided that no two developers are working on the same node. Also, by separating each phase of the handling process in this way, we are able to implement the nodes "piecewise", meaning: a developer can define the AST node, create a PR, and merge it; the developer can then define the CST node, create a PR, and merge it; they can then implement a converter, and so in. These can all be done as individual, discrete, atomic steps done (mostly) independently of each other. 
+
+## appendix
+
+### parsing
+
+Parser combinators were tried using the Sprache nuget package. Though combinators are very flexible, the code is trivially readable, and development is exceedingly fast, combinators result in potentially incorrect parsing on grammars that are not context-free; the OData ABNF is not a context-free grammar, so we cannot rely on combinators. You can see an attempt at this implementation [here](https://github.com/OData/odata.net/blob/corranrogue9/framework/interfacesanddatatypes/odata/Root/OdataResourcePath/CombinatorParsers/OdataRelativeUriParser.cs).

odata.sln

@@ -0,0 +1,28 @@
+
+Microsoft Visual Studio Solution File, Format Version 12.00
+# Visual Studio Version 17
+VisualStudioVersion = 17.12.35514.174
+MinimumVisualStudioVersion = 10.0.40219.1
+Project("{FAE04EC0-301F-11D3-BF4B-00C04F79EFBC}") = "odata", "odata\odata.csproj", "{6E3F11CA-8949-47C1-894C-CE61D911A2F2}"
+EndProject
+Project("{FAE04EC0-301F-11D3-BF4B-00C04F79EFBC}") = "odata.tests", "odata.tests\odata.tests.csproj", "{07CA58E0-3FA7-4BC2-8CB1-6974049CB5CF}"
+EndProject
+Global
+	GlobalSection(SolutionConfigurationPlatforms) = preSolution
+		Debug|Any CPU = Debug|Any CPU
+		Release|Any CPU = Release|Any CPU
+	EndGlobalSection
+	GlobalSection(ProjectConfigurationPlatforms) = postSolution
+		{6E3F11CA-8949-47C1-894C-CE61D911A2F2}.Debug|Any CPU.ActiveCfg = Debug|Any CPU
+		{6E3F11CA-8949-47C1-894C-CE61D911A2F2}.Debug|Any CPU.Build.0 = Debug|Any CPU
+		{6E3F11CA-8949-47C1-894C-CE61D911A2F2}.Release|Any CPU.ActiveCfg = Release|Any CPU
+		{6E3F11CA-8949-47C1-894C-CE61D911A2F2}.Release|Any CPU.Build.0 = Release|Any CPU
+		{07CA58E0-3FA7-4BC2-8CB1-6974049CB5CF}.Debug|Any CPU.ActiveCfg = Debug|Any CPU
+		{07CA58E0-3FA7-4BC2-8CB1-6974049CB5CF}.Debug|Any CPU.Build.0 = Debug|Any CPU
+		{07CA58E0-3FA7-4BC2-8CB1-6974049CB5CF}.Release|Any CPU.ActiveCfg = Release|Any CPU
+		{07CA58E0-3FA7-4BC2-8CB1-6974049CB5CF}.Release|Any CPU.Build.0 = Release|Any CPU
+	EndGlobalSection
+	GlobalSection(SolutionProperties) = preSolution
+		HideSolutionNode = FALSE
+	EndGlobalSection
+EndGlobal

odata.tests/MSTestSettings.cs

@@ -0,0 +1 @@
+[assembly: Parallelize(Scope = ExecutionScope.MethodLevel)]

odata.tests/Test1.cs

@@ -0,0 +1,226 @@
+namespace odata.tests
+{
+    using AbnfParser.CstNodes.Core;
+    using Root;
+    using Root.OdataResourcePath.CombinatorParsers;
+    using Root.OdataResourcePath.Transcribers;
+    using Sprache;
+    using System.Text;
+
+    [TestClass]
+    public sealed class Test1
+    {
+        [TestMethod]
+        public void TestMethod1()
+        {
+            var url = "$metadata";
+            var cst = OdataRelativeUriParser.Instance.Parse(url);
+
+            var ast = Root.OdataResourcePath.CstToAstConverters.OdataRelativeUriConverter
+                .Default
+                .Visit(cst, default);
+
+            var newCst = Root.OdataResourcePath.AstToCstConverters.OdataRelativeUriConverter
+                .Default
+                .Visit(ast, default);
+
+            var stringBuilder = new StringBuilder();
+            OdataRelativeUriTranscriber.Default.Visit(newCst, stringBuilder);
+
+            Assert.AreEqual(url, stringBuilder.ToString());
+        }
+
+        [TestMethod]
+        public void GenerateAlphaTranscribers()
+        {
+            var range = (0x01, 0x7E);
+
+            var builder = new StringBuilder();
+            for (int i = range.Item1; i <= range.Item2; ++i)
+            {
+                var className = $"x{i:X2}";
+                builder.AppendLine($"public sealed class {className}Transcriber");
+                builder.AppendLine("{");
+                builder.AppendLine($"private {className}Transcriber()");
+                builder.AppendLine("{");
+                builder.AppendLine("}");
+                builder.AppendLine();
+                builder.AppendLine($"public static {className}Transcriber Instance {{ get; }} = new {className}Transcriber();");
+                builder.AppendLine();
+                builder.AppendLine($"public Void Transcribe({className} node, StringBuilder context)");
+                builder.AppendLine("{");
+                builder.AppendLine($"context.Append((char)0{className});");
+                builder.AppendLine("return default;");
+                builder.AppendLine("}");
+                builder.AppendLine("}");
+            }
+
+            File.WriteAllText(@"C:\Users\gdebruin\code.txt", builder.ToString());
+        }
+
+        [TestMethod]
+        public void GenerateTranscriber()
+        {
+            var ranges = new[]
+            {
+                (0x20, 0x3D),
+                (0x3F, 0x7E),
+            };
+            var elementName = "ProseVal";
+
+            var builder = new StringBuilder();
+            builder.AppendLine("using System.Text;");
+            builder.AppendLine();
+            builder.AppendLine("using AbnfParser.CstNodes.Core;");
+            builder.AppendLine("using Root;");
+            builder.AppendLine();
+            builder.AppendLine($"public sealed class {elementName}Transcriber : {elementName}.Visitor<Void, StringBuilder>");
+            builder.AppendLine("{");
+            builder.AppendLine($"private {elementName}Transcriber()");
+            builder.AppendLine("{");
+            builder.AppendLine("}");
+            builder.AppendLine();
+            builder.AppendLine($"public static {elementName}Transcriber Instance {{ get; }} = new {elementName}Transcriber();");
+            builder.AppendLine();
+            foreach (var range in ranges)
+            {
+                GenerateAccepts(builder, elementName, range.Item1, range.Item2);
+            }
+
+            builder.AppendLine("}");
+
+            File.WriteAllText(@"C:\Users\gdebruin\code.txt", builder.ToString());
+        }
+
+        private void GenerateAccepts(StringBuilder builder, string elementName, int start, int end)
+        {
+            for (int i = start; i <= end; ++i)
+            {
+                var className = $"x{i:X2}";
+                builder.AppendLine($"protected internal override Void Accept({elementName}.{className} node, StringBuilder context)");
+                builder.AppendLine("{");
+                builder.AppendLine($"return {className}Transcriber.Instance.Transcribe(node.Value, context);");
+                builder.AppendLine("}");
+                builder.AppendLine();
+            }
+        }
+
+        [TestMethod]
+        public void Generate()
+        {
+            var ranges = new[]
+            {
+                (0x41, 0x5A),
+                (0x61, 0x7A),
+            };
+            var elementName = "ProseVal";
+
+            var builder = new StringBuilder();
+            builder.AppendLine("public abstract TResult Dispatch<TResult, TContext>(Visitor<TResult, TContext> visitor, TContext context);");
+            builder.AppendLine();
+            builder.AppendLine("public abstract class Visitor<TResult, TContext>");
+            builder.AppendLine("{");
+            builder.AppendLine($"public TResult Visit({elementName} node, TContext context)");
+            builder.AppendLine("{");
+            builder.AppendLine("return node.Dispatch(this, context);");
+            builder.AppendLine("}");
+            builder.AppendLine();
+            foreach (var range in ranges)
+            {
+                GenerateAccepts(builder, range.Item1, range.Item2);
+            }
+
+            builder.AppendLine("}");
+            builder.AppendLine();
+            foreach (var range in ranges)
+            {
+                GenerateClasses(builder, elementName, range.Item1, range.Item2);
+            }
+
+
+            File.WriteAllText(@"C:\Users\gdebruin\code.txt", builder.ToString());
+        }
+
+        private void GenerateAccepts(StringBuilder builder, int start, int end)
+        {
+            for (int i = start; i <= end; ++i)
+            {
+                var className = $"x{i:X2}";
+                builder.AppendLine($"protected internal abstract TResult Accept({className} node, TContext context);");
+            }
+        }
+
+        private void GenerateClasses(StringBuilder builder, string elementName, int start, int end)
+        {
+            for (int i = start; i <= end; ++i)
+            {
+                var className = $"x{i:X2}";
+                builder.AppendLine($"public sealed class {className} : {elementName}");
+                builder.AppendLine("{");
+                builder.AppendLine($"public {className}(Core.x3C lessThan, Core.{className} value, Core.x3E greaterThan)");
+                builder.AppendLine("{");
+                builder.AppendLine("LessThan = lessThan;");
+                builder.AppendLine("Value = value;");
+                builder.AppendLine("GreaterThan = greaterThan;");
+                builder.AppendLine("}");
+                builder.AppendLine();
+                builder.AppendLine($"public Core.x3C LessThan {{ get; }}");
+                builder.AppendLine($"public Core.{className} Value {{ get; }}");
+                builder.AppendLine($"public Core.x3E GreaterThan {{ get; }}");
+                builder.AppendLine();
+                builder.AppendLine($"public sealed override TResult Dispatch<TResult, TContext>(Visitor<TResult, TContext> visitor, TContext context)");
+                builder.AppendLine("{");
+                builder.AppendLine("return visitor.Accept(this, context);");
+                builder.AppendLine("}");
+                builder.AppendLine("}");
+                builder.AppendLine();
+            }
+        }
+
+        [TestMethod]
+        public void CoreRules()
+        {
+            var coreRulesPath = @"C:\github\odata.net\odata\AbnfParser\core.abnf";
+            var coreRulesText = File.ReadAllText(coreRulesPath);
+            var cst = AbnfParser.CombinatorParsers.RuleListParser.Instance.Parse(coreRulesText);
+
+            //// TODO if the ABNF is missing a trailing newline, the last rule will be dropped
+
+            var stringBuilder = new StringBuilder();
+            AbnfParser.Transcribers.RuleListTranscriber.Instance.Transcribe(cst, stringBuilder);
+            var transcribedText = stringBuilder.ToString();
+            Assert.AreEqual(coreRulesText, transcribedText);
+        }
+
+        [TestMethod]
+        public void AbnfRules()
+        {
+            var coreRulesPath = @"C:\github\odata.net\odata\AbnfParser\core.abnf";
+            var coreRulesText = File.ReadAllText(coreRulesPath);
+            var abnfRulesPath = @"C:\github\odata.net\odata\AbnfParser\abnf.abnf";
+            var abnfRulesText = File.ReadAllText(abnfRulesPath);
+            var fullRulesText = string.Join(Environment.NewLine, abnfRulesText, coreRulesText);
+            var cst = AbnfParser.CombinatorParsers.RuleListParser.Instance.Parse(fullRulesText);
+
+            var stringBuilder = new StringBuilder();
+            AbnfParser.Transcribers.RuleListTranscriber.Instance.Transcribe(cst, stringBuilder);
+            var transcribedText = stringBuilder.ToString();
+            Assert.AreEqual(fullRulesText, transcribedText);
+        }
+
+        [TestMethod]
+        public void OdataRules()
+        {
+            var odataRulesPath = @"C:\github\odata.net\odata\odata.abnf";
+            var odataRulesText = File.ReadAllText(odataRulesPath);
+            var cst = AbnfParser.CombinatorParsers.RuleListParser.Instance.Parse(odataRulesText);
+
+            var stringBuilder = new StringBuilder();
+            AbnfParser.Transcribers.RuleListTranscriber.Instance.Transcribe(cst, stringBuilder);
+            var transcribedText = stringBuilder.ToString();
+
+            File.WriteAllText(odataRulesPath, transcribedText);
+            Assert.AreEqual(odataRulesText, transcribedText);
+        }
+    }
+}