Multiple commits to ultimately support removal of SafeHandle for global handles (#321)

* Adapted to new model with handle generation in this repo
* Sets the stage for removing use of SafeHandle as it makes the generation of the handle code local to this repo so it's easier to maintain.
* Converted generator to using Ubiquity.NET.CommandLine for args parsing.
* Added scripting to generate the wrappers off-line
* Added specialized target to generate the response file for the generator.
    * The project has a package reference to the LibLLVM package so that it can find the headers to parse.
* Adjusted stylecop.json to get expected behavior with regard to internal interfaces.
    * See: https://github.com/DotNetAnalyzers/
StyleCopAnalyzers/issues/3100
* Fixed `EnsureBuildOutputPaths` target in `Directory.Build.targets` so it actually creates the `NuGet` folder even in IDE builds.
* Removed unused 'Using" namespace statements.
* Renamed `LLJit` for consistency
* Misc. Updates to support testing/validation of ref counting problems
* Fixed debug record tests to operate without asserts in debug mode
* Added wrapper to re-order API parameters for `LLVMOrcJITLookup()`
* Renamed `GlobalHandleBaseExtensions.CloneWithAddref()` to `GlobalHandleBaseExtensions.CloneWithManagedAddRef()` to help express and clarify the intent.
* Fixed links in docs for clarity
* Updated dictionary ignored words list
* Clarified/Corrected doc comments
* Updated to resolve latest analyzer issues
* Commiting changes for ref counting
* Not yet complete but allows merging stashed changes for removal of safehandles.
    * Removing safe handles avoids confusion on addref for managed wrapper and addref for the native ABI handle. These are NOT the same and can't be used interchangeably for `StringPoolEntry` handles as some APIs assume "move" semantics, others have only temp access, while others will perform an addref on their own [It's all VERY confusing and poorly documented - but users of this library should not care about these subtleties. A major point of this lib is to resolve them consistently.
* Final remove of safe handle for global handles
* Global handles are IDisposable structs
    * While this is sometimes considered controversial it is primarily to leverage built in support for analyzers and use of I Disposable. Additionally, it makes the disposal consistent across the handle types.
* Fixed a LOT of issues with reference counting of `SymbolStringPoolEntry`
    * Handling of those was not correct in a LOT of places. This was ignored in a release build of the native libraries, but with a debug build it asserted... a lot! 🤦 This fixes #320
* ContextHandle was renamed to WrappedHandle for both global and aliased handles.
* Code generator was updated to use a single template that generates code for struct handles from the same template. IFF it is a global handle will it be generated with IDisposable support. Ultimately the structs are just a wrapper around a `nint`.
* Added support for C#14 field keyword to make some code simpler and easier to read/maintain.
    * due to bugs in support of `extension` that keyword IS NOT used. It was tried, even in the VS2026 preview, it's just too broken in the IDE and analyzers etc... to be worth the hassle.
        * It isn't necessary for anything other than properties or statics so not all that useful in the end.
* Moved constant enumerations from LibLLVMValueKind to allow debugger to show original value not named symbolic.
* Fixed handling of `LLVMErrorRef` it had ref count and crash issues that were identified in debug builds and testing of conversion to value type for handles.
    * `LLVMErrorRef` itself is a ref type `class` as it is mutable to handle the weird ownership semantics of the error object and message. The wrapper types handle all the idiosyncrasies of this low-level weird behavior.
        * Added special marshaling behavior for the error ref as it isn't like the other handles
         * Added Correct support for TypeId. Technically the only supported form is a string. Other values include an ID but there's nothing available in the LLVM-C API to do anything with them. (It's all very C++ specific). Fortunately, they are generally swallowed internally and the only form that is supposed to surface out of the C API is that of a string.
* Added extension property to interop handle `LLVMValueRef` to allow getting the kind of value.
    * This is mostly used for debugging to find the type of a value while debugging. It isn't generally useful and not exposed from upper layers.
* Added support for getting symbols where the symbol name itself includes the `:` character. Parsing of the string representation is flaky at best and really just a diagnostic/debug tool. (But it was a really helpful one!)
* Added Dispose of the ICodeGenerator to common Kaleidoscope REPL so that it will trigger cleanup (and possible debug asserts for ref count issues)
* Continued application of pattern for naming internal handle in wrapper types.
    * Member is named `Handle` and for types that need to transfer ownership to native there is an `InvalidateAfterMove` to allow for scenarios where move only happens on success.
* Added SrcGeneration library to support source generation and `IndentedTextWriter` extensions.
    * This allows for code only replacement for T4 templates. While T4 is great for some cases it can get unruly and downright impenetrable for more complex cases. Thus, this support library allows for common cases of C# code generators.
        * The handle generator doesn't currently use this as it uses T4 with a fairly simple template but that may change over time. It is used by `ScopeStack` to format the stack as nested data string for diagnostics.

Author: smaillet

Directory.Build.targets

@@ -1,9 +1,9 @@
-<Project InitialTargets="VerifyProjectSettings;ShowBuildParameters">
+<Project InitialTargets="VerifyProjectSettings;ShowBuildParameters;EnsureBuildOutputPaths">
     <!--
     Since Nuget.config is configured to include the build output location this
     will ensure the folder exists during restore so that it won't fail.
     -->
-    <Target Name="EnsureBuildOutputPaths" BeforeTargets="Restore;Build;Rebuild">
+    <Target Name="EnsureBuildOutputPaths" Condition="!Exists($(PackageOutputPath))">
         <MakeDir Directories="$(PackageOutputPath)"/>
     </Target>
 

Directory.Packages.props

@@ -6,7 +6,8 @@
   <ItemGroup>
     <GlobalPackageReference Include="Ubiquity.NET.Versioning.Build.Tasks" Version="5.0.7-alpha.0.1" />
     <GlobalPackageReference Include="IDisposableAnalyzers" Version="4.0.8" Condition="'$(NoCommonAnalyzers)' !=' true'" />
-    <GlobalPackageReference Include="MustUseRetVal" Version="0.0.2" />
+    <GlobalPackageReference Include="MustUseRetVal" Version="0.0.2" Condition="'$(NoCommonAnalyzers)' !=' true'" />
+
     <!--
     NOTE: This analyzer is sadly, perpetually in "pre-release mode". There have been many issues/discussion on the point
     and it has all fallen on deaf ears. So policies regarding "NO-Prerelease" components need to be overruled on this one
@@ -34,10 +35,13 @@
 
     <!-- Common packages for solution -->
     <PackageVersion Include="System.Linq.Async" Version="6.0.3" />
+    <PackageVersion Include="Ubiquity.NET.LibLLVM" Version="20.1.8" />
     <PackageVersion Include="Ubiquity.NET.Versioning" Version="6.0.2-beta" />
     <PackageVersion Include="Antlr4BuildTasks" Version="12.10.0" />
     <PackageVersion Include="Antlr4.Runtime.Standard" Version="4.13.1" />
     <PackageVersion Include="OpenSoftware.DgmlBuilder" Version="2.1.0" />
+    <PackageVersion Include="CppSharp" Version="1.1.5.3168" />
+    <PackageVersion Include="System.CodeDom" Version="9.0.7" />
 
     <!-- Tests all use the same framework versions -->
     <PackageVersion Include="Microsoft.NET.Test.Sdk" Version="17.14.1" />

Generate-HandleWrappers.ps1

@@ -0,0 +1,51 @@
+<#
+.SYNOPSIS
+    Generates the wrappers for handles from the headers in the Ubiquity.NET.LibLLVM NuGet Package
+
+.DESCRIPTION
+    This is used 'off-line' (by a developer) to generate the source for the handle wrappers. It is
+    NOT run during an automated build as the generator itself is dependent on the CppSharp library
+    that only supports the X64 architecture. Automated builds may (at some point in the future)
+    run on any architecture supported by .NET so cannot generate the sources at build time. A developer
+    machine generating the wrappers is assumed X64 (Windows, Linux, or Mac)
+
+.PARAMETER SkipRun
+    This parameter is used for inner loop testing to generate the response file so the paths
+    match the actual version of the package used. It skips actually building/running the
+    generator so that a developer can debug the run of that stage.
+#>
+Param(
+    [switch]$SkipRun
+)
+
+Push-Location $PSScriptRoot
+$oldPath = $env:Path
+try
+{
+    $ErrorActionPreference = 'stop'
+    $generatorProj = '.\src\Interop\LlvmBindingsGenerator\LlvmBindingsGenerator.csproj'
+    $rspPath = Join-Path $PSScriptRoot 'generator.rsp'
+
+    Write-Information "Generating response file: $rspPath via GenerateResponseFile target in $generatorProj"
+    dotnet msbuild -restore -target:GenerateResponseFile -property:HandleGeneratorResponeFilePath=`""$rspPath"`" $generatorProj
+
+    if(!$SkipRun)
+    {
+        Write-Information 'Generating Handle wrapper source...'
+        dotnet run --no-restore --project $generatorProj -- @$rspPath
+    }
+}
+catch
+{
+    # Everything from the official docs to the various articles in the blog-sphere say this isn't needed
+    # and in fact it is redundant - They're all WRONG! By re-throwing the exception the original location
+    # information is retained and the error reported will include the correct source file and line number
+    # data for the error. Without this, only the error message is retained and the location information is
+    # Line 1, Column 1, of the outer most script file, which is, of course, completely useless.
+    throw
+}
+finally
+{
+    Pop-Location
+    $env:Path = $oldPath
+}

IgnoredWords.dic

@@ -16,6 +16,7 @@ attrib
 Attribs
 AttributeSet
 Attrs
+automatable
 baz
 binaryop
 binlogs
@@ -32,6 +33,7 @@ buildtransitive
 builtinop
 byref
 byval
+canonicalization
 castable
 cibuild
 Cmp
@@ -67,11 +69,13 @@ foo
 fullsrc
 func
 getelementptr
+getters
 gh
 github
 Globalization
 Hashtable
 Identifier
+ifunc
 Impl
 initializer
 inline
@@ -86,12 +90,15 @@ LibLLVM
 Llilum
 llvm
 llvmversion
+lookups
 LValue
 malloc
 marshallers
 marshalling
+materializer
 memcopy
 memcpy
+memset
 metadata
 Mips
 msbuild
@@ -106,6 +113,7 @@ Nullable
 Nvidia
 online
 optimizenone
+outdent
 pages
 paren
 perf
@@ -119,6 +127,7 @@ readonly
 refactor
 refcount
 referenceable
+Relocations
 repl
 repo
 RMW
@@ -136,10 +145,12 @@ structs
 Subrange
 Sym
 telliam
+templated
 tl
 trx
 typdef
 Typedef
+typedefs
 typelib
 typeof
 uid
@@ -151,6 +162,7 @@ Unhandled
 uniqued
 uniqueing
 unmarshal
+unreferenced
 Unshipped
 userdefinedop
 Users

global.json

@@ -1,6 +1,6 @@
 {
     "sdk": {
-        "version": "9.0.201",
+        "version": "9.0.305",
         "rollForward": "latestMinor",
         "allowPrerelease": false
     },

src/Interop/InteropTests/ABI/libllvm-c/DataLayoutBindingsTests.cs

@@ -4,8 +4,10 @@
 using Microsoft.VisualStudio.TestTools.UnitTesting;
 
 using Ubiquity.NET.InteropHelpers;
+using Ubiquity.NET.Llvm.Interop.ABI.StringMarshaling;
 
 using static Ubiquity.NET.Llvm.Interop.ABI.libllvm_c.DataLayoutBindings;
+using static Ubiquity.NET.Llvm.Interop.ABI.llvm_c.Error;
 
 namespace Ubiquity.NET.Llvm.Interop.ABI.libllvm_c.UT
 {
@@ -15,11 +17,15 @@ public class DataLayoutBindingsTests
         [TestMethod]
         public void LibLLVMParseDataLayoutTest( )
         {
-            using(var errorRef = LibLLVMParseDataLayout( "badlayout"u8, out LLVMTargetDataRef retVal ))
-            using(retVal)
+            using(LLVMErrorRef errorRef = LibLLVMParseDataLayout( "badlayout"u8, out LLVMTargetDataRef retVal ))
             {
-                Assert.IsTrue( retVal.IsInvalid );
+                Assert.IsTrue( retVal.IsNull );
                 Assert.IsTrue( errorRef.Failed );
+                Assert.IsFalse( errorRef.Success);
+                Assert.IsFalse( errorRef.IsNull );
+                Assert.IsTrue( errorRef.IsString );
+                Assert.AreEqual(LLVMGetStringErrorTypeId(), errorRef.TypeId);
+                Assert.AreNotEqual( 0, errorRef.DangerousGetHandle() );
                 string errMsg = errorRef.ToString();
                 Assert.IsFalse( string.IsNullOrWhiteSpace( errMsg ), "Failure should have an error message" );
             }
@@ -31,7 +37,7 @@ public void LibLLVMParseDataLayoutTest( )
             {
                 Assert.IsFalse( errorRef.Failed );
                 Assert.IsTrue( errorRef.Success );
-                Assert.IsFalse( retVal.IsInvalid );
+                Assert.IsFalse( retVal.IsNull );
                 string errMsg = errorRef.ToString();
                 Assert.IsTrue( string.IsNullOrWhiteSpace( errMsg ), "Valid layout should NOT have an error message" );
             }

src/Interop/InteropTests/ContextHandleMarshallerTests.cs

@@ -11,7 +11,7 @@ public class ContextHandleMarshallerTests
         [TestMethod]
         public void ConvertToManagedTest( )
         {
-            var handle = ContextHandleMarshaller<LibLLVMMDOperandRef>.ConvertToManaged(NativeABIValue);
+            var handle = WrappedHandleMarshaller<LibLLVMMDOperandRef>.ConvertToManaged(NativeABIValue);
             Assert.IsFalse( handle.IsNull );
             Assert.AreEqual( NativeABIValue, handle.DangerousGetHandle() );
             Assert.AreEqual( NativeABIValue, (nint)handle );
@@ -24,7 +24,7 @@ public void ConvertToUnmanagedTest( )
 
             // Validate FromABI() method AND verify assumptions made in subsequent asserts...
             Assert.AreEqual( NativeABIValue, handle.DangerousGetHandle() );
-            nint abiHandleVal = ContextHandleMarshaller<LibLLVMMDOperandRef>.ConvertToUnmanaged(handle);
+            nint abiHandleVal = WrappedHandleMarshaller<LibLLVMMDOperandRef>.ConvertToUnmanaged(handle);
             Assert.AreEqual( NativeABIValue, abiHandleVal );
         }
 

src/Interop/InteropTests/DebugRecordTests.cs

@@ -1,6 +1,8 @@
 // Copyright (c) Ubiquity.NET Contributors. All rights reserved.
 // Licensed under the Apache-2.0 WITH LLVM-exception license. See the LICENSE.md file in the project root for full license information.
 
+using System;
+
 using Microsoft.VisualStudio.TestTools.UnitTesting;
 
 using Ubiquity.NET.Llvm.Interop.ABI.libllvm_c;
@@ -60,6 +62,24 @@ public void DebugRecordEnumerationSucceeds( )
             // Now attach debug records to the malloc.
             // first create the information to attach (It's a lot...)
             using LLVMDIBuilderRef diBuilder = LLVMCreateDIBuilder(module);
+            LLVMMetadataRef diFile = LLVMDIBuilderCreateFile(diBuilder, "testfile.cs"u8, Environment.CurrentDirectory);
+            LLVMMetadataRef diCU = LLVMDIBuilderCreateCompileUnit(
+                    diBuilder,
+                    LLVMDWARFSourceLanguage.LLVMDWARFSourceLanguageC_sharp,
+                    diFile,
+                    Producer: null,
+                    isOptimized: false,
+                    Flags: null,
+                    RuntimeVer: 0,
+                    SplitName: null,
+                    LLVMDWARFEmissionKind.LLVMDWARFEmissionFull,
+                    DWOId: 0,
+                    SplitDebugInlining: false,
+                    DebugInfoForProfiling: false,
+                    SysRoot: null,
+                    SDK: null
+                    );
+
             LLVMMetadataRef int32DiType = LLVMDIBuilderCreateBasicType(
                 diBuilder,
                 "int32_t"u8,
@@ -76,7 +96,7 @@ public void DebugRecordEnumerationSucceeds( )
                 Scope: default,
                 Name: "TestFunc"u8,
                 LinkageName: "_TestFunc"u8,
-                File: default,
+                File: diFile,
                 LineNo: 0,
                 Ty: diFuncType,
                 IsLocalToUnit: true,
@@ -85,11 +105,23 @@ public void DebugRecordEnumerationSucceeds( )
                 LLVMDIFlags.LLVMDIFlagPrivate,
                 IsOptimized: false
             );
+
+            var diLocalVar = LLVMDIBuilderCreateAutoVariable(
+                diBuilder,
+                scope,
+                Name: string.Empty,
+                File: diFile,
+                LineNo: 0,
+                int32PtrDiType,
+                AlwaysPreserve: false,
+                LLVMDIFlags.LLVMDIFlagArtificial,
+                4);
+
             LLVMMetadataRef diLocation = LLVMDIBuilderCreateDebugLocation(ctx, 123,4, scope, default);
 
             // Now attach the record to the result of the malloc call
             // and retest the status as it should be different now
-            LLVMDbgRecordRef dbgRecord = LLVMDIBuilderInsertDbgValueRecordBefore(diBuilder, mallocInst, int32PtrDiType, emptyExpression, diLocation, mallocInst);
+            LLVMDbgRecordRef dbgRecord = LLVMDIBuilderInsertDbgValueRecordBefore(diBuilder, mallocInst, diLocalVar, emptyExpression, diLocation, mallocInst);
             Assert.IsTrue( LibLLVMHasDbgRecords( mallocInst ) );
 
             // this should not crash now... [It shouldn't ever but that's another story...]