Creating Runtime Module for IR to IO , adding libsvm support(wip) by pantShrey · Pull Request #5 · armanbilge/vilcacora

pantShrey · 2025-07-15T18:21:00Z

The outputs are inconsistent
Sometimes i get proper response like

Loading model from: static_svm.onnx
Model loaded. Translating to ModelIR...
Translation successful.

--- Running Interpreter ---
Inference complete!

--- Inference Outputs ---
Output tensor 'label': Array(1)
Output tensor 'probabilities': Array(1.1038835E-38, 1.4E-45)
Process 32574 exited with status = 0 (0x00000000)

But majorly I am getting error

Loading model from: static_svm.onnx
Model loaded. Translating to ModelIR...
Translation successful.

--- Running Interpreter ---
Process 32570 stopped
* thread #5, stop reason = EXC_BAD_ACCESS (code=1, address=0x0)
    frame #0: 0x0000000100b5d38c libsvm.4.dylib`svm_predict_values + 160
libsvm.4.dylib`svm_predict_values:
->  0x100b5d38c <+160>: ldr    x1, [x26], #0x8
    0x100b5d390 <+164>: mov    x0, x22
    0x100b5d394 <+168>: mov    x2, x19
    0x100b5d398 <+172>: bl     0x100b58cd4    ; Kernel::k_function(svm_node const*, svm_node const*, svm_parameter const&)

armanbilge · 2025-07-16T05:15:06Z

+            Right(List.empty[Allocation])
+        }
+      }
+    } yield newAllocs.flatten


I wonder if instead of flatten can use flatTraverse.

armanbilge · 2025-07-16T05:27:24Z

+          case "SVMClassifier" =>
+            for {
+              _ <- checkArity(node, 1, 2) // Ensure SVMClassifier has 2 outputs
+              scoresOutputName = node.output(1)


Some of the repeated logic is unfortunate here. Not for this PR, but in a follow-up PR I wonder if we can combine both these phases in single-pass. So, the translator would generate both IR operations and manual allocations simultaneously as it traverses the ONNX operations.

armanbilge · 2025-07-16T05:35:10Z

 lazy val runtime = project
  .enablePlugins(ScalaNativePlugin, ScalaNativeBrewedConfigPlugin)
-  .dependsOn(ir)
+  .dependsOn(ir, onnx)


Is this just for testing? I think the runtime shouldn't directly depend on ONNX.

Suggested change

.dependsOn(ir, onnx)

.dependsOn(ir, onnx % Test)

armanbilge · 2025-07-16T05:37:06Z

+  /** Copies input Scala Arrays into their corresponding pre-allocated C memory. */
+  private def copyInputsToMemory(


Instead of copying the values, we can use Scala Native array.unsafeAt(0) operation to get a Ptr. That is the magic of Scala Native, we can directly do interop without needing to copy data between Scala and native data structures :)

armanbilge · 2025-07-16T05:37:28Z

+  /** Copies final results from C memory back into new Scala Arrays. */
+  private def copyOutputsFromMemory(model: ModelIR, memory: MemoryMap): IO[Map[String, Array[_]]] =


Same thing here, instead of copying the memory back, we can directly write into the result array.

armanbilge · 2025-07-16T05:48:51Z

+      case other =>
+        IO.raiseError(
+          new NotImplementedError(s"Operation not implemented: ${other.getClass.getSimpleName}"),
+        )


Similar idea here, we can actually raise this error outside of the IO so that it surfaces earlier.

armanbilge · 2025-07-16T05:50:43Z

+  private def handleElementWise(op: Operation, memory: MemoryMap, model: ModelIR)(
+      f: (Float, Float) => Float,
+  ): IO[Unit] = IO {


Unfortunate the f: (Float, Float) => Float will be allocated as a lambda and use dynamic dispatch. The Scala Native Interflow optimizer may be able to specialize this and avoid the lambda allocation and dispatch, but unfortunately it would be better to write these as separate methods with the operation hard-coded, for the guaranteed best performance.

armanbilge · 2025-07-16T05:53:17Z

+        // This entire block is now a single IO action that performs the C interop.
+        predictionResult <- IO {
+          // 1. Fill the svm_node struct with input data from the input tensor.
+          for (i <- 0 until numFeatures) {


We should use a while loop with manually-incremented counter instead of a for (which desugars to a Range allocation, foreach, and lambda with dynamic dispatch).

armanbilge · 2025-07-16T05:55:14Z

+          // 2. Prepare for and call the prediction function.
+          val nrClass = op.classLabels.size
+          val decValuesCount = nrClass * (nrClass - 1) / 2
+          val decisionValuesPtr = stackalloc[CDouble](decValuesCount.toUInt)


I see that you are returning decisionValuesPtr at the end of the IO, but this is not safe to do with stackallocated memory. The stack is popped at the end of the IO, so that memory is no longer safe to access.

armanbilge · 2025-07-16T05:55:52Z

+      // Using a for-comprehension here breaks the logic into clean, sequential IO steps.
+      for {


Actually, I think it would be okay to write the logic within a single IO.

…lator

armanbilge · 2025-07-18T15:58:50Z

+    * @return
+    *   An IO containing a map of output tensor names to their resulting Scala arrays.
+    */
+  def execute(model: ModelIR, inputs: Map[String, Array[_]]): IO[Map[String, Array[_]]] = {


Suggested change

def execute(model: ModelIR, inputs: Map[String, Array[_]]): IO[Map[String, Array[_]]] = {

def execute(model: ModelIR, inputs: Map[String, Array[_]]): Resource[IO, IO[Map[String, Array[_]]]] = {

armanbilge · 2025-07-18T15:59:00Z

+    memoryResource(model, inputs, outputArrays)
+      .use { memoryMap =>


Suggested change

memoryResource(model, inputs, outputArrays)

.use { memoryMap =>

memoryResource(model, inputs, outputArrays)

.flatMap { memoryMap =>

armanbilge · 2025-07-18T15:59:28Z

+      op: Operation.SVMClassifier,
+      memory: MemoryMap,
+      model: ModelIR,
+  ): IO[Unit] = {


Suggested change

): IO[Unit] = {

): Resource[IO, IO[Unit]] = {

armanbilge · 2025-07-18T15:59:36Z

+      svmInputNode <- malloc[svm_node](sizeof[svm_node] * (numFeatures + 1).toUSize)
+    } yield (modelPtr, svmInputNode)
+
+    svmResources.use { case (modelPtr, svmInputNode) =>


Suggested change

svmResources.use { case (modelPtr, svmInputNode) =>

svmResources.map { case (modelPtr, svmInputNode) =>

…ately executing it added test suite for interpreter

pantShrey added 2 commits July 15, 2025 23:47

Creating Runtime Module for IR to IO , adding libsvm support(wip)

0684cb8

corrected the translator test cases regarding data type 6 and 7

6a2ac93

armanbilge reviewed Jul 16, 2025

View reviewed changes

pantShrey added 3 commits July 18, 2025 17:50

removed memcpy from inputs and outputs and used flatTraverse in trans…

8a31831

…lator

replaced for with while and hard coded separate methods

6e17ab1

logic placed in a single IO in handleSvmClassifier

573e47f

armanbilge reviewed Jul 18, 2025

View reviewed changes

pantShrey added 2 commits July 25, 2025 17:06

corrected libsvm implementation returned a resource inplace of immedi…

aff54e2

…ately executing it added test suite for interpreter

fixing the libsvm warning

f6b8d88

armanbilge marked this pull request as ready for review August 4, 2025 19:58

		/** Copies input Scala Arrays into their corresponding pre-allocated C memory. */
		private def copyInputsToMemory(

		/** Copies final results from C memory back into new Scala Arrays. */
		private def copyOutputsFromMemory(model: ModelIR, memory: MemoryMap): IO[Map[String, Array[_]]] =

		// Using a for-comprehension here breaks the logic into clean, sequential IO steps.
		for {

	def execute(model: ModelIR, inputs: Map[String, Array[_]]): IO[Map[String, Array[_]]] = {
	def execute(model: ModelIR, inputs: Map[String, Array[_]]): Resource[IO, IO[Map[String, Array[_]]]] = {

		memoryResource(model, inputs, outputArrays)
		.use { memoryMap =>

	svmResources.use { case (modelPtr, svmInputNode) =>
	svmResources.map { case (modelPtr, svmInputNode) =>

Conversation

pantShrey commented Jul 15, 2025

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants