TornadoVM for newbies?

[mikymigs]

Hello,

I've been struggling trying to run my own code ("Hello World" type of project) using TornadoVM. To understand why I'm struggling, let me give you a little bit of context:

• I'm an experienced java developer but I've never used GraalVM or any kind of JDK "plugin" as you call it. My standard usage of Java is to use official JDK versions with Maven for dependencies.
• I'm a complete newbie on the topic of GPU programming. I just recently got interested in the topic and wanted to try if I could run a simple "Hello World" project on my GPU for fun and, who knows, to see if I could use this later on for professional projects.
• I have successfully installed TornadoVM following your guide for Windows and I'm able to run the tests from the example package.

Now after that I'm completely lost on how I can actually use TornadoVM for my own project. I've tried adding the maven dependencies in my pom.xml which didn't work (error: Could not find artifact tornado:tornado-api:jar:0.09 in universityOfManchester-graal (https://raw.githubusercontent.com/beehive-lab/tornado/maven-tornadovm) so I ended up adding the jar dependencies manually from the TornadoVM folder (TornadoVM\tornado-api\target\tornado-api-0.10-dev-bf7b95b.jar and TornadoVM\matrices\target\tornado-matrices-0.10-dev-bf7b95b.jar).
This seems to work but I have no idea where to go from here... I saw online that we could use annotations like @parallel or use "Dynamic Reconfiguration" with ts.execute(Policy.PERFORMANCE); but I feel like I'm just copy and pasting some code without really understanding what I'm doing. Also I don't understand why I had to install all this GraalVM stuff if I just need to import a jar in my project... How can I be sure my program is actually running in the TornadoVM environnement?

All that to say. Is there any "Hello World" type of tutorial that exists for newbies that allows using TornadoVM in a standard Java project? Is there any API type of documentation that explains your libs, the classes, the methods and what they are for? I mean, all I got is a few code sample and an annotation. There must be something I've missed! If you could give me some pointers or links to some additional documentation that would be great.

I hope you can help me move forward in my quest,

Thanks in advance!

[jjfumero]

HI @mikymigs ,
thank you for trying out TornadoVM.

There are a lot of question (that's good)., I will try to reply all of them.

Now after that I'm completely lost on how I can actually use TornadoVM for my own project. I've tried adding the maven dependencies in my pom.xml which didn't work (error: Could not find artifact tornado:tornado-api:jar:0.09 in universityOfManchester-graal (https://raw.githubusercontent.com/beehive-lab/tornado/maven-tornadovm) so I ended up adding the jar dependencies manually from the TornadoVM folder (TornadoVM\tornado-api\target\tornado-api-0.10-dev-bf7b95b.jar and TornadoVM\matrices\target\tornado-matrices-0.10-dev-bf7b95b.jar).

M... That's an error from our side. I will check. In any case, you can download the latest version (from master). At the time of writing this, we are in v0.10-dev.

This seems to work but I have no idea where to go from here...

So, TornadoVM can accelerate mainly data-parallel applications on GPUs. Although you can also use the multi-core CPU with OpenCL underneath. We have shown acceleration for Matrix-Multiplications (widely used on Machine Learning and Deel Learning), applications for computer vision, applications for NPL and face detection among many others.

To get inspired, here a few link:

• TornadoVM examples for compute: https://github.com/beehive-lab/TornadoVM/tree/master/examples/src/main/java/uk/ac/manchester/tornado/examples/compute
• TornadoVM with Microsoft Kfusion for computer vision: https://github.com/beehive-lab/kfusion-tornadovm/
• TornadoVM running Cascade Algorithms for face detection: Gary Frost has a really good article about this with TornadoVM. It seems the website is down but I will ask and update the link here.

With TornadoVM v0.9, we extended the TornadoVM API to express kernel parallelism. Examples here: https://github.com/beehive-lab/TornadoVM/tree/master/examples/src/main/java/uk/ac/manchester/tornado/examples/tornadovmcontext/compute

I saw online that we could use annotations like @parallel or use "Dynamic Reconfiguration" with ts.execute(Policy.PERFORMANCE);

TornadoVM has different modes of working. The dynamic reconfiguration is a brute force process in which TornadoVM automatically detects the fastest device for each data size.

For introductory material, we have an introductory article at InfoQ: https://www.infoq.com/articles/tornadovm-java-gpu-fpga/

We also have another intro-article for Jaxenter: https://jaxenter.com/tornado-vm-java-162460.html

Also I don't understand why I had to install all this GraalVM stuff if I just need to import a jar in my project... How can I be sure my program is actually running in the TornadoVM environnement?

You don't need Graal. TornadoVM can run with OpenJDK. We have tested:

• OpenJDK 8, 11 - 14
• Red Hat Mandrel
• Windows JDK 11
• Amazon Corretto 11
• GraalVM

TornadoVM is just a java program. If you type tornado --printFlags, you will see something like this (I am using OpenJDK 8 with JVMCI)

$ tornado --pringFlags
/home/juan/manchester/oracleGraal/openjdk1.8.0_262-jvmci-20.2-b03/bin/java -server -XX:-UseCompressedOops \
 -XX:+UnlockExperimentalVMOptions \
-XX:+EnableJVMCI \
-Djava.library.path=/home/juan/manchester/tornado/tornado/bin/sdk/lib \
-Djava.ext.dirs=/home/juan/manchester/tornado/tornado/bin/sdk/share/java/tornado  \
-Dtornado.load.api.implementation=uk.ac.manchester.tornado.runtime.tasks.TornadoTaskSchedule \
-Dtornado.load.runtime.implementation=uk.ac.manchester.tornado.runtime.TornadoCoreRuntime \
-Dtornado.load.tornado.implementation=uk.ac.manchester.tornado.runtime.common.Tornado \
-Dtornado.load.device.implementation.opencl=uk.ac.manchester.tornado.drivers.opencl.runtime.OCLDeviceFactory \
-Dtornado.load.device.implementation.ptx=uk.ac.manchester.tornado.drivers.ptx.runtime.PTXDeviceFactory \
-Dtornado.load.annotation.implementation=uk.ac.manchester.tornado.annotation.ASMClassVisitor \
-Dtornado.load.annotation.parallel=uk.ac.manchester.tornado.api.annotations.Parallel   \
-XX:-UseJVMCIClassLoader

As you can see, you need some implementations that are loaded at runtime, as well as the .so for OpenCL and PTX connectors of TornadoVM (JNI libraries). So, as soon as you have this, you can invoke TornadoVM with any tested OpenJDK.

All that to say. Is there any "Hello World" type of tutorial that exists for newbies that allow using TornadoVM in a standard Java project? Is there any API type of documentation that explains your libs, the classes, the methods and what they are for? I mean, all I got is a few code sample and an annotation. There must be something I've missed! If you could give me some pointers or links to some additional documentation that would be great.

Apart from the ones I mentioned, we have this documentation:

• FAQ: https://github.com/beehive-lab/TornadoVM/blob/master/assembly/src/docs/15_FAQ.md
• Documentation: https://github.com/beehive-lab/TornadoVM/tree/master/assembly/src/docs
• Examples: https://github.com/beehive-lab/TornadoVM/blob/master/assembly/src/docs/2_EXAMPLES.md
• Tutorial developed for InfoQ: https://github.com/jjfumero/qconlondon2020-tornadovm
• InfoQ presentation: https://www.infoq.com/presentations/tornadovm-java-gpu-fpga/

Hope this helps.
Juan

[mikymigs]

Thank you for your answer, this makes more sense. I managed to run a program I had compiled with maven successfully. I don't know if that's the best method but the way I did was to build with mvn package and then run the jar using tornado -jar myjar.jar

I wish there was a way however to run the program directly in my IDE (IntelliJ) also for debugging purposes. I'm also wondering how this can work if I want to use this on an existing project. Let's assume I isolate a piece of code that can run on GPUs and I already have a whole project using Spring and a bunch of other libs potentially even running on a server already that is not a "tornado compatible" VM. Then how can I call this piece of code that can only run within this specific tornado VM environment? Should I build some kind of a micro service? Actually is it even possible to run a web app with tornado VM?

[jjfumero]

Thank you for your answer, this makes more sense. I managed to run a program I had compiled with maven successfully. I don't know if that's the best method but the way I did was to build with mvn package and then run the jar using tornado -jar myjar.jar

Yes. that should work as well.

I wish there was a way however to run the program directly in my IDE (IntelliJ) also for debugging purposes. I'm also wondering how this can work if I want to use this on an existing project.

You can run TornadoVM with IntelIiJ. You just need to setup a new configuration (Add Configuration).

• Select (+) Application.
• Set the main class you want to run
• Press Modify options and add VM options (Alt+ V): here you add all flags that tornado --printFlags gives you.

I think that's all you need.

As an example, here how we run on Intel FPGAs from IntelliJ with TornadoVM: https://www.youtube.com/watch?v=lYtpi7zcp1g (no audio)

Let's assume I isolate a piece of code that can run on GPUs and I already have a whole project using Spring and a bunch of other libs potentially even running on a server already that is not a "tornado compatible" VM. Then how can I call this piece of code that can only run within this specific tornado VM environment? Should I build some kind of a micro service? Actually is it even possible to run a web app with tornado VM?

We have some demos running with NodeJS, if that helps:

Live demo: https://youtu.be/Q-_eB86hPPA?t=1267
Code: https://github.com/jjfumero/qconlondon2020-tornadovm/tree/master/nodejs

[mikymigs]

You can run TornadoVM with IntelIiJ. You just need to setup a new configuration (Add Configuration).

Select (+) Application.
Set the main class you want to run
Press Modify options and add VM options (Alt+ V): here you add all flags that tornado --printFlags gives you.

I had tried that before but I ended up with this error: java.lang.UnsatisfiedLinkError: C:\...\TornadoVM\bin\sdk\lib\tornado-opencl.dll: Can't find dependent libraries so I assumed it wasn't possible but I will look more into making this worked based on the link you provided.

You did answer most of my questions, thank you for that, the rest of the work is on my end now to figure out :)

Have a great day,

[fafayqa]

@jjfumero There is no tornado-api nor tornado-matrices on Maven.
In fact, there is absolutely no way how/where to get compiled Tornado files (jar)...mindbogling!

[jjfumero]

@fafayqa , TornadoVM is not on Maven.

To compile TornadoVM: https://github.com/beehive-lab/TornadoVM/blob/master/INSTALL.md

• TornadoVM with JDK 8 with JVMCI support: see the installation guide here.
• TornadoVM with GraalVM (either with JDK 8 or JDK 11): see the installation guide here.

Note: To run TornadoVM in Windows OS, install TornadoVM with GraalVM. More information here.

Note: To run TornadoVM on ARM Mali, install TornadoVM with GraalVM and JDK 11. More information here.

Additionally, all JAR files can be obtained from here:

https://github.com/beehive-lab/TornadoVM/tree/maven-tornadovm