Skip to content

Latest commit

 

History

History

graalpy-javase-guide

README.md

Using Python Packages in a Java SE Application

Python libraries can be used in and shipped with plain Java applications. The GraalPy Maven artifacts and GraalVM Polyglot APIs allow flexible integration with different project setups.

Using Python packages in Java projects often requires a bit more setup, due to the nature of the Python packaging ecosystem. GraalPy provides a python-embedding package that simplifies the required setup to ship Python packages as Java resources or in separate folders. The important entry points to do so are the VirtualFileSystem and the GraalPyResources classes.

1. Getting Started

In this guide, we will add a small Python library to generate QR codes to a Java GUI application: Screenshot of the app

2. What you will need

To complete this guide, you will need the following:

  • Some time on your hands
  • A decent text editor or IDE
  • A supported JDK1, preferably the latest GraalVM JDK

3. Solution

We recommend that you follow the instructions in the next sections and create the application step by step. However, you can go right to the completed example.

4. Writing the application

You can start with any Maven or Gradle application that runs on JDK 17 or newer. We will demonstrate on both build systems. A default Maven application generated from an archetype.

mvn archetype:generate -DarchetypeGroupId=org.apache.maven.archetypes \
  -DarchetypeArtifactId=maven-archetype-quickstart -DarchetypeVersion=1.5 \
  -DgroupId=example -DartifactId=javase -Dpackage=org.example \
  -Dversion=1.0-SNAPSHOT -DinteractiveMode=false

And a default Gradle Java application generated with the init task.

gradle init --type java-application --dsl kotlin --test-framework junit-jupiter \
    --package org.example --project-name javase --java-version 17 \
    --no-split-project --no-incubating

4.1 Dependency configuration

Add the required dependencies for GraalPy in the <dependencies> section of the POM or to the dependencies block in the build.gradle.kts file.

pom.xml

<dependencies>
  <dependency>
    <groupId>org.graalvm.polyglot</groupId>
    <artifactId>python</artifactId> <!-- ① -->
    <version>24.1.2</version>
    <type>pom</type> <!-- ② -->
  </dependency>

  <dependency>
    <groupId>org.graalvm.python</groupId>
    <artifactId>python-embedding</artifactId> <!-- ③ -->
      <version>24.1.2</version>
  </dependency>
</dependencies>

build.gradle.kts

dependencies {
  implementation("org.graalvm.python:python:24.1.2") // ①
  implementation("org.graalvm.python:python-embedding:24.1.2") // ③
}

❶ The python dependency is a meta-package that transitively depends on all resources and libraries to run GraalPy.

❷ Note that the python package is not a JAR - it is simply a pom that declares more dependencies.

❸ The python-embedding dependency provides the APIs to manage and use GraalPy from Java.

4.2 Adding packages

Most Python packages are hosted on PyPI and can be installed via the pip tool. The Python ecosystem has conventions about the filesystem layout of installed packages that need to be kept in mind when embedding into Java. You can use the GraalPy plugins for Maven or Gradle to manage Python packages for you.

pom.xml

<build>
  <plugins>
    <plugin>
      <groupId>org.graalvm.python</groupId>
      <artifactId>graalpy-maven-plugin</artifactId>
      <version>24.1.2</version>
      <executions>
        <execution>
          <configuration>
            <packages> <!-- ① -->
              <package>qrcode==7.4.2</package>
            </packages>
            <pythonHome> <!-- ② -->
              <includes>
              </includes>
              <excludes>
                <exclude>.*</exclude>
              </excludes>
            </pythonHome>
            <pythonResourcesDirectory> <!-- ③ -->
              ${project.basedir}/python-resources
            </pythonResourcesDirectory>
          </configuration>
          <goals>
            <goal>process-graalpy-resources</goal>
          </goals>
        </execution>
      </executions>
    </plugin>
  </plugins>
</build>

build.gradle.kts

plugins {
    application
    id("org.graalvm.python") version "24.1.2"
}

graalPy {
    packages = setOf("qrcode==7.4.2") // ①
    pythonHome { includes = setOf(); excludes = setOf(".*") } // ②
    pythonResourcesDirectory = file("${project.projectDir}/python-resources") // ③
}

❶ The packages section lists all Python packages optionally with requirement specifiers. In this case, we install the qrcode package and pin it to version 7.4.2.

❷ The GraalPy plugin can copy the Python standard library resources. This is mainly useful when creating a GraalVM Native Image, a use-case that we are not going to cover right now. We disable this by specifying that we want to exclude all standard library files matching the regular expression .*, i.e., all of them, from the included Python home.

❸ We can specify where the plugin should place Python files for packages and the standard library that the application will use. Omit this section if you want to include the Python packages into the Java resources (and, for example, ship them in the Jar). Later in the Java code we can configure the GraalPy runtime to load the package from the filesystem or from resources.

Note that due to a bug in the 24.1.2 version of the org.graalvm.python plugin for Gradle you need to include a resource. A simple workaround is to add a src/main/resources/META-INF/MANIFEST.MF:

Manifest-Version: 1.0

4.3 Creating a Python context

GraalPy provides APIs to make setting up a context to load Python packages from Java as easy as possible.

GraalPy.java

package org.example;

import java.nio.file.Path;

import org.graalvm.polyglot.Context;
import org.graalvm.python.embedding.utils.*;

public class GraalPy {
    static VirtualFileSystem vfs;

    public static Context createPythonContext(String pythonResourcesDirectory) { // ①
        return GraalPyResources.contextBuilder(Path.of(pythonResourcesDirectory))
            .option("python.PythonHome", "") // ②
            .build();
    }

    public static Context createPythonContextFromResources() {
        if (vfs == null) { // ③
            vfs = VirtualFileSystem.newBuilder().allowHostIO(VirtualFileSystem.HostIO.READ).build();
        }
        return GraalPyResources.contextBuilder(vfs).option("python.PythonHome", "").build();
    }
}

❶ If we set the pythonResourcesDirectory property in our build config, we use this factory method to tell GraalPy where that folder is at runtime.

❷ We excluded all of the Python standard library from the resources in our build config. The GraalPy VirtualFileSystem is set up to ship even the standard library in the resources. Since we did not include any standard library, we set the "python.PythonHome" option to an empty string.

❸ If we do not set the pythonResourcesDirectory property, the GraalPy Maven plugin will place the packages inside the Java resources. Because Python libraries assume they are running from a filesystem, not a resource location, GraalPy provides the VirtualFileSystem, and API to make Java resource locations available to Python code as if it were in the real filesystem. VirtualFileSystem instances can be configured to allow different levels of through-access to the underlying host filesystem. In this demo we use the same VirtualFileSystem instance in multiple Python contexts.

4.3 Using a Python library from Java

After reading the qrcode docs, we can write Java interfaces that match the Python types we want to use and methods we want to call on them. GraalPy makes it easy to access Python objects via these interfaces. Java method names are mapped directly to Python method names. Return values are mapped according to a set of generic rules. The names of the interfaces can be chosen freely, but it makes sense to base them on the Python types, as we do below.

QRCode.java

package org.example;

interface QRCode {
    PyPNGImage make(String data);

    interface PyPNGImage {
        void save(IO.BytesIO bio);
    }
}

IO.java

package org.example;

import org.graalvm.polyglot.io.ByteSequence;

interface IO {
    BytesIO BytesIO();

    interface BytesIO {
        ByteSequence getvalue();
    }
}

Using these interfaces and the GraalPy class, we can now create QR-codes and show them in, for example, a JLabel.

App.java

package org.example;

import java.io.*;
import javax.imageio.ImageIO;
import javax.swing.*;

public class App {
    public static void main(String[] args) throws IOException {
        if (System.getProperty("graalpy.resources") == null) {
            System.err.println("Please provide 'graalpy.resources' system property.");
            System.exit(1);
        }
        try (var context = GraalPy.createPythonContext(System.getProperty("graalpy.resources"))) { // ①
            QRCode qrCode = context.eval("python", "import qrcode; qrcode").as(QRCode.class); // ②
            IO io = context.eval("python", "import io; io").as(IO.class);

            IO.BytesIO bytesIO = io.BytesIO(); // ③
            qrCode.make("Hello from GraalPy on JDK " + System.getProperty("java.version")).save(bytesIO);

            var qrImage = ImageIO.read(new ByteArrayInputStream(bytesIO.getvalue().toByteArray())); // ④
            JFrame frame = new JFrame("QR Code");
            frame.getContentPane().add(new JLabel(new ImageIcon(qrImage)));
            frame.setDefaultCloseOperation(WindowConstants.EXIT_ON_CLOSE);
            frame.setSize(400, 400);
            frame.setVisible(true);
        }
    }
}

❶ If we do not want to ship the directory with the Python package separately and pass the location at runtime, we can embed them and use the virtual filesystem constructor.

❷ Python objects are returned using a generic Value type. We cast the io and qrcode packages to our declared interfaces so we can use Java typing and IDE completion features.

❸ Method calls on our interfaces are transparently forwarded to the Python objects, arguments and return values are coerced automatically.

❹ Python code returns the generated PNG as an array of unsigned bytes, which we can process on the Java side.

5. Running the application

If you followed along with the example, you can now compile and run your application from the commandline:

With Maven:

./mvnw compile
./mvnw exec:java -Dexec.mainClass=org.example.App -Dgraalpy.resources=./python-resources

With Gradle:

Update the build script to pass the necessary Java property to the application:

build.gradle.kts

application {
    mainClass = "org.example.App"
    applicationDefaultJvmArgs = listOf("-Dgraalpy.resources=" + System.getProperty("graalpy.resources"))
}

Run from command line:

./gradlew assemble
./gradlew run

6. Next steps

Footnotes

  1. Oracle JDK 17 and OpenJDK 17 are supported with interpreter only. GraalVM JDK 21, Oracle JDK 21, OpenJDK 21 and newer with JIT compilation. Note: GraalVM for JDK 17 is not supported. ↩