TornadoVM 7 Enables GPU-Native Workloads From Java Applications

TornadoVM, an open-source heterogeneous GPU programming framework for Java has reached version 7.3, a major milestone for the project.

InfoQ previously covered the project in December 2025 and in 2022.

TornadoVM automatically accelerates Java programs on multi-core CPUs and GPUs. Rather than replacing existing JVMs, it works alongside them and provides a rich GPU programming ecosystem in Java that you can use with any mainstream JDK distribution such as OpenJDK, Graalvm, Temurin, or Zulu.

It offloads Java code to hardware backends, manages memory between the Java heap and accelerators, and runs the resulting compute kernels. This makes it a useful building block for modern cloud and machine learning workloads.

During development, TornadoVM compiles the source Java code to OpenCL and Apple Metal backends. The idea is that you can develop locally on your laptop and use the hardware available locally, and then deploy into a server that has Nvidia GPUs, and achieve major performance improvements - a form of "Write Once, Run Anywhere" for modern ML workloads.

Version 7 brings several architectural changes. The backends now use the Foreign Function & Memory (FFM) API in place of the Java Native Interface (JNI). The experimental JVM Compiler Interface (JVMCI) dependency has also been removed. According to the team, TornadoVM now sources "all compiler metadata through a reflection + ASM + Unsafe layer" which is the single, uniform path on every supported JDK (21, 25, 26 and 27).

The release also deepens integration with the Nvidia ecosystem through an optimized CUDA backend. This includes support for the cuTile programming model (via a new TileContext) and the ability to call CUDA libraries such as cuBLAS and CUTLASS directly from within TornadoVM TaskGraphs. Developers can swap JIT-generated kernels for CUDA library calls, or chain the two together, depending on their performance and portability needs. Similarly, TornadoVM adds the same functionality for Apple's MXL library again via the hybrid API. Finally, support for multi-GPU execution has been added.

Not every Java computation benefits from offloading. The best candidates are data-parallel workloads, such as loops with no dependencies between iterations, because their iterations can run in parallel on the accelerator.

One of the goals of recent TornadoVM versions is to produce an idiomatic JVM environment for accessing the libraries that Nvidia have been tuning for different hardware platforms. The aim is for Java developers to tap into things like Linear Algebra, Fast Fourier Transforms (for Signal Analysis), Deep Neural Networks and hand-optimized matrix multiplication for sparse matrices. TornadoVM has library integrations with Quarkus, Lang4j, Flink, Spring AI and others and is also available through SDKMan.

For example, consider the integration of Apache Flink with TornadoVM - the main idea is that the developer writes a standard query as they normally would with the Table API. A configuration flag is used to indicate that this query should potentially be accelerated on a GPU.

Then, when the computation reaches the task manager, the framework checks if there is a GPU available in the node it is executing on and, if so, it connects to a library provider - which analyzes the intermediate representation (IR) of the query to see if the computation is actually suitable for GPU execution.

In total, TornadoVM offers four ways to program Nvidia GPUs as a Java programmer - Firstly, the parallel API which is really just an annotation that annotates a loop, and provides a hint for the compiler that this loop can be potentially paralyzed. This simple API did not prove to be sufficient in the long-term to get the desired performance for production applications, so additional APIs were added - such as the kernel context API and the already-mentioned TileContext.

Alongside the 7.3 release, the TornadoVM team introduced jitLLM v2.0, its next-generation Java AI inference engine built on TornadoVM. The team positions jitLLM as a direct competitor to llama.cpp and Ollama in performance, and it ships as a single JAR file.

Both TornadoVM and jitLLM are being commercialized through Beestream.ai, a spin-off founded by the TornadoVM team.

Further details can be found on the TornadoVM site, and also in recent videos from Devoxx Belgium, including: "Tap into the NVIDIA Ecosystem from Java with TornadoVM's Hybrid API".