Overview

Java 21 (To be precise JDK 21) reached General Availability status on 19 September 2023. JDK 21 is also an LTS release, so it is safe to migrate to it. JDK 21 comes with a nice set of new features that have become GA, but also some interesting previews. This article focuses on Virtual Threads. But first, let’s take a glimpse at the whole JEP’s list:

  • JEP 444: Virtual Threads
  • JEP 431: Sequenced Collections
  • JEP 439: Generational ZGC
  • JEP 440: Record Patterns
  • JEP 441: Pattern Matching for switch
  • JEP 449: Deprecate the Windows 32-bit x86 Port for Removal
  • JEP 451: Prepare to Disallow the Dynamic Loading of Agents
  • JEP 452: Key Encapsulation Mechanism API

Preview releases:

  • JEP 453: Structured Concurrency (Preview)
  • JEP 430: String Templates (Preview)
  • JEP 442: Foreign Function & Memory API (Third Preview)
  • JEP 443: Unnamed Patterns and Variables (Preview)
  • JEP 445: Unnamed Classes and Instance Main Methods (Preview)
  • JEP 446: Scoped Values (Preview)
  • JEP 448: Vector API (Sixth Incubator)

Now back to Virtual threads…

JEP 444: Virtual Threads Intro

Java has regarded platform threads as lightweight abstractions built upon operating system (OS) threads (see drawing below). The creation of these platform threads incurred a significant cost due to the allocation of resources at the OS level (kernel). To mitigate this overhead, in Java, we traditionally used thread pools. Furthermore, it’s necessary to place constraints on the number of platform threads since these resource-intensive threads can potentially impact the overall machine performance. In effect Java is not itself managing the state of these threads, it rather relies on the OS in this regards with all the pros and cons. On one side Java doesn’t need to manage scheduling and blocking. On the other side of this, java process becomes dependent on the OS scheduling, handling of blocking I/O etc and cannot really make any assumption here. Also, the scheduling algorithms would differ from OS to OS.

Those limitations are grounded in the fact that platform threads are directly mapped on a one-to-one basis to OS threads.

Virtual threads introduce a solution that addresses this limitation by associating Java threads with carrier threads responsible for handling thread operations by mounting and unmounting them onto an OS thread (new thread scheduling mechanism). In contrast, the carrier thread interfaces directly with the OS thread, offering an abstraction layer that provides developers with enhanced flexibility and control.

Virtual Threads have been developed with the following main goals in mind:

  • Enable server applications written in the simple thread-per-request style to scale with near-optimal hardware utilization.
  • Enable existing code that uses the java.lang.Thread API to adopt virtual threads with minimal change.

Looks like the design goals were reached. Let’s consider why it might matter.

In cases where Operation ‘blocked’ the thread, such as InputStream#read(), Thread.sleep(long), the new runtime will detect the block and put the blocked operation aside to be monitored. Once the blocking cause (I/O) has finished, it’s put on any available thread, allowing the program to continue executing. The new key concept is that when a blocking operation is called, in a virtual thread, you no longer monopolize an actual operating system thread.

What the change

# DummyWorkload is Runnable in all examples

# Constructiong Virtual threads using Thread.Builder
Thread thread = Thread.ofVirtual()
                      .unstarted(new DummyWorkload("New virtual thread"));

#using ofVirtual()
Thread.ofVirtual().start(new DummyWorkload("Easy start"));

#starting directly
Thread.startVirtualThread(new DummyWorkload("Easy start"));

Here is also a very handy Executors API to run tasks as virtual threads.

try (var executor = Executors.newVirtualThreadPerTaskExecutor()) {
	for (int i = 0; i < numberOfThreads; i++) {
		xecutor.submit(new DummyWorkload());
	}
}

More working examples you’ll find here

Good to know

Virtual threads are cheap and plentiful, and thus should never be pooled: A new virtual thread should be created for every application task. Most virtual threads will thus be short-lived and have shallow call stacks, performing as little as a single HTTP client call or a single JDBC query. Platform threads, by contrast, are heavyweight and expensive, and thus often must be pooled. They tend to be long-lived, have deep call stacks, and be shared among many tasks.

Virtual threads support thread-local behavior the same way as platform threads, but because the virtual threads can be created in millions, thread-local variables should be used only after careful consideration. The use of semaphores should be fine.

The JDK’s virtual thread scheduler is a work-stealing ForkJoinPool that operates in FIFO mode. The parallelism of the scheduler is the number of platform threads available for the purpose of scheduling virtual threads. By default, it is equal to the number of available processors, but it can be tuned with the system property jdk.virtualThreadScheduler.parallelism. This ForkJoinPool is distinct from the common pool that is used, for example, in the implementation of parallel streams, and which operates in LIFO mode.

Furthermore

  • The public Thread constructors cannot create virtual threads.
  • Virtual threads are always daemon threads.
  • Virtual threads have a fixed priority of Thread.NORM_PRIORITY. The Thread.setPriority(int) method has no effect on virtual threads. This limitation may be revisited in a future release.
  • Virtual threads are not active members of thread groups. When invoked on a virtual thread, Thread.getThreadGroup() returns a placeholder thread group with the name “VirtualThreads”.
  • Virtual threads have no permissions when running with a SecurityManager set.

Virtual Threads impact

Using virtual threads does not require learning new concepts, though it may require unlearning habits developed to cope with today’s high cost of threads. Virtual threads will not only help application developers — they will also help framework designers provide easy-to-use APIs that are compatible with the platform’s design without compromising on scalability.

Spring Support

Spring Framework 6.1 which focuses on JDK 21 has become GA around mid of November 23. and it brings:

Particularly with Spring Boot 3.2, you can specify spring.threads.virtual.enabled=true and Spring Boot will replace the ExecutorService in use in places like Jetty, Tomcat, Kafka, RabbitMQ, and others, with a virtual-thread-backed ExecutorService.

Note that Spring Framework 6.1 provides a first-class experience on JDK 21 and Jakarta EE 10 at runtime while retaining a JDK 17 and Jakarta EE 9 baseline in the source code. Spring also tracks the evolution of GraalVM for JDK 21 with refined metadata inference while remaining compatible with GraalVM 22.3 for the time being.