Java Virtual Threads: Why, not What?
Java Virtual Thread is the new concurrency paradigm in Java, developed under the Project Loom umbrella, along with JEP 437: Structured Concurrency and JEP 429: Scoped Values features. Virtual threads are lightweight enough to run millions of threads concurrently. The main objective of Virtual threads is to allow Java developers to achieve high-throughput servers with a simple ‘thread-per-request’ style code, instead of dealing with different kinds of asynchronous programming. The ‘thread-per-request’ coding style enables developers to have excellent tooling and observability.
The purpose of this post is not to introduce Java Virtual Threads but to discuss why Java Virtual Threads matter in our Java ecosystem and why they are important. Before that, let’s have a quick recap of Traditional Java Threads and the Asynchronous programming paradigm in Java.
Traditional Java Thread
Java’s traditional thread is a sort of abstraction of an OS thread since it is just a simple wrapper around the OS thread. However, it’s costly and limits us from achieving high throughput because Java threads have become a limited resource, often restricted to 1000–10,000 threads in most systems.
To illustrate this, try running Thread.sleep(0) * 1000 times sequentially and then using Java threads. You will clearly see how Java threads are more costly compared to single-thread sequential results. (See Table: 1)
The following Java Threads code snippet is from the benchmarking project we used to compare Java concurrency options.
import java.time.Duration;
import java.time.Instant;
public class ThreadAction {
public Long runThreadAction(int iterations, long sleepTime){
Thread threads[] = new Thread[iterations];
Instant actionStart = Instant.now();
// Create and start multiple threads in a loop
for (int i = 0; i < iterations; i++) {
final int index = i;
threads[i] = new Thread(() -> {
// Perform the task within the thread
// Simulate some work
try {
Thread.sleep(sleepTime);
} catch (InterruptedException e) {
e.printStackTrace();
}
});
threads[i].start(); // Start the thread
}
for (Thread thread : threads) {
try {
thread.join(); // Wait for thread to complete
} catch (InterruptedException e) {
e.printStackTrace();
}
}
return Duration.between(actionStart, Instant.now()).toMillis();
}
}Asynchronous Programming
Since Java Thread is so costly, Java has moved towards Asynchronous programming to achieve higher throughput with smaller units of concurrency. However, Async programming also has its limitations. One of the most impactful drawbacks of Async programming is that you have to rewrite your entire solution to adapt to it. Additionally, poor observability is another drawback of asynchronous programming.
The following Java Completable Future asynchronous programming code snippet from the benchmarking project we used to compare Java concurrency options.
import java.time.Duration;
import java.time.Instant;
import java.util.concurrent.CompletableFuture;
import java.util.concurrent.ExecutionException;
public class CompletableFutureAction {
public Long runCompletableFutureAction(int iterations, long sleepTime) {
// Create an array of CompletableFuture
CompletableFuture<String>[] completableFutures = new CompletableFuture[iterations];
Instant actionStart = Instant.now();
// Execute tasks asynchronously in a loop
for (int i = 0; i < iterations; i++) {
final int index = i;
completableFutures[i] = CompletableFuture.supplyAsync(() -> {
try {
Thread.sleep(sleepTime);
} catch (InterruptedException e) {
throw new RuntimeException(e);
}
return "Task " + index;
});
}
// Wait for all CompletableFuture to complete and collect the results
CompletableFuture<Void> allFutures = CompletableFuture.allOf(completableFutures);
// Get the results from all CompletableFuture
try {
allFutures.get(); // Wait for all tasks to complete
for (CompletableFuture<String> future : completableFutures) {
future.get();
}
} catch (InterruptedException | ExecutionException e) {
e.printStackTrace();
}
return Duration.between(actionStart, Instant.now()).toMillis();
}
}Java Virtual Thread
To avoid the cost of platform threads and the complexity and observability limitations of Asynchronous programming, Project Loom introduced Virtual threads. These Virtual threads retain the traditional flavor of platform threads but with a smaller cost. Ultimately, this approach allows us to achieve high throughput with low cost, less complexity, and a maintainable codebase.
The following Java Virtual Thread code snippet from the benchmarking project we used to compare Java concurrency options.
import java.time.Duration;
import java.time.Instant;
import java.util.concurrent.ExecutionException;
import java.util.concurrent.ExecutorService;
import java.util.concurrent.Executors;
import java.util.concurrent.Future;
public class VirtualThreadAction {
public Long runVirtualThreadAction(int iterations, long sleepTime) {
ExecutorService executor = Executors.newVirtualThreadPerTaskExecutor();
Future<?>[] virtualThreads = new Future[iterations];
Instant actionStart = Instant.now();
// Create and start multiple threads in a loop
for (int i = 0; i < iterations; i++) {
final int index = i;
virtualThreads[i] = executor.submit(() -> {
// Perform the task within the thread
// Simulate some work
try {
Thread.sleep(sleepTime);
} catch (InterruptedException e) {
e.printStackTrace();
}
});
}
for (Future thread : virtualThreads) {
try {
thread.get(); // Wait for thread to complete
} catch (InterruptedException | ExecutionException e) {
e.printStackTrace();
}
}
return Duration.between(actionStart, Instant.now()).toMillis();
}
}NOTE: You should have Java version 19 or 20 to run the above code. Since Java Virtual Thread is a preview feature, make sure to provide the --enable-preview option when you run it. You can ignore this message if you are running on a version higher than Java 20.Benchmark — Sequential vs Threads vs Async vs Virtual Threads
Let’s evaluate the performance of Sequential, Platform Threads, Asynchronous Programs, and the newly introduced Virtual Threads in different scenarios.
For the benchmarking project, we have three different actions. We used Thread.sleep() with different parameters to evaluate how context switching is costly in sequential vs. concurrent approaches. To simulate IO-intensive tasks, we read 1000 text files containing 1000 string lines each. Additionally, to simulate CPU-intensive tasks, we calculated the sum of squares from 1 to 1000000.
TL;DR — Little’s law
Let’s take a look at the theoretical relationship between the number of threads, requests per second, and response time. This is because we should follow the ‘measure, don’t assume’ approach in engineering life.
L = λW
λ = Number of requests reached in second
W = Number of Requests that can be processed in one second
L = ConcurrencyIn a nutshell, let’s assume we have a request that takes 900ms to process. We receive that kind of request 30,000 times in any given second. In order to serve without a bottleneck, we can apply Little’s law and check what level of concurrency is required in the system.
λ = 30000 Req/Sec
W = 900ms
Apply Little's Law
L = λW
L = 1000 * 0.05S
= 27000If we consider the above scenario, in order to serve without a bottleneck, we need 27,000 threads. If we use traditional Java Threads, we will encounter limitations in thread creation because the creation of the thread 1:1 is coupled to the platform thread. The number of platform threads we can create depends on the system specifications. However, lightweight Virtual threads don’t have such a resource limitation!
If you are interested, you can have a look at the below project which I used to do the benchmarking.
Happy Coading…
