Java 23: Virtual Threads and Structured Concurrency in Production

What’s New in Java 23

Java 23, released in September 2024, marks a turning point for concurrent programming on the JVM. Virtual threads—lightweight threads managed by the Java runtime instead of the OS—are now a stable, production-ready feature. Combined with structured concurrency APIs, Java 23 enables developers to write highly scalable server applications without the complexity of reactive frameworks or thread pools.

This release addresses one of Java’s longest-standing challenges: how to scale I/O-bound workloads efficiently. Traditional Java threads are expensive (each OS thread consumes ~2MB of memory), making it impractical to spawn thousands of them. Virtual threads solve this by allowing millions of lightweight, user-mode threads that are automatically scheduled by the runtime.

Understanding Virtual Threads

Virtual threads are Java’s answer to Python’s asyncio and Go’s goroutines. Unlike goroutines or async/await syntax, virtual threads preserve the familiar synchronous programming model while providing near-unlimited concurrency.

The Problem Virtual Threads Solve

Consider a traditional web server handling 10,000 concurrent requests. With platform threads (OS threads), you’d need 10,000 threads, each consuming 2-4MB of memory:

// Traditional approach: one platform thread per request
ExecutorService executor = Executors.newFixedThreadPool(100);

for (int i = 0; i < 10000; i++) {
    executor.submit(() -> {
        // Handle HTTP request
        String response = fetchDataFromDatabase();
        String enriched = callExternalAPI(response);
        sendResponse(enriched);
    });
}

This approach creates a bottleneck: you can only create ~100 threads before memory and context-switching overhead become prohibitive. Any additional requests queue up, waiting for a thread to become available.

With virtual threads, you can spawn millions:

// Java 23: virtual threads
ExecutorService executor = Executors.newVirtualThreadPerTaskExecutor();

for (int i = 0; i < 1_000_000; i++) {
    executor.submit(() -> {
        // Handle HTTP request
        String response = fetchDataFromDatabase();
        String enriched = callExternalAPI(response);
        sendResponse(enriched);
    });
}

Each virtual thread costs only ~100 bytes of memory. The runtime automatically schedules virtual threads onto platform threads, pausing them when they block on I/O (database calls, HTTP requests) and resuming them when I/O completes.

How Virtual Threads Work

Virtual threads are implemented using continuations—the runtime captures and pauses execution when a virtual thread blocks on I/O:

1. Creation: Thread.ofVirtual().start(() -> { ... })
2. Blocking I/O: Virtual thread calls database.query() → runtime captures state and pauses
3. Scheduling: Runtime parks the virtual thread, schedules another on the freed platform thread
4. I/O Completion: When I/O finishes, the virtual thread resumes execution

This happens transparently—your code remains synchronous and straightforward.

Practical Example: Building a High-Concurrency Web Service

Let’s build a simple microservice that fetches user data from multiple sources concurrently:

import java.net.http.HttpClient;
import java.net.http.HttpRequest;
import java.net.http.HttpResponse;
import java.util.concurrent.ExecutorService;
import java.util.concurrent.Executors;
import java.util.stream.IntStream;

public class UserDataAggregator {
    private static final HttpClient client = HttpClient.newBuilder()
        .version(HttpClient.Version.HTTP_2)
        .build();
    
    public static void main(String[] args) throws Exception {
        // Create executor with virtual threads
        ExecutorService executor = Executors.newVirtualThreadPerTaskExecutor();
        
        // Simulate 10,000 concurrent requests
        IntStream.range(0, 10000).forEach(userId -> {
            executor.submit(() -> handleUserRequest(userId));
        });
        
        executor.close();
    }
    
    static void handleUserRequest(int userId) {
        try {
            // Fetch from multiple APIs concurrently
            String userData = fetchUserProfile(userId);
            String orders = fetchUserOrders(userId);
            String preferences = fetchUserPreferences(userId);
            
            System.out.println("User " + userId + ": " + 
                userData + " | " + orders + " | " + preferences);
        } catch (Exception e) {
            e.printStackTrace();
        }
    }
    
    static String fetchUserProfile(int userId) throws Exception {
        HttpRequest request = HttpRequest.newBuilder()
            .uri(new java.net.URI("https://api.example.com/users/" + userId))
            .GET()
            .build();
        
        // This blocks the virtual thread, not the platform thread
        HttpResponse<String> response = client.send(request, 
            HttpResponse.BodyHandlers.ofString());
        return response.body();
    }
    
    static String fetchUserOrders(int userId) throws Exception {
        // Similar implementation
        return "[orders]"; 
    }
    
    static String fetchUserPreferences(int userId) throws Exception {
        // Similar implementation
        return "[preferences]";
    }
}

Running this code with Java 23, you can comfortably handle 10,000 concurrent requests. Compare this to pre-virtual-thread Java: you’d hit memory limits or use a reactive framework like Spring WebFlux (which requires learning async/await patterns).

Structured Concurrency: Managing Complex Task Workflows

While virtual threads make concurrency cheap, managing multiple concurrent tasks reliably is still hard. Structured concurrency (in preview in Java 23) solves this by providing a clear, parent-child task relationship.

The Problem: Unstructured Concurrency

Traditional approaches make it hard to reason about when tasks complete:

List<Future<?>> futures = new ArrayList<>();
futures.add(executor.submit(() -> task1()));
futures.add(executor.submit(() -> task2()));
futures.add(executor.submit(() -> task3()));

// Did all tasks finish? Were there exceptions?
// Hard to reason about—futures can leak, exceptions get lost
for (Future<?> f : futures) {
    try {
        f.get(); // Blocks indefinitely if something goes wrong
    } catch (Exception e) {
        // Handle exception
    }
}

The Solution: StructuredTaskScope

Structured concurrency uses StructuredTaskScope to enforce a clear parent-child relationship:

import java.util.concurrent.StructuredTaskScope;

public class ConcurrentUserFetch {
    
    public record UserData(
        String profile,
        String orders,
        String preferences
    ) {}
    
    static UserData fetchUserConcurrently(int userId) throws Exception {
        // Task scope auto-closes and ensures all child tasks complete
        try (var scope = new StructuredTaskScope.ShutdownOnFailure()) {
            // Submit child tasks
            var profileTask = scope.fork(() -> fetchUserProfile(userId));
            var ordersTask = scope.fork(() -> fetchUserOrders(userId));
            var prefsTask = scope.fork(() -> fetchUserPreferences(userId));
            
            // Join all tasks (with timeout safety)
            scope.joinUntil(Instant.now().plusSeconds(5));
            
            // If any task failed, exception is thrown automatically
            return new UserData(
                profileTask.resultNow(),
                ordersTask.resultNow(),
                prefsTask.resultNow()
            );
        }
    }
    
    public static void main(String[] args) throws Exception {
        UserData data = fetchUserConcurrently(42);
        System.out.println(data);
    }
    
    static String fetchUserProfile(int userId) throws Exception {
        Thread.sleep(100); // Simulate I/O
        return "Profile for user " + userId;
    }
    
    static String fetchUserOrders(int userId) throws Exception {
        Thread.sleep(150);
        return "Orders for user " + userId;
    }
    
    static String fetchUserPreferences(int userId) throws Exception {
        Thread.sleep(80);
        return "Prefs for user " + userId;
    }
}

Key benefits:

• Automatic scope closure: Tasks are guaranteed to complete before the scope exits
• Exception propagation: If any task fails, the exception propagates immediately
• Timeout safety: Built-in timeout prevents indefinite waits
• Clear semantics: Parent-child relationship makes code intent obvious

Getting Started with Java 23

Installation

Download Java 23 from oracle.com or use a package manager:

# macOS with Homebrew
brew install openjdk@23

# Linux (Ubuntu/Debian)
sudo apt-get install openjdk-23-jdk

# Verify installation
java -version

Enable Preview Features

Structured concurrency is still in preview, so enable preview features:

# Run with preview features
java --enable-preview UserDataAggregator.java

# Or in Maven
<build>
    <plugins>
        <plugin>
            <groupId>org.apache.maven.plugins</groupId>
            <artifactId>maven-compiler-plugin</artifactId>
            <version>3.11.0</version>
            <configuration>
                <source>23</source>
                <target>23</target>
                <compilerArgs>
                    <arg>--enable-preview</arg>
                </compilerArgs>
            </configuration>
        </plugin>
    </plugins>
</build>

Common Pitfalls and Best Practices

Pitfall 1: Blocking on CPU-Intensive Work

Virtual threads excel at I/O concurrency, not CPU concurrency. Never block a virtual thread on CPU-bound work:

// BAD: CPU-bound work blocks the virtual thread
executor.submit(() -> {
    int sum = 0;
    for (int i = 0; i < 1_000_000_000; i++) {
        sum += i; // CPU work
    }
});

// GOOD: Use ForkJoinPool for CPU-bound work
ForkJoinPool.commonPool().submit(() -> {
    int sum = 0;
    for (int i = 0; i < 1_000_000_000; i++) {
        sum += i;
    }
});

Pitfall 2: Neglecting Exception Handling

With thousands of concurrent tasks, exception handling becomes critical:

// BAD: Exceptions silently lost
try (var scope = new StructuredTaskScope.ShutdownOnFailure()) {
    scope.fork(() -> riskyOperation());
    scope.join(); // Missing exception check
}

// GOOD: Explicit exception handling
try (var scope = new StructuredTaskScope.ShutdownOnFailure()) {
    scope.fork(() -> riskyOperation());
    scope.joinUntil(Instant.now().plusSeconds(10));
} catch (IOException e) {
    logger.error("Task failed", e);
    // Handle gracefully
}

Pitfall 3: Thread-Local Variables

Be cautious with thread-local storage—each virtual thread gets its own copy:

private static final ThreadLocal<Connection> dbConnection = 
    ThreadLocal.withInitial(() -> createConnection());

// Each virtual thread gets a separate connection
// This can exhaust connection pools quickly!
executor.submit(() -> {
    Connection conn = dbConnection.get(); // Creates new connection per thread
});

For 1 million virtual threads, this creates 1 million database connections—likely causing a connection pool exhaustion.

Why Virtual Threads Matter Now

Before Java 21 (Virtual Threads Preview)

Scaling Java servers required one of these approaches:

1. Reactive frameworks (Spring WebFlux, Vert.x): Learn new async patterns, higher cognitive load
2. Thread pools: Limited concurrency, context-switching overhead
3. Other languages: Abandon Java’s ecosystem and tooling

After Java 23 (Virtual Threads Production-Ready)

You can write synchronous, readable code that scales to millions of concurrent connections using only Java’s standard library.

Real-World Impact

Companies like Uber and Twitter have already tested virtual threads at scale. Preliminary benchmarks show:

• Memory usage: 10-100x reduction per concurrent task
• Throughput: 2-5x improvement on I/O-bound workloads
• Response latency: Reduced tail latencies due to better scheduling

Testing Virtual Thread Applications

When testing concurrent code, use Mock Data Generator to create realistic test scenarios, and validate API responses with JSON Formatter for correctness.

Here’s a minimal test harness:

@Test
void testConcurrentUserFetch() throws Exception {
    ExecutorService executor = Executors.newVirtualThreadPerTaskExecutor();
    List<Future<String>> results = new ArrayList<>();
    
    // Submit 1000 tasks
    for (int i = 0; i < 1000; i++) {
        results.add(executor.submit(() -> fetchUserConcurrently(42)));
    }
    
    // Verify all completed successfully
    for (Future<String> result : results) {
        assertNotNull(result.get(5, TimeUnit.SECONDS));
    }
    
    executor.close();
}

Migration Path from Pre-Java 23

If you’re running Java 17 or 21, adopting Java 23 is straightforward:

1. Update Maven/Gradle:

   <maven.compiler.source>23</maven.compiler.source>
   <maven.compiler.target>23</maven.compiler.target>

2. Replace thread pool creation:

   // Before
   ExecutorService executor = Executors.newFixedThreadPool(100);
   
   // After
   ExecutorService executor = Executors.newVirtualThreadPerTaskExecutor();

3. Test thoroughly: Virtual threads expose concurrency bugs that platform threads masked. Use Diff Checker to compare benchmark results before/after migration.

4. Monitor carefully: Track virtual thread counts, pinning events (when a virtual thread blocks a platform thread), and task completion times.

Debugging Virtual Threads

Java 23 includes enhanced debugging support. View virtual thread stacks with:

jstack <pid> | grep "virtual"

Or in JFR (Java Flight Recorder):

jcmd <pid> JFR.start name=vthread_profile duration=30s
jcmd <pid> JFR.dump name=vthread_profile filename=profile.jfr
jfr view profile.jfr

Conclusion

Java 23 represents a maturation of the JVM’s concurrency model. Virtual threads and structured concurrency enable developers to write scalable, maintainable code without sacrificing readability or diving into reactive frameworks.

For I/O-bound applications—microservices, APIs, data pipelines—Java 23 is now a compelling choice alongside Go, Rust, and Node.js. The learning curve is minimal if you’re already familiar with Java, and the performance gains are substantial.