Java 23 Virtual Threads and Structured Concurrency: A Deep Dive for Backend Developers

Why Java 23 Matters for Concurrency

Java 23, released in September 2024, marks a turning point in how Java developers approach concurrent programming. For decades, Java’s threading model relied on expensive OS threads—creating thousands of them would quickly exhaust system resources. Virtual threads change this fundamentally, allowing millions of lightweight threads to run on a small number of OS threads.

This isn’t a minor optimization. Structured concurrency and virtual threads represent the biggest shift in Java’s concurrency story since the introduction of the java.util.concurrent package in Java 5. If you’re building APIs, microservices, or any I/O-bound backend systems, understanding these features is critical.

What Are Virtual Threads?

A virtual thread is a lightweight thread managed by the Java runtime rather than the operating system. Virtual threads are cheap to create, require minimal memory overhead (~100 bytes vs ~2MB for platform threads), and the JVM can schedule thousands of them on a handful of OS threads.

Here’s the key insight: you no longer need to optimize thread creation. You can write code as if spawning a thread is free, because it essentially is.

Creating Virtual Threads

In Java 19–22, virtual threads were a preview feature. Java 23 makes them finalized—you can use them in production without the --enable-preview flag.

Before Java 23, you’d create threads like this:

// Platform thread (expensive)
Thread platformThread = new Thread(() -> {
    System.out.println("Running on OS thread");
});
platformThread.start();

Virtual threads use the same API but run on the lightweight scheduler:

// Virtual thread (cheap, production-ready in Java 23)
Thread virtualThread = Thread.ofVirtual().start(() -> {
    System.out.println("Running on virtual thread");
});

// Wait for completion
virtualThread.join();

Or using the ThreadFactory pattern:

ThreadFactory virtualFactory = Thread.ofVirtual().factory();
ExecutorService executor = Executors.newThreadPerTaskExecutor(virtualFactory);

// Submit 10,000 tasks—no problem
for (int i = 0; i < 10000; i++) {
    executor.submit(() -> {
        System.out.println("Task running on virtual thread");
    });
}

executor.shutdown();
executor.awaitTermination(1, TimeUnit.MINUTES);

Structured Concurrency: The Game Changer

Structured concurrency is about imposing structure on concurrent programs. Instead of spawning threads that live for arbitrary periods, you organize them into well-defined scopes with clear entry and exit points.

Without structure, concurrent code becomes unpredictable:

// Without structure: threads escape their logical scope
ExecutorService executor = Executors.newFixedThreadPool(4);

Future<String> result1 = executor.submit(() -> fetchUserData(userId));
Future<String> result2 = executor.submit(() -> fetchOrders(userId));

// What if one throws an exception? 
// What if the main thread exits before these complete?
// The executor keeps running in the background...

Structured concurrency changes this with the StructuredTaskScope API:

// With structure: tasks are scoped to a block
try (var scope = new StructuredTaskScope.ShutdownOnFailure()) {
    Callable<String> userData = scope.fork(() -> fetchUserData(userId));
    Callable<String> orders = scope.fork(() -> fetchOrders(userId));
    
    // Wait for all to complete; cancel others on failure
    scope.joinUntil(Instant.now().plusSeconds(5));
    
    return new UserProfile(
        userData.resultNow(),
        orders.resultNow()
    );
} catch (TimeoutException e) {
    System.err.println("Request timed out");
    throw e;
}

Notice the try-with-resources: the scope automatically cancels any remaining tasks when the block exits. This prevents task leaks and ensures proper cleanup.

Step-by-Step Guide: Building a Concurrent Web Service

Let’s build a practical example: a user service that fetches data from multiple sources concurrently.

1. Set Up the Environment

Ensure you’re on Java 23 or later:

java -version
# openjdk version "23" 2024-09-17

2. Create a Data Fetcher Service

import java.util.concurrent.Callable;

public class DataService {
    
    public String fetchUserData(int userId) {
        // Simulate network call
        try {
            Thread.sleep(100);
        } catch (InterruptedException e) {
            Thread.currentThread().interrupt();
        }
        return "User #" + userId;
    }
    
    public String fetchUserOrders(int userId) {
        try {
            Thread.sleep(150);
        } catch (InterruptedException e) {
            Thread.currentThread().interrupt();
        }
        return "3 orders for user #" + userId;
    }
    
    public String fetchUserPreferences(int userId) {
        try {
            Thread.sleep(80);
        } catch (InterruptedException e) {
            Thread.currentThread().interrupt();
        }
        return "Preferences for user #" + userId;
    }
}

3. Use Structured Concurrency

import java.util.concurrent.Callable;
import java.util.concurrent.StructuredTaskScope;
import java.time.Instant;
import java.util.concurrent.TimeoutException;

public class UserProfileService {
    private final DataService dataService = new DataService();
    
    public UserProfile getUserProfile(int userId) throws TimeoutException, InterruptedException {
        try (var scope = new StructuredTaskScope.ShutdownOnFailure()) {
            // Fork three concurrent tasks
            Callable<String> userData = scope.fork(() -> dataService.fetchUserData(userId));
            Callable<String> orders = scope.fork(() -> dataService.fetchUserOrders(userId));
            Callable<String> preferences = scope.fork(() -> dataService.fetchUserPreferences(userId));
            
            // Wait for all with timeout
            scope.joinUntil(Instant.now().plusSeconds(2));
            
            return new UserProfile(
                userData.resultNow(),
                orders.resultNow(),
                preferences.resultNow()
            );
        }
    }
    
    record UserProfile(String user, String orders, String preferences) {}
}

4. Benchmark the Difference

Sequential version:

long start = System.nanoTime();
String user = dataService.fetchUserData(1);
String orders = dataService.fetchUserOrders(1);
String prefs = dataService.fetchUserPreferences(1);
long sequential = System.nanoTime() - start;
// ~330ms (100 + 150 + 80)

Structured concurrency version:

long start = System.nanoTime();
var profile = userProfileService.getUserProfile(1);
long concurrent = System.nanoTime() - start;
// ~150ms (max of 150, 100, 80)

That’s a 2x speedup with minimal code complexity.

Advanced Patterns: Handling Exceptions

Structured concurrency provides different exception handling strategies:

ShutdownOnFailure

Cancels remaining tasks if any fails:

try (var scope = new StructuredTaskScope.ShutdownOnFailure()) {
    scope.fork(() -> operationA());
    scope.fork(() -> operationB());
    scope.joinUntil(deadline);
} catch (StructureViolationException e) {
    // One of the tasks threw an exception
    System.err.println("A task failed, others cancelled");
}

ShutdownOnSuccess

Cancels remaining tasks when first succeeds:

try (var scope = new StructuredTaskScope.ShutdownOnSuccess<String>()) {
    scope.fork(() -> fetchFromPrimaryDatabase());
    scope.fork(() -> fetchFromBackupDatabase());
    
    String result = scope.joinUntil(deadline);
    // First one to succeed wins
} catch (TimeoutException e) {
    System.err.println("No source responded in time");
}

Real-World Considerations

1. Thread-Local Variables

Virtual threads can still use ThreadLocal, but be aware that a single virtual thread may be scheduled on different platform threads during execution. For truly thread-affine data, use ScopedValue instead:

// Old approach (works but not ideal)
private static ThreadLocal<RequestContext> context = new ThreadLocal<>();

// New approach (designed for virtual threads)
private static final ScopedValue<RequestContext> context = ScopedValue.newInstance();

public void handleRequest(RequestContext ctx) {
    ScopedValue.where(context, ctx).run(() -> {
        // Code here has access to ctx
        processRequest();
    });
}

2. Pinned Threads

If a virtual thread enters native code or holds a lock while calling synchronized, it pins to its underlying platform thread. This defeats the purpose of virtual threads. Avoid:

// BAD: synchronized blocks pin virtual threads
synchronized (lock) {
    virtualThread.run();
}

// GOOD: use ReentrantLock instead
Lock lock = new ReentrantLock();
lock.lock();
try {
    virtualThread.run();
} finally {
    lock.unlock();
}

3. Monitoring and Debugging

Use JFR (Java Flight Recorder) to monitor virtual thread behavior:

java -XX:StartFlightRecording=filename=recording.jfr MyApp

Then analyze with JDK Mission Control to see virtual thread scheduling, blocking operations, and pinning events.

Common Pitfalls

Pitfall 1: Ignoring Scope Semantics

Not waiting for scopes to complete can lead to silent failures:

// WRONG: scope exits without waiting
try (var scope = new StructuredTaskScope.ShutdownOnFailure()) {
    scope.fork(() -> importantOperation());
    // Scope exits here, cancels the task
}

// CORRECT: wait for completion
try (var scope = new StructuredTaskScope.ShutdownOnFailure()) {
    scope.fork(() -> importantOperation());
    scope.join(); // Wait for all tasks
}

Pitfall 2: Resource Exhaustion

While virtual threads are cheap, they’re not free. Creating millions simultaneously can still cause memory pressure. Always bound your concurrent tasks:

// WRONG: unbounded concurrency
for (int i = 0; i < 1_000_000; i++) {
    Thread.ofVirtual().start(() -> processItem(i));
}

// CORRECT: use a semaphore to bound concurrency
Semaphore limiter = new Semaphore(1000);
for (int i = 0; i < 1_000_000; i++) {
    limiter.acquire();
    Thread.ofVirtual().start(() -> {
        try {
            processItem(i);
        } finally {
            limiter.release();
        }
    });
}

Pitfall 3: Mixing Old and New APIs

Don’t mix ExecutorService patterns with structured concurrency in the same codebase without clear boundaries:

// MIXED (confusing):
var scope = new StructuredTaskScope.ShutdownOnFailure();
ExecutorService executor = Executors.newVirtualThreadPerTaskExecutor();
// Now you have two different concurrency patterns active

// CLEAN: pick one pattern
// For simple cases: structured concurrency
// For complex, long-lived services: ExecutorService

Why It Matters

Java’s concurrency model has been the limiting factor for scaling. With virtual threads and structured concurrency, Java becomes competitive with async runtimes (Go, Node.js) while maintaining the simplicity of threaded code.

For API developers, this means:

• Better resource utilization: handle 100,000 concurrent connections on modest hardware
• Simpler code: no callback hell, no async/await boilerplate
• Better debugging: virtual threads appear in stack traces
• Easier migration: existing thread-based code often works with minimal changes

Testing Your Understanding

If you want to verify your understanding of structured concurrency scope semantics, you can test JSON payloads from your microservices using JSON Formatter to validate the structure of concurrent API responses.

For debugging token-based authentication in virtual thread contexts, the JWT Decoder tool can help you validate token structures in concurrent scenarios.

Next Steps

1. Upgrade to Java 23 if you haven’t already
2. Refactor one service to use structured concurrency—start small
3. Profile with JFR to understand your concurrency patterns
4. Read JEP 453 (Structured Concurrency) for exhaustive details
5. Experiment with different StructuredTaskScope strategies for different patterns

Virtual threads and structured concurrency aren’t just performance improvements—they’re a new way of thinking about concurrent program correctness. Embrace them, and your systems will be more scalable and maintainable.

Resources

• JEP 444: Virtual Threads (19–23)
• JEP 453: Structured Concurrency
• Project Loom Documentation
• Java 23 Release Notes