Quarkus Vs Spring Boot: A Real-world Performance Comparison

Quarkus vs Spring Boot - A real world JVM Performance comparison for Kotlin and Java on JDK 14
A performance comparison before and after switching from Spring Boot to Quarkus in production: Heap and Non-Heap Memory Usage, Garbage Collection count and time, loaded classes count and thread count, time to first request, request per second & request latency.

Introduction

After reading a lot about Quarkus and doing several promising benchmark tests (like this feature and load test comparison with Spring Boot), I decided to migrate my monitoring microservice web app from Spring Boot to Quarkus. The tech stack used initially by this web app is as follows:

After the migration, the app's stack looks like this:

For those interested, I have shared many Quarkus migration tips in the following article: https://simply-how.com/migrate-spring-boot-app-to-quarkus

This article will compare several key JVM metrics collected during the first 24 hours of the deployment in production (with a 30 minute offset from startup): Heap Memory, Non-Heap memory, Garbage Collection count and time, loaded class count and thread count. Elasticsearch and Kibana were used to collect and display the graphics.

Memory usage

As demonstrated by the illustrations, the heap memory usage was reduced by about:

  • 55 MB (Average)
  • 70 MB (Max)
  • 38 MB (Min)

The non-heap memory usage was reduced by about 12 MB.

The Spring Boot app's committed heap memory already reached the max (the max memory was originally tuned for it), while Quarkus freed up 111 MB to the operating system.

Overall, I was able to save about 24% of the original memory (Heap).


Quarkus vs Spring Boot Performance - 1- Memory metrics
Memory metrics
Quarkus vs Spring Boot Performance - 2- Heap Memory Usage (Average Aggregation)
Heap Memory Usage (Average Aggregation)
Quarkus vs Spring Boot Performance - 3- Heap Memory Usage (Max Aggregation)
Heap Memory Usage (Max Aggregation)
Quarkus vs Spring Boot Performance - 4- Non-Heap Memory Usage
Non-Heap Memory Usage

Other metrics

In a 24 hour time frame, Spring Boot had done 131 GCs that lasted a total of 2 seconds and 616 milliseconds, while Quarkus did only 39 GCs that lasted a total of 1 seconds and 299 milliseconds.

We can also see that Spring Boot does about 1 GC each 10 minute while Quarkus does 1 GC each 30 minutes.

Quarkus loaded 2119 fewer classes and had 6 less median thread count.

The average CPU usage is similar, with a slightly higher peak for Spring Boot.


Quarkus vs Spring Boot Performance - 1- Garbage Collection, loaded classes and thread metrics
Garbage Collection, loaded classes and thread metrics
Quarkus vs Spring Boot Performance - 2- Garbage Collection Count Evolution
Garbage Collection Count Evolution
Quarkus vs Spring Boot Performance - 3- Average CPU load
Average CPU load

Benchmarks

Time to first request

You can find the script used to collect the time to first request in the following repository.

Quarkus is by far the winner in this category with more than 6 seconds of difference.

Request per second

As suggested by this official Quarkus blog post, the hello endpoint used for the benchmarking was implemented using reactive routes in order to run purely on the IO thread.

Loadtest

Quarkus had a higher request per second and lower latencies using the loadtest tool.

Wrk

Quarkus is still the clear winner with twice as much rps as Spring Boot when using the wrk tool.


Quarkus vs Spring Boot Performance - 1- First Response Time
First Response Time
Quarkus vs Spring Boot Performance - 2- loadtest - Spring Boot
Spring Boot benchmarking with loadtest
Quarkus vs Spring Boot Performance - 3- loadtest - Quarkus
Quarkus benchmarking with loadtest
Quarkus vs Spring Boot Performance - 4- wrk - Spring Boot
Spring Boot load testing with wrk
Quarkus vs Spring Boot Performance - 5- wrk - Quarkus
Quarkus load testing with wrk

Conclusion

Overall, Quarkus had better memory usage, garbage collection & JVM metrics, time to first request and simple request throughput.

Quarkus is the clear winner for this use case.

I have open sourced a big part of the web app presented in this tutorial, more specifically the analytics microservice:

  • The Sprint Boot version's source code can be found in this repository
  • The Quarkus version's source code can be found in this repository

For even more performance gains, the next step would be to migrate Quarkus from JDK to GraalVM native image. I have shared in this article some tips to help you fix some issues that may block you when building using GraalVM Native.