Off-Heap Native Filters is the first feature we added to Heliosearch, a new open source project designed to bring Solr performance to the next level.

Big JVM heaps can be Big Trouble

JVMs have never been good at dealing with large heaps. Large heaps mean lots of garbage collection work, and often means some pretty long stop-the-world GC pauses where nothing else can proceed. This can cause query/request timeouts, or even zookeeper session timeouts in SolrCloud mode.

Off-Heap Filters

Heliosearch/Solr has some pretty advanced filter caching, but it can take up a significant amount of memory, depending on the application. This is exactly the type of large, longer lived objects that can benefit by being moved off the JVM heap and explicitly managed. Off-heap memory is invisible to the garbage collector.

Heliosearch filters (Solr DocSet objects) are now allocated off-heap and reference counted so they can be freed as soon as they are no longer being used. The JVM GC no longer needs to waste time copying around these blocks of memory. This helps to both eliminate the long GC pauses as well as increase request throughput.

Test Setup

I expected that I’d have to try a lot of different things to re-create the stop-the-word GC pauses reported by others, but they happened on my first try! They weren’t as big as others reported, but my heap size is small as those things go. Bigger heaps are correlated with bigger GC pauses.

Test Details

• Ubuntu Linux server, 8GB RAM, 4 CPU Cores, Java 1.7 64 bit
• Client: 8 threads, each doing a query of an id with a random filter (500 different filters)
• filterCache: size=1000, large enough to hold all filters w/o evictions
• Index: 3.8GB, 50M docs

 
Apache Solr command line:

java -jar -Xmx4G start.jar

 
Heliosearch/Solr command line:

java -jar start.jar

 
I had to set the heap size to 4GB when running Apache Solr to avoid OOM exceptions. Since the maximum amount of RAM on the box was 8G, I wanted to leave the remaining memory for the OS to cache the index files (else things would get really slow).

GC Results

Here are the graphs of the resulting GC activity for a run of 20,000 query requests.
The grey bars represent time spent in a GC. The red line is the actual size of the heap, and the blue line represents the actual amount of the heap in use.

Solr

It was even easy to externally see the stop-the-world pauses on Solr while the test was running. Logging was enabled, so every request left a log message, causing the terminal to rapidly scroll. Whenever a major GC compaction hit, the terminal abruptly stopped scrolling.

Heliosearch

The Heliosearch GC graph completes sooner because less time is spent doing garbage collection. Notice the almost complete absence of stop-the-world full GC pauses, and greatly reduced other GC pauses.

Query Latency

This chart shows percentile query latencies of the second 10,000 queries in a 20,000 query run (just to ensure hotspot and the caches were all warmed up).

Query Throughput

The Query Throughput graph illustrates just how much CPU time is spent in garbage collection that can be freed up with off-heap data structures. This is an extreme result of course. The throughput increase caused by off-heaping data structures would be more moderate if one is not experiencing frequent large garbage collections.

Memory Use

The maximum resident memory of the process (monitored externally via top) was measured over 5 runs.

Heliosearch, with it’s off-heap filters, had a more stable memory profile and used less memory on average. This left more memory free for the operating system to cache index files, which is very important for good performance.

Conclusion

In this simple test, off-heap filters eliminated long GC pauses, made requests more predictable by reducing large outliers, and increased overall query throughput.

Try it yourself and give us feedback in the Heliosearch Forum!