Solr 4.10 and Heliosearch .07 have added a terms query (or terms filter) to more efficiently match many terms in a single field. A large number of terms are often useful for things like access control lists or security filters. Previously, the only way to do this was a large […]
performance
Native code faceting for Solr has just been added to Heliosearch, and benchmarks show an impressive 2x performance increase! This is faceting code written in C++ and statically compiled for maximum performance, and loaded into the JVM via JNI (Java Native Interface). nCache, Heliosearch’s off-heap version of the Lucene/Solr FieldCache, […]
Native Code Faceting
Lucene/Solr background Lucene has a segmented architecture – when a small amount of documents are added to an existing index, this will often just add an additional small segment to the index. Caching data structures at the segment level (e.g. field values used for sorting) is often desirable so that […]
Off-Heap FieldCache Faceting and Sorting
Heliosearch’s off-heap FieldCache was previously introduced and benchmarked for integer fields. Support for all numeric field types as well as string fields has now been completed, and this post will focus on the performance of string fields. A review of nCache (n is for “native”) features and goals: nCache has […]
Heliosearch/Solr Off-Heap FieldCache Performance
nCache Heliosearch has a new replacement for the Lucene FieldCache currently used by Solr for sorting, faceting, and function queries. Introducing nCache (n is for “native”): nCache has Off-Heap Data-structures, just like the Off-Heap Filters to lower garbage collection pauses and GC overhead. nCache is a managed cache, meaning […]
nCache: Heliosearch/Solr Off-Heap FieldCache
Off-Heap Native Filters is the first feature we added to Heliosearch, a new open source project designed to bring Solr performance to the next level. Big JVM heaps can be Big Trouble JVMs have never been good at dealing with large heaps. Large heaps mean lots of garbage collection work, […]