Noggit is the world’s fastest streaming JSON parser for Java. Noggit is the streaming JSON parser used in Solr. It lives here on github. JSON features and extensions Noggit supports a number of extensions to the JSON grammar. All of these extensions are optional and may be disabled. Comments Unquoted […]
java
Background I needed a really good hash function for the distributed indexing in SolrCloud. Since it is be used for partitioning documents, it needed to be really high quality (well distributed) since we don’t want uneven shards. It also needed to be cross-platform, so a client could calculate this hash […]
MurmurHash3 for Java
With CPU cores constantly increasing, there has been some major work done in Lucene/Solr to increase the scalability under multi-threaded load. Read-only IndexReaders One bottleneck was synchronization around the checking of deleted docs in a Lucene IndexReader. Since another thread could delete a document at any time, the IndexReader.isDeleted() call […]
Solr scalability improvements
Having performance issues with Solr’s faceted search and certain types of fields? Help has arrived in the form of a new Solr faceting algorithm! This new faceting implementation dramatically improves the performance of faceted search, making it suitable for a much wider range of applications. The existing multivalued field faceting […]
Solr Faceted Search Performance Improvements
I was surprised to discovered that there isn’t a good cross-platform hash function defined for strings. MD5, SHA, FVN, etc, all define hash functions over bytes, meaning that it’s under-specified for strings. So I set out to create a standard 32 bit string hash that would be well defined for […]
lookup3ycs : a standard high performance string hash
A new chapter in Solr scalability has been opened with the addition of distributed search! http://wiki.apache.org/solr/DistributedSearch Distributed Search splits an index into multiple shards, and queries across all the shards, combining the results and presenting a single merged response that looks like it came from a single server. Solr’s current […]