Forum: Building APIs & Wire Protocol

Post: Solr considerations

Solr considerations
jeff_i
Aug 11, 2010
Any thought to a solr api to voltdb?
Lucene/Solr
rbetts
Aug 11, 2010
We heard one use-case for lucene/text indexes over a subset of a VoltDB table. We have not explored this in depth.


I see a few possible integrations.


1. Generate text search indexes on selected columns at a partition. This would allow text searches within a partition relatively efficiently. It would not make global multi-partiton text search fast (it would be dominated by the mult-partition transaction cost). Our storage layer is C++ based. Integrating lucene, a Java project, at this layer is non-trivial.


2. Allow Solr/Lucene to pull and index VoltDB data. This is requires exporting data from VoltDB in an efficient way for storage and indexing in a solr/lucene store. We have live export functionality in VoltDB (you can pull a stream of table changes while VoltDB is running). We have not explored how well export would work with Solr or what kind of update stream would be friendliest to Solr.


I'm not familiar with Solr, beyond a couple hours of reading. Perhaps if (1) were accomplished, a more sophisticated integration would be feasible, with Solr using VoltDB as its backing-store. This is speculation on my part and not based on a technical understanding of Solr.
Take a look at solandra - cassandra + solr/lucene
malsmith
Nov 11, 2011
http://blog.sematext.com/2010/02/09/lucandra-a-cassandra-based-lucene-backend/


https://github.com/tjake/Solandra --


This works really well - but the idea could translate well to using VoltDb as a backend - in many ways it may be a better fit the cassandra is.