Neo4j and ElasticSearch
There are various ways to integrate Neo4j with ElasticSearch, here we will list some approaches and point to solutions that enable you to reuse your existing ES infrastructure with Neo4j.
You should have a sound understanding of both ElasticSearch and Neo4j, each data model and APIs to leverage them effectively together.
Intermediate
General Observations
There are several way of integrating Neo4j with ElasticSearch. In the past the easiest were River plugins which have been discontinued. You can integrate with Neo4j’s TransactionEventHandler to push graph changes directly to ElasticSearch. Another option is to supply sources and sinks for LogStash.
Another option would be to implement a full index-provider that uses ES as storage. This is a tight integration which might suffer from the roundtrip latencies to ElasticSearch’s REST API.
If you plan to connect Neo4j to ElasticSearch using the default Java driver, please be aware that there are incompatibilities between the Lucene version used by Neo4j and ElasticSearch, so you might need to go with a REST based solution like JEST. |
Push to ElasticSearch
Implementing a Neo4j Transaction Handler provides you with all the changes that were made within a transaction.
With the afterCommit
notification method, we can make sure that we only send data to ElasticSearch that has been committed to the graph.
Using labels as filtering mechanism, you can render a node’s properties as a JSON document and insert it asynchronously in bulk into ElasticSearch.
To register the Kernel Extension, just drop the jar in Neo4j’s classpath or plugins
directory and configure the ElasticSearch-URL and the label/property combinations to trigger and render the update.
elasticsearch.host_name=http://localhost:9200 elasticsearch.index_spec=people:Person(first_name,last_name), places:Place(name)
We provided a sample implementation below, that you can use out of the box or extend for further customization and fine-tuning.
LogStash integration
Pere Urbon who’s experienced both in Neo4j and ElasticSearch, wrote open-source implementations of sinks and sources provided Neo4j.
-
[Introduction]
-
[Documentation]
-
[Source Code]
Was this page helpful?