Neo4j and ElasticSearch

Goals
There are various ways to integrate Neo4j with ElasticSearch, here we will list some approaches and point to solutions that enable you to reuse your existing ES infrastructure with Neo4j.
Prerequisites
You should have a sound understanding of both ElasticSearch and Neo4j, each data model and APIs to leverage them effectively together.

Intermediate

General Observations

There are several way of integrating Neo4j with ElasticSearch. In the past the easiest were River plugins which have been discontinued. You can integrate with Neo4j’s TransactionEventHandler to push graph changes directly to ElasticSearch. Another option is to supply sources and sinks for LogStash.

Another option would be to implement a full index-provider that uses ES as storage. This is a tight integration which might suffer from the roundtrip latencies to ElasticSearch’s REST API.

If you plan to connect Neo4j to ElasticSearch using the default Java driver, please be aware that there are incompatibilities between the Lucene version used by Neo4j and ElasticSearch, so you might need to go with a REST based solution like JEST.

Push to ElasticSearch

Implementing a Neo4j Transaction Handler provides you with all the changes that were made within a transaction. With the afterCommit notification method, we can make sure that we only send data to ElasticSearch that has been committed to the graph. Using labels as filtering mechanism, you can render a node’s properties as a JSON document and insert it asynchronously in bulk into ElasticSearch.

To register the Kernel Extension, just drop the jar in Neo4j’s classpath or plugins directory and configure the ElasticSearch-URL and the label/property combinations to trigger and render the update.

neo4j.properties
elasticsearch.host_name=http://localhost:9200
elasticsearch.index_spec=people:Person(first_name,last_name), places:Place(name)

We provided a sample implementation below, that you can use out of the box or extend for further customization and fine-tuning.

GraphAware Neo4j-To-Elasticsearch

Neo4j plugin providing automatic replication to Elasticsearch

Elasticsearch plugin providing integration with Neo4j

LogStash integration

Pere Urbon who’s experienced both in Neo4j and ElasticSearch, wrote open-source implementations of sinks and sources provided Neo4j.

  • [Introduction]

  • [Documentation]

  • [Source Code]