Neo4j 1.3 “Abisko Lampa” M01 – HA growing up

Michael Hunger

Welcome to the first milestone release of the new 1.3 version of Neo4j. Release 1.3 is codenamed “Abisko Lampa” – the lamp of Abisko. Starting with this major release we’re using code names borrowed from the Swedish train stations along this track, lighting the way down from the northernmost tip of Sweden. And it is still dark in Abisko.

For the “Abisko Lampa” we have a number of big things coming.

We want to support large datasets with more than 4 billion nodes, relationships and properties all sharing the id space. Another focus is improvements in the HA area, especially making it easier for ops to start and maintain Neo4j HA clusters. Regarding remote Neo4j servers, we will tackle the efficiency of the remote interface as well as improving client libraries. An interesting and convenient feature we will work on is the automatic indexing, allowing you to configure which properties are automatically synchronized with the index when created, updated or deleted. One fruitful task ahead are the planned improvements of the Traversal API, including the ability to persist Traversal Descriptions. And of course, we want to ease the usage and installation of Neo4j server for all of you.

The Neo4j 1.2 release brought you preliminary HA support. As every parent knows, a baby lives and smiles but takes a lot of time and effort to care for. To avoid that you leave our HA newborn in the backyard because it is not cute enough to justify the effort, the Neo4j team spent the last two weeks improving HA maturity and usability.

We are happy to celebrate the 1.3.M01^st birthday with you.

For our release this week, we’ve worked on these things:

Neo4j server running in an HA cluster

By configuring the Neo4j database to run in HA mode, i.e. providing server name, machine id, and zookeeper servers and by setting the database mode to ha (org.neo4j.server.database.mode=ha) in the server config you can your Neo4j server on top of a HA cluster.

Transfer big amounts of data in HA

A limit in the existing HA implementation regarding the transfer size that affected big transactions, long running ones and transaction replay was removed by adding support for chunked data transfer. This also allows that files for new instances are automatically copied from master.

Indexing bug fixes

We fixed a NPE in IndexHits#size(), which was sometimes thrown. A bug during recovery was removed that occurred when only relationship index commands were executed before the crash (Thanks to Andreas Bauer for reporting that).The index can now participate in full online backups, which is also used for the “copy from master” functionality in HA.

Improvements to the shell

The Neo4j-shell got some attention, we improved the exception messages and added an ‘eval’ app that can evaluate arbitrary javascript code. Here is a small example:

neo4j-shell (0)$ eval node = db.createNode(); current.createRelationshipTo(node,
   DynamicRelationshipType.withName("KNOWS"));

neo4j-shell (32)$ eval
' traverser = Traversal.description().breadthFirst().traverse(current).iterator();
' while ( traverser.hasNext() ) {
'     out.println( "" + traverser.next() );
' }
'

Neo4j sources available on GitHub

Starting with the next milestone 1.3M.02 we will move the Neo4j embedded and Neo4j server product source repositories from Subversion to the Neo4j.org github account (github.com/Neo4j). The Subversion repository will still be available, but only in read only mode.
Publishing the Neo4j sources on GitHub and leveraging its collaboration infrastructure will enable even better community involvement, feedback and improvements.

Java6 Requirement

Starting with this version, Neo4j requires Java6. This should be a non-issue as most Neo4j embedded users have moved on from Java5.

As always, thanks to everyone for contributions!

Michael


Want to learn more about graph databases? Click below to get your free copy of O’Reilly’s Graph Databases ebook and discover how to use graph technologies for your application today.

Download My Ebook