Neo4j 1.7.M02 – Cache Cachet, Matching Matchers, and Debian Debs

Neo4j 1.7 Milestone 2 introduces a trio of interesting advances: a new cache scheme, targeted pattern matching in Cypher, and Debian install packages. Faster, smarter, and more accessible. 

Under the hood, Neo4j runs on the JVM (that’s the ‘J’ in ‘Neo4j’). And as every java developer knows: the Garbage Collector is your friend, the Garbage Collector is your enemy. The Garbage Collector (GC) helpfully relieves the developer from worrying about memory management.  Unhelpfully, garbage collection introduces unpredictability in an application’s responsiveness. While partial garbage collection can be a nuisance,  full garbage collection can be disastrous, pausing an application for uncomfortably long durations (from a few to far too many seconds). Sadly, there are no really good solutions to consistently avoid full GC pauses or to even prevent the GC from kicking in at inconvenient times.

Neo4j tweaks the impact of garbage collection with cache strategies suited for different scenarios (see the manual section on caches). Our newest innovation under the hood is the Atomic Array Cache (AAC), designed to keep GC troubles at bay. The ‘Array’ part occupies a fixed amount of Java heap space, managed with concurrency friendly ‘Atomic’ operations. In action, the AAC prevents GC from collecting anything in the cache without explicit management by Neo4j.

The AAC is compact, fitting 10x more primitives into the same memory as the other cache types. And the performance is very exciting, up to 10x faster. Best of all, when configured correctly there will be no GC pauses.

Want to give it a try? Grab the 1.7.M02 release, then check the docs for proper configuration of the caches. Let us know your experiences, whether woo-hoo or oh-darn moments.

By describing what you want and leaving the how-to-do-it up to someone else, a declarative language like Cypher presents many opportunities for optimizations. Our brilliant Michael Hunger recently joined Cypher master Andres Taylor for a hard look at pattern matching, initiating a classification of common use cases. Then the duo targeted different pattern matcher implementations to fit each type of query. The results are promising. 

CypherQueryWithAggregationCase :

start a=node(*) where a.value? < 5 
    return coalesce(a.value?,0), count(*) 
    order by count(*) desc limit 10

Before => 25 ops/s

After  => 31 ops/s

CypherQueryWithPatternsCase:

start a=node({ids}) 
    match a-[:`0`]->b<--d-->a 

    return b

Before => 2 ops/s

After => 44 ops/s

CypherSimpleLongPathCase:

start a = node(0)
    match a-[:AB]->b-[:BC]->c-[:CD]->d-[:DE]->e 
    return e

Before => 1 ops/s

After => 7 ops/s

Considering use cases, Cypher now has support for multiple relationship types within a single relationship segment, so you can express things like:

START person=node(3)
    MATCH (person)-[:LIVES_IN|WORKS_IN]->(city)
    RETURN city

Please report back on your performance impressions of 1.7.M02, so we can continue to expand our use case coverage. 

Ops people rejoice, because Neo4j is now a simple `apt-get` away for any Debian-based Linux distro (like the ever popular Ubuntu). The debian installer is now part of the regular build and deploy chain, pushing out to our own debian repository. See https://debian.neo4j.org/ for details, following these steps to install:

$ sudo -s
$ echo 'deb https://debian.neo4j.org/repo binary/' >> /etc/apt/sources.list.d/neo4j.list
$ aptitude update
$ aptitude install neo4j-enterprise neo4j-coordinator

To see a list of resolved issues and minor additions included in 1.7.M02, refer to the github issues. Many of these originate from community reporting, so don’t be shy: let us know about any problems you encounter.

Thanks again to everyone in the community for contributing to the conversation in our google group, particularly when helping out new graphistas. We often hear kind compliments about how thoughtful, helpful, and outright attractive our community is. Absolutely agreed, you all are remarkable.

OK, time to download Neo4j 1.7.M02 to enjoy the fresh new abilities. 

Cheers,

Peter