The new query optimizer far exceeds the performance of the old rule-based planner and the more recent cost-based planner (nicknamed “Ronja“) for Cypher queries of any length or complexity. We’re confident that our enterprise customers and community users alike will enjoy the benefits of this newer, faster and more humanly accessible query optimizer.
The Backstory
The development roots of the Neo4j graph database go all the way back to the turn of the millennium, but the development roots of our new query optimizer date all the way back to 1975 (when East Germany was still a thing).
Early on in building the company that would later become Neo4j, Inc. (even before we hired employee #10), CEO Emil Eifrem knew he needed a blazing fast way to optimize graph queries for his new graph database. His reasoning was that if the most powerful graph database on the planet is the human brain, then the best query optimizer would be one that could operate at the speed of thought.
In order to make this dream a reality, Emil did what most of us would have done in similar circumstances: Embark on a “geek cruise” around the Baltic Sea (seriously though, who does that?).
Hours after presenting his graph database to a bunch of fellow geeks on the cruise, a connection was made: A way was introduced to integrate this sort of heuristic optimization approach into Neo4j. The year was 2008.
Two years later, that approach would become more deeply embedded within the company, even while other Cypher query optimizers were also in development. And now, a full decade after that world-changing moment, we’re proud to release this new query optimizer to the world. (For a grand total of 43 years of development!)
Introducing the Der Hunger Query Optimizer
In the spirit of our other Northern-European-nicknamed query optimizers (looking at you, Ronja), we’ve christened this new one the Der Hunger optimizer.
Why that name? First, it just seemed natural. When Emil first made that connection back in 2008, the name just came with the optimizer. Second, Der Hunger is hungry. Hungry for graphs, and hungry for helpfulness.
Der Hunger is 10000x faster than our standard cost-based optimizer (hit the bricks, Ronja!) and will be slowly rolled out to all Neo4j instances as quickly as we’re able to scale the solution (pending FDA and probably OSHA approvals). Until then, optimization times will just have to wait their turn.
The Der Hunger optimizer will also fully integrate with all aspects of the Neo4j Graph Platform, including even the lesser-known Neo5j graph database, Neo4j Matrix Edition, Neo 4 Java and NullDB features.
While planning and optimizing your queries as fast as Der Hunger can manage, its AI-comparable algorithms also answer millions of questions on Stack Overflow, Slack and Twitter. Why? Because it never sleeps. No really, we can’t emphasize it enough: Der Hunger never sleeps. Like ever. (“He” was born that way.)
In fact, we’ve even been able to tune the neural network functionality behind Der Hunger in order to get it to write a few blog posts for us. You’ve probably read one without even knowing it. (Der Hunger writes under a pen name for obvious reasons.)
How Can You Use the New Optimizer?
Similar to other experimental features in our Cypher runtime, the new optimizers can be enabled with a query prefix.
This will cause your query to be sent for processing by the new optimizer to create a highly efficient query plan which is then re-incorporated into your execution plan cache for further use.
Just use:
cypher planner=DerHunger MATCH … my complex query …
Der Hunger’s Exclusive Beta Release This Spring
With the start of Q2 2018, Der Hunger will be made available in an exclusive beta process to both our enterprise customers and community users alike.
Why not just customers exclusively? Because the Der Hunger optimizer loves graphs and values relationships too much; he simply can’t exclude optimizing someone’s queries just because they’re not a paying customer.
While innovating with query optimization, we figured we may as well innovate in our business model as well. During the beta stage, all maintenance fees for Der Hunger optimizer will be paid in coffee and sushi.
Unfortunately, the beta release will be limited because if everyone jumps on using the new optimizer, it will result in a worldwide DoH (Denial of Hunger) attack across all Neo4j instances.
Finally, even though Neo4j data is built to maintain strict ACID compliance, the Herr Hunger optimizer will use an eventually consistent approach until our IRB finally allows us to just clone him already.
Ready to sign up for the beta? Click the button below to get added to the waitlist:
Optimize My Queries, Herr Hunger!
Optimize My Queries, Herr Hunger!
***
Update: This is an April Fool’s Day article; please don’t take it seriously! While Neo4j’s Cypher optimizer gets better in every version, to get help from Michael Hunger and other folks, please post your optimization questions to our Customer Support Portal or our Discussion Forum.
