Multi Tenancy in Neo4j: A Worked Example

If you haven’t already, download Neo4j.

We will need to have a database running and open Neo4j Browser to walk through this guide. If you are unsure how to create and start a database, step-by-step instructions for doing so in Neo4j Desktop are provided in this guide.

We’re going to use the APOC Library in this guide, and in order to process local files, we’ll need need to add the following entry to the apoc.conf file:

We’ll also be using Neo4j Fabric, which we can enable by adding the following properties in the neo4j.conf file:

We’re going to use a dataset published by Carrefour, a French retailer, as part of their Delighting Customers Challenge Basket Data. Rik van Bruggen recently wrote a series of blog posts showing how to import and analyse this data in Neo4j 3.5.

The data is available as a JSON file (1GB) curated by Rik. Once we’ve downloaded the file, we need to put it into the import folder of Neo4j.

The dataset is accompanied by a data dictionary that describes each of the fields that we see in the file:

From looking at these descriptions, mall seems like a good property to use to partition the data. We can store the tickets printed in mall 1 in one database, and the tickets printed in mall 2 in another database.

We can now process this file using the apoc.load.json procedure.

We can see that some items are from mall 1 and some from mall 2. Let’s have a look how many tickets each mall printed:

It looks like we’ll have two quite evenly populated databases. Let’s get those databases created!

We’re going to store the data for each mall in a different Neo4j database. We’ll create these databases using the CREATE DATABASE command, which we need to run against the system database. Let’s first switch to that database:

Once we’ve done that we can run the following statements to create the databases:

We can check that those databases have been created by running the following command:

We’re going to import the data into the following graph model:

We’ll have nodes representing products, clients, tickets, and ticket items, and relationships that indicate which client purchased a ticket, the items that comprise a ticket, and the product that each item was for.

We’ll use the same apoc.load.json procedure to import the data into each of our databases. We’ll filter the JSON file by the mall property so that we load the data appropriately.

Let’s switch to the mall1 database and setup parameters that we’ll use in our import query:

And now we’ll create that indexes that we’re going to need:

Now we’re ready to process the JSON file. We’ll wrap our call to apoc.load.json inside a call to apoc.periodic.iterate so that our import is batched, which will avoid out of memory exceptions:

And now let’s do the same thing for mall 2. We’ll switch to that database and setup our parameters:

And now we’ll create that indexes that we’re going to need:

And now let’s import the tickets sold in this store:

We can see a sample of the imported graph in the Neo4j Browser visualization below:

Now that we’ve imported the data, let’s explore it by writing some queries. We’ll show the results of running some queries on both databases and some just on one of them.

The following query returns the biggest spending clients in the mall:

Interestingly, the biggest spending client in both malls is the same person.

The spendPerTicket also looks interesting. The following query shows the products bought by the client that has the highest spendPerTicket in mall 2 :

On the left hand side of the diagram we can see highlighted nodes that have skewed the average spend! My Spanish isn’t great, but I can see an iPad Air among the items purchased. ASP ROB LG HOMB S5 is a robot vacuum cleaner and I think POR APP MB MJVE2YA is an Apple Macbook. The other items look more like the type of products you’d buy in a normal grocery shop.

Let’s go back to mall 1 and see which products are purchased most often. We can write the following query to compute this:

All the top items are priced at less than 30 euros each. We can see a couple of beers (cerveza) in positions 3 and 4, and then Roscon seems to be a range of tasty looking pastries.

We can also write queries across databases using Neo4j Fabric. This functionality is useful for executing aggregate queries across multiple databases.

Let’s first change to the fabric database by running the following command:

Now we’re going write a query that returns the total spend on products in each of our databases.

We’ll prefix our queries with the USE clause, which tells the Fabric engine which database to execute the query against. The syntax of this clause is as follows:

USE <fabric.database.name>.<fabric.graph.*.database>

Or in our case USE fabric.mall1 and USE fabric.mall2.

The following query finds the total spend and number of purchases in each mall, combines the results using the UNION ALL clause, and then orders the results by total spend:

The total spend and number of purchases are almost identical between the malls! We can also write a version of this query that reduces duplication by using the built-in <fabric-database-name>graphIds function and the CALL{} clause, as shown below:

We can also aggregate the results returned by the sub queries executed on each of the mall databases. The following query finds the highest selling products in each mall, and then shows the top 10 products across both malls:

The top selling product in both malls is ROSCON NATA MEDIANO, but it’d be good if the sales for each product was aggregated so that we have only one row per product. We can do this using the SUM aggregation function:

The table below shows the results of running the aggregation query: