How-To: Neo4j ETL Tool

Now that the application is all set up, there are only a few steps to import data.

When you first load the ETL Tool, you might be asked to allow some background processes to run. We will choose Allow here, which continues loading the application and brings up the main screen.

On this screen, a default project should show and the database instances in that project. If you do not have any graph database instances created, you will see an image like the one below. You will need to create a graph^ before the next steps with the ETL tool. The ETL Tool supports local and remote instances (including Neo4j AuraDB).

Once you have a database created, you will need to select a project to work within. This determines which Neo4j databases will populate as options for you to import the data in a later step. The screenshot below shows the steps, with the red highlighting and numbering to show where to click and in what order.

After choosing a project, you will need to set up a connection to a relational database. You can do this by clicking the Add Connection button in the lower left of the pane.

When you click to add a new connection, you will see a form to enter the relational database connection details. We recommend first choosing the type of database from the dropdown box in the middle of the page, as that will automatically fill in some of the other fields for you.

The image below shows the list of relational databases included in the tool. If you are using a database that is not specified in the dropdown list, you can still connect by choosing the JDBC Driver option and filling in the details.

Both the MySQL and PostgreSQL connections pre-populate many of the configurations for you. For any of the other database options, you will also specify the driver file for that database.

Once you have filled in the form, click the Test and Save Connection button at the bottom. The results for whether it was successful or not will show in a blue (success) or red (error) message bar at the top of your window.

To begin the ETL process, we need to tell the application which relational database we want as the source and which Neo4j database we want as the target. In the screenshot below, you can see the list of relational databases on the left side of the pane, and the Neo4j database options on the right side of the pane.

Choose your relational database, then the Neo4j database and click the Start Mapping button in the lower righthand side. Just as before, a blue message bar will show at the top of your screen if the step was successful or a red one will show if step failed. In this case, it was successful, so our Start Mapping button inactivates, and the Next button activates to proceed to the next step!

This step is where the actual translation of the relational data into graph data happens. There are three rules the tool uses to convert from relational to graph.

Those rules create a graph data model like the one below.

This is where you can edit the mapping to change any of the translations, such as property names, data types, and relationships.

You can use the mouse to zoom in on the model or drag the image to focus on certain areas of the graph. If you do not see a component you are looking for, you can start typing the entity name in the search bar on the left side of the pane. Any matching results will show immediately.

Nodes and relationships are listed in respective tabs on the left side. To update, simply click the entity in the list. The tool also allows you to skip any nodes or relationships that you do not want to import to the graph. In the screenshot below, we have chosen to skip the UsState nodes because there are no relationships, so the data is less valuable to us in graph format. However, we could still choose to import those nodes and later refactor the graph to create relationships from those nodes to others in the graph.

To edit the details of mappings, click on the pencil icon next to entity in the list or double click on the entity in the visualization on the right. A popup box will list the fields and offer options for any changes. You can click Save to apply your changes to the graph.

Oftentimes, the relationship types will be defaulted to meaningless names for a graph, so we can update them with more meaningful names for improved data context. The image below shows an example of some changes to relationship types.

Once you have made any changes here, you can click Save Mapping. The status of the change will show in a blue (success) or red (error) message bar at the top of your window. If successful, you can click Next to go to the import step.

There are a couple of different ways that the ETL Tool can import data to Neo4j, depending on the status of your graph database. Each method has certain requirements and advantages, which are listed below.

We will show the online vs. offline import methods to see the different options that come with each.

After you choose an import method from the dropdown box, you can choose if you want to import to a specific database (Neo4j v4.0+ only).

Though there is only one import type when the database is shut down, there are a couple more things you can specify. You can tell the import to go to a specific database (Neo4j v4.0+ only), whether it exists or not. If that database name does not exist, you can choose to manually create the database after the import (with Cypher command CREATE DATABASE <dbname> on system database) or you can click the checkbox at the bottom to have the ETL Tool create the database for you with the import.

With either method, once you have filled out the form as much as you need or want, click the Import Data button in the lower right corner to start the load. If the import is successful, you will see a screen similar to the one below (this example used the Online Direct import method). If it fails, you will see a red error message at the top of the screen, and you can troubleshoot the issue with the logs by clicking the See Logs button at the bottom.

Now, you can query the Neo4j database or use Neo4j Browser to verify the data loaded to properly. Your relational data has now been transformed to a graph!

If you want to test the ETL Tool, and you do not already have a data set, you can use the Northwind example, as we did here. We have included links to download both PostgreSQL and MySQL, if you don’t already have a relational database in mind.

There are also other options for ETL. Feel free to check out some partner integrations, the LOAD CSV functionality, and the APOC developer library.