Data ingestionEnterprise EditionNot available on AuraIntroduced in 2025.10
There are several ways to load data into a sharded property database:
-
Creating a sharded property database from delimited files using
neo4j-admin database import. -
Creating a sharded property database by splitting an existing Neo4j database.
-
Loading data transactionally into an existing sharded property database.
-
Incrementally updating an existing sharded property database.
Offline ingestion
You can use the offline ingestion methods to initially import data into a sharded property database. This is useful when you want to import in bulk before starting your application, for incremental imports later on, or for splitting an existing database into shards.
Initial import from delimited files
You can use the neo4j-admin database import full command to import data from delimited files into a sharded property database as in a standard Neo4j database.
This is particularly useful for large datasets that you want to import in bulk before starting your application or for incremental imports later on.
You can specify the --property-shard-count option to define the number of property shards you want to create.
This will help distribute the data across multiple servers in a Neo4j cluster.
|
If you are creating the property shards on a self-managed server, the server that executes the |
The following example shows how to import a set of CSV files, back them up to S3 using the --target-location and --target-format options, and then create a database using those seeds in S3.
-
Using the
neo4j-admin database importcommand, import data into thefoo-shardeddatabase, creating one graph shard and three property shards. If the process is running on the same server as another Neo4j DBMS process, the latter must be stopped. The--target-locationand--target-formatoptions take the outputs of the import, turn them into uncompressed backups, and upload them to a location ready to be seeded from.neo4j-admin database import full foo-sharded --nodes=nodes.csv --nodes=movies.csv --relationships=relationships.csv --input-type=csv --property-shard-count=3 --schema=schema.cypher --target-location=s3://bucket/folder/ --target-format=backup -
Create the database foo-sharded as a sharded property database by seeding it from your backups in the AWS S3 bucket:
CREATE DATABASE `foo-sharded` DEFAULT LANGUAGE CYPHER 25 PROPERTY SHARDS { COUNT 3 } OPTIONS { seedUri: `s3://bucket/folder/` };
The cluster automatically distributes the data across its servers. For more information on seed providers, see Create a database from a URI.
Incremental import / offline updates
You can use the neo4j-admin database import incremental command to import data into an existing database.
This is particularly useful for large batches of data that you want to add to an existing sharded property database.
It allows you to do faster updates than is possible transactionally.
-
Stop the
foo-shardeddatabase if it is running. See Starting and stopping a sharded property database for instructions. -
Run the
neo4j-admin database import incrementalcommand by specifying the--property-shard-countoption to define the number of property shards you want to create, the--target-locationand--target-formatoptions to upload the resulting stores to a location ready for re-creating the databases from, and the CSV files that you wish to update your existing data with. See Incremental import for more information and instructions.
neo4j-admin database import incremental foo-sharded --nodes=nodes.csv --nodes=movies.csv --relationships=relationships.csv --input-type=csv --property-shard-count=3 --schema=schema.cypher --target-location=s3://bucket/folder/ --target-format=backup
-
Re-create your database using the
dbms.recreateDatabase()or follow step 2 of Splitting an existing database into shards and creating a new database with the resulting stores the same way you would for a normal offline incremental import.
Importing data using LOAD CSV
You can use LOAD CSV to import data into a sharded property database.
This is especially useful when you want to import small to medium-sized datasets (up to 10 million records) from local and remote files, including cloud URIs.
For more information, see Cypher Manual → LOAD CSV and Getting Started guide → Tutorial: Import CSV data using LOAD CSV.
|
Transactional Cypher statements involving |
Splitting an existing database into shards
You can use the neo4j-admin database copy command to split an existing database into shards.
It works in the same way as a standard database copy with a few additional arguments.
You must specify --property-shard-count to be > 0 to indicate that you want to create a sharded property database.
If --to-format is a value other than spd_block, a warning will be printed in the log that the given format will be ignored.
If --to-format is spd_block and --property-shard-count is not set, an exception will be thrown to specify the number of shards.
The following example shows how to split the existing foo database into a new database called foo-sharded, which has 3 property shards in a cluster deployment.
If you are using a standalone server, you can skip steps 2 and 3.
-
On one of the servers, copy the data from the
foodatabase into the databasefoo-sharded, creating one graph shard and three property shards. Thefoodatabase must be stopped.neo4j-admin database copy foo foo-sharded --copy-schema --property-shard-count 3 --target-location=s3://bucket/folder/ --target-format=backupFor more information about the syntax and options of the
neo4j-admin database copycommand, see Copy a database store. -
Create the database
foo-shardedas a sharded property database by seeding it from your dumps in AWS S3 bucket:CREATE DATABASE `foo-sharded` DEFAULT LANGUAGE CYPHER 25 PROPERTY SHARDS { COUNT 3 } OPTIONS { seedUri: `s3://bucket/folder/` };The cluster automatically distributes the data across its servers. For more information on seed providers, see Create a database from a URI.
Online ingestion
You can use the online ingestion methods to import data into a sharded property database. This is useful for smaller datasets or when you want to create a new database from an existing one.
Creating an empty sharded property database
You can create an empty sharded database using the CREATE DATABASE command.
The command allows you to specify the number of property shards and the topology of the graph shard.
The following examples show how to create an empty sharded database with different configurations.
Example 1: Create an empty sharded database with the default topology (1 primary, no secondaries, and 1 replica per property shard)
CYPHER 25 CREATE DATABASE `foo-sharded`
PROPERTY SHARDS { COUNT 3 };
Creating a sharded database from a URI
You can create a new sharded property database from an existing database with seeding from one or more URIs. This is useful when you want to create a new database as a copy of an existing one, or when you want to seed a new database with data from another source. For more information on how seed from URI works, see the Create a database from a URI.
The following example shows how to create a shared database with seeding from one or several URIs.
Example 1: Create a sharded database with seeding from one URI
CYPHER 25 CREATE DATABASE `foo-sharded`
PROPERTY SHARDS { COUNT 3 }
OPTIONS { seedURI: “s3://bucket/folder/” };
Example 2: Create a sharded database with seeding from one URI with a different name
This one is similar to example 1, but the system looks for other-database-g000, etc.
CYPHER 25 CREATE DATABASE `foo-sharded`
PROPERTY SHARDS { COUNT 3 }
OPTIONS { seedURI: “s3://bucket/folder/”, seedSourceDatabase: “other-database” };
Example 3: Create a sharded database with seeding from multiple URIs
The URIs need to be keyed by the shard name they should be used to seed.
The shard names will be databaseName-g000 and databaseName-p000 to databaseName-px, where x is the number of property shards -1.
CYPHER 25 CREATE DATABASE `foo-sharded`
PROPERTY SHARDS { COUNT 3 }
OPTIONS { seedUri: {
`foo-sharded-g000`: "s3://bucket/folder/foo-g000.dump",
`foo-sharded-p000`: "s3://bucket/folder/foo-p001.dump",
`foo-sharded-p001`: "s3://bucket/folder/foo-p002.dump"
} };