Back up an online database

Remember to plan your backup carefully and back up each of your databases, including the system database.

Note that it is not allowed to take a backup of a database alias, only physical databases can be backed up.

Command

A Neo4j database can be backed up in online mode using the backup command of neo4j-admin. The command must be invoked as the neo4j user to ensure the appropriate file permissions.

It is best practice, but not mandatory, to perform the backup from a server on the same network as the database, but that is not part of the cluster. You should install Neo4j on that machine to make the neo4j-admin command available. This machine is known as a backup client.

Neo4j 5 introduces a new version of the backup command which produces immutable backup artifacts (as opposed to mutable folders as in previous versions).

Backup artifact

The neo4j-admin database backup command produces one backup artifact file per database each time it is run. A backup artifact file is an immutable file containing the backup data of a given database along with some metadata like the database name and ID, the backup time, the lowest/highest transaction ID, etc.

Backup artifacts can be of two types:

a full backup containing the whole database store or
a differential backup containing a log of transactions to apply to a database store contained in a full backup artifact.

Backup chain

The first time the backup command is run, a full backup artifact is produced for a given database. On the other hand, differential backup artifacts are produced by the subsequent runs.

A backup chain consists of a full backup optionally followed by a sequence of n contiguous differential backups.

Figure 1. Backup chain

Usage

The neo4j-admin database backup command can be used for performing an online full or differential backup from a running Neo4j Enterprise server. The produced differential backup artifact contains transaction logs that can be replayed and applied to stores contained in full backup artifacts when restoring a backup chain.

Neo4j’s backup service must have been configured on the server beforehand. The command can be run both locally and remotely. However, it uses a significant amount of resources, such as memory and CPU. Therefore, it is recommended to perform the backup on a separate dedicated machine. The neo4j-admin database backup command also supports SSL/TLS. For more information, see Online backup configurations.

neo4j-admin database backup is not supported in Neo4j Aura.

Syntax

neo4j-admin database backup [-h] [--expand-commands] [--verbose]
                            [--compress[=true|false]] [--keep-failed[=true|false]]
                            [--parallel-recovery[=true|false]]
                            [--additional-config=<file>]
                            [--include-metadata=none|all|users|roles]
                            [--inspect-path=<path>] [--pagecache=<size>] [--temp-path=<path>]
                            [--to-path=<path>] [--type=<type>] [--from=<host:port>[,<host:
                            port>...]]... [<database>...]

Description

Perform an online backup from a running Neo4j enterprise server. Neo4j’s backup service must have been configured on the server beforehand.

Parameters

Table 1. `neo4j-admin database backup` parameters
Parameter	Description	Default
`[<database>…]`	Name(s) of the remote database(s) to backup. Supports globbing inside of double quotes, for example, "data*". (<database> is required unless `--inspect-path` is used.)	`neo4j`

If <database> is "*", neo4j-admin will attempt to back up all databases of the DBMS.

Options

Option Description Default

--additional-config=<file>^[1]

Configuration file with additional configuration.

--compress[=true|false]

Request backup artifact to be compressed. Compression can yield a backup artefact many times smaller, but the exact reduction depends upon many factors, including the database format and the kind of data stored. If disabled, the size of the produced artifact will be approximately equal to the size of the backed-up database. The speed of the backup operation is affected by compression, but which is faster depends upon the relative performance of CPU and storage. If backup speed is important, consider evaluating both options - with compression enabled and disabled.

true

--expand-commands

Allow command expansion in config value evaluation.

--from=<host:port>[,<host:port>…]

Comma-separated list of host and port of Neo4j instances, each of which are tried in order.

-h, --help

Show this help message and exit.

--include-metadata=none|all|users|roles

Include metadata in the file. This cannot be used for backing up the system database. Possible values are:

roles - include commands to create the roles and privileges (for both database and graph) that affect the use of the database.
users - include commands to create the users that can use the database and their role assignments.
all - include both roles and users.
none - does not include any metadata.

Privileges specific to the DBMS and not to the backed-up database are not included in the backup. For instance, GRANT ROLE MANAGEMENT ON DBMS TO $role will not be backed up.

Accordingly, roles and users that do not have database-related privileges are not included in the backup (e.g. those with only DBMS or no privileges).

It is recommended to use SHOW USERS, SHOW ROLES, and SHOW ROLE $role PRIVILEGES AS COMMANDS to get the complete list of users, roles and privileges in these situations.

all

--inspect-path=<path>

List and show the metadata of the backup artifact(s). Accepts a folder or a file.

--keep-failed[=true|false]

Request failed backup to be preserved for further post-failure analysis. If enabled, a directory with the failed backup database is preserved.

false

--pagecache=<size>

The size of the page cache to use for the backup process.

--parallel-recovery[=true|false]

Allow multiple threads to apply pulled transactions to a backup in parallel. For some databases and workloads, this may reduce backup times significantly. Note: this is an EXPERIMENTAL option. Consult Neo4j support before use.

false

--temp-path=<path>

Introduced in 5.24 Provide a path to a temporary empty directory for storing backup files until the command is completed. The files will be deleted once the command is finished.

--to-path=<path>

Directory to place backup in (required unless --inspect-path is used). It is possible to back up databases into AWS S3 buckets, Google Cloud storage buckets, and Azure using the appropriate URI as the path.

--type=<type>

Type of backup to perform. Possible values are: FULL, DIFF, AUTO. If none is specified, the type is automatically determined based on the existing backups. If you want to force a full backup, use FULL.

AUTO

--verbose

Enable verbose output.

1. See Tools → Configuration for details.

The --to-path=<path> option can also back up databases into AWS S3 buckets (from Neo4j 5.19), Google Cloud storage buckets (from Neo4j 5.21), and Azure buckets (from 5.24). For more information, see Back up a database to a cloud storage.

Neo4j 5.24 introduces the --temp-path option to address potential issues related to disk space when performing backup-related commands, especially when cloud storage is involved.

If --temp-path is not set, a temporary directory is created inside the directory specified by the --path option.

If you don’t provide the --path option or if your provided path points to a cloud storage bucket, a temporary folder is created inside the current working directory for Neo4j. This fallback option can cause issues because the local filesystem (or the partition where Neo4j is installed) may not have enough free disk to accommodate the intermediate computation.

Therefore, it is strongly recommended to provide a --temp-path option when executing a backup-related command, especially if the folder provided in the --path option points to a cloud storage bucket.

Exit codes

Depending on whether the backup was successful or not, neo4j-admin database backup exits with different codes. The error codes include details of what error was encountered.

Table 3. Neo4j Admin backup exit codes when backing up one database
Code	Description
`0`	Success.
`1`	Backup failed, or succeeded but encountered problems such as some servers being uncontactable. See logs for more details.

Table 4. Neo4j Admin backup exit codes when backing multiple databases
Code	Description
`0`	All databases are backed up successfully.
`1`	One or several backups failed, or succeeded with problems.

Online backup configurations

Checkpointing

When a full backup is requested, it always triggers a checkpoint. The backup cannot proceed until the checkpoint finishes.

While the server is checkpointing, the backup job receives no data, which may lead to the backup timeout. To extend the backup timeout, modify the dbms.cluster.catchup.client_inactivity_timeout setting, which restricts the network inactivity. It controls the timeout duration of the catchup protocol, which is the underlying protocol of multiple catchup processes, including backups.

You can also tune up the Checkpoint settings or check that your disks are performant enough to handle the load. For more information, see Checkpoint IOPS limit.

To read more about checkpointing, see Database internals → Checkpointing and log pruning.

Server configuration

The table below lists the basic server parameters relevant to backups. Note that by default, the backup service is enabled but only listens on localhost (127.0.0.1). This needs to be changed if backups are to be taken from another machine.

Table 5. Server parameters for backups
Parameter name	Default value	Description
`server.backup.enabled`	`true`	Enable support for running online backups.
`server.backup.listen_address`	`127.0.0.1:6362`	Listening server for online backups.

Memory configuration

The following options are available for configuring the memory allocated to the backup client:

Configure heap size for the backup::

HEAP_SIZE configures the maximum heap size allocated for the backup process. This is done by setting the environment variable HEAP_SIZE before starting the operation. If not specified, the Java Virtual Machine chooses a value based on the server resources.

Configure page cache for the backup::

The page cache size can be configured by using the --pagecache option of the neo4j-admin database backup command.

You should give the Neo4J page cache as much memory as possible, as long as it satisfies the following constraint:

Neo4J page cache + OS page cache < available RAM, where 2 to 4GB should be dedicated to the operating system’s page cache.

For example, if your current database has a Total mapped size of 128GB as per the debug.log, and you have enough free space (meaning you have left aside 2 to 4 GB for the OS), then you can set --pagecache to 128GB.

Computational resource configurations

Transaction log files

The transaction log files, which keep track of recent changes, are rotated and pruned based on a provided configuration. For example, setting db.tx_log.rotation.retention_policy=3 files keeps 3 transaction log files in the backup. Because recovered servers do not need all of the transaction log files that have already been applied, it is possible to further reduce storage size by reducing the size of the files to the bare minimum. This can be done by setting db.tx_log.rotation.size=1M and db.tx_log.rotation.retention_policy=3 files. You can use the --additional-config parameter to override the configurations in the neo4j.conf file.

Removing transaction logs manually can result in a broken backup.

Security configurations

Securing your backup network communication with an SSL policy and a firewall protects your data from unwanted intrusion and leakage. When using the neo4j-admin database backup command, you can configure the backup server to require SSL/TLS, and the backup client to use a compatible policy. For more information on how to configure SSL in Neo4j, see SSL framework.

Configuration for the backup server should be added to the neo4j.conf file and configuration for backup client to the neo4j-admin.conf file. SSL settings should be set identically between both to ensure compatibility.

The default backup port is 6362, configured with key server.backup.listen_address. The SSL configuration policy has the key of dbms.ssl.policy.backup.

As an example, add the following content to your neo4j.conf and neo4j-admin.conf files:

dbms.ssl.policy.backup.enabled=true
dbms.ssl.policy.backup.tls_versions=TLSv1.2
dbms.ssl.policy.backup.ciphers=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384
dbms.ssl.policy.backup.client_auth=REQUIRE

Neo4j 5.24 also supports TLSv1.3. To use both TLSv1.2 and TLSv1.3 versions, you must specify which ciphers to be enforced for each version. Otherwise, Neo4j could use every possible cipher in the JVM for those versions, leading to a less secure configuration.

For a detailed list of recommendations regarding security in Neo4j, see Security checklist.

It is very important to ensure that there is no external access to the port specified by the setting server.backup.listen_address. Failing to protect this port may leave a security hole open by which an unauthorized user can make a copy of the database onto a different machine. In production environments, external access to the backup port should be blocked by a firewall.

Cluster configurations

In a cluster topology, it is possible to take a backup from any server hosting the database to backup, and each server has two configurable ports capable of serving a backup. These ports are configured by server.backup.listen_address and server.cluster.listen_address respectively. Functionally, they are equivalent for backups, but separating them can allow some operational flexibility, while using just a single port can simplify the configuration. It is generally recommended to select secondary servers to act as backup servers since they are more numerous than primary servers in typical cluster deployments. Furthermore, the possibility of performance issues on a secondary server, caused by a large backup, does not affect the performance or redundancy of the primary servers. If a secondary server is not available, then a primary can be selected based on factors, such as its physical proximity, bandwidth, performance, and liveness.

Use the SHOW DATABASES command to learn which database is hosted on which server.

To avoid taking a backup from a cluster member that is lagging behind, you can look at the transaction IDs by exposing Neo4j metrics or via Neo4j Browser. To view the latest processed transaction IDs (and other metrics) in Neo4j Browser, type :sysinfo at the prompt.

Targeting multiple servers

It is recommended to provide a list of multiple target servers when taking a backup from a cluster, since that may allow a backup to succeed even if some server is down, or not all databases are hosted on the same servers. If the command finds one or more servers that do not respond, it continues trying to backup from other servers and continues backing up other requested databases, but the exit code of the command is non-zero, to alert the user to the fact there is a problem. If a name pattern is used for the database together with multiple target servers, all servers contribute to the list of matching databases.

Examples

The following are examples of how to perform a backup of a single database and multiple databases. The target directory /mnt/backups/neo4j must exist before calling the command and the database(s) must be online.

Back up a single database

You do not need to use the --type option to specify the type of backup. By default, the type is automatically determined based on the existing backups.

bin/neo4j-admin database backup --to-path=/path/to/backups/neo4j neo4j

Perform a forced full backup of a single database.

If you want to force a full backup after several differential backups, you can use the --type=full option.

bin/neo4j-admin database backup --type=full --to-path=/path/to/backups/neo4j neo4j

Back up multiple databases

To back up several databases that match database a pattern you can use name globbing. For example, to backup all databases that start with n from your three-node cluster, run:

bin/neo4j-admin database backup --from=192.168.1.34:6362,192.168.1.35:6362,192.168.1.36:6362 --to-path=/mnt/backups/neo4j --pagecache=4G "n*"

Back up a list of databases

To back up several databases by name, you can provide a list of database names.

neo4j-admin database backup --from=192.168.1.34:6362,192.168.1.35:6362,192.168.1.36:6362 --to-path=/mnt/backups/neo4j --pagecache=4G "test*" "neo4j"

Back up a database to a cloud storage

The following examples show how to back up a database to a cloud storage bucket using the --to-path option.

Neo4j uses the AWS SDK v2 to call the APIs on AWS using AWS URLs. Alternatively, you can override the endpoints so that the AWS SDK can communicate with alternative storage systems, such as Ceph, Minio, or LocalStack, using the system variables aws.endpointUrls3, aws.endpointUrlS3, or aws.endpointUrl, or the environments variables AWS_ENDPOINT_URL_S3 or AWS_ENDPOINT_URL.

Install the AWS CLI by following the instructions in the AWS official documentation — Install the AWS CLI version 2.
Create an S3 bucket and a directory to store the backup files using the AWS CLI:
```
aws s3 mb --region=us-east-1 s3://myBucket
aws s3api put-object --bucket myBucket --key myDirectory/
```
For more information on how to create a bucket and use the AWS CLI, see the AWS official documentation — Use Amazon S3 with the AWS CLI and Use high-level (s3) commands with the AWS CLI.
Verify that the ~/.aws/config file is correct by running the following command:
```
cat ~/.aws/config
```
The output should look like this:
```
[default]
region=us-east-1
```
Configure the access to your AWS S3 bucket by setting the aws_access_key_id and aws_secret_access_key in the ~/.aws/credentials file and, if needed, using a bucket policy. For example:
1. Use aws configure set aws_access_key_id aws_secret_access_key command to set your IAM credentials from AWS and verify that the ~/.aws/credentials is correct:
  cat ~/.aws/credentials
  The output should look like this:
  [default] aws_access_key_id=this.is.secret aws_secret_access_key=this.is.super.secret
2. Additionally, you can use a resource-based policy to grant access permissions to your S3 bucket and the objects in it. Create a policy document with the following content and attach it to the bucket. Note that both resource entries are important to be able to download and upload files.
  { "Version": "2012-10-17", "Id": "Neo4jBackupAggregatePolicy", "Statement": [ { "Sid": "Neo4jBackupAggregateStatement", "Effect": "Allow", "Action": [ "s3:ListBucket", "s3:GetObject", "s3:PutObject", "s3:DeleteObject" ], "Resource": [ "arn:aws:s3:::myBucket/*", "arn:aws:s3:::myBucket" ] } ] }
Run the neo4j-admin database backup command to back up your database to your AWS S3 bucket:
```
bin/neo4j-admin database backup --to-path=s3://myBucket/myDirectory/ mydatabase
```

Ensure you have a Google account and a project created in the Google Cloud Platform (GCP).
1. Install the gcloud CLI by following the instructions in the Google official documentation — Install the gcloud CLI.
2. Create a service account and a service account key using Google official documentation — Create service accounts and Creating and managing service account keys.
3. Download the JSON key file for the service account.
4. Set the GOOGLE_APPLICATION_CREDENTIALS and GOOGLE_CLOUD_PROJECT environment variables to the path of the JSON key file and the project ID, respectively:
  export GOOGLE_APPLICATION_CREDENTIALS="/path/to/keyfile.json" export GOOGLE_CLOUD_PROJECT=YOUR_PROJECT_ID
5. Authenticate the gcloud CLI with the e-mail address of the service account you have created, the path to the JSON key file, and the project ID:
  gcloud auth activate-service-account service-account@example.com --key-file=$GOOGLE_APPLICATION_CREDENTIALS --project=$GOOGLE_CLOUD_PROJECT
  For more information, see the Google official documentation — gcloud auth activate-service-account.
6. Create a bucket in the Google Cloud Storage using Google official documentation — Create buckets.
7. Verify that the bucket is created by running the following command:
  gcloud storage ls
  The output should list the created bucket.
Run neo4j-admin database backup command to back up your database to your Google bucket:
```
bin/neo4j-admin database backup --to-path=gs://myBucket/myDirectory/ mydatabase
```

Ensure you have an Azure account, an Azure storage account, and a blob container.
1. You can create a storage account using the Azure portal.
  For more information, see the Azure official documentation on Create a storage account.
2. Create a blob container in the Azure portal.
  For more information, see the Azure official documentation on Quickstart: Upload, download, and list blobs with the Azure portal.
Install the Azure CLI by following the instructions in the Azure official documentation — Azure official documentation.
Authenticate the neo4j or neo4j-admin process against Azure using the default Azure credentials.
See the Azure official documentation on default Azure credentials for more information.
```
az login
```
Then you should be ready to use Azure URLs in either neo4j or neo4j-admin.

To validate that you have access to the container with your login credentials, run the following commands:

# Upload a file:
az storage blob upload --file someLocalFile  --account-name accountName - --container someContainer --name remoteFileName  --auth-mode login

# Download the file
az storage blob download  --account-name accountName --container someContainer --name remoteFileName --file downloadedFile --auth-mode login

# List container files
az storage blob list  --account-name someContainer --container someContainer  --auth-mode login

Run neo4j-admin database backup command to back up your database to your Azure container:

bin/neo4j-admin database backup --to-path=azb://myStorageAccount/myContainer/myDirectory/ mydatabase