Getting Started with Neo4j and Flask

Goals

This course provides an overview on everything that you need to build a Neo4j application with the Python programming language.

Prerequisites

You should have Python 3 installed on your system. You should have cloned Python Movie Demo repository onto your machine.

This course provides an overview of what you need to know to write a Flask application using a Neo4j database. To keep concepts simple, the application is an IMDB clone with basic account authentication and movie recommendation functionality.

Before We Start - Why Neo4j?

Neo4j is the world’s most popular graph database. Graph databases offer a number of advantages:

Neo4j provides a schema-optional representation of both entities and relationships between entities.
Relationships between entities are traversed rather than joined. Traversals explore the local subgraph, meaning that query times stay the same even as your database grows.
Because of the traversal paradigm, we can think in terms of the complex relationships in our data, without worrying as much about how to model it.

Neo4j makes it easy to create nodes and relationships in an intuitive way, but you can also always change the structure of your database with a query.

Introduction to Neo4j

Connected information is everywhere in our world. Neo4j was built to efficiently store, handle, and query highly-connected elements in your data model. With a powerful and flexible data model, you can represent your real-world, variably-structured information without a loss of fidelity. The property graph model is easy to understand and work with, especially for object-oriented and relational developers.

The property graph model consists of:

Nodes, which have:

properties: schemaless key/value pairs
labels: describe and group nodes much like tables group rows, except nodes can also have multiple labels

Relationships, which connect two nodes directionally and have:

properties: schemaless key/value pairs
A type: describes how it connects the two nodes

While relationships are directional, querying relationships in either direction has no associated performance cost.

Cypher

Cypher is Neo4j’s built-in query language. Cypher queries look like the following code block:

MATCH (p:Person)-[:ACTED_IN]->(m:Movie)
RETURN p, m.title

The MATCH clause is the most common starting point for Cypher queries. It defines a pattern to search the database for, and returns one result for each match. With the RETURN clause, we would end up returning a table such as:

Table 1. Result of Cypher query
p	m.title
{name: "Denise"}	"Toy Story"
{name: "Denise"}	"Animal House"

As you can see here, we can return entire entities from our database rather than just their properties.

This is very handy, but it would also be nice to avoid the duplication of our Person node. So, you can perform the same match but instead use the collect function to aggregate the values:

MATCH (p:Person)-[:ACTED_IN]->(m:Movie)
RETURN p, collect(m.title)

Table 2. Result of Cypher aggregation query
p	m.title
{name: "Denise"}	["Toy Story", "Animal House"]

While it’s possible to get started using Neo4j without learning Cypher, it is a very powerful tool to query a Neo4j database and is worth learning. Also, since this project works by making Cypher queries to Neo4j, it is helpful to understand Cypher as your queries get more complex. Here is a Cypher tutorial, if you would like to learn more.

The Neo4j Movie Flask/React App

Let’s jump right into it. You’re a Python developer interested in Neo4j and want to build a web app, microservice, or mobile app. You’ve already read up on Neo4j, played around with some datasets, and learned enough Cypher to get started. Now you’re looking for a demo app or template to start putting those skills into practice.

If you haven’t already, clone the Python Move Demo repository onto your machine. This tutorial post will walk you through rating a movie on a sample movie rating application, from the initial setup to viewing the list of movies you’ve rated. The following video will walk briefly walk you through the various features of the movie app.

The Database

This project uses a classic Neo4j dataset: the movie database. It includes Movie, Actor, Director, and Genre nodes, connected by relationships as described below:

(:Movie)-[:HAS_GENRE]→(:Genre)
(:Actor)-[:ACTED_IN]→(:Movie)
(:Director)-[:DIRECTED]→(:Movie)

Additionally, users can create accounts, log in, and add their ratings to movies:

(:User)-[:RATES]->(:Movie)

The API

The Flask portion of the application interfaces with the database and presents data to the React.js front-end via a RESTful API. You can find the flask API in the /flask-api directory in the repo.

The Front-End

The front-end, built in React.js, consumes the data provided by the Flask API and presents it through some views to the end user, including:

Home page
Movie detail page
Actor and Director detail page
User detail page
Sign-up and Login pages

You can find the front-end code in the web directory.

Setup

To get the project running, clone the repository and follow along with these instructions, which are recapped in the video:

First, Start the Database!

Your app will need a database, and the easiest way to access a database that’s already full of data is by connecting directly to the "Recommendations" database in Neo4j Sandbox.

Log in to Neo4j Sandbox by visiting https://sandbox.neo4j.com/, either using social authentication or your email and password.

After logging in to Neo4j Sandbox, tap "New Project" and select "Recommendations," then tap the blue "Launch Project" button to start the database you will be connecting to.

In order to connect to the database from the environment from which you’ll be running the app (presumably your local machine), you’ll need credentials. You can find those under the "Connection details" and/or the "Connect via drivers" tab:

Note the section that looks like this - you’ll need to copy+paste the credentials in the driver section to connect to the database from your local machine. For example, if the driver line contains the following:

driver = GraphDatabase.driver("bolt://52.72.13.205:47929", auth=basic_auth("neo4j", "knock-cape-reserve"))

Then, in your text editor, open and/or create flask-api/.env and enter the appropriate information into the variables: DATABASE_USERNAME, DATABASE_PASSWORD, and DATABASE_URL. Then save the file.

DATABASE_USERNAME = 'your usernamer'
DATABASE_PASSWORD = 'your password'
DATABASE_URL = 'your URL'

To start the Flask API, run:

cd flask-api
pip3 install -r requirements.txt
export FLASK_APP=app.py
flask run

Verify that the endpoints are running as expected by taking a look at the docs at: http://localhost:5000/docs

Start the React.js Front-End

With the database and backend running, open a new terminal tab or window and move to the project’s /web subdirectory. Run nvm use to ensure you’re using the node version specified for this project. If you don’t have the recommended version of node installed, follow the prompt to install the recommended version. After verifying you are using the recommended user, run:

npm install
cp src/config/settings.example.js src/config/settings.js
npm start

Navigate to view the app at http://localhost:3000/

Click on a movie poster to see its corresponding movie detail page.

Click on a cast or crew member to see that person’s profile, which includes biographical information, related people, and more movies the person has acted in, directed, written, or produced:

Going Through The Endpoints

Let’s look at how we would request a list of all the established genres from the database. The GenreList class queries the database for all Genre nodes, serializes the results, and then returns them via /api/v0/genres:

class GenreList(Resource):
    @swagger.doc({
        'tags': ['genres'],
        'summary': 'Find all genres',
        'description': 'Returns all genres',
        'responses': {
            '200': {
                'description': 'A list of genres',
                'schema': GenreModel,
            }
        }
    })
    def get(self):
        def get_genres(tx):
            return list(tx.run('MATCH (genre:Genre) SET genre.id=id(genre) RETURN genre'))
        db = get_db()
        result = db.read_transaction(get_genres)
        return [serialize_genre(record['genre']) for record in result]

def serialize_genre(genre):
    print(genre)
    return {
        'id': genre['id'],
        'name': genre['name'],
    }

api.add_resource(GenreList, '/api/v0/genres')

What’s Going on with the Serializer?

If you’ve only used a non-Bolt Neo4j driver before, these bolt-driver responses may be different from what you’re used to. In the "GET all Genres" example described above, result = db.read_transaction(get_genres) returns a series of records:

[<Record genre=<Node id=1 labels=frozenset({'Genre'}) properties={'name': 'Adventure', 'id': 1}>>, <Record genre=<Node id=2 labels=frozenset({'Genre'}) properties={'name': 'Animation', 'id': 2}>>, <Record genre=<Node id=3 labels=frozenset({'Genre'}) properties={'name': 'Children', 'id': 3}>>, <Record genre=<Node id=4 labels=frozenset({'Genre'}) properties={'name': 'Comedy', 'id': 4}>>, <Record genre=<Node id=6 labels=frozenset({'Genre'}) properties={'name': 'Fantasy', 'id': 6}>>, <Record genre=<Node id=9 labels=frozenset({'Genre'}) properties={'name': 'Romance', 'id': 9}>>, <Record genre=<Node id=10 labels=frozenset({'Genre'}) properties={'name': 'Drama', 'id': 10}>>, <Record genre=<Node id=13 labels=frozenset({'Genre'}) properties={'name': 'Action', 'id': 13}>>, <Record genre=<Node id=14 labels=frozenset({'Genre'}) properties={'name': 'Crime', 'id': 14}>>, <Record genre=<Node id=16 labels=frozenset({'Genre'}) properties={'name': 'Thriller', 'id': 16}>>, <Record genre=<Node id=23 labels=frozenset({'Genre'}) properties={'name': 'Horror', 'id': 23}>>, <Record genre=<Node id=33 labels=frozenset({'Genre'}) properties={'name': 'Mystery', 'id': 33}>>, <Record genre=<Node id=37 labels=frozenset({'Genre'}) properties={'name': 'Sci-Fi', 'id': 37}>>, <Record genre=<Node id=49 labels=frozenset({'Genre'}) properties={'name': 'Documentary', 'id': 49}>>, <Record genre=<Node id=51 labels=frozenset({'Genre'}) properties={'name': 'IMAX', 'id': 51}>>, <Record genre=<Node id=56 labels=frozenset({'Genre'}) properties={'name': 'War', 'id': 56}>>, <Record genre=<Node id=63 labels=frozenset({'Genre'}) properties={'name': 'Musical', 'id': 63}>>, <Record genre=<Node id=161 labels=frozenset({'Genre'}) properties={'name': 'Western', 'id': 161}>>, <Record genre=<Node id=162 labels=frozenset({'Genre'}) properties={'name': 'Film-Noir', 'id': 162}>>, <Record genre=<Node id=7745 labels=frozenset({'Genre'})]

The serializer parses these slightly results into the processed data we need:

def serialize_genre(genre):
    return {
        'id': genre['id'],
        'name': genre['name'],
    }

Voila! You get an array of genres at /genres.

Beyond the `/Genres` Endpoint

Of course, an app that just shows movie genres isn’t very interesting. Take a look at the routes and models used to build the home page, movie detail page, and person detail page.

The User Model

Aside from creating themselves and authenticating with the app, Users can rate Movies with the :RATED relationship, illustrated below.

User Properties

password: The hashed version of the user’s chosen password api_key: The user’s API key, which the user uses to authenticate requests username: The user’s chosen username

:RATED Properties

rating: an integer rating between 1 and 5, with 5 being love it and 1 being hate it

Users Can Create Accounts

Before a User can rate a Movie, the user has to exist, i.e. someone has to sign up for an account. The sign-up process will create a node in the database with a User label, along with the properties necessary for logging in and maintaining a session.

The registration endpoint is located at /api/v0/register. The app automatically submits a request to the register endpoint when a user fills out the "Create an Account" form and taps "Create Account." Assuming you have the API running, you can test requests either by using the interactive docs at /3000/docs/ or by using cURL.

Naturally, you should replace the placeholder fields throughout with your chosen username and password.

Example: Create a New User

Request

curl -X POST --header 'Content-Type: application/json' --header 'Accept: application/json' -d '{ "username": "Mary Jane", "password": "SuperPassword"}' 'http://localhost:5000/api/v0/register'

Response

{
   "id":"e1e157a2-1fb5-416a-b819-eb75c480dfc6",
   "username":"Mary333 Jane",
   "avatar":{
      "full_size":"https://www.gravatar.com/avatar/b2a02..."
   }
}

Example: Try to Create a New User but Username is Already Taken

Request

curl -X POST --header 'Content-Type: application/json' --header 'Accept: application/json' -d  '{ "username": "Mary Jane", "password": "SuperPassword"}' 'http://localhost:5000/api/v0/register'

Response

{
    "status":400,
    "username":"username already in use"
}

User registration logic is implemented in /flask-api/app.py as described below:

User Registration

class Register(Resource):
    @swagger.doc({
        'tags': ['users'],
        'summary': 'Register a new user',
        'description': 'Register a new user',
        'parameters': [
            {
                'name': 'body',
                'in': 'body',
                'schema': {
                    'type': 'object',
                    'properties': {
                        'username': {
                            'type': 'string',
                        },
                        'password': {
                            'type': 'string',
                        }
                    }
                }
            },
        ],
        'responses': {
            '201': {
                'description': 'Your new user',
                'schema': UserModel,
            },
            '400': {
                'description': 'Error message(s)',
            },
        }
    })
    def post(self):
        data = request.get_json()
        username = data.get('username')
        password = data.get('password')
        if not username:
            return {'username': 'This field is required.'}, 400
        if not password:
            return {'password': 'This field is required.'}, 400

        db = get_db()

        results = db.run(
            '''
            MATCH (user:User {username: {username}}) RETURN user
            ''', {'username': username}
        )
        try:
            results.single()
        except ResultError:
            pass
        else:
            return {'username': 'username already in use'}, 400

        results = db.run(
            '''
            CREATE (user:User {id: {id}, username: {username},
                               password: {password},
                               api_key: {api_key}}) RETURN user
            ''',
            {
                'id': str(uuid.uuid4()),
                'username': username,
                'password': hash_password(username, password),
                'api_key': binascii.hexlify(os.urandom(20)).decode()
            }
        )
        user = results.single()['user']
        return serialize_user(user), 201

Users Can Log In

Now that users are able to register for an account, we can define the view that allows them to login to the site and start a session.

The registration endpoint is located at /api/v0/login. The app submits a request to the login endpoint when a user fills in the username and password text boxes and taps "Create Account." Assuming you have the API running, you can test requests either by using the interactive docs at /5000/docs/ or by using cURL.

Example: Login

Request

curl -X POST --header 'Content-Type: application/json' --header 'Accept: application/json' -d '{"username": "Mary Jane", "password": "SuperPassword"}' 'http://localhost:5000/api/v0/login'

Response

{
    "status":200,
    "token":"31361a8d0f479f3da6c3a04793744c70f998be11"
}

Example: Wrong Password

Request

curl -X POST --header 'Content-Type: application/json' --header 'Accept: application/json' -d '{ "username": "Mary Jane", "password": "SuperPassword"}' 'http://localhost:5000/api/v0/register'

Response

{
    "password":"wrong password",
    "status":400
}

Example: See Myself

Request

curl -X GET --header 'Accept: application/json' --header 'Authorization: Token 5a85862fb28a316ea6a1' 'http://localhost:5000/api/v0/users/me'

Response

{
    "avatar": {
        "full_size":"https://www.gravatar.com/avatar/c2eab..."
    },
    "id": "94a604f7-3eab-4f28-88ab-12704c228936",
    "status":200,
    "username":"Mary Jane"
}

The code here is similar to that of /register. There is a similar form to fill out, where a user types in their username and password. With the given username, a User is initialized. The password they filled out in the form is then verified against the hashed password that was retrieved from the corresponding :User node in the database. If the verification is successful, the program will return a token. The user is then directed to an authentication page, from which they can navigate through the app, view their user profile, and rate movies.

Example: Users Can Rate Movies

Once a user has logged in and navigated to a page that displays movies, they can select a star rating for any movie in the page or remove any of their previous movie ratings.

The user can access their previous ratings and the respective movies that were rated through both their user profile and the movie detail page in question.

Example: Rate a Movie

Request

curl -X POST --header 'Content-Type: application/json' --header 'Accept: application/json' --header 'Authorization: Token ce40f63e79344f017a48b205db27aeaa301ae2b6' -d '{"rating":4}' 'http://localhost:5000/api/v0/movies/15602/rate'

Response

{"status":200}

Python Implementation: Rate a Movie

class RateMovie(Resource):
    @login_required
    def post(self, id):
        parser = reqparse.RequestParser()
        parser.add_argument('rating', choices=list(range(0, 6)),
                            type=int, required=True,
                            help='A rating from 0 - 5 inclusive (integers)')
        args = parser.parse_args()
        rating = args['rating']

        db = get_db()
        results = db.run(
            '''
            MATCH (u:User {id: {user_id}}),(m:Movie {id: {movie_id}})
            MERGE (u)-[r:RATED]->(m)
            SET r.rating = {rating}
            RETURN m
            ''', {'user_id': g.user['id'], 'movie_id': id, 'rating': rating}
        )
        return {}

    @login_required
    def delete(self, id):
        db = get_db()
        db.run(
            '''
            MATCH (u:User {id: {user_id}})
                          -[r:RATED]->(m:Movie {id: {movie_id}}) DELETE r
            ''', {'movie_id': id, 'user_id': g.user['id']}
        )
        return {}, 204

Example: See All of My Ratings

Request

curl -X GET --header 'Accept: application/json' --header 'Authorization: Token ce40f63e79344f017a48b205db27aeaa301ae2b6' 'http://localhost:5000/api/v0/movies/rated'

Response

[
  {
    "duration": 101,
    "id": "15602",
    "my_rating": 4,
    "poster_image": "https://image.tmdb.org/t/p/w440_and_h660_face/1FSXpj5e8l4KH6nVFO5SPUeraOt.jpg",
    "rated": 6.6,
    "released": "1995-12-22",
    "summary": "John and Max resolve to save their beloved bait shop from turning into an Italian restaurant, just as its new female owner catches Max's attention.",
    "tagline": "John and Max resolve to save their beloved bait shop from turning into an Italian restaurant, just as its new female owner catches Max's attention.",
    "title": "Grumpier Old Men"
  }
]

Python Implementation: See My Ratings

class MovieListRatedByMe(Resource):
    @login_required
    def get(self):
        db = get_db()
        result = db.run(
            '''
            MATCH (:User {id: {user_id}})-[rated:RATED]->(movie:Movie)
            RETURN DISTINCT movie, rated.rating as my_rating
            ''', {'user_id': g.user['id']}
        )
        return [serialize_movie(record['movie'],
        record['my_rating']) for record in result]

...

def serialize_movie(movie, my_rating=None):
    return {
        'id': movie['tmdbId'],
        'title': movie['title'],
        'summary': movie['plot'],
        'released': movie['released'],
        'duration': movie['runtime'],
        'rated': movie['imdbRating'],
        'tagline': movie['plot'],
        'poster_image': movie['poster'],
        'my_rating': my_rating,
    }

Example: My Recommendations

Request

curl -X GET --header 'Accept: application/json' --header 'Authorization: Token ce40f63e79344f017a48b205db27aeaa301ae2b6' 'http://localhost:5000/api/v0/movies/recommended'

Response

[
  {
    "duration": 82,
    "id": "45523",
    "my_rating": null,
    "poster_image": "https://image.tmdb.org/t/p/w440_and_h660_face/8mJMrrT4tkfZLMFvKQ0Hq6jlXbp.jpg",
    "rated": 8.6,
    "released": "2010-01-26",
    "summary": "In this unique and dynamic live concert experience, Louis C.K.'s exploration of life after 40 destroys politically correct images of modern life with thoughts we have all had...but would rarely admit to.",
    "tagline": "In this unique and dynamic live concert experience, Louis C.K.'s exploration of life after 40 destroys politically correct images of modern life with thoughts we have all had...but would rarely admit to.",
    "title": "Louis C.K.: Hilarious"
  },
  {
    "duration": 100,
    "id": "38757",
    "my_rating": null,
    "poster_image": "https://image.tmdb.org/t/p/w440_and_h660_face/1uPxRO0iYwW02lzwatRhkugWBYs.jpg",
    "rated": 7.8,
    "released": "2010-11-24",
    "summary": "The magically long-haired Rapunzel has spent her entire life in a tower, but now that a runaway thief has stumbled upon her, she is about to discover the world for the first time, and who she really is.",
    "tagline": "The magically long-haired Rapunzel has spent her entire life in a tower, but now that a runaway thief has stumbled upon her, she is about to discover the world for the first time, and who she really is.",
    "title": "Tangled"
  },
...
]

@login_required
def get(self):
    def get_movies_list_recommended(tx, user_id):
        return list(tx.run(
            '''
            MATCH (me:User {id: $user_id})-[my:RATED]->(m:Movie)
            MATCH (other:User)-[their:RATED]->(m)
            WHERE me <> other
            AND abs(my.rating - their.rating) < 2
            WITH other,m
            MATCH (other)-[otherRating:RATED]->(movie:Movie)
            WHERE movie <> m
            WITH avg(otherRating.rating) AS avgRating, movie
            RETURN movie
            ORDER BY avgRating desc
            LIMIT 25
            ''', {'user_id': user_id}
        ))
    db = get_db()
    result = db.read_transaction(get_movies_list_recommended, g.user['id'])
    return [serialize_movie(record['movie']) for record in result]

The React Front-end

You can take a look at the React front-end code at in the /web/ subdirectory. The React front-end is very simple, and is composed of the following parts, as described in the video above:

Home Page
Authentication Page
Movie Detail Page
Person (Actor, Director) Page
User Profile Page

Home Page

The home page is a relatively simple page making calls to two endpoints: the "GET movies by genre endpoint" and the "GET movie by ID" endpoint.

The "Featured Movies" portion at the top calls three hard-coded movies.

renderFeatured() {
    var {movies} = this.props;

    return (
      <div className="nt-home-featured">
        <h3 className="nt-home-header">Featured Movies</h3>
        <ul>
          { _.compact(movies.featured).map(f => {
            return (
              <li key={f.id}>
                <Link to={`/movie/${f.id}`}>
                  <img src={f.posterImage} alt="" />
                </Link>
              </li>
            );
          })}
        </ul>
      </div>
    );
  }

static getFeaturedMovies() {
    return Promise.all([
        axios.get(`${apiBaseURL}/movies/13380`),
        axios.get(`${apiBaseURL}/movies/15292`),
        axios.get(`${apiBaseURL}/movies/11398`)
    ]);
}

Movie and Person Detail

The Movie and Person detail are visually very similar pages - both with a poster image on the left and carousels on the bottom. However, the Movie page is different depending on whether the user is authenticated, as the authenticated user is able to mark their rating on each movie.

User Profile

The User Profile page allows the user to re-rate or un-rate their movies, and view more movie recommendations based on those ratings.

Deployment

Deploying the Recommendations Database with AuraDB

This section demonstrates how to deploy the Recommendations database to AuraDB

Download the Dump File
Create an account on AuraDB
Upload the dump and start the database

Deploying the Backend and Front-end with Heroku

You will have to create two apps on Heroku: one for the backend and one for the front-end.

Starting with the backend, create a new app on Heroku. On your local machine, add the Heroku repo as a remote. On Heroku > Settings > Config Vars, add the credentials to connect to the database hosted Neo4j AuraDB (or the sandbox if you haven’t migrated to AuraDB).

Then, create another Heroku app for the front-end. Add another git remote pointed to the Heroku app dedicated to the front-end app. Under Heroku > Settings > Config Vars, add the environment variables for the REACT_APP_API_BASE_URL and REACT_APP_PROXY_URL fields.

Under Heroku > Settings > Buildpacks, add mars/create-react-app to load dependencies.

Check out the Makefile in the root directory of the project. It contains the commands needed to deploy the project. You can run deploy-api to deploy the Rlask API and deploy-web to run deployment on the React site.

Makefile

deploy-api:
    git branch -f heroku-api
    git branch -D heroku-api
    git subtree split --prefix flask-api -b heroku-api
    git push heroku-api heroku-api:master --force

deploy-web:
    git branch -f heroku-web
    git branch -D heroku-web
    git subtree split --prefix web -b heroku-web
    git push heroku-web heroku-web:master --force

Next Steps

Fork the repo and hack away! Find directors that work with multiple genres, or find people who tend to work with each other frequently as writer-director pairs. Did you find a way to improve the application or the Python driver? Create a GitHub Issue and/or submit a pull request.

Was this page helpful?

Getting Started with Neo4j and Flask

Before We Start - Why Neo4j?

Introduction to Neo4j

Cypher

The Neo4j Movie Flask/React App

The Database

The API

The Front-End

Setup

Start the React.js Front-End

Going Through The Endpoints

What’s Going on with the Serializer?

Beyond the /Genres Endpoint

The User Model

User Properties

:RATED Properties

Users Can Create Accounts

Users Can Log In

The React Front-end

Home Page

Movie and Person Detail

User Profile

Deployment

Deploying the Recommendations Database with AuraDB

Deploying the Backend and Front-end with Heroku

Next Steps

Beyond the `/Genres` Endpoint