Xfinity xFi & User Personalization with Graphs

Editor’s Note: This presentation was given by Jessica Lowing at GraphConnect New York in September 2017.

Presentation Summary

We’ve come a long way towards making our homes smarter – but we still have a long way to go. Many of the tasks we want our homes to perform, such as “turn out the lights in my kid’s’ room,” requires insight into complex semantic and social relationships.

Comcast has embarked on creating and perfecting its xFi smart home prototype, which – based on research – they knew had to include connected devices, connected people, rich interfaces and automation to be truly helpful. Some products that grew out of this prototype include tools like Kidwatch, which notifies you when your child arrives home from school, and Porchcam, which displays on your TV screen the person who just rang your doorbell.

The xFi team quickly learned that the key to making these tools successful was to use rich definitions for their data in order to leverage relationships that makes their tools as useful as possible. This led to the Comcast allows customers to personalize their smart homes:



Have you ever fallen asleep with the lights on in another room? Or left the house wondering if the oven was on or if the windows or garage door were open? Or left the house without taking an umbrella or jacket? Or been at the grocery store and wondered whether or not you had any chicken left in the fridge?

If our homes were actually smart, it would do all of this stuff for us. Today, there’s a lot of things our homes could be doing for us that they aren’t. And unfortunately, the only progress we’ve made so far is to connect a bunch of things. But in reality, we still have a long way to go towards making our homes truly interactive and helpful.

When it comes down to it, our homes still aren’t very smart.

Smart Home, Turn Off My Kid’s Lights!

Let’s dive into an example with my BFFs River and Lily. Let’s say we want our smart home to turn off the lights in Lily’s room, which is a pretty simple request that your average smart home could probably do. “Alexa, turn off the floor lamp in Lily’s room.” Alexa has that covered.

How about, “Start the washer and dryer after this movie is over?” This request gets a bit trickier, so Alexa probably couldn’t handle this one. If you live in a giant house, you might also want to know whether or not the girls are home, which is something your smart home can’t currently tell you.

Our light example showcases a strong relationship to semantics and social relationships, where we have to know a lot of information: who is speaking, who are the kids, what’s in the kids’ room, do they have lights in their room, etc.

Let’s look at this from a computer’s point of view:



The answers to our questions above are as follows: I am Maggie (a red node), my kids are Lily and River (also red nodes), the room I’m talking about is the kids’ bedroom (purple node), and the subset of light devices in the room I want to control are a floor lamp, nightstand light and ceiling light (green nodes).

If you were truly in a smart home, you would be able to speak naturally to it. It would know you and your family, the context of your real life, and the world you live in — and have the ability to do the things you want, when you want. As a parent, you want to feel safe, comfortable, in control of the sea of devices in your home, as well as connected to the people outside of your home.

The experience of a smart home should be easy and rewarding, not a burden. This is the big problem we’ve been looking at for a long time.

Building the Smart Home Prototype

Comcast is a large telecommunications company that provides cable television, ISP, products for home security, mobile phones and telephony. We own NBCUniversal and have syndication partners across North America.

In 2013, we started a new team based out of Sunnyvale, California – with part of the team in Philadelphia, where Comcast is headquartered – to research and prototype a smart home of the future. We named our team “Jarvis” for none other than Tony Stark’s smart home in Iron Man, and I joined the team as a technical product manager. My masters from MIT’s AI lab in distributed robotic control systems helped me bring insight to this connected home and the problem space in which we’re working.

We started by taking a long, hard look at the question, “What would the connected home of the future really look like?” We based our prediction off of university research, trends in the market and products offered by different companies, which together led to some core themes:



“Connected devices” refers to the ability for all the devices in your home to naturally interact; connected people, which amounts to integrating the natural relationships between people in the real world; rich interfaces, which refers to all the voices and interfaces of different devices; and automation to control those devices.

Next, we set out to develop some functional prototypes that would incorporate these core themes:



These included:

• PorchCam on TV: When my doorbell rings, I want to see my PorchCam on my X1 television and tell my smart home to unlock the door.
• Where’s Waldo: Ask my smart home, “Where is my daughter?” and see a map of Lily’s coordinates within the city, displayed on my TV.
• BYODevice: Bring any device to my home, and quickly and easily integrate it into my smart home of other devices
• Kidwatch: Tell me when my youngest daughter comes home from school, or if she fails to arrive by 4:00 PM.
• Jawbone Granny: Jawbone is like a FitBit but with an open API. Has my grandmother exercised today? If not, we should plan a family walk around dinnertime.

From Prototype to Product

We held a series of internal demos within Comcast, and the response was very positive – so positive that we were told to go build these tools. Our next step was to evaluate the gap between prototype and production by examining the current services available at Comcast, and on the broader market, and determining the work it would take to bridge that gap.

We started with Xfinity Home, a home security product with a suite of devices such as cameras, lights and door locks that is controlled via an interface. But this system doesn’t include a socially-embedded functional network, because it only included the subset of devices that came off the truck with your technician.

In 2015, we launched Works with Xfinity Home, which let you add other devices to your Xfinity Home. Now “Turn off all the lights” could include your Philips Hue or your Lutron dimmer. In 2016, we tackled automation by building the Xfinity Rules Engine, Rulio. It’s a really powerful, open-source tool that lets you ask questions like, “Tell me when River comes home from school.”

Let’s review how we accomplish this within Xfinity Home:



I want to know when the kids come home from school, so I’ll set up a push notification or SMS whenever the door opens or the system is disarmed. I only have two rules to manage, and all is good in the world. Now I live in a smart home.

But what happens when you add more components, like facial and voice recognition, or when specific devices join your home WiFi network?



No human in their right mind is going to want to manage this number of device-based rules.

When it comes down to it, device automation is not the solution here. So even though we’ve built a lot of connected tools, we still haven’t built a smart home yet.

The Key to Success: Rich Definitions

Circling back to our initial example, we want to know when River comes home. But this notification doesn’t currently differentiate between who is arriving home. It could be River, or the dog-walker or the babysitter.

To begin making these differentiations, we need a richer definition of what a person is within our systems:



A person is not just an ID. A person is a set of relationships to personal information, locations, people and devices.

Similarly, we also needed a rich definition of a home:



What is a home? It’s a physical building in a specific location with a unique topology. There are people who live there and come and go, devices that come and go, and services that support those devices.

Why do we need this rich definition? Since people are at the center of these smart homes, they also need to be at the center of our automation – which brings us back to modeling social and semantic relationships.

This takes us to the XFINITY profile graph, which is a scalable, flexible, multi-tenant user-profile service for extending personal information and relationships across XFINITY products. It lets us model our customers’ real-life relationships, and provides the context so that XFINITY applications provide a more personalized experience for users.

The XFINITY Profile Graph

Design Principles

Before I get into the details, I want to cover a few core ideas and design principles we used when building this user graph:



Big idea #1: We need to build a shared platform at the household level that can be used by any Xfinity application to provide the user(s) the same set of information.

Big idea #2: We need to model this set of relationships as a graph data structure. This data is richly connected, and the real value provided is the set of relationships between the data points.

Big idea #3: We need a native graph database. A bad design would be to support graph semantics on a relational database, or in a non-relational database with indexing. But in either of those structures, the graph complexity would grow along with the data, which would require giant crawls through the database to take advantage of those relationships. In a native graph, we bypass this problem.

GraphQL and Neo4j

The Xfinity profile graph is built with GraphQL APIs on Neo4j, a natural fit. This allows platform developers to build generic, expressive APIs for our clients, and provides client developers with the benefit of being intuitive and flexible.

It provides the ability to query for information each of the different applications are interested in, and supports the specific traversal through that graph. For the end customer, it provides applications that support unique experiences for each customer based off that same data service, but with unique information for each household.

Let’s take a look at this again: “Turn off the lights in my kid’s room.” It was kind of tricky before, but it’s not so bad when you look at it through GraphQL:



We can build this and give it to our applications.

The User Interface

Let’s take a look at how this can surface to an end customer, taking a person’s WiFi network as an example. This is traditionally how people manage their home WiFi network: you write your network name and your password on a sticky note and give it to whoever needs to use it. This provides no insight into your network, or any sort of connection to the people in your home.

Xfinity xFi is Comcast’s personalized WiFi experience that is tailored to customers’ individual needs. xFi has introduced user profiles, which lets you name people in your home and assign devices to them. There’s also a concept of “guest” and “household.”



xFi lets you manage your network in terms of the people in your home, a novel way of managing network management that allows you to really dig in and see what’s going on with their devices:



For each device, you can go in and see individual usage patterns and pause individual devices. Now you can pause all of your kids’ devices when it’s dinnertime, which is really the people-based experience parents are after.

Overall, it gives you an easy way to see what’s going on and control devices at the household, person, and device level. You can learn more about what Xfinity xFi does here. We also made some big announcements about the future of xFi at CES in February of 2018.

Apply Smart Home Technology to Your Industry

If you knew more about your customers, how could you change the products you’re building today? You are just as empowered as my team was back in 2013 to take a hard look at the state of the world and make predictions about where it’s going, to explore, to try things out and prototype.

Connect with your users. Let your product fit naturally into their lives because every customer’s life is going to be unique and different. Go build great personalized products, because that’s what’s getting us closer to a smart home. And you are empowered to do exactly that.