How to Use GraphQL DataLoader

GraphQL offers a very convenient way to handle database relations. But with this simplicity comes with an issue that actually easy to fix, but sometimes people forget about it. And that issue is known as the “N+1” problem. In this tutorial, we will learn on how to fix this problem using DataLoader.

Getting Started

The only thing we need to get started with this project is a blank folder with npm package initialized. So, lets create one!

$ mkdir learn-dataloader
$ cd learn-dataloader
$ npm init -y

Now, lets install some packages.

$ npm install apollo-server graphql

Here, we’re installing Apollo Server and GraphQL to build our GraphQL server.

Or, I have already published the source code of this entire project on my GitHub. Go ahead and clone this repo into your computer.

$ git clone https://github.com/rahmanfadhil/learn-dataloader.git

GraphQL Server

First, we need to create a GraphQL server. Let’s do that inside index.js file.

const { ApolloServer, gql } = require("apollo-server")
const typeDefs = require("./typeDefs")
const resolvers = require("./resolvers")

const server = new ApolloServer({
	typeDefs,
	resolvers,
})

server.listen().then(() => console.log("Server has started!"))

If you haven’t created a GraphQL server before, it consists of three parts, the server, resolver, and type definitions. You can read more about it right here.

And, here is our type definitions inside typeDefs.js.

const { gql } = require("apollo-server")

module.exports = gql`
	type Query {
		hello: String
	}
`

For now, it only contains a single query called hello, which will then returns a string.

Finally, here is our resolvers inside resolvers.js.

module.exports = {
	Query: {
		hello: () => "world",
	},
}

It just resolves the value of hello query that we define in the typeDefs.js.

Now, let’s see if our server is actually working.

Data

Let’s say we want to create a server that provides a list of books and those authors. We can store it inside the data.js.

const authors = [
	{ id: 1, name: "Simon Sinek", email: "simon@example.com" },
	{ id: 2, name: "Malcolm Gladwell", email: "malcolm@example.com" },
]

const books = [
	{ id: 1, title: "Start with Why", author: 1 },
	{ id: 2, title: "The Tipping Point", author: 2 },
	{ id: 3, title: "Leaders Eat Last", author: 1 },
	{ id: 4, title: "Outliers", author: 2 },
]

module.exports = { authors, books }

Here, we have an array of authors that has a name and email. Each of them also has their own unique id which allows us to relate them into another data type, which in this case is the books.

In the book data type, we have the title as well as the id. We also have the author, which refers to the author of that book.

Now, I hope you get the idea right. An author can have multiple books, but a book can only have a single author.

We also want to export those variables, so that we can use it inside another files.

Data resolvers

Let’s provide these data in our GraphQL server. We can start by adding the type definitions.

const { gql } = require("apollo-server")

module.exports = gql`
	type Query {
		books: [Book]
	}

	type Book {
		id: ID
		title: String
		author: Author
	}

	type Author {
		name: String
		email: String
	}
`

Then, we define the resolver of the books query, which is pretty straightforward.

const { authors, books } = require("./data")

module.exports = {
	Query: {
		books: () => books,
	},
}

Now, let’s try to run the server and fetch all books and those authors.

As you can see, our GraphQL API works just fine until we try to fetch the author of those books.

And the reason why get null in the author field is because we don’t tell the server how to get those authors. So, let’s make some adjustments in our resolvers.

const { authors, books } = require("./data")

module.exports = {
	Query: {
		books: () => books,
	},
	Book: {
		author: (parent) => authors.find((author) => author.id === parent.author),
	},
}

Now, we can get the author by using the parent value that passed to the resolver function and find the author by that id.

If you try that query again, you can see that now we get those authors.

The N+1 Problem

Now, here is the thing. When we try fetching the author of those books, we’re getting a common relational problem called “N+1”. And here is how it looks like.

Try to print out the author ids while fetching them.

const { authors, books } = require("./data")

module.exports = {
	Query: {
		books: () => books,
	},
	Book: {
		author: (parent) => {
			console.log(`fetching author ${parent.id}`)
			return authors.find((author) => author.id === parent.author)
		},
	},
}

Then, run the server and execute the previous GraphQL query to fetch all books and those authors.

$ node index.js
Server has started!
fetching author 1
fetching author 2
fetching author 1
fetching author 2

Even though we only have two authors, we’re fetching them four times. And this is not an efficient approach because we’re trying to fetch the same author over and over again.

Imagine if these data are stored inside a database. How long does it takes to run those queries, even though we can get all of them at the same time with a single query.

This is where GraphQL DataLoader comes very handy. It allows us to bulk the process of fetching those relational data, so that we can query them at the same time.

DataLoader

First, we can install the dataloader package using NPM.

$ npm install dataloader

Then, we can make a new loader in the author resolver to get our book’s author.

const DataLoader = require("dataloader")

module.exports = {
	Query: {
		books: () => books,
	},
	Book: {
		author: (parent) => {
			const authorLoader = new DataLoader((keys) => {
				const result = keys.map((authorId) => {
					return authors.find((author) => author.id === authorId)
				})

				return Promise.resolve(result)
			})

			return authorLoader.load(parent.author)
		},
	},
}

Here, we’re constructing a new DataLoader instance where we define how to grab every single authors that need to be displayed at the same time. First, we loop through the keys, which contains an array of author ids. Then, we find each of those authors.

I don’t know why, but the loader function MUST returns a promise. That’s why we use Promise.resolve to transform our result into a promise.

That’s it!

You can try that out, and you will get the same result. But this time, the query only happened once. Now, the problem is solved by simply using DataLoader to grab relational data in our GraphQL API.

Tags: Javascript, Nodejs, GraphQL