Sourcing local .json files with Gatsby

• Node.js
• Gatsby
• JavaScript
• JSON

Hi all, I recently had a discussion with Rahul on Twitter regarding building a Gatsby blog using only locally sourced .json files.

There are a number of ways to achieve this so here’s a demo and a repo if you’d prefer to dive straight in.

• Demo
• Repo Branches
  • main - (single source .json)
  • multiple-json-files
  • gatsby-plugins

These are three main methods i’ll be discussing

1. Sourcing data from a single .json file using Node.js
2. Sourcing data from multiple .json files using Node.js
3. Sourcing data from multiple .json files using Gatsby plugins

The first two methods use the sourceNodes extension point which is just one of the many Gatsby Node API methods

The first two methods will also use the createNode action to inject “data” found in the .json source files into Gatsby’s GraphQL layer.

The last method will use only Gatsby Plugins.

All methods will be using the File System Route API and all of the above methods will eventually create post and posts in Gatsby’s GraphQL layer which will be used by the “curly brace” syntax collection routes method to create each of the “post” pages seen in the demo

Single .json file

The code for this example can be found on this branch: main, and the directory structure looks like this

content
   |-- MOCK_DATA.json

// gatsby-node.js

const MOCK_DATA = require('./content/MOCK_DATA.json');

exports.sourceNodes = ({ actions: { createNode }, createContentDigest }) => {
  MOCK_DATA.forEach((data, index) => {
    createNode({
      ...data,
      id: `post-${index}`,
      internal: {
        type: `post`,
        contentDigest: createContentDigest(data),
      },
    });
  });
};

From top to bottom here’s what’s going on.

1. Fist I require a single .json file called MOCK_DATA.json
2. Then by using the sourceNodes extension point I can access Gatsby’s Actions and add my own code
3. I’ve used forEach to iterate over every object contained within the MOCK_DATA.json file and “for each” object found I use createNode to inject the “data” from each object into Gatsby’s GraphQL layer along with some other required parameters, E.g id, internal.type and internal.contentDigest

Once Gatsby has created the node it’s then query-able using GraphQL. An example query in GraphiQL would look like this 👇

// gets all posts

  allPost {
    nodes {
      id
      title
    }
  }

// get a single post by `id`

  post(id: {eq: "post-0"}) {
    title
  }

The query names post and allPost are created by Gatsby using the name provided by internal.type

Multiple .json files

The code for this example can be found on this branch: multiple-json-files, and the directory structure looks like this

content
   |-- 01-Mar-2021.json
   |-- 05-Jul-2020.json
   |-- 07-May-2020.json
   ...

// gatsby-node.js

const { readdirSync, readFileSync } = require('fs');

exports.sourceNodes = ({ actions: { createNode }, createContentDigest }) => {
  const DIR = './content';

  files = readdirSync(DIR);

  files.forEach((file, index) => {
    // ignore files starting with a dot
    if (!/^\..*/.test(file)) {
      let data = JSON.parse(readFileSync(`${DIR}/${file}`));

      createNode({
        ...data,
        id: `post-${index}`,
        internal: {
          type: `post`,
          contentDigest: createContentDigest(data),
        },
      });
    }
  });
};

This approach is a bit more of a rascal because each object from the previous example has now been extracted into individual .json files and named by date.

1. First I use the sourceNodes extension point so I can access Gatsby’s Actions plus add my own code
2. I’ve used Node’s readdirSync to read the directory
3. Then “for each” over each file found in that directory and use readFileSync and JSON.parse to read the data contained within the file before using createNode to inject the “data” into Gatsby’s GraphQL layer

As above once Gatsby has created the node it’s then query-able using GraphQL. An example query in GraphiQL would look the same as above but here it is again 👇

// gets all posts

  allPost {
    nodes {
      id
      title
    }
  }

// get a single post by `id`

  post(id: {eq: "post-0"}) {
    title
  }

Gatsby Plugins

The code for this example can be found on this branch: gatsby-plugins, and the directory structure looks like this

content
   |-- 01-Mar-2021.json
   |-- 05-Jul-2020.json
   |-- 07-May-2020.json
   ...

Using this final approach it’s possible to avoid gatsby-node.js altogether and instead achieve the exact same thing using gatsby-transform-json and gatsby-source-filesystem and configuring them in gatsby-config.js

// gatsby-config.js

module.exports = {
  plugins: [
    {
      resolve: `gatsby-transformer-json`,
      options: {
        typeName: `post`,
      },
    },
    {
      resolve: `gatsby-source-filesystem`,
      options: {
        path: `${__dirname}/content`,
        ignore: [`**/\.*`], // ignore files starting with a dot
      },
    },
  ],
};

By setting the typeName in the gatsby-transformer-json options to ‘post’ Gatsby will again create post and allPost meaning the nodes continue to be query-able as mentioned above using GraphQL. An example query in GraphiQL would again look the same as above E.g 👇

// gets all posts

  allPost {
    nodes {
      id
      title
    }
  }

// get a single post by `id`

  post(id: {eq: "post-0"}) {
    title
  }

Final Thoughts

It took me a little while to understand how GraphQL nodes were created by Gatsby and it’s fair to say that a lot of the source plugins remove the need for you to ever understand what’s going on under the hood but my hope is by explaining how Gatsby does this will make your life a bit easier when you come to work with these nodes when using either page queries, useStaticQuery or as i’ve done in this demo the File System Route API and syntax collection routes method.

If you’ve found any of the above confusing please do come find me on Twitter as i’m always happy to discuss.

Cheerio!