Tuesday, November 17, 2020 By Paul Scanlon

Use Netlify Functions and the Twitter API v2 as a CMS for your Gatsby blog

Apologies in advance for the rather long-winded blog title but as it suggests in this post i’m going to explain how you can use Netlify Functions to access your Twitter profile data using the Twitter v2 API and display it on your Gatsby blog.

A rather unique requirement

This might be a specific to me but I wanted to solve a little problem I was having with my “digital footprint”. As you can see I have this blog: https://paulie.dev and a commercial portfolio: https://www.pauliescanlon.io

Both sites are built on top of my Gatsby theme: gatsby-theme-terminal which is Open source and can be found on my GitHub

Using a Gatsby Theme solves one of my issues as I’m able to have two sites that look and work pretty much the same way and any changes I make to the theme are inherited by both my sites. It’s kind of like managing your own multi brand design system, but just for yourself.

There was one other problem though. 🤔

I wanted both sites to have the same “intro” section, but every time I made a change to one I had to make the same change to the other site to ensure they were both displaying the same intro text.

This might be fine if I weren’t a developer but doing something twice is one time too many IMO.

It was also a little frustrating because I also wanted my Twitter profile description to be in sync with both the sites so, again another place to remember to update my personal blurb.

One option I considered would have been to hook up a Content Management System, and this would have been fine and it would have kept both my sites in sync but it wouldn’t have been able to update my Twitter profile blurb…

So, I’ve decided to reverse engineer the Twitter API and use that as a CMS to populate both my sites. The idea is quite simple. I’ll use the Twitter profile description as though it were a field from a CMS. Naturally any changes I make to this will appear on my Twitter profile and below is how I pull that same info into both of my sites.

Demo

Here’s what I’ll be showing you how to build:

App / API https://gatsby-netlify-twitter.netlify.app
GitHub repo https://github.com/PaulieScanlon/gatsby-netlify-twitter

… but the actual API I use for my blog and site is here: https://paulie-api.netlify.app

Tech

Netlify Functions

“Power your site without managing servers” is how Netlify describe Functions and for all intents and purposes thats exactly what they are. Similar to how you might create an Express app and deploy it somewhere but without the hassle of having to setup server side environments and more crucially any really dweeby server uptime monitoring.

Twitter API v2

A set of endpoints that can be used to get data from Twitter. Any Twitter requests must be done server side and use a set of keys and tokens. You can’t unfortunately hit the Twitter API from the browser so we need a “server” or as mentioned above, a Netlify Function

Using both of the above i’ve made my own API endpoint which goes off and hits the Twitter API and returns my Profile information which I can then display in the intro section of my blog and site. I’ve deployed this API to Netlify and it’s completely de-coupled from either of my sites but will return data which can be fetched from client side “fetch” request from within my site and blog. That url again is here: https://paulie-api.netlify.app

Before we start

Before we get started there’s a couple of things you’ll need to have in place.

Twitter API v2

Apply for access to the Twitter API. This is quite a lengthy process so strap in and also bookmark this post as it might take a few days for Twitter to accept your application.

Once you have access you can head over to the Developer Portal and create a new project, and within the project you can create an “app”, I called mine “paulie-api”.

In here you’ll find all the API keys and tokens required to access the Twitter API. Make a note of them somewhere as we’ll be using them later.

Netlify CLI

To run Netlify Functions we’ll be using netlify dev rather than gatsby develop or yarn develop so you’ll need to install the Netlify CLI

The Build

In order to develop you own API I found it easiest to have some kind of “site” running at the same time which will access the API endpoint and render the response on the page. In the demo repo you’ll see i’ve set up a really simple Gatsby Site with one page that uses “fetch” to, er fetch and then render the data.

I’ve used Theme UI for the style but naturally you can choose whatever you like to do this.

Whether you’re starting from scratch or adding Netlify Functions to an existing project you’ll need to start by adding a functions dir to the root of your project.

|-- functions
  |-- package.json
|-- src
package.json

functions is kind of it’s own application so it’ll need it’s own package.json and will have one dependency on twitter-v2

// ./functions/package.json

{
  "name": "gatsby-netlify-twitter-api",
  "version": "1.0.0",
  "description": "An api for the Twitter v2 api",
  "main": "index.js",
  "scripts": {
    "test": "echo \"Error: no test specified\" && exit 1"
  },
  "keywords": [],
  "author": "",
  "license": "ISC",
  "dependencies": {
    "twitter-v2": "^0.1.2"
  }
}

Next have a look at .env.example. You’ll need to create your own .env file and add the environment variables as seen in the .env.example. Naturally you’ll want to change the GATSBY_TWITTER_USERNAME to your own Twitter username and the Twitter keys and tokens will be what I referenced earlier which are provided by the Twitter Developer Portal

// ./.env

GATSBY_API_URL=./.netlify/functions
GATSBY_TWITTER_USERNAME=

TWITTER_API_KEY=
TWITTER_API_KEY_SECRET=
TWITTER_ACCESS_TOKEN=
TWITTER_ACCESS_TOKEN_SECRET=

Next create a Twitter client, this is what we’ll use to pass the keys and tokens onto the Twitter API when we make a request

// ./functions/client.js

const Twitter = require('twitter-v2');

module.exports = {
  client: new Twitter({
    consumer_key: process.env.TWITTER_CONSUMER_KEY,
    consumer_secret: process.env.TWITTER_CONSUMER_KEY_SECRET,
    access_token: process.env.TWITTER_ACCESS_TOKEN,
    access_token_secret: process.env.TWITTER_ACCESS_TOKEN_SECRET,
  }),
};

You should now be looking at something similar to the below

...
|-- functions
  |-- client.js
  |-- package.json
|-- src
package.json
.env
...

Now we need to create the “endpoint” that our frontend will hit, which in turn goes off and grabs the data from the Twitter API.

I created a dir called twitter-user and inside I create a new file and called it twitter-user.js

...
|-- functions
  |-- client.js
  |-- twitter-user
    |-- twitter-user.js
  |-- package.json
|-- src
package.json
.env
...

It’s in here where we can use the client.js to hit a Twitter API endpoint and pass with it the required keys and tokens from the client

// ./functions/twitter-user/twitter-user.js

const { client } = require('../client');

exports.handler = async (event, context, callback) => {
  const { data } = await client.get(`users/by/username/${process.env.GATSBY_TWITTER_USERNAME}`, {
    user: {
      fields:
        'created_at,description,entities,id,location,name,pinned_tweet_id,profile_image_url,protected,public_metrics,url,username,verified,withheld',
    },
  });

  callback(null, {
    headers: {
      'Access-Control-Allow-Origin': '*',
    },
    statusCode: 200,
    body: JSON.stringify({ user: data }),
  });
};

In the above you can see we use our client to hit the users/by/username Twitter API endpoint which you can read more about here, which returns a data object which I pass on to the callback body as { user: data }

This is the object that’ll we receive in our frontend

The next bit will greatly depend on how you’ve set up your frontend but in the Demo I have one page called index.js which uses a useEffect to “fetch” the data from the Netlify Function.

The example file contains a few extra bits for isLoading and hasError but the below should be enough to allow you hit to the Netlify Function which in turn hits the Twitter API and returns your profile information data.

// ./src/pages/index.js

import React, { useState } from 'react';

const IndexPage = () => {
  const [response, setResponse] = useState({ user: null });

  useEffect(() => {
    fetch(`${process.env.GATSBY_API_URL}/twitter-user`)
      .then((response) => response.json())
      .then((response) => {
        setResponse({ user: response.user });
      })
      .catch((error) => {
        console.error({ error });
      });
  }, []);

  return (
    <pre>
      <code>{JSON.stringify(response.user, null, 2)}</code>
    </pre>
  );
};

export default IndexPage;

process.env.GATSBY_API_URL is the path to the Netlify Function we added earlier to .env and i’ve hard-coded /twitter-user in the component / page as you might want to create different endpoints that return different data on different pages.

You might be wondering why this environment variable is prefixed with GATSBY_. This is so Gatsby can access it from the frontend. You can read more about Gatsby environment variables here

IMPORTANT

In order for Netlify Functions to work both locally and when deployed we need to ensure we’ve got netlify-lambda installed and have added both a "start" and "postinstall" script to the root package.json (not the package.json in ./functions)

npm install netlify-lambda --save -dev

// ./package.json

...
  "scripts": {
    "develop": "gatsby develop",
    "build": "gatsby build",
    "clean": "gatsby clean",
    "serve": "gatsby serve",
+    "start": "npm run develop",
+    "postinstall": "netlify-lambda install"
  },
   "devDependencies": {
+   "netlify-lambda": "^1.6.3",
  }
...

Before we get too carried away, it’s important to note that we’ll no longer be using gatsby develop or yarn develop to start the Gatsby app, if you do that our Netlify Function won’t be running and you’ll get an error.

Instead, run netlify dev this is so both the Gatsby site and the Netlify Function are run at the same time.

Instead of visiting the usual http://localhost:8000/ we’ll now be visiting http://localhost:8888/

And to ensure when we deploy everything works as it should you’ll need to modify your netlify.toml

For the most part netlify dev will attempt to automatically determine which static site generator you’re using by looking for certain files in the project root. If you don’t already have a gatsby-config.js at the root of your project add one as i’ve done here… you might not want to use Theme UI so ignore that if you’re using some other CSS method.

As mentioned above Netlify Functions are kind of their own application so when you deploy this to Netlify we need to ensure that the dependencies get installed, and you tell Netlify where the functions are in the directory structure

You’ll also need to ensue you’ve added all the same .env variables to the Netlify environment. Have a look in your Site settings and find “Environment” under “Build & Deploy”

Also make sure you’ve correctly set the “Sensitive variable policy” I just set mine as “Deploy without restrictions” because i’m edgy!

// ./netlify.toml

[build]
-  command = "yarn build"
+  command = "yarn build && cd functions && yarn"
+  functions = "functions"
  publish = "public/

With all that set you should be able to run netlify dev visit http://localhost:8888/ and see your own Twitter data rendered on the page, and once you deploy you “should” find everything continues to work as it did locally.

I’m not 100% on this but I think you might have to deploy at least once just so netlify dev knows where the functions are… it’s a bit weird even though you’re developing locally, however it seems to error until you deploy at least once.

Fingers crossed this is enough to get you up and running with Netlify Functions, but if you feel like any of this is a bit “draw the rest of the fucking owl” feel free to find me on Twitter.

Hey!

Leave a reaction and let me know how I'm doing.