Paul Scanlon's Site
By Paul Scanlon

Use Netlify Functions and the Twitter API v2 as a CMS for your Gatsby blog

Apologies in advance for the rather long-winded blog title but as it suggests in this post iā€™m going to explain how you can use Netlify Functions to access your Twitter profile data using the Twitter v2 API and display it on your Gatsby blog.

A rather unique requirement

This might be a specific to me but I wanted to solve a little problem I was having with my ā€œdigital footprintā€. As you can see I have this blog: https://paulie.dev and a commercial portfolio: https://www.pauliescanlon.io

Both sites are built on top of my Gatsby theme: gatsby-theme-terminal which is Open source and can be found on my GitHub

Using a Gatsby Theme solves one of my issues as Iā€™m able to have two sites that look and work pretty much the same way and any changes I make to the theme are inherited by both my sites. Itā€™s kind of like managing your own multi brand design system, but just for yourself.

There was one other problem though. šŸ¤”

I wanted both sites to have the same ā€œintroā€ section, but every time I made a change to one I had to make the same change to the other site to ensure they were both displaying the same intro text.

This might be fine if I werenā€™t a developer but doing something twice is one time too many IMO.

It was also a little frustrating because I also wanted my Twitter profile description to be in sync with both the sites so, again another place to remember to update my personal blurb.

One option I considered would have been to hook up a Content Management System, and this would have been fine and it would have kept both my sites in sync but it wouldnā€™t have been able to update my Twitter profile blurbā€¦

So, Iā€™ve decided to reverse engineer the Twitter API and use that as a CMS to populate both my sites. The idea is quite simple. Iā€™ll use the Twitter profile description as though it were a field from a CMS. Naturally any changes I make to this will appear on my Twitter profile and below is how I pull that same info into both of my sites.

Demo

Hereā€™s what Iā€™ll be showing you how to build:

ā€¦ but the actual API I use for my blog and site is here: https://paulie-api.netlify.app

Tech

Netlify Functions

ā€œPower your site without managing serversā€ is how Netlify describe Functions and for all intents and purposes thats exactly what they are. Similar to how you might create an Express app and deploy it somewhere but without the hassle of having to setup server side environments and more crucially any really dweeby server uptime monitoring.

Twitter API v2

A set of endpoints that can be used to get data from Twitter. Any Twitter requests must be done server side and use a set of keys and tokens. You canā€™t unfortunately hit the Twitter API from the browser so we need a ā€œserverā€ or as mentioned above, a Netlify Function

Using both of the above iā€™ve made my own API endpoint which goes off and hits the Twitter API and returns my Profile information which I can then display in the intro section of my blog and site. Iā€™ve deployed this API to Netlify and itā€™s completely de-coupled from either of my sites but will return data which can be fetched from client side ā€œfetchā€ request from within my site and blog. That url again is here: https://paulie-api.netlify.app

Before we start

Before we get started thereā€™s a couple of things youā€™ll need to have in place.

Twitter API v2

Apply for access to the Twitter API. This is quite a lengthy process so strap in and also bookmark this post as it might take a few days for Twitter to accept your application.

Once you have access you can head over to the Developer Portal and create a new project, and within the project you can create an ā€œappā€, I called mine ā€œpaulie-apiā€.

In here youā€™ll find all the API keys and tokens required to access the Twitter API. Make a note of them somewhere as weā€™ll be using them later.

Netlify CLI

To run Netlify Functions weā€™ll be using netlify dev rather than gatsby develop or yarn develop so youā€™ll need to install the Netlify CLI

The Build

In order to develop you own API I found it easiest to have some kind of ā€œsiteā€ running at the same time which will access the API endpoint and render the response on the page. In the demo repo youā€™ll see iā€™ve set up a really simple Gatsby Site with one page that uses ā€œfetchā€ to, er fetch and then render the data.

Iā€™ve used Theme UI for the style but naturally you can choose whatever you like to do this.

Whether youā€™re starting from scratch or adding Netlify Functions to an existing project youā€™ll need to start by adding a functions dir to the root of your project.

|-- functions
  |-- package.json
|-- src
package.json

functions is kind of itā€™s own application so itā€™ll need itā€™s own package.json and will have one dependency on twitter-v2

// ./functions/package.json

{
  "name": "gatsby-netlify-twitter-api",
  "version": "1.0.0",
  "description": "An api for the Twitter v2 api",
  "main": "index.js",
  "scripts": {
    "test": "echo \"Error: no test specified\" && exit 1"
  },
  "keywords": [],
  "author": "",
  "license": "ISC",
  "dependencies": {
    "twitter-v2": "^0.1.2"
  }
}

Next have a look at .env.example. Youā€™ll need to create your own .env file and add the environment variables as seen in the .env.example. Naturally youā€™ll want to change the GATSBY_TWITTER_USERNAME to your own Twitter username and the Twitter keys and tokens will be what I referenced earlier which are provided by the Twitter Developer Portal

// ./.env

GATSBY_API_URL=./.netlify/functions
GATSBY_TWITTER_USERNAME=

TWITTER_API_KEY=
TWITTER_API_KEY_SECRET=
TWITTER_ACCESS_TOKEN=
TWITTER_ACCESS_TOKEN_SECRET=

Next create a Twitter client, this is what weā€™ll use to pass the keys and tokens onto the Twitter API when we make a request

// ./functions/client.js

const Twitter = require('twitter-v2');

module.exports = {
  client: new Twitter({
    consumer_key: process.env.TWITTER_CONSUMER_KEY,
    consumer_secret: process.env.TWITTER_CONSUMER_KEY_SECRET,
    access_token: process.env.TWITTER_ACCESS_TOKEN,
    access_token_secret: process.env.TWITTER_ACCESS_TOKEN_SECRET,
  }),
};

You should now be looking at something similar to the below

...
|-- functions
  |-- client.js
  |-- package.json
|-- src
package.json
.env
...

Now we need to create the ā€œendpointā€ that our frontend will hit, which in turn goes off and grabs the data from the Twitter API.

I created a dir called twitter-user and inside I create a new file and called it twitter-user.js

...
|-- functions
  |-- client.js
  |-- twitter-user
    |-- twitter-user.js
  |-- package.json
|-- src
package.json
.env
...

Itā€™s in here where we can use the client.js to hit a Twitter API endpoint and pass with it the required keys and tokens from the client

// ./functions/twitter-user/twitter-user.js

const { client } = require('../client');

exports.handler = async (event, context, callback) => {
  const { data } = await client.get(`users/by/username/${process.env.GATSBY_TWITTER_USERNAME}`, {
    user: {
      fields:
        'created_at,description,entities,id,location,name,pinned_tweet_id,profile_image_url,protected,public_metrics,url,username,verified,withheld',
    },
  });

  callback(null, {
    headers: {
      'Access-Control-Allow-Origin': '*',
    },
    statusCode: 200,
    body: JSON.stringify({ user: data }),
  });
};

In the above you can see we use our client to hit the users/by/username Twitter API endpoint which you can read more about here, which returns a data object which I pass on to the callback body as { user: data }

This is the object thatā€™ll we receive in our frontend

The next bit will greatly depend on how youā€™ve set up your frontend but in the Demo I have one page called index.js which uses a useEffect to ā€œfetchā€ the data from the Netlify Function.

The example file contains a few extra bits for isLoading and hasError but the below should be enough to allow you hit to the Netlify Function which in turn hits the Twitter API and returns your profile information data.

// ./src/pages/index.js

import React, { useState } from 'react';

const IndexPage = () => {
  const [response, setResponse] = useState({ user: null });

  useEffect(() => {
    fetch(`${process.env.GATSBY_API_URL}/twitter-user`)
      .then((response) => response.json())
      .then((response) => {
        setResponse({ user: response.user });
      })
      .catch((error) => {
        console.error({ error });
      });
  }, []);

  return (
    <pre>
      <code>{JSON.stringify(response.user, null, 2)}</code>
    </pre>
  );
};

export default IndexPage;

process.env.GATSBY_API_URL is the path to the Netlify Function we added earlier to .env and iā€™ve hard-coded /twitter-user in the component / page as you might want to create different endpoints that return different data on different pages.

You might be wondering why this environment variable is prefixed with GATSBY_. This is so Gatsby can access it from the frontend. You can read more about Gatsby environment variables here

IMPORTANT

In order for Netlify Functions to work both locally and when deployed we need to ensure weā€™ve got netlify-lambda installed and have added both a "start" and "postinstall" script to the root package.json (not the package.json in ./functions)

npm install netlify-lambda --save -dev

// ./package.json

...
  "scripts": {
    "develop": "gatsby develop",
    "build": "gatsby build",
    "clean": "gatsby clean",
    "serve": "gatsby serve",
+    "start": "npm run develop",
+    "postinstall": "netlify-lambda install"
  },
   "devDependencies": {
+   "netlify-lambda": "^1.6.3",
  }
...

Before we get too carried away, itā€™s important to note that weā€™ll no longer be using gatsby develop or yarn develop to start the Gatsby app, if you do that our Netlify Function wonā€™t be running and youā€™ll get an error.

Instead, run netlify dev this is so both the Gatsby site and the Netlify Function are run at the same time.

Instead of visiting the usual http://localhost:8000/ weā€™ll now be visiting http://localhost:8888/

And to ensure when we deploy everything works as it should youā€™ll need to modify your netlify.toml

For the most part netlify dev will attempt to automatically determine which static site generator youā€™re using by looking for certain files in the project root. If you donā€™t already have a gatsby-config.js at the root of your project add one as iā€™ve done hereā€¦ you might not want to use Theme UI so ignore that if youā€™re using some other CSS method.

As mentioned above Netlify Functions are kind of their own application so when you deploy this to Netlify we need to ensure that the dependencies get installed, and you tell Netlify where the functions are in the directory structure

Youā€™ll also need to ensue youā€™ve added all the same .env variables to the Netlify environment. Have a look in your Site settings and find ā€œEnvironmentā€ under ā€œBuild & Deployā€

Also make sure youā€™ve correctly set the ā€œSensitive variable policyā€ I just set mine as ā€œDeploy without restrictionsā€ because iā€™m edgy!

// ./netlify.toml

[build]
-  command = "yarn build"
+  command = "yarn build && cd functions && yarn"
+  functions = "functions"
  publish = "public/

With all that set you should be able to run netlify dev visit http://localhost:8888/ and see your own Twitter data rendered on the page, and once you deploy you ā€œshouldā€ find everything continues to work as it did locally.

Iā€™m not 100% on this but I think you might have to deploy at least once just so netlify dev knows where the functions areā€¦ itā€™s a bit weird even though youā€™re developing locally, however it seems to error until you deploy at least once.

Fingers crossed this is enough to get you up and running with Netlify Functions, but if you feel like any of this is a bit ā€œdraw the rest of the fucking owlā€ feel free to find me on Twitter.

Hey!

Leave a reaction and let me know how I'm doing.

  • 0
  • 0
  • 0
  • 0
  • 0
Powered byNeon