• Skip to primary navigation
  • Skip to main content
  • Skip to primary sidebar

I Like Kill Nerds

The blog of Australian Front End / Aurelia Javascript Developer & brewing aficionado Dwayne Charrington // Aurelia.io Core Team member.

  • Home
  • Aurelia 2
  • Aurelia 1
  • About
  • Aurelia 2 Consulting/Freelance Work

Recursively Fetch Data From The GitHub API Using Octokit

Javascript · August 18, 2020

Recently I began updating TidyFork to be written in Aurelia 2. As such, I took the opportunity to change how the repositories were loaded in the UI. There are numerous Octokit libraries provided by GitHub which let you make requests to the GitHub API.

Using the request.js package, I make calls to the GitHub API. I also use the Parse Link Header package to parse the pagination headers that GitHub returns on its API responses.

Install the needed packages by running: npm install @octokit/request parse-link-header

Here is the code in all of its glory. It’s a function which will recursively load all user-owned GitHub repositories, it will keep loading additional repos until there are no more to load.

import { request } from '@octokit/request';
import * as parse from 'parse-link-header';

async function getUserRepos(username, page = 1) {
	const repos = [];
  
  	// Call the GitHub API and get all repositories
    const result = await request(`GET /user/repos`, {
      headers: {
        authorization: `token ${localStorage.getItem('github_auth_token')}`,
      },
      visibility: 'all',
      sort: 'created',
      direction: 'desc',
      per_page: 100,
      page: page
    });
    
    // Push the result into the repos array above
    // this will be called for the initial function call
    repos.push(...result.data);

  	// Parse the pagination headers
    const pagination = parse(result.headers.link);
  
    // If we have a next property, we have more results to load
    if (pagination.next) {
      // Recursively call this function again
      const response = await getUserRepos(username, parseInt(pagination.next.page));
      repos.push(...response);
    }
  
    // Return the loaded repos
    return repos
      .reduce((acc, repo) => {
        // If the repo owner matches the logged in user, allow it
        if (repo.owner.login === username) {
          acc.push(repo);
        }
        return acc;
      }, []);
}

The majority of the functionality inside of our function happens before the reduce call at the bottom. We make the request, push the data into an array, check if there is pagination data and get the next page number. We then recursively call the function again, passing in the username and next page number.

It’s possible you might not even need to parse the link headers, the Octokit package might have functionality that handles this for you. This is just the result I came up with.

Dwayne

Leave a Reply Cancel reply

0 Comments
Inline Feedbacks
View all comments

Primary Sidebar

Popular

  • Testing Event Listeners In Jest (Without Using A Library)
  • How To Get The Hash of A File In Node.js
  • Which Neural DSP Archetype Plugins Should You Buy?
  • Smoke Detector Randomly Goes Off Early Hours of The Morning
  • How To Mock uuid In Jest
  • Neural DSP Reveal Details About the Long-Awaited Quad Cortex Desktop Editor
  • Web 3.0 may have died before it even started
  • Deno Raises $21M - but is anyone using it yet?
  • NBN Box Installed Inside of Garage, Where Do You Put The Modem?
  • How To Install Eufy Security Cameras Without Drilling or Using Screws

Recent Comments

  • Jay on Neural DSP Reveal Details About the Long-Awaited Quad Cortex Desktop Editor
  • john on Deno Raises $21M – but is anyone using it yet?
  • Oranges on How To Store Users In Firestore Using Firebase Authentication
  • Precious on Fixing Sequel Pro SQL Encoding Error For Imported SQL Files
  • James on A List of WordPress Gutenberg Core Blocks

Copyright © 2022 · Dwayne Charrington · Log in

wpDiscuz