Ad-hoc reports from GitHub in F#

At Bitrise we have multiple organizations registered on GitHub with several open source repositories. Our developers and managers often need to access a quick overview of the current status of repositories and/or issues. Read about our methods for processing data from GitHub API.

We can process data the way we did with ClickUp, using JSON and HTTP type providers or we can opt for the official GitHub library, Octokit to communicate with the API. With Octokit, once authenticated you can easily build your reports in Ruby, Node.js and .NET languages but there are third party libraries for other frameworks too.

The next few paragraphs will introduce methods for processing data from GitHub API.

Octokit in F#

The .NET documentation encourages us to write code in C# but a lazy coder like me prefers fewer lines. Also, we do not need to compile the code, our beloved F# Interactive will do the job instead.

We need a couple of packages:


open Octokit
open Deedle
open Xplot.Googlecharts
Copy code

Connect

You can try the API without authentication but that will limit you to 60 requests an hour. Authentication is easy once you registered your application at GitHub. GitHub requires a valid User-Agent value, so use the name of your application. Use your token the following way to establish a connection.


let appName = "Bitrise-GitHub-App"

let client = GitHubClient(new ProductHeaderValue(appName));

let token = 'abc123...'
let tokenAuth = Credentials(token)
let client.Credentials = tokenAuth
Copy code

Get that data

client.Repository.GetAllForOrg() method needs only the name of the organization as a parameter. The signature of Task<IReadOnlyList<Repository>> tells us this is a task that is generating a list of repositories. The following lines count the repositories of bitrise-steplib where you can find the code of our workflow steps.


let org = "bitrise-steplib"

let repos  = client.Repository.GetAllForOrg(org)

// execute task
let repoResults = repos.Result

// number of repositories
repoResults.Count
Copy code

Process the data

For F# Deedle is what pandas are for Python: it has all the functions to process data in a tabular format. Let's draw a table of the number of forks, open issues and stargazers to understand which step has the most active community.


repoResults

// filter attribute
|> Seq.map (fun r -> (r.FullName, r.ForksCount, r.OpenIssuesCount, r.StargazersCount))

// build a data frame from F# record
|> Frame.ofRecords

// unfortunately it will not inherit the keys
|> Frame.indexColsWith ["repository"; "forks"; "open issues"; "stargazers"]

// descending order
|> Frame.sortRowsBy "stargazers" (fun v -> -v)

// Google Charts table
|> Chart.Table
Copy code

We can easily collect high-level figures with Deedle and its Stats namespace. This last contains members for several statistical indicators like mean, standard deviation or skewness etc.


let options = Options(colors = [|"#6B25BC"; "#60D0C4"|])

repoResults
|> Seq.map (fun r -> (r.Owner.Login, r.ForksCount, r.OpenIssuesCount))
|> Frame.ofRecords
|> Frame.indexColsWith ["organization"; "forks"; "open issues"]

// total numbers by organization
|> Frame.aggregateRowsBy ["organization"] ["forks"; "open issues"] Stats.sum
// we need only the numerical columns for the chart
|> Frame.dropCol "organization"
|> Chart.Column 
|> Chart.WithTitle "Total number of forks and open issues on repositories of bitrise-steplib"
|> Chart.WithOptions options
|> Chart.WithLegend true
Copy code

Stay tuned for the next article on data wizardry titled Understanding Variance in User Attributes coming soon.

No items found.

Explore more topics

App development

Best practices from engineers on how to use Bitrise to build better apps, faster.

Community

Meet other Bitrise engineers, technology experts, power users, partners and join our BUGs.

Company

All the updates about Bitrise events, sponsorships, employees, and more.

Insights

Mobile development, latest tech, industry insights, and interviews with experts.

Mobile DevOps

Learn why mobile development is unique and requires a set of unique practices.

Releases

Stay tuned for the last updates, new features, and product improvements.

Get the latest from Bitrise

Join other Mobile DevOps engineers who receive regular emails from Bitrise, filled with tips, news, and best practices.