Migrating a Decade of Work to Gitlab

By Matthew Cowie  ·   July 21, 2015  ·  3 minute read

We’re in the process of refreshing many of our internal tools here and the cornerstone is a trusty, old, plain Gitolite server. It sat behind the scenes faithfully chugging away, but as we pile on more and more projects, collaborators, and general legacy cruft, we were looking for a clean break.

We’re in the process of refreshing many of our internal tools here and the cornerstone is a trusty, old, plain Gitolite server. It sat behind the scenes faithfully chugging away, but as we pile on more and more projects, collaborators, and general legacy cruft, we were looking for a clean break.

Weighing our options

There are quite a few options for code repository services out there. Github is by far the most popular today, but their pricing is geared towards large teams on a small number of projects, whereas we’re the opposite. BitBucket is friendlier to smaller teams, but ideally, we’d have something we could host ourselves. Which led us to Gitlab.

Gitlab offered everything we could want: friendly web dashboards, user management, email and Slack notifications, open-source, and easy to maintain and upgrade. Not to mention the web interface is a Rails application, which we happen to know a thing or two about.  

Preparation

After getting our server up and secured, we were ready to migrate our repositories. I found a nice blog post and used it as a jumping-off point. I wrote a Quick ’n’ Dirty™ Ruby script based on this structure. The main points are:

  1. Get the list of repositories
  2. Create a Gitlab repository
  3. Mirror the legacy repository locally
  4. Push the mirror up to the new Gitlab repo

Getting the list of repositories out of Gitolite isn’t too hard, assuming you have access to all the repos, you can generate a nice list by SSH’ing the server

ssh git@yourgitoliteurl.com

Previously, we had all our repos at the root level, but Gitlab seems to discourage root level repos, and having some semblance of organization wasn’t something I wanted to fight. I came up with a few logical groupings of projects, created them as groups in Gitlab, and created a mapping file like so

cogswell clients/cogswell rails-app-template internal/blastoff

Given URLs are going to break anyway, it’s a good opportunity to give tools the cheeky names they deserve. Now that we know where things are coming from and where they’re going we can get started!

Migration

The script itself isn’t too complex; it iterates through the mapping file line by line repeating the steps we laid out earlier. You’ll need a Gitlab admin’s private token (found at profile > settings > account). First, we use Gitlab’s JSON API to create the project. To make sure it’s added under the proper group we need the ID of that group, so we search groups by name. If this weren’t the one-time-use Quick ’n’ Dirty™ method you’d cache that value in a lookup table instead of querying the app over and over. 

Now that the project’s set up, we mirror clone the old repo to grab all of the things, add Gitlab as a remote, and push it all up to our new Gitlab project.

While this script was written specifically for our migration, it should be a useful jumping-off point for anyone looking to do something similar.  The Gitlab API has a ton of options if you want to further specify settings for your projects. Also handy would be this Ruby wrapper for the API that I did not discover until after I was done, but then what’s the fun of not slinging together a few HTTP requests?