Pruning packages using the API

Jul 05, 2015 · 4 min read

packagecloud-how-to

TL;DR

Since releasing the Packages & Versions API, several customers have used it prune packages in their repository, automatically yanking old packages as new ones are uploaded.

This blog post will show an example of how to implement pruning yourself, using Ruby.

Requirements

To follow along, you’ll need a packagecloud.io account, an API Token, and Ruby.

The Goal

Lets say we have a Jenkins job that automatically pushes to packagecloud.io every time there is a new build. However, we only want to keep the last 5 builds.

To accomplish this, we will check the package versions API after every push to see if we have hit our limit (5), if so, yank the oldest package.

The script: `prune.rb`

We’ll begin by creating a basic prune.rb script and listing all of our debian packages.

To keep our script simple, we’ll install the rest-client gem to help us with our API calls.

$ gem install rest-client

If you haven’t done so already, you can get your Packagecloud API token here.

Listing all debian packages

#!/usr/bin/env ruby

require 'json'
require 'pp'
require 'rest-client'

API_TOKEN = '11111111111111111111111111111111'
USER = 'my_username'
REPOSITORY = 'my_repo'

url = "https://#{API_TOKEN}:@packagecloud.io/api/v1/repos/#{USER}/#{REPOSITORY}/packages/deb.json"

packages = RestClient.get(url)

pp JSON.parse(packages)

Run the script and the output should resemble something like:

[{"name"=>"redis",
  "versions_count"=>3,
  "versions_url"=>
  "/api/v1/repos/my_username/my_repo/package/deb/ubuntu/utopic/redis/amd64/versions.json",
  "repository_url"=>"/api/v1/repos/my_username/my_repo",
  "repository_html_url"=>"/my_username/my_repo"}]

This is a list of all your debian packages for the repository, among them is a redis package with 3 versions for ubuntu/utopic.

For the purposes of this example, this is the package we’ve decided to prune.

Let’s update our prune.rb to use the versions_url returned above to get all the versions for redis.

Listing all `redis` versions

#!/usr/bin/env ruby

require 'json'
require 'pp'
require 'rest-client'

API_TOKEN = '11111111111111111111111111111111'
USER = 'my_username'
REPOSITORY = 'my_repo'
PACKAGE = 'redis'
DIST = 'ubuntu/utopic'

base_url = "https://#{API_TOKEN}:@packagecloud.io/api/v1/repos/#{USER}/#{REPOSITORY}"

package_url = "/package/deb/#{DIST}/#{PACKAGE}/amd64/versions.json"

url = base_url + package_url

redis_versions = RestClient.get(url)

pp JSON.parse(redis_versions)

This should print out a list of all the different versions for our redis package.

Now, let’s sort this list by the created_at date (ascending is the default), and yank the first entry if we are over our limit of 5 packages.

You could also sort by version, or release here, if you wish.

Sort `redis` versions by `created_at`

#!/usr/bin/env ruby

require 'json'
require 'pp'
require 'rest-client'
require 'time'

API_TOKEN = '11111111111111111111111111111111'
USER = 'my_username'
REPOSITORY = 'my_repo'
LIMIT = 5
PACKAGE = 'redis'
DIST = 'ubuntu/utopic'

base_url = "https://#{API_TOKEN}:@packagecloud.io/api/v1/repos/#{USER}/#{REPOSITORY}"

package_url = "/package/deb/#{DIST}/#{PACKAGE}/amd64/versions.json"

url = base_url + package_url

redis_versions = RestClient.get(url)

parsed_redis_versions = JSON.parse(redis_versions)

sorted_redis_versions = parsed_redis_versions.sort_by { |x| Time.parse(x["created_at"]) }

if sorted_redis_versions.size >= LIMIT
  to_yank = sorted_redis_versions.first
  pp to_yank
end

Run this script again, and if you are at the LIMIT, you should see the oldest entry printed out.

If it looks correct, we go ahead and prune the package. Here is the final version of the script:

Prune oldest `redis` version if above `LIMIT`

#!/usr/bin/env ruby

require 'json'
require 'pp'
require 'rest-client'
require 'time'

API_TOKEN = '11111111111111111111111111111111'
USER = 'my_username'
REPOSITORY = 'my_repo'
LIMIT = 5
PACKAGE = 'redis'
DIST = 'ubuntu/utopic'

base_url = "https://#{API_TOKEN}:@packagecloud.io/api/v1/repos/#{USER}/#{REPOSITORY}"

package_url = "/package/deb/#{DIST}/#{PACKAGE}/amd64/versions.json"

url = base_url + package_url

redis_versions = RestClient.get(url)

parsed_redis_versions = JSON.parse(redis_versions)

sorted_redis_versions = parsed_redis_versions.sort_by { |x| Time.parse(x["created_at"]) }

if sorted_redis_versions.size >= LIMIT
  to_yank = sorted_redis_versions.first

  distro_version = to_yank["distro_version"]
  filename = to_yank["filename"]
  yank_url = "/#{distro_version}/#{filename}"
  url = base_url + yank_url

  puts "yanking #{url}"
  result = RestClient.delete(url)
  if result == {}
    puts "successfully yanked #{filename}!"
  end
end

Conclusion

By combining the package versions and destroy API with some simple scripting, we were able to achieve our goal. Make sure to explore the rest of our API to find other ways you can improve your software delivery pipeline using Packagecloud.

Don’t hesitate to contact us with any questions or issues you have using our API or service, we love hearing from our users.