APT repository internals

TL;DR

Following up on our previous post about yum repository internals, this blog post will dive into the internals of APT repositories. Similarly, we’ll cover what each index file does and take a look at how a user can inspect and verify the metadata themselves.

What is an APT repository?

An APT repository is a collection of deb packages with metadata that is readable by the apt-* family of tools, namely, apt-get. Having an APT repository allows you to perform package install, removal, upgrade, and other operations on individual packages or groups of packages.

APT repositories are essential for storing, managing, and delivering software to Debian and Ubuntu systems.

Create an APT repository with reprepro

Generate GPG key for signing an APT repository

(Feel free to skip this section if you already have a GPG key you wish to use)

We’ll start by creating a GPG key to sign our repository metadata for a fuller, more illustrative example:

$ gpg --gen-key

Since this is a throwaway blog example, we’ll just pick all the defaults without a passphrase.

NOTE: You’ll definitely want a passphrase for production deployments and ask-passphrase in your apt/debian/conf/options below.

Now when we gpg --list-keys, we should see:

$ gpg --list-keys
pub   2048R/78AF53BD 2015-08-02
uid                  Test Repo User <testy@example.org>
sub   2048R/3F0EF11C 2015-08-02

Installation of reprepro

Before taking a closer look at the repository metadata itself, let’s show how to get a repository set up by using the open source reprepro command line tool.

You can create an APT repository by using the command line tool reprepro which you can install on Ubuntu and Debian systems by running:

$ sudo apt-get install reprepro

Once installed, let’s create a directory for our APT repository and reprepro configuration.

Configuring reprepro

Let’s create the directory structure necessary for reprepro and it’s configuration.

$ mkdir -p my_test_repo/apt/debian/conf

In the conf directory, create a file named distributions, here we will list all the distributions and architectures our repository will support. For this example, we’ll only support one distribution, debian squeeze:

Origin: my_test_repo
Label: my_test_repo
Codename: squeeze
Architectures: i386 amd64
Components: main
Description: Apt repository for my_test_repo
SignWith: 3F0EF11C
Contents: percomponent

The value of SignWith being the subkey ID from the GPG key we wish to use (the one we generated above, in this case)

This defines a my_test_repo for debian squeeze.

Now let’s download an example package to add to this repository:

$ wget -q 'https://packagecloud.io/computology/packagecloud-test-packages/packages/ubuntu/precise/packagecloud-test_1.1-2_amd64.deb/download' -O /tmp/packagecloud-test_1.1-2_amd64.deb

Generate APT repository with reprepro

To add that package to this repository, we’lll call reprepro with the desired distribution and path to debian package, from inside my_test_repo/apt/debian directory, like so:

$ reprepro includedeb squeeze /tmp/packagecloud-test_1.1-2_amd64.deb

This will create and sign all the necessary repository metadata APT needs to install and verify this repository on another computer.

Hosting GPG key and APT repository over HTTP

Once the package has been added and repository indices have been generated, we’ll need to serve them up over HTTP so that APT can read it.

First, let’s export our GPG key and host it along with our repository, that way anyone installing this repository can also import our GPG public key.

$ gpg --export --armor 3F0EF11C > my_test_repo/gpg.key

Then, let’s serve up our repository using python’s built in HTTP server.

Navigate to my_test_repo and run:

$ sudo python -m SimpleHTTPServer 80

(Do this in a screen session if you’d like use the same terminal session for the rest of the tutorial)

Installing Package from APT repository

Finally, we can install a package from our configured and hosted repository!

Create an /etc/apt/sources.list.d/my_test_repo.list on the computer you wish you install our package on and add the following line, substituted with the IP of the computer hosting the repository:

deb http://my_test_repo.example/apt/debian squeeze main

Let’s also import our GPG key into APT:

$ wget -O - http://my_test_repo.example/gpg.key | sudo apt-key add -

Now we can run apt-get update to fetch the repository metadata and install our package:

$ sudo apt-get update
$ sudo apt-get install packagecloud-test=1.1-2

On the server, you should see requests coming in for various repository metadata as APT tries install this package:

[02/Aug/2015 17:54:59] "GET /apt/debian/dists/squeeze/Release.gpg HTTP/1.1" 200 -
[02/Aug/2015 17:54:59] "GET /apt/debian/dists/squeeze/main/i18n/Translation-en.bz2 HTTP/1.1" 404 -
[02/Aug/2015 17:54:59] "GET /apt/debian/dists/squeeze/Release HTTP/1.1" 200 -
[02/Aug/2015 17:54:59] "GET /apt/debian/dists/squeeze/main/binary-amd64/Packages.bz2 HTTP/1.1" 404 -
[02/Aug/2015 17:54:59] "GET /apt/debian/dists/squeeze/main/binary-amd64/Packages.lzma HTTP/1.1" 404 -
[02/Aug/2015 17:54:59] "GET /apt/debian/dists/squeeze/main/binary-amd64/Packages.gz HTTP/1.1" 200 -

On the client, you can verify it successfully installed our package, by running:

$ packagecloud_hello

Of course, using packagecloud is a much faster and simpler solution :) with SSL, GPG, authentication, collaboration, and everything else you need ready to go!

APT repository metadata

Now that we have verified our APT repository works by installing our package on another computer, let’s dive into the mechanics of how an APT repository works.

Let’s take a look at our reprepro generated directory structure from the root of my_test_repo/apt/debian and what each file does:

• ./dists/squeeze/Contents-<arch>.gz: A gzipped index containing a directory listing of all packages for this architecture, across all components.
• ./dists/squeeze/Release: The primary index which contains the locations and checksums of the architecture-specific Packages files.
• ./dists/squeeze/Release.gpg: If SignWith was provided, reprepro will generate a detached signature for the above Release index.
• ./dists/squeeze/InRelease: Also only generated if SignWith was provided, it’s a version of Release with a clearsign signature, used by newer versions of APT.
• ./dists/squeeze/main/binary-<arch>/Packages: Lists metadata about each package available for a given architecture <arch>.
• ./dists/squeeze/main/Contents-<arch>.gz: A gzipped index containing a directory listing of all packages for this architecture <arch>, for the main component.
• ./pool/main/*: Directory where the actual package files reside.

Debugging

The examples below will show how you can view the index yourself to verify it contains what you expect.

GPG Verification of signed APT repository metadata

APT automatically verifies GPG signed metadata if it knows the public key (which we imported above), but we can also verify manually to be extra sure.

Let’s import the public key into our GPG keyring instead of APT’s:

wget -O - http://my_test_repo.example/gpg.key | gpg --import

Download the Release file and its signature:

$ wget http://my_test_repo.example/apt/debian/dists/squeeze/Release.gpg
$ wget http://my_test_repo.example/apt/debian/dists/squeeze/Release

Verify it with GPG:

$ gpg --verify Release.gpg Release

For more information on GPG signatures and APT packages, check out our post on gpg sign and verify deb packages.

Inspecting Release metadata for APT repository

Let’s get the Release for ubuntu precise for the packagecloud test packages repository:

$ curl -L https://packagecloud.io/computology/packagecloud-test-packages/ubuntu/dists/precise/Release

It will return a huge list of all the architectures and their relevant Packages files and checksums, so for binary-amd64, for our example we have:

94fb8d5d4355c0c891a8756453d0ac85cf34dff1fc461abfbd02ec7f85044e46 658 main/binary-amd64/Packages
6d54859644489197158046e0afd1bf5816eda8713d31227607df9fa23d52e1cb 486 main/binary-amd64/Packages.bz2
63db7747a633bb3f16cfa7996648f03b8d33aff027216302b02f0ac00aa54fb0 445 main/binary-amd64/Packages.gz
f28c0ba65f233f4352ec3e33a7c6c5578d5ed4f862e4107706a82c22efbc8c38 524 main/binary-amd64/Packages.xz

Inspecting Packages metadata for APT repository

Let’s get the Packages for ubuntu precise and binary-amd64 for the packagecloud test packages repository:

$ curl -L https://packagecloud.io/computology/packagecloud-test-packages/ubuntu/dists/precise/main/binary-amd64/Packages

We can also verify the checksum (SHA256 in this case) for this Packages file contained in Release above:

$ curl -sL https://packagecloud.io/computology/packagecloud-test-packages/ubuntu/dists/precise/main/binary-amd64/Packages | shasum -a 256

This should return the expected checksum, which is:

94fb8d5d4355c0c891a8756453d0ac85cf34dff1fc461abfbd02ec7f85044e46

Once the checksum is verified, we can look at the actual metadata, which looks like:

Package: packagecloud-test
Priority: optional
Section: misc
Installed-Size: 39
Maintainer: Joe Damato <joe@packagecloud.io>
Architecture: amd64
Version: 1.1-2
Depends: libc6 (>= 2.2.5)
Filename: pool/precise/main/p/packagecloud-test/packagecloud-test_1.1-2_amd64.deb
Size: 2830
MD5sum: b709d6a5b2e6883ee97803e5fd1b204e
SHA1: ba4500d3b385aced8c8ed3c48981eba2dd2005b4
SHA256: 8ed8bb73915088d422c26ce3e92c6477b3afdfcd7377713880db8ba68a697289
SHA512: afcde64a1bfc5fdce1fff00002a2007a7f05eef51e45eb223885b89e2deb45cefd51dd596d79dea1daf020312afb14967eaef2d1a4c45f7da28ec045bfc2efc6
Description: packagecloud-test
Description-md5: 49d80a99b15b8081983c8a12416577a7

Inspecting Contents metadata for APT repository

Lastly, let’s check out the Contents file for the packagecloud test packages repository and amd64:

$ curl -sL https://packagecloud.io/computology/packagecloud-test-packages/ubuntu/dists/precise/Contents-amd64.gz | zless

This will list out all the files associated with each package:

usr/local/bin/packagecloud_hello misc/packagecloud-test=1.1-2
usr/share/doc/packagecloud-test/changelog.Debian.gz misc/packagecloud-test=1.1-2
usr/share/doc/packagecloud-test/copyright misc/packagecloud-test=1.1-2

Conclusion

APT repository metadata is comprised of a set of index files, checksums, and in some cases a GPG signature. The metadata describes which packages can be found in a repository, various attributes about each package, file and directory listings.

APT repository metadata can be manually examined and verified on the command line using a combination of curl, less/zless, gpg, and shasum. This is useful if you need to debug some sort of issue with your repository (missing package, missing dependency, incorrect version, etc) or are curious about the inner workings of one of the most important pieces to your operating system.

Set up your own package repository.

Fast, reliable, and secure software starts here.

Try Packagecloud