What are source packages?
A package is a collection of binaries, scripts, and associated data that is installed by your package manager.
Packages are typically generated from source code and a set of a metadata written by the package maintainer. Occasionally, the source may by patched by the package maintainer at build time.
A source package captures the source code and patches as they were at build time.
On Debian-based systems (including Ubuntu), source packages are called “Debian source packages” and are described by a plain-text DSC file, and a set of tarballs representing the original source code, patches, and Debian packaging files.
Why are source packages useful?
Source packages are extremely useful for:
- Regenerating binary packages
- Debugging issues with applications and libraries
- Modifying existing applications to add additional logging
- Confirming whether or not a particular security fix has been applied to the source
The engineers at packagecloud.io spend a lot of time sifting through the source code of apt-get and other Ubuntu and Debian tools to find work-arounds for bugs, read the implemenation details of undocumented options, and learn more about package management internals.
We strongly recommend that people and organizations running applications in production become familiar with source packages so that when a bug appears, system operators and developers will have the tools to quickly and easily obtain the source and debug the issue.
Working with Debian and Ubuntu source packages
If you intend to rebuild or modify a source package, it is strongly recommended that you install the devscripts package which contains many useful tools for working with Debian source packages:
$ sudo apt-get install devscripts
Getting Debian source packages
Getting the source for a package on a Debian-based system is straight-forward. You can run this command as a normal user and the source code will be written to the current directory.
For example, to get the source for redis run:
$ apt-get source redis
This will create a directory with the Redis source and all Debian-specific patches applied to it.
This way, you can look through the source code for Redis precisely as it was when Redis was compiled and put into the package that you have installed on your system.
So, if you need to track down a bug, cryptic error message, undocumented feature, or to check if your package has a security update applied, you can run a command to obtain the source quickly and easily.
Installing build dependencies
In order to rebuild a Debian source package, you will need to begin by first installing all the build dependencies:
$ sudo apt-get build-dep redis
This will install any necessary packages for the build process (compilers, scripting languages, and so on).
Next, you can make whatever changes (if any) you want to the source. Apply patches, adjust the debian/rules file, or add extra debugging output.
Modifying the source and applying patches
Debian source packages can have patches applied in different ways, but the most common way for patches to be applied and organized is with a program called quilt.
quilt organizes patches in the debian/patches directory. Patches are applied as they are listed in the debian/patches/series file.
You can find more information about working with quilt by reading this great guide.
Updating the Debian package version number
Debian package version numbers are derived from entries in the debian/changelog file in the source tree. In order to release a new version, you must modify this file. To help modify the changelog, you can use the program dch.
There are many ways to use dch, so please check the manpage, but a common usage is:
$ dch -i
This will attempt to increment the version number, add a black changelog entry for you, and open your editor to allow you to fill in the specific details.
Rebuilding the package
Now, you are ready to rebuild the package!
You can do this by running:
$ debuild -us -uc
In the top level source directory. This will produce your Debian package files and a new Debian source package.
Upload Debian Source Packages to packagecloud.io
You can now upload your Debian binary and source packages to packagecloud.
You simply push the DSC file, and the package_cloud CLI will find and upload the tarballs listed in the file automatically.
For example:
$ package_cloud push user/repo/ubuntu/trusty redis_2.2.12-1build1.dsc
Looking for repository at user/repo... success!
Checking source package redis_2.2.12-1build1.dsc... success!
Pushing redis_2.2.12-1build1.dsc... success!
This is handled programmatically in two steps. First the package contents API is used to gather the list of tarballs and checksums in the source package. Next, the package create API is used to upload the dsc file and associated tarballs.
Conclusion
The Debian and Ubuntu package toolchain enables users to quickly and easily recreate the source tree for a package as it was at build time. This is incredibly useful for debugging strange application behavior and for adding additional logging output to applications that aren’t telling you what you need to know when they run.
Becoming familiar with the workflow of obtaining source code via the package manager is essential for debugging and administering infrastructure of any size.
