Making a Debian Metapackage for One Command OS Setup


This tutorial will guide through making a Debian Metapackage (i.e. a package that only has dependencies on other packages and installs no files) and setting up your configuration so that it's easily instatiable on a new system.

Introduction

This tutorial will guide through making a Debian Metapackage (i.e. a package that only has dependencies on other packages and installs no files) and setting up your configuration so that it’s easily instatiable on a new system.

You can find the code we’ll write in this tutorial at Github.

Our project will include two main parts: the metapackage, in the deb subdirectory of our project, and the system-wide configuration files under the system/debian subdirectory. This separation is because (i) we want out metapackage to be a pure metapackage and (ii) if we ever need to replace a system-wide configuration file that belongs to another package, the debian packaging system won’t let us. In my configuration I also have a subdirectory for my dotfiles, another one for executable scripts that appears in $PATH, another one for guix configuration, and another one for Raspberry Pi stuff; and system/rpi has system-wide configuration files for it. I list these as an example to show that you can combine in this project multiple setups for different operating systems and other configuration files and scripts as you wish. Other than these we’ll have a script to initialise the system for us using this setup, doing also some housekeeping for us (not so) transparently. We’ll also include an nginx setup to demonstrate how to use this system to install our configuration, and also Syncthing to demonstrate how to install a package from a third-party apt repository. These are examples and you might just remove them or maybe use them in your setup if you find them useful to you.

One note is that we won’t build a perfect Debian package nor bother with maintaining it according to the requirements of the Debian or Ubuntu projects. This is because this is meant to be a tool for one’s personal setup. It’s perfectly possible to adhere to these regulations, in which case I refer you to relevant documentation.

The last note before I proceed with the tutorial is that while I use Ubuntu Gnome and have built my config on it, this setup should work on all Debian-based operating systems with little or no modification. Package names may differ from one of those to another, so beware of these pitfalls if you are going to try to maintain a cross-platform configuration. One big difference between Ubuntu and Debian is that the former has sudo installed by default whereas the latter may not depending on how you installed it, and the scripts and makefiles in this setup use sudo, so you may want to either install it beforehand or modify them to use su(1) instead.

The Metapackage

The source code for the metapackage will reside in the deb/DEBIAN subdirectory of our project. Any file in the deb directory other than the DEBIAN subdirectory will be part of the Debian package and installed in the root directory of your filesystem. You may include configuration files in say deb/etc/ and they’ll go under /etc when you install the package, but if those files happen to coincide with other packages’ files, as noted above, our package won’t install; and the package won’t anymore be a metapackage. If you really want to include stuff here, be careful to not cause clashes, and install configuration under directories like /etc/nginx/conf.d instead of /etc/nginx. But do avoid this.

The first file we’ll create is the deb/DEBIAN/copyright, which is useless if you won’t publish this configuration, but the packaging system requires us to have this, so we will.

deb/DEBIAN/copyright:

Format: http://anonscm.debian.org/viewvc/dep/web/deps/dep5.mdwn?revision=174
Upstream-Name: myconfig

Files: *
Copyright: 2017 Joe Maintainer <example.com>
License: Ciao

License: Ciao
 Humans may not use this program!

Done, you can just copy and paste this, and modify with your personal information. Now we’re going to create deb/DEBIAN/control.in, which is going to provide some parameters for the Debian package we’re creating. Again, you can just copy and paste this, as this is just boilerplate that we need.

deb/DEBIAN/control.in:

Package: myconfig
Standards-Version: 3.6.2
Version: __VERSION__
Section: misc
Priority: standard
Architecture: all
Maintainer: __MAINT__
Description: My base system.
 Install all the packages that I use.
Depends: __DEPS__

We’re going to process this file and generate the deb/DEBIAN/control file, which is the actual file the packaging system will read. We do this because we don’t want to change the Version: field every time we update our configuration.

The next file is deb/DEBIAN/changelog which is yet more boilerplate that we have to have and that you can just copy from below and modify for your personal information if you will.

deb/DEBIAN/changelog:

myconfig (0.1) unstable; urgency=medium

  * Initiate a package.

-- Joe Maintainer <joe@example.com>  Mon, 13 Mar 2017 20:23:44 +0300

We won’t ever need to touch this file again.

Now that we have all the boring boilerplate in place, we may go on to creating the actual bits that we’ll use. The first is the deb/DEBIAN/makefile which will contain the build recipe for transforming control.in to control with input from variables that are going to come from the toplevel makefile 1 and another file that we’ll create in a bit.

deb/DEBIAN/makefile:

FILES=control
include deps.mk # DEPS defined here
_DEPS!=echo $(DEPS) | sed 's: :, :g'

all: $(FILES)

deps:
    @echo $(DEPS)

control: control.in deps.mk makefile
    sed 's,__MAINT__,$(MAINT),' > control.in |\
    sed 's,__VERSION__,$(VERSION),' |\
    sed "s/__DEPS__/$(_DEPS)/"> $@;

clean:
    rm -rf $(FILES)

.PHONY: all deps control clean

You can just copy this verbatim. The rules are simple: the deps.mk provides a variable called $DEPS, which is a space separated list of dependencies (space-separation will prove convenient in a bit) which we’ll convert to a comma separated one as Debian expects in the control file, into which we will interpolate that and some other variables. The rule for this file depends on control.in, on the makefile and on deps.mk which we’ll create in a bit, but uses the first one only. This is for it to run whenever we update our setup or the makefile, otherwise we’d need to touch(1) control.in all the time.

And finally we’ll create deb/DEBIAN/deps.mk, which will list our dependencies in a convenient way. If we didn’t do this, we’d have to have them in the control.in file, but its format requires that all the dependencies are listed on one line as a comma separated list, which is rather inconvenient to edit, and does not allow categorising and commenting our dependencies. Many times one forgets why they included a given package, especially when it’s used indirectly, so comments are important. So, let’s go and create it.

deb/DEBIAN/deps.mk:

### Base:
DEPS=nginx syncthing gnutls-bin
### VCS:
DEPS+=git

This file is basically a makefile fragment that we include in the makefile we created above.

At this moment we’re done with the control files for the Debian metapackage we’re creating. Now we’ll add the system/debian tree and populate it with examples. You can skip this step, and omit this subtree entirely if you think you won’t have any configuration files installed system-wide, in which case in the toplevel makefile you will have to edit the debian-init rule to not depend on debian-config.

The Configuration Tree

The system/debian tree will resemble the root file system of a Debian system and include files to be installed in such a path that a file system/debian/XXX will be copied to /XXX. We’ll make numbered backups 2 of target files so if we overwrite a file mistakenly, we will always be able to get it back. I recommend that you only use this to install configuration files into etc and maybe some other files. If you want to install an application that’ll go into /opt, you might want to add the -u option to the call to cp command in the rule debian-config of the toplevel makefile we’ll create so that it will skip files where the destination is not newer than the source file (see the cp(1) man page for further information).

We’ll create two files under this tree as examples of its use. These will be configuration files for Nginx. You’ll probably want to omit these and use your own configuration files, but you can use these ones however you like if you like them. Let’s go:

system/debian/etc/nginx/sites-enabled/myserver.site:

# nginx websites config.
## Docs:
# http://wiki.nginx.org/Pitfalls
# http://wiki.nginx.org/QuickStart
# http://wiki.nginx.org/Configuration
# /usr/share/doc/nginx-doc/examples/
##

# Default server configuration

server {
    listen 6878 default_server;
    listen [::]:6878 default_server;

    set $root_dir /igk/www;
    rewrite ^(/~[^/]+)$ $1/ redirect;
    rewrite ^/~(<user>[^/]+)(.+) $2;
    if ($user) {
       set $root_dir /home/$user/public_html;
    }
    root $root_dir;

    # Add index.php to the list if you are using PHP
    index index.html README;

    server_name myserver;

    location / {
        # First attempt to serve request as file, then
        # as directory, then fall back to displaying a 404.
        try_files $uri $uri/ =404;
    }

    # deny access to .htaccess files, if Apache's document root
    # concurs with nginx's one
    location ~ /\.ht {
        deny all;
    }
}

system/debian/etc/nginx/nginx.conf:

user www-data;
worker_processes auto;
pid /run/nginx.pid;
include /etc/nginx/modules-enabled/*.conf;

events {
    worker_connections 768;
}

http {
    sendfile on;
    tcp_nopush on;
    tcp_nodelay on;
    keepalive_timeout 65;
    types_hash_max_size 2048;

    include /etc/nginx/mime.types;
    default_type application/octet-stream;

    ssl_protocols TLSv1 TLSv1.1 TLSv1.2; # Dropping SSLv3, ref: POODLE
    ssl_prefer_server_ciphers on;

    access_log /var/log/nginx/access.log;
    error_log /var/log/nginx/error.log;

    gzip on;
    gzip_disable &quot;msie6&quot;;

    include /etc/nginx/conf.d/*.conf;
    include /etc/nginx/sites-enabled/*.site;
}

Now it’s time we write the toplevel makefile for our project.

The Makefile

We’ll have a concise and self documenting makefile that’ll have all the recipes for operating this configuration. When first initiating your system, you’ll be better off using the script that we’ll create next, so that you can create users, add third-party apt repositories, enable and disable services &c; but thereafter, when you want to add or remove a package to your system, you’ll just edit deb/DEBIAN/deps.mk, and run make deb-inst at the project root.

I’ll include the entire makefile below and explain it in detail.

makefile:

export VERSION!=date +'%Y%m%d%H%M'
export MAINT="Joe Maintainer <joe@example.com>"
export DEB=myconfig.deb
HERE=$(PWD)

all: help

help:
    @echo &quot;Targets:&quot;;\
    echo "  deb     build the Debian package ($(DEB))";\
    echo "  deb-inst    install $(DEB), generating it if necessary";\
    echo "  debian-config   install system configuration for Debian/Ubuntu";\
    echo "  debian-init initialise new Debian/Ubuntu system";

debian-init: deb-inst debian-config

debian-config:
    sudo cp -RPv --preserve=mode --backup=numbered system/debian/* /

deb: $(DEB)

$(DEB): deb-config deb/DEBIAN/control
    dpkg-deb -b deb $@

deb-config:
    cd deb/DEBIAN; $(MAKE) $(MAKEFLAGS); cd $(HERE)

deb-touch:
    touch deb/DEBIAN/control.in

deb-check: deb
    lintian $(DEB)

deb-inst: deb
    sudo apt-get install ./$(DEB) && sudo apt-get autoremove

clean:
    rm -rf *.deb; cd deb/DEBIAN; $(MAKE) $(MAKEFLAGS) clean;\
    cd $(HERE)

.PHONY: all deb deb-config deb-check deb-inst deb-touch clean
.PHONY: debian-config debian-init

Up top we have the $VERSION variable, which will generate a numeric version number for our package each time we generate and regenerate it, so that we don’t have to manually increment it. You can use whatever versioning scheme you might wish should you want to properly version the package, but for personal use this ever-increasing number should suffice and is going to be more convenient. If you will use a proper version number, mind that you should change the assignment operator from != to =, as the former runs the rvalue as a shell command and assigns its result to the variable (see the man page for make(1)). I believe that this makefile is POSIX compatible, but I only ever use it with GNU make and do not intentionally try to keep it so.

You can set $MAINT to your identity, but if I’m not wrong, the identities in the files under the deb/DEBIAN directory should match one another, so you probably will have to edit copyright and changelog to have this identity too.

The variable $DEB is the name of the debian package file that this makefile will generate. If you change this name, make sure that the initialisation script will use the correct name too.

The $HERE variable is the path to the project root, we use it to return there from a call to a recursive make process.

The all rule is the default rule (because it is the first one in the makefile), and we make it run the help rule, which just prints out an overview of the public rules; but if you want, you can change this to make it run a rule that does some concrete things.

The help rule, as we said, documents how to use this makefile.

debian-init runs the two most important rules in correct order when first initialising the system. We’ll explain these rules below.

The debian-config rule installs the system/debian tree to the system root directory like was explained in section [[The Configuration Tree]] . It does not dereference symbolic links (i.e. copies them verbatim; the -P flag), preserves the mode setting of the source file (the --preserve=mode option), and creates numbered backups of the target files (the --backup=numbered option). See the man page for cp(1) for detailed explanation of the flags. One possible modification here is that, as was said before, you can add the -u flag to the call to cp, which will not copy files if they are not newer than the target files they match.

deb is just an handy alias for the value of the $DEB variable, which contains the name of the debian file.

The $(DEB) rule generates the Debian package we define in the deb subdirectory of the project. It depends on the deb-config rule which runs make in the deb/DEBIAN tree, and on the deb/DEBIAN/control file, generated by the call to the recursive make by deb-config. We still depend on that file in case we want to modify it directly for debugging purposes.

deb-touch is a shortcut for updating the modification time of deb/DEBIAN/control.in without actually modifying it. We can use this like make deb-touch deb to force regeneration of the package without making any actual changes. Might be useful when $MAINT or $VERSION are changed.

deb-check executes a Debian package linting tool called lintian on the package. You probably will get many warnings as we skip many things that’d be required for a Debian package that you’d want to add to some official repository or publish yourself, for the sake of simplicity.

deb-inst installs the generated package, and runs apt-get autoremove to remove any package that is not a dependency of another one or not manually installed by you. So, when you remove a package from deb/DEBIAN/deps.mk and run this rule, it’ll remove that package. This probably is the rule you’ll use most often.

clean removes the generated package and cleans the deb/DEBIAN tree.

.PHONY is a special rule in make that lists all the rules that don’t directly generate a file from some inputs. See the man page for make(1) for details.

And this is the makefile for our project. You can copy it verbatim and modify as you need. At the moment you should be able to run make deb-inst and install the package if you have make and dpkg-deb commands available. Now we go on to create the initialisation script for our system.

The Initialisation Script

This script is optional, but may be helpful. Also, take this as an example, as you needs might vary quite a bit. Thus, I’ll document this rather lightly. Always run this script at the project root.

scripts/init.sh

# Initialise a Debian/Ubuntu installation.

set -e

. /etc/os-release

say () {
    echo === $@
}

### Add third-party repos:
#### Syncthing:
curl -s https://syncthing.net/release-key.txt | sudo apt-key add -
echo 'deb https://apt.syncthing.net/ syncthing stable' |\
    sudo tee /etc/apt/sources.list.d/syncthing.list
sudo chmod 644 /etc/apt/sources.list.d/syncthing.list

### Install packages:
say Installing system packages...
sudo apt-get update

(
    if which make >/dev/null 2>/dev/null; then
    make debian-init
    else
    sudo apt-get install ./myconfig.deb && make debian-config
    fi \
    || (echo Failed installing system programmes && exit 1)
)

### Add the user:
u=myuser
if id g 2>;/dev/null >;/dev/null; then
    say User $u exists already.
else
    say Adding user $u...
    sudo useradd -c 'Joe Maintainer' -d /home/$u \
     -G sudo,adm,cdrom,dip,plugdev,lpadmin -m -s /bin/bash -U -u 1881 $u \
    || echo Failed adding user $u &amp;&amp; exit 1;
    sudo groupmod -g 1881 $u || echo Failed setting GID for $u && exit
fi

### Start system services:
say Starting system services...
for service in nginx syncthing@$u; do
    (systemctl status $service | grep 'enabled;') >/dev/null \
    || sudo systemctl enable $service;
    (systemctl status $service | grep 'active (running)') >/dev/null \
    || sudo systemctl start $service;
done

say Done.  You may want to reboot your computer.

First, we set -e so that the script immediately terminates should an error happen. Then we source /etc/os-release which defines some useful variables on a Debian-like system. After, we define say for debugging. The next three command lines add the Syncthing repository. Remove this if you don’t use that application, but you can use these commands as a template for adding the repositories that you use.

Next we try to install the packages using the makefile. If make is available, we run make debian-init, otherwise we try to install from a previously generated package (it is a good idea to keep the latest build around for bootstrapping your setup on a fresh install; IIRC Ubuntu does not come with make, and Debian may lack both make and dpkg-deb). If both fail, we report error and exit with status of 1.

If the last step was successful, we create a user. During a fresh installation of Ubuntu I usually create a dummy user to run this script which thereafter I delete.

Lastly we enable and start the services we need. You should modify the service list to include the services you like, between for service in and ; do.

Conclusion

So this should be it! If you followed all the instuctions, you should have a working setup for a Debian metapackage to initialise your system and install your software with a single command. You can part from this to generate your personal setup. Hope it’s going to be useful to you!

Please e-mail me if you find any problems anywhere in the post. Happy hacking!

Footnotes

1 We’ll use recursive makefiles, which some define as an antipattern; but this is out of our scope as we’re not interested in fast and reliable builds of a computer programme (in which case that recursive makefiles are bad is a matter of debate, not a fact), but generation of a single file from a single output via plain text processing and some shell scripting depending on that. On actual programming projects where make is used to compile programmes though, there are better alternatives, e.g. GNU Make allows having files in subdirectories in targets and sources when defining rules.

2 See the GNU coreutils manual.