Creating and maintaining mkbuilds and SlackBuilds

Mkbuild is simplepkg's tool for creating SlackBuild script from mkbuild spec files and SlackBuild models. Mkbuild files are for simplepkg what an ​ebuild is for ​Gentoo, what an ​spec file is for ​rpm and what ​rules is for ​Debian.

In short, a .mkbuild file has the minimum information needed to tell the mkbuild program how to build an SlackBuild for a given program in a way that this resulting SlackBuild script is able to build a package for this program by itself, i.e, the resulting SlackBuild is an independent script capable to create a package without the need of any additional program to run it. But, as you probably already saw at the createpkg howto, simplepkg provides also a tool to easy the process of creating packages from SlackBuilds and managing binary repositories.

Mkbuild manages two different trees: a mkbuild tree composed by mkbuild model files and a SlackBuild tree.

Mkbuild example

Let's start with a real example. Below we have the whole content of a mkbuild file that can produce a SlackBuild for the ​Kino video editing software. We'll use kino as an example as it's fully compliant with ​autotools way of "./configure && make && make DESTDIR=$PKG install" to do things. So it's a nice example to start but take in mind that mkbuild is not limited in fully compliant installation methods as we'll see during this tutorial.

Here comes the code for kino.mkbuild:

#
#  This mkbuild is free software; you can redistribute it and/or modify it under the
#  terms of the GNU General Public License as published by the Free Software
#  Foundation; either version 2 of the License, or any later version.
#
#  This mkbuild is distributed in the hope that it will be useful, but WITHOUT ANY
#  WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR
#  A PARTICULAR PURPOSE.  See the GNU General Public License for more details.
#
#  You should have received a copy of the GNU General Public License along with
#  this program; if not, write to the Free Software Foundation, Inc., 59 Temple
#  Place - Suite 330, Boston, MA 02111-1307, USA
#

#------------------------
# Construction Variables
#------------------------

# Author name
[[SLACKBUILD AUTHOR]]="Silvio Rhatto"

# Initials author name
[[SLACKBUILD AUTHOR INITIALS]]="rha"

# Complete URL address or URL base address ( without $SRC_NAME-$VERSION... )
[[DOWNLOAD FOLDER URL]]="http://downloads.sourceforge.net/kino/kino-1.3.2.tar.gz"

# Documentation files. Null ("") to auto-set commom doc-files:
# NEWS TODO README AUTHORS INSTALL ChangeLog MAINTAINERS COPYING readme.*
[[DOCUMENTATION FILES]]="BUGS AUTHORS COPYING ChangeLog INSTALL NEWS README TODO"

# Other configure arguments
[[OTHER CONFIGURE ARGS]]="--enable-quicktime --sysconfdir=/etc --with-docdir=/usr/doc/$PACKAGE-$VERSION"

# SlackBuild model
[[SLACKBUILD MODEL]]="generic.mkSlackBuild"

# SlackBuild PATH in Slack.Sarava tree, defaults to "others/unclassified/[[PKG NAME]]"
[[SLACKBUILD PATH]]="media/video/kino"

# Dependency list input
# Enter string: "dependency_1 [condiction_1] [versio_1]: dependency_2 [condiction_2] [versio_2]"
[[SLACK REQUIRED]]="libsamplerate: libraw1394: libavc1394: libdv: libiec61883: libquicktime"

#---------------------
# SlackBuild Sections
#---------------------
# Default sections:
#      head, set_variables, slkflags, start_structure, untar_source,
#      make_package, install_package, build_package
# Warning: don't remove '#>>' and "#<<" tags.
#>> Start SlackBuild Sections:
 on: head
 on: slackbuildrc
 on: set_variables
 on: slkflags
 on: error_codes
 on: start_structure
off: create_build_user_and_group
 on: download_source
off: md5sum_download_and_check_0
off: md5sum_download_and_check_1
off: gpg_signature_check
 on: manifest_check
 on: untar_source
 on: patch_source
 on: configure
 on: make_package
 on: install_package
 on: strip_binaries
off: compress_manpages
off: compress_info_files
off: copy_init_scripts
 on: install_documentation
 on: slackdesc
off: move_config_files
off: postinstall_script
 on: slack_required
 on: build_package
 on: clean_builds
#<< End SlackBuild Sections

#------------------
# Sections changes
#------------------
#>slackdesc
kino: kino (Kino is a non-linear DV editor for GNU/Linux)
kino:
kino: Kino is a non-linear DV editor for GNU/Linux. It features excellent 
kino: integration with IEEE-1394 for capture, VTR control, and recording 
kino: back to the camera. It captures video to disk in Raw DV and AVI 
kino: format, in both type-1 DV and type-2 DV (separate audio stream) 
kino: encodings.
kino: 
kino: 
kino: http://www.kinodv.org/
kino: 
#<slackdesc

Some more concepts before proceed

Before continuing explaining the bits of kino.mkbuild, let's talk a bit more about mkbuild concepts. Roughly speaking, a mkbuild file is split into three sections:

1. Construction variable assignment.
2. SlackBuild model section switches.
3. SlackBuild model section changes.

What mkbuild basically does is to apply the specs set at the .mkbuild file into a SlackBuild model file (.mkSlackBuild). By default, mkbuild uses the generic.mkSlackBuild model found at /etc/simplepkg/defaults/mkbuild but other models are provided (for example, a model file intended to build packages for perl modules) and you can even create or modify models to fit your own needs.

Mkbuild combines a spec file with a model, producing a result SlackBuild that can then be used to build the actual package:

Mkbuild spec ---->----.
                       \
SlackBuild Model -->-----> SlackBuild ---> Binary Package

A SlackBuild model file is basically a shell script template split into sections and written to support a simple pattern replace of some of its parameters. What the three mkbuild sections mentioned above does is just to assign specific values to a SlackBuild model, turn on or off some of it's sections and even modify partially or entirely some of its sections.

But why not just managing SlackBuilds?

At this part of the tutorial you're probably asking: why these people choose to keep an additional spec file instead of just managing SlackBuild files? Why they added this amount of complexity?

The answer is: imagine you have a repository with hundreds of SlackBuilds and you realized that all your SlackBuilds lacks some functionality or they need to be rewritten to be compatible with some third party software. For minor changes, grep and sed can work, but if you're developing a large system and are planning to make deep changes in your SlackBuilds, this method can be a pain.

The aim to create a mkbuild spec file is to concentrate just in the way a package needs to be built and let all specific details in how to do it do a SlackBuild model file. If you want to change all your SlackBuilds, just change the model and let mkbuild rebuild all your SlackBuild repository. A SlackBuild model can be understood as the generalization of the procedures used to create a package, and each mkbuild spec can be understood as a special configuration of this generalized process.

Back to our example

Let's get back to our kino.mkbuild example. The first mkbuild "section" (Construction variable assignment) set some of the SlackBuild parameters accordingly to the values needed to build a package for kino. Some of its parameters are self-explaining, like SLACKBUILD AUTHOR, SLACKBUILD AUTHOR INITIALS, DOWNLOAD FOLDER URL and DOCUMENTATION FILES, while others need at least a short explanation:

• OTHER CONFIGURE ARGS: this parameter is optional and adds more options to the configure script than the default provided by the SlackBuild model.
• SLACKBUILD MODEL: sets which SlackBuild model should be used.
• SLACKBUILD PATH: sets the SlackBuild and mkbuild path inside its respective trees.
• SLACK REQUIRED: if this parameter is set, mkbuild will also create a slack-required file containing the required dependencies to build the package.

In this howto we'll not explain about all existing and possible parameters for a mkbuild spec. Have a look at the mkbuild documentation? instead.

The next mkbuild spec section we'll talk about is the SlackBuild model section switches. In kino.mkbuild case, we have:

#---------------------
# SlackBuild Sections
#---------------------
# Default sections:
#      head, set_variables, slkflags, start_structure, untar_source,
#      make_package, install_package, build_package
# Warning: don't remove '#>>' and "#<<" tags.
#>> Start SlackBuild Sections:
 on: head
 on: slackbuildrc
 on: set_variables
 on: slkflags
 on: error_codes
 on: start_structure
off: create_build_user_and_group
 on: download_source
off: md5sum_download_and_check_0
off: md5sum_download_and_check_1
off: gpg_signature_check
 on: manifest_check
 on: untar_source
 on: patch_source
 on: configure
 on: make_package
 on: install_package
 on: strip_binaries
off: compress_manpages
off: compress_info_files
off: copy_init_scripts
 on: install_documentation
 on: slackdesc
off: move_config_files
off: postinstall_script
 on: slack_required
 on: build_package
 on: clean_builds
#<< End SlackBuild Sections

Each SlackBuild model has its sections. What this mkbuild section do is just to swith which ones should be used and which ones not. At kino's case, the resulting mkbuild will not have a section to check the source file with gnupg (gpg_signature_check) for example, but will have a section to strip the complied binary files (strip_binaries).

The last "section" is about SlackBuild model section changes, where any SlackBuild model section can be changed. In kino's case, jut the slack-desc section is changed to contain its short description:

#------------------
# Sections changes
#------------------
#>slackdesc
kino: kino (Kino is a non-linear DV editor for GNU/Linux)
kino:
kino: Kino is a non-linear DV editor for GNU/Linux. It features excellent 
kino: integration with IEEE-1394 for capture, VTR control, and recording 
kino: back to the camera. It captures video to disk in Raw DV and AVI 
kino: format, in both type-1 DV and type-2 DV (separate audio stream) 
kino: encodings.
kino: 
kino: 
kino: http://www.kinodv.org/
kino: 
#<slackdesc

Creating your first SlackBuild

If you want to try out mkbuild with our example, just save it's contents to a file called kino.mkbuild and then run

mkbuild kino.mkbuild

Mkbuild will apply the specifications from kino.mkbuild at the generic.mkSlackBuild model and make a kino.SlackBuild. As we set the SLACK REQUIRED parameter, mkbuild will create a slack-required for it. By default, mkbuild will also try find the kino source code or download it from DOWNLOAD FOLDER URL in order to create a Manifest file where integrity metadata about the source code should be stored.

After mkbuild created kino.SlackBuild, you can test running it as root

./kino.SlackBuild

And the SlackBuild should be able to build kino if you have all its dependencies already installed on you system. If you don't have all dependencies already installed, try first to install then using createpkg and then run kino.SlackBuild again.

Creating a mkbuild

You may want also to create mkbuilds for the programs you like or need, so we'll give short instructions about how to do that. Please refer to the mkbuild documentation? for more complete information.

First, create a working environment:

mkdir /tmp/mkbuild && cd /tmp/mkbuild

Then, create a mkbuild containing default configuration:

mkbuild -n program-name

The above command creates a file called program-name.mkbuild. Edit this file making the appropriate changes, especially DOWNLOAD FOLDER URL and SLACKBUILD PATH. You don't need to fill and use all parameters: most of them are optional and just needed if the program you want to build has a different way to be built than the SlackBuild model is ready for.

Once you made your changes, type

mkbuild program-name.mkbuild

And a program-name.SlackBuild should be built.

Managing mkbuild and SlackBuild repositories

Mkbuild is also able to manage a whole mkbuild and SlackBuild repositories, with or without revision control. By default, mkbuild is configured to store mkbuilds at /var/simplepkg/mkbuilds and SlackBuilds at /var/simplepkg/SlackBuild (note that the SlackBuild folder used by mkbuild is the same used by createpkg when looking for package building scripts).

To submit a mkbuild you just made to its repository folder, type

mkbuild --submit-mkbuild program-name.mkbuild

or simply

mkbuild -sm program-name.mkbuild

To submit a SlackBuild to its repository folder, try

mkbuild --submit-slackbuild program-name.mkbuild

or simply

mkbuild -sa program-name.mkbuild

In all these cases, mkbuilds are by default sent to /var/simplepkg/mkbuilds and SlackBuilds to /var/simplepkg/slackbuilds. If you want to simultaneously submit both mkbuild and SlackBuild files, try

mkbuild --submit-all program-name.mkbuild

or simply

mkbuild -sa program-name.mkbuild

Mkbuild will store both mkbuild spec and the SlackBuild script in subfolders defined by SLACKBUILD PATH mkbuild parameter, which defaults to "others/unclassified/program-name" if absent from the mkbuild spec file.

Repositories under version control

One of the most useful features of mkbuild -- and of course simplepkg -- is it's ability to manage mkbuild and SlackBuild trees under version control. You can easily turn your trees into version control system using

mkbuild --import

If you just want version control at the mkbuild tree, use

mkbuild --import-mkbuilds

The same works if you just want version control at the SlackBuilds tree:

mkbuild --import-slackbuilds

These commands will create subversion repositories at /var/svn/mkbuilds and/or /var/svn/slackbuilds, import existing mkbuilds/SlackBuilds to them and then make /var/simplepkg/mkbuilds and/or /var/simplepkg/slackbuilds as a working copies. If you want to import your mkbuilds and/or Slackuilds to diferent places, just tell mkbuild to do so, for example

mkbuild --import /some/other/subversion/path

You can even import your stuff to a remote repository using a command like

mkbuild --import svn+ssh://server/path

Instead keeping your own version control system with just your mkbuilds and SlackBuilds, probably would be better to benefit and contribute to a collective effort and use the repositories maitained by the ​Slack Sarava Community. In order to do so, forget the above import commands and first checkout from the mkbuilds repository using

mkbuild --sync

And then sync with the SlackBuilds repository using

createpkg --sync

Note that if you already imported your files to new repositories, then --sync will just update simplepkg's working copies, so if you want to switch from your own repositories to the ones maintained by the community you'll need to move away your current /var/simplepkg/mkbuilds and/or /var/simplepkg/slackbuilds.

Once you imported the repositories, you can commit changes made to and existing or new mkbuild using

mkbuild --commit-all program-name "commit message"

To check local changes before commiting, use

mkbuild --status

All these basic repository commands (import, sync, commit and status) makes mkbuild able to easily manage mkbuilds and SlackBuild repositories for you, no need to worry about doing things by hand.

Other useful commands

Mkbuild was written to make package maintainer's life easier, so has commands to make management of a whole tree of mkbuilds or SlackBuilds really easy. For example, when you do

mkbuild program-name

mkbuild first search for a file called package-name.mkbuild at your current folder. If it can't find it, then it tries to search at the mkbuild tree (by default, /var/simplepkg/mkbuilds).

All other mkbuild commands that operates with a mkbuild file has the same behaviour and therefore you can manage all your mkbuilds and SlackBuilds without need to navigate through your repositories.

If you want to edit an existing mkbuild, just type

mkbuild -e program-name

and then mkbuild will open program-name.mkbuild (if found) at the editor of your choice (set with $EDITOR environment variable).

How it makes one's life easier

As an example of how mkbuild makes one's life easier, let's consider a real situation where both mkbuild and SlackBuild trees are under version control in a shared development environment (i.e, with multiple developers) and that you want to add a new software (rsnapshot) at both trees.

First you begin syncing your mkbuild repository, then you setup a work folder where you start editing your new mkbuild. After that, you let mkbuild create the SlackBuild script and submit both mkbuild and SlackBuild to their respective trees:

$ mkbuild --sync # sync with mkbuild repository
$ mkdir ~/work && cd ~/work # setup a workfolder
$ mkbuild rsnapshot # this creates a new mkbuild
$ mkbuild -e rsnapshot # edit especially source URL
$ mkbuild -sa rsnapshot
[...]

Submiting rsnapshot.SlackBuild
A         app/backup/rsnapshot
A         Manifest
A         rsnapshot.SlackBuild

Submiting rsnapshot.mkbuild
A         app/backup/rsnapshot
A         Manifest
A         rsnapshot.mkbuild

Then, before commiting, you can try to compile and it to see if everything happens as expected:

createpkg rsnapshot

If the package is ok, then you can submit both mkbuild and SlackBuild to their trees using:

$ mkbuild -ca rsnapshot "adding rsnapshot"
Adding         app/backup/rsnapshot
Adding         app/backup/rsnapshot/Manifest
Adding         app/backup/rsnapshot/rsnapshot.SlackBuild
Transmitting file data .
Committed revision 1983.
Adding         app/backup/rsnapshot
Adding         app/backup/rsnapshot/Manifest
Adding         app/backup/rsnapshot/rsnapshot.mkbuild
Transmitting file data .
Committed revision 274.

Now, if other users sync their trees with the repository, then they will be able to create the rsnapshot package you just added a mkbuild/SlackBuild pair.

If you realized that you made a mistake or if you want to update rsnapshot mkbuild and/or SlackBuild, just edit it, submit and commit:

$ mkbuild -e rsnapshot.mkbuild
$ mkbuild -sa rsnapshot

Submiting rsnapshot.SlackBuild

Submiting rsnapshot.mkbuild

$ mkbuild -ca rsnapshot "fixing key url"
Sending        app/backup/rsnapshot/rsnapshot.SlackBuild
Transmitting file data .
Committed revision 1984.
Sending        app/backup/rsnapshot/rsnapshot.mkbuild
Transmitting file data .
Committed revision 275.

Manifest files

Mkbuild don't just make things easier but also safer with the use of Manifest files very similar to ​the ones used by Gentoo. The idea is to have an integrity checking for each source file used by mkbuild/createpkg to prevent compilation, installation and usage of compromised code.

When you run mkbuild, by default it already tries to build hashes from the program source code. If it don't find the source, it attempt to download it. From our example above, we'll have:

$ mkbuild rsnapshot
Updating Manifest...
Trying to find rsnapshot-1.3.1.tar.gz at /tmp/rsnapshot /var/simplepkg/sources /tmp...
File rsnapshot-1.3.1.tar.gz not found, trying to download it...
[...]  

2009-01-19 16:37:17 (40.8 KB/s) - `/var/simplepkg/src/rsnapshot/rsnapshot-1.3.1.tar.gz' saved [215806/215806]

Please make sure that the following hashes are correct:
DIST rsnapshot-1.3.1.tar.gz 215806 MD5 md5_hash SHA1 sha1_hash SHA256 sha256_hash SHA512 sha512_hash

So, before you sending mkbuilds and SlackBuilds to the repository, check for the source code metadata at the Manifest file. When createpkg (or even running the SlackBuild by hand) attempts to build the package, it will first check if the source code it has available has the same hashes that are at the Manifest file.

If you want to, mkbuild is also capable of digitally signining Manifest files using gpg, although that is not enabled by default (se mkbuild documentation? for more information). If you do so, the generated SlackBuild will be able to check if the Manifest signature is valid.