2012/03/03

Home made pkgng repositories

By popular demand I have been requested a tutorial about how to maintain your own pkgng repositories.

Currently the best solution for that is to use poudriere (ports-mgmt/poudriere).

Let say you want to maintain 2 repositories: 8.2-RELEASE i386 and 9.0-RELEASE amd64.

For that you will need an amd64 (to support both amd64 and i386 box with 9.0 binary support and 8.2 binary support, a default 9.0-RELEASE amd64 should be enough :)

poudriere only depends on zfs and sh, so you won't need much, just a zfs pool available.

First: install poudriere:

$ make -C /usr/ports/ports-mgmt/poudriere install clean

To be able to build the repository the host would also need to have pkgng installed (no need to convert it to pkgng anyway.)

$ make -C /usr/ports/ports-mgmt/pkg install clean

Now configure your poudriere, defining some configuration to /usr/local/etc/poudriere.conf:

BASEFS=/poudriere
ZPOOL=myzpool
FTPHOST=ftp.freebsd.org
POUDRIERE_DATA=/poudriere_data
RESOLV_CONF=/etc/resolv.conf
DISTFILES_CACHE=/usr/ports/distfiles

This should be enough (see poudriere.conf.sample for more informations)

Create the default ports tree:

$ poudriere ports -c

Create the two jails (in fact it will be chroot :))

$ poudriere jails -c -j 82i386 -v 8.2-RELEASE -a i386
$ poudriere jails -c -j 90amd64 -v 9.0-RELEASE -a amd64

Make them pkgng aware

$ mkdir /usr/local/etc/poudriere.d
$ echo "WITH_PKGNG=yes" > /usr/local/etc/poudriere.d/82i386-make.conf
$ echo "WITH_PKGNG=yes" > /usr/local/etc/poudriere.d/90amd64-make.conf

Add the list of packages you want to build:

$ cat ~/mylist1
editors/vim
www/nginx
$ cat ~/mylist2
www/firefox
editors/libreoffice

If you want special options just add them to you different make.conf in poudriere.d

to share options between all the jails managed by poudriere, add them to /usr/local/etc/poudriere.d/make.conf

You can now create your packages:

$ poudriere bulk -f ~/mylist1 -j 82i386
$ poudriere bulk -f ~/mylist2 -j 90amd64

When finished you will get 2 pkgng repositories in /poudrieredata/packages/82i386-default and /poudrieredata/packages/90amd64-default

You can now provide them through your webserver.

On you client boxes just:

$ echo "packagesite: http://yoururl/82i386-default" >> /usr/local/etc/pkg.conf

or

$ echo "packagesite: http://yoururl/90amd64-default" >> /usr/local/etc/pkg.conf

If your client box already has packages from an old installation, first convert it to pkgng

$ fetch http://yoururl/90amd64-default/Latest/pkg.txz
$ tar xf ./pkg.txz -s ",/.*/,,g" "*/pkg-static"
$ ./pkg-static add ./pkg.txz
$ pkg2ng

Now you can forget about pkg_install and pkg2ng

Just use pkgng (for example):

$ pkg update
$ pkg upgrade
$ pkg install firefox

To update your repository:

$ poudriere ports -u # this update your default ports tree
$ poudriere bulk -f ~/mylist1 -j 82i386 -k
$ poudriere bulk -f ~/mylist2 -j 90amd64 -k

bonus poudriere will rebuild only what has changed and what is impacted by this change.

Once it is done simply upgrade your client box:

$ pkg update
$ pkg upgrade

More informations on poudriere here

rqsulfates wrote on 2012/03/05 :

Thanks for your share!!


Roman wrote on 2012/03/17 :

So there's no way to use it without zfs?


bapt wrote on 2012/04/08 :

No there is no way without zfs


Jon wrote on 2012/04/23 :

Please specify how to build all ports tree. Thanks in advance.


Jon wrote on 2012/04/23 :

Please specify how to build all ports tree. Thanks in advance.


Bapt wrote on 2012/04/24 :

Currently the only way to build all the ports tree would be to list all the compotent into the -f


Bryan+Drewery wrote on 2012/05/01 :

Great guide. Thank you!


Vick Khera wrote on 2012/07/12 :

Just gave this a try. Failed here:

# poudriere bulk -f /tmp/packages -j 90amd64
kldload: can't load linprocfs: No such file or directory

Looked through the sources and found that it unconditionally demand-loads linprocfs and linsysfs when running the builk command, and there is no flag to disable that.

You should list linprocfs in your requirements.


ANDREYF wrote on 2012/07/23 :

http://git.etoilebsd.net/poudriere/tree/README error 404 :(


bobstar wrote on 2012/12/12 :

hi
are there An changes that a public repo (packagesite) like pkgbeta Will Ben available/usable soon)?

i won't doubt many people are waiting for that.

thanx anyway


Cody121 wrote on 2013/01/27 :

If I add a port to 'mylist', and re-run poudriere, it proceeds to delete all existing .txz files claiming that it is 'missing dependencies'. It then and re-builds every single one of the ports it just deleted unless I tell it to skip sanity checks (-s).

Lets say after I update my ports tree with poudriere -u, should every package be rebuilt at that time regardless if it was actually updated? Can i skip the sanity check after an update? Sounds insane to skip the sanity check, but rebuilding the same code over and over and over is tedious and perhaps unnecessary?

Any clarification would be greatly appreciated. Or maybe just a best practices for adding a port.

poudriere version 2.3.1
pkg-1.0.6
9.1-RELEASE

Thanks!


Cody121 wrote on 2013/02/13 :

I dont know what I did different, but following these instructions seem to have it update correctly without updating everything.
https://fossil.etoilebsd.net/poudriere/doc/trunk/doc/pkgng_repos.wiki
(they are probably the same, just I botched these)


Name :
URL (optional) :
Write here: "ETOOMUCHSPAM"
Comments :