pyalpm : archlinux pacman bindings in python

Wanted to toy with python and pacman. Starting with subprocess calls to `pacman` and ugly string processing. But the google fairy told me I was heading in the wrong direction.

sudo pacman -S pyalpm

5 minutes later, I choke. I can't find a way to instantiate anything. ipython helps navigating modules, 5 more minutes I find a "hack" to get a Handle and then the local package database.

import pyalpm

PATH="/var/lib/pacman"

def init():
    "() -> db"
    h = pyalpm.Handle(PATH,PATH)
    return h.get_localdb()

DB = init()

def installed_pkgs():
    return DB.pkgcache()

In [115]: pkgs = installed_pkgs()

In [116]: pkgs[:10]
Out[116]: 
[<alpm.Package("ghc-7.6.3-1-i686") at 0xb5a1b130>,
 <alpm.Package("qemu-1.5.2-1-i686") at 0xb5a1ce50>,
 <alpm.Package("chromium-29.0.1547.65-1-i686") at 0xb5a18ab0>,
 <alpm.Package("emacs-24.3-4-i686") at 0xb5a18ae0>,
 <alpm.Package("faenza-icon-theme-1.3.1-2-any") at 0xb5a18d40>,
 <alpm.Package("python-3.3.2-2-i686") at 0xb5a1ccb0>,
 <alpm.Package("jre7-openjdk-headless-7.u40_2.4.1-3-i686") at 0xb5a1b8f0>,
 <alpm.Package("gcc-4.8.1-3-i686") at 0xb5a1b080>,
 <alpm.Package("python2-2.7.5-2-i686") at 0xb5a1cd50>,

 <alpm.Package("linux-3.10.10-1-i686") at 0xb5a1c420>]

<yay/>

archlinux :: mint 15 month long swap

update: 1 pacman -Su and 2GB later, boots ok, runs fine. One little detail, kernel warns about root being mounted rw when it shouldn't .. I couldn't read, boots too fast.

Fed up by, well, my mediocrity, I boot my archlinux machine again. Firefox shows a cached version of Google News July 30th 2013. It's been a month already. I just needed a debian based distro to try docker.io, arch is supported but it needs an aufs kernel module, which requires quite a lot of disk space to compile, my partition setup forbis such an amount. Anyway, Mint 15 was a agreable experience, after a while I forgot about arch, the overall better desktop configuration (better colors schemes, fonts, themes) made it feel stabler. I greatly suffered the windows manager intercepting all my emacs keybindings. also Xmonad screens suits me better than floating windows ala Mate.

The Mint 15 machine is less old, core duo => core 2 duo, with twice the ram and a non sluggish hard drive. A lot of things were faster. But a few days ago, I messed up the user/groups in Mint, I can't sudo anymore, booting singler user is useless since I believe root doesn't have a password, which is required by ubuntu/mint recovery mode. I don't even know what group I belonged to in the first place. Now I boot arch, the disk screams but 10 seconds later I'm under Xmonad. The terminal fires up fast, emacs fires up fast. Everything seems a little more reactive. This core duo can't decode 720p without choking but for most things everything is a little better. It's not even an engineered and polished system, I blended configuration examples, tweaks from wikis until it was good enough (c).

I have to admit, I forget a lot of things. Passwords took a little time to come back. I don't remember what I told Xmonad for launching the browser or the editor, fortunately I left the default dmenu one. Day saved.

Mint is not rolling, I missed the constant contact with arch repos. I'm in for a treat now. Pacman says I have 163 presents to open before midnight. Let's see if arch breaks as easily as I read so often on r/archlinux.

$ sudo pacman -Su
:: Starting full system upgrade...
:: Replace python-distribute with extra/python-setuptools? [Y/n]
resolving dependencies...
looking for inter-conflicts...

Packages (167): ack-2.08-1  alsa-plugins-1.0.27-2  archlinux-keyring-20130818-1
                bash-4.2.045-5  binutils-2.23.2-3  bison-3.0-1
                ca-certificates-java-20130815-1  cairo-1.12.16-1  cantarell-fonts-0.0.14-1
                chromaprint-0.7-6 chromium-29.0.1547.62-1  cloc-1.60-1
                cmus-2.5.0-3  cpupower-3.10-2  cryptsetup-1.6.2-1
                curl-7.32.0-1  device-mapper-2.02.100-1  dhcpcd-6.0.5-1
                dvdauthor-0.7.1-6  emacs-24.3-4  ethtool-1:3.10-1
                fbida-2.09-3 ffmpeg-1:2.0.1-1  file-roller-3.8.4-1
                fluidsynth-1.1.6-2  fontconfig-2.10.95-1  fuse-2.9.3-1
                gcc-4.8.1-3  gcc-libs-4.8.1-3  gdb-7.6.1-1
                gettext-0.18.3.1-1  ghostscript-9.10-1  giflib-5.0.4-2
                git-1.8.4-1  glew-1.10.0-2 glib2-2.36.4-1
                glibc-2.18-3  gnokii-0.6.31-6  gnome-icon-theme-3.8.3-1
                gnome-settings-daemon-3.8.4-2  gnupg-2.0.21-1  gnuplot-4.6.3-1
                gnutls-3.2.4-1  go-2:1.1.2-2  gpgme-1.4.3-1
                graphviz-2.32.0-1  groff-1.22.2-5 grub-2.00.5086-1
                gst-libav-1.0.10-1  gst-plugins-bad-1.0.10-1  gst-plugins-base-1.0.10-1
                gst-plugins-base-libs-1.0.10-1  gst-plugins-good-1.0.10-1  gst-plugins-ugly-1.0.10-1
                gstreamer-1.0.10-1  gtk3-3.8.4-1 guvcview-1.7.1-2
                hwdetect-2013.08-2  iftop-1.0pre2-1  imagemagick-6.8.6.9-1
                imlib2-1.4.5-5  intel-dri-9.2.0-1  inxi-1.9.14-1
                iproute2-3.10.0-1  iputils-20121221-3  ipython-1.0.0-3
                isl-0.12.1-1  iso-codes-3.44-1 jdk7-openjdk-7.u40_2.4.1-3
                jre7-openjdk-7.u40_2.4.1-3  jre7-openjdk-headless-7.u40_2.4.1-3  jshon-20130815-1
                kbd-2.0.0-1  keychain-2.7.1-3  kmod-15-1
                lftp-4.4.9-1  libarchive-3.1.2-2  libbsd-0.6.0-1
                libgexiv2-0.6.1-2 libmariadbclient-5.5.32-2  libmng-2.0.2-2
                librsvg-1:2.37.0-1  libsamplerate-0.1.8-2  libtiff-4.0.3-3
                libtracker-sparql-0.16.2-1  libusbx-1.0.16-2  libvdpau-0.7-1
                libwbclient-4.0.9-1  libwebp-0.3.1-3  linux-3.10.10-1
                linux-api-headers-3.10.6-1  lirc-utils-1:0.9.0-58  llvm-libs-3.3-1
                logrotate-3.8.6-1  lshw-B.02.17-1  lvm2-2.02.100-1
                man-pages-3.53-1  mariadb-5.5.32-2  mariadb-clients-5.5.32-2
                mercurial-2.7-1  mesa-9.2.0-1 mesa-libgl-9.2.0-1
                mkinitcpio-0.15.0-1  mkinitcpio-busybox-1.21.1-2  moreutils-0.49-1
                mpd-0.17.5-1  mplayer-36285-4  mupdf-1.3-4
                nasm-2.10.09-1  netctl-1.3-1  nodejs-0.10.17-1
                nspr-4.10-2  nss-3.15.1-1 ntp-4.2.6.p5-17
                openresolv-3.5.6-1  pacman-mirrorlist-20130830-1  parallel-20130722-1
                perl-5.18.1-1  perl-error-0.17021-1  perl-file-next-1.12-1
                perl-time-duration-1:1.1-3  php-5.5.3-1  php-apache-5.5.3-1
                php-pear-5.5.3-1  pixman-0.30.2-1  powertop-2.4-1
                python-distribute-0.6.45-1 [removal]  python-numpy-1.7.1-2
                python-pip-1.4.1-2  python-pygments-1.6-2  python-setuptools-1.0-1
                python-tornado-3.1.1-1 python-virtualenv-1.10.1-1  python2-mako-0.8.1-2
                python2-markupsafe-0.18-2  python2-paramiko-1.11.0-1  racket-5.3.6-1
                redis-2.6.16-1  redshift-1.7-7  reiserfsprogs-3.6.24-1
                run-parts-4.4-1  sbcl-1.1.10-1  serf-1.3.0-1
                sip-4.15.1-1  smartmontools-6.2-1  smbclient-4.0.9-1
                sqlite-3.8.0.1-1  subversion-1.8.1-2  syslog-ng-3.4.3-1
                tcl-8.6.0-5  testdisk-6.14-1  tk-8.6.0-2
                ttf-dejavu-2.34-1  unrtf-0.21.5-1  util-linux-2.23.2-1
                valgrind-3.8.1-3  vim-7.4.0-2  vim-runtime-7.4.0-2
                wayland-1.2.1-1  x264-20130702-2  xdebug-2.2.3-3
                xf86-video-intel-2.21.15-1  xorg-xprop-1.2.2-1  xorg-xset-1.2.3-1
                xorg-xwd-1.0.6-1  youtube-dl-2013.08.30-1 zsh-5.0.2-3

Total Download Size:    484.72 MiB
Total Installed Size:   2021.43 MiB
Net Upgrade Size:       27.16 MiB

:: Proceed with installation? [Y/n]

yet another fat archlinux update (with added dangerous migration juicy bits)

For once I had the reflex to check archlinux.org for additional informations. There was a filesystem structure change that needed some care.

# update of the day

luser@x60s_GPT ~ (master*) $ pacman -Qu
apache 2.2.24-1
bash 4.2.045-1
bluez 4.101-1
bridge-utils 1.5-1
cdrkit 1.1.11-2
coreutils 8.21-1
cryptsetup 1.6.1-1
dash 0.5.7-3
device-mapper 2.02.98-3
dhcpcd 5.6.8-1
e2fsprogs 1.42.7-1
ecryptfs-utils 103-1
ed 1.8-1
filesystem 2013.03-2
fuse 2.9.2-1
gconf 3.2.6-1
glibc 2.17-5
inetutils 1.9.1-5
iproute2 3.9.0-1
iptables 1.4.18-2
iptraf-ng 1.1.3.1-1
kmod 13-1
libatasmart 0.19-1
libsasl 2.1.26-3
lsof 4.87-1
lvm2 2.02.98-3
net-tools 1.60.20120804git-2
openresolv 3.5.4-2
pm-utils 1.4.1-5
ppp 2.4.5-5
rtmpdump 20121203-1
sed 4.2.2-2
shadow 4.1.5.1-5
systemd 204-2
systemd-sysvcompat 204-2
sysvinit-tools 2.88-10
tar 1.26-3
udisks2 2.1.0-2
usbmuxd 1.0.8-1
util-linux 2.23.1-1
v4l-utils 0.9.5-1
wpa_actiond 1.4-1
wpa_supplicant 2.0-3
zsh 5.0.2-1

# had to remove these alien packages because of the filesystem upgrade

luser@x60s_GPT ~ (master*) $ pacman -Qqo /bin /sbin /usr/sbin | pacman -Qm -
brightd 0.4.1-2
consolekit 0.4.6-4
ddcxinfo-arch 0.8-2
hwd 5.5.2-1
localepurge 0.6.2+nmu2-3
pmtools 20101124-2

another archlinux massive upgrade push

luser@x60s_GPT ~ (master*) $ pacman -Qu
avahi 0.6.31-6
cairo 1.12.14-3
cairomm 1.10.0-2
chromium 26.0.1410.63-1
dvdauthor 0.7.1-4
emacs 24.3-1
fbida 2.09-1
feh 2.9.1-1
gd 2.0.36RC1-5
gdk-pixbuf2 2.28.1-1
ghostscript 9.07-1
graphviz 2.30.1-2
gst-plugins-good 1.0.7-1
gstreamer0.10-bad 0.10.23-3
gstreamer0.10-bad-plugins 0.10.23-3
gstreamer0.10-good 0.10.31-1
gstreamer0.10-good-plugins 0.10.31-1
gthumb 3.2.1-1
guvcview 1.7.0-1
imagemagick 6.8.4.10-1
imlib2 1.4.5-2
jbig2dec 0.11-5
jdk7-openjdk 7.u21_2.3.9-1
jre7-openjdk 7.u21_2.3.9-1
jre7-openjdk-headless 7.u21_2.3.9-1
libgdiplus 2.10-3
libpng 1.5.15-1
librsvg 2.37.0-1
libtracker-sparql 0.16.0-4
libwebp 0.3.0-1
mjpegtools 2.0.0-2
mplayer 35920-1
php 5.4.14-1
php-apache 5.4.14-1
php-pear 5.4.14-1
poppler 0.22.3-2
qemu 1.4.1-1
qt4 4.8.4-16
rrdtool 1.4.7-5
sdl_image 1.2.12-2
slim 1.3.5-2
tumbler 0.1.27-2
virtualbox 4.2.12-1
webkitgtk2 1.10.2-4
webkitgtk3 2.0.1-1
wxgtk 2.8.12.1-4
xloadimage 4.1-12
xv 3.10a-16
luser@x60s_GPT ~ (master*) $

correcting pacman scriptlet syntax error with gdb

On Nov 18 2012, archlinux openntpd-3.9p1-20-i686 package install script had a syntax error. An empty bash function (it seems) caused upgrade abortion. I thought I could intercept package decompression so that I can correct the script but pacman is fine grained and clean everything as soon as it can. So I used that as an excuse to mess with it using gdb. I never ran a real program in gdb before, not even one without debugging symbols, so it wasn't a breeze. After 10 minutes of useless `continue` I tried to locate function calls of interest. Not knowing the linux C ecosystem I started with a guess `mkdir`, then saw `mkdtemp`. I was monitoring /tmp for new entries made by pacman. But by the time I see a .INSTALL it's already too late. After poking through pacman github's repo, I saw new names but these are non loaded shared libraries and I don't understand how to declare a breakpoint on these. Anyway the script execution is done in a forked process, so the .INSTALL file still exists at the time a `b fork` will pop, that's where vim comes into play for a little on-the-fly monkeypatch.

tl;dr: gdb breakpoints on mkdtemp and fork leaves an open window to edit pacman's .INSTALL script, and successfully install this package, one that people on #archlinux consider obsolete, too late, I'll brag anyway.

references:

https://projects.archlinux.org/pacman.git/tree/lib/libalpm/util.c

edit: few wordings, and forgot the end. * facepalm *

pacman-key errors with nightly isos of Arch Linux

tl;dr :

While Setup still running, change tty, Edit /tmp/pacman.conf to add 'SigLevel = Never' somewhere below '[core]', retry the package install step.

----

Out of curiosity I tried a nightly iso from here : http://releng.archlinux.org/isos/
The setup fetch the packages but fails to install them (throwing many "key could not be looked up remotely"), warning that "pacman-key --init should probably be run".
Running in vbox , pacman-key --init asks for entropy, usually playing with the system ( mouse, IO ) push the entropy above the needed threshold, but in my case nothing worked. Arch wiki suggests to use rngd from AUR, but the setup isos cannot build from AUR. Reading the output in the logging tty I finally catch the `pacman` command mentionning /tmp/pacman.conf. Adding the 'SigLevel = Never'.

The post-reboot pacman.conf is also using 'SigLevel = Never'.

- build rngd
- pacman-key --init
- switch SigLevel to TrustedOnly or anything that looks paranoid
- install something and see how it goes