A Raspbian read-only root-fs HOWTO

Geschrieben am 13. März 2014 in Debian GNU/Linux, Linux, Raspberry Pi von giggls || Keine Kommentare

In embedded applications it is often a requirement, that the device must be able to sustain a power cycle almost any time.

Unfortunately this is not something which modern operating systems (including GNU/Linux) like very much.

Fortunately in Linux there are workarounds. While there are specialized filesystems like f2f2, the most simple approach is still to just run the OS from a read-only root filesystem.

So here is the solution I made for my brewing hardware.

We bootup our fresh raspbian image install available at http://downloads.raspberrypi.org.

On the HDMI console expand the filesystem and setup i18n (german keyboard in my case).

All steps starting from here can now be done via ssh as well as the HDMI console.

  • Remove some stuff which is not needed or unsuitable for readonly operation:
  • apt-get remove --purge wolfram-engine triggerhappy
    apt-get remove --purge cron anacron logrotate dbus dphys-swapfile
    
  • Remove X-Server and related stuff:
  • apt-get remove --purge xserver-common lightdm
    insserv -r x11-common
    
  • auto-remove some X11 related libs
  • apt-get autoremove --purge
    
  • Install busybox syslog instead of rsyslog
  • The reason for doing this is because we do not want logfiles, but we want to be able to do some debugging (read logfiles). busybox-syslogd does not write logfiles but logs to a ring-buffer in memory which can be displayed using the logread command:

    apt-get install busybox-syslogd
    dpkg --purge rsyslog
    

The following steps are important, because we do not want any filesystem checks on our headless system at all!

  • Comment do_start in /etc/init.d/checkroot.sh
  • Comment do_start in /etc/init.d/checkfs.sh
  • ...
    case "$1" in
      start|"")
            #do_start
            ;;       
      restart|reload|force-reload)
            echo "Error: argument '$1' not supported" >&2
            exit 3
            ;;    
      stop)
            # No-op
            ;;     
      *)
            echo "Usage: checkfs.sh [start|stop]" >&2
            exit 3
            ;;    
    esac
    ...
    
  • Comment Operations in /etc/init.d/checkroot-bootclean.sh
  • ...
    case "$1" in
      start|"") 
            # Clean /tmp, /lib/init/rw, /run and /run/lock.  Remove the
            # .clean files to force initial cleaning.  This is intended
            # to
            # allow cleaning of directories masked by mounts while the
            # system was previously running, which would otherwise
            # prevent
            # them being cleaned.
            #rm -f /tmp/.clean /lib/init/rw/.clean /run/.clean /run/lock/.clean
    
            #clean_all
            exit $?   
            ;;
      restart|reload|force-reload)
            echo "Error: argument '$1' not supported" >&2
            exit 3
            ;;
      stop)   
            # No-op
            ;;
      *)
            echo "Usage: checkroot-bootclean.sh [start|stop]" >&2
            exit 3
            ;;
    esac
    ...
    
  • Comment swaponagain ‘swapfile’ in /etc/init.d/mountall.sh
  • Remove a couple of startup scripts:
  • insserv -r bootlogs
    insserv -r sudo
    insserv -r alsa-utils
    insserv -r console-setup
    insserv -r fake-hwclock 
    
  • Change /etc/fstab as follows:
  • proc              /proc           proc    defaults     0       0
    /dev/mmcblk0p1    /boot           vfat    defaults,ro  0       2
    /dev/mmcblk0p2    /               ext4    defaults,ro  0       1
    tmpfs             /tmp            tmpfs   defaults     0       0
    
  • append ro in /boot/cmdline.txt:
  • ...  elevator=deadline rootwait ro
    
  • Make dhclient write its leases file to /tmp instead of /var/lib/dhcp/:
  • rm -rf /var/lib/dhcp/
    ln -s /tmp /var/lib/dhcp
    

That’s it, have fun with your read-only Raspbian. As far as my brewing software is concerned, there is automated remount-rw/ro support included (see sample configfile).

“git bisect” the perfect tool for knocking down bugs

Geschrieben am 16. Januar 2013 in Linux von giggls || Kommentare deaktiviert

The most annoying bugs are those of the “this did already work in the past” type.

Doing manual bisecting has also been an annoying task in the past. Especially when talking about Kernel bugs which will require frequent reboots.

Fortunately this step while still somewhat time consuming got much easier in recent times thanks to two very handy tools. One of them is git, the top-notch revision control system, the other one is virtualisation (virtualbox in my case).

So here is how I search the broken commit using theses tools:

Lets assume we have a problem which we discovered in kernel 3.7 and which did not appear in kernel 3.6.

First get a stable kernel tree if you do not already have one:

git clone git://git.kernel.org/pub/scm/linux/kernel/git/stable/linux-stable.git

The issue the following 3 commands:

git bisect start
git bisect good v3.7
git bisect bad v3.6

Our source tree is now at a state where we must check if this kernel works or not. For this purpose I usually run Debian’s make-kpkg command which will build a debian package from a given kernel source.

Afterwards install this kernel in your virtual Linux system and reboot the VM.

Login to the VM and check if the bug is present there. I so issue a git bisect bad command, if not call git bisect good.

Repeat these two steps until git tells you exactly which commit did cause the problem.

Now all you need to do is mail the person responsible for the faulty commit.

Sven

The perfect Gitolite-Server (with Kerberos Authentication and more)

Geschrieben am 4. Dezember 2012 in FOSS, kerberos, Linux, Tipp/hint von giggls || 2 Kommentare

Back in Juli I wrote a blog-post about how I set up a Gitolite-Server using Kerberos-Authentication.

As this post seems to be the only documentation on the web about how to do this, I got quite some feedback. In a recent email conversation I have been asked, if I know about a method, which would not require a patched Version of ssh.

Well, I did not know of one immediately, but now I have implemented one, which does not only make it unnecessary to patch sshd, but will also make the server a little bit more elegant to use from a users perspective :)

So here is a new Version of my Gitolite Server+Kerberos HOWTO

Login is now possible with your usual login name (username@servername), using gitolite@servername is obsolete and disabled by this setup.

Supported login-methods are:

  • password authentication (password is checked by whatever active Pluggable Authentication Module, pam_krb5 in my case)
  • authentication without password using an ssh public key
  • authentication without password using kerberos/gssapi

How to setup the system:

We once again start from a system which has a working Kerberos installation. We will however not need something like libnss-ldapd or libnss-sss. I assume that we are working as root, so just use sudo bash on Ubuntu and derivates.

  • Add a local user gitolite to your system with “*” in passwd field
  • Download and compile libnss-catchall [1]:
  • git clone git://git.geggus.net/nss-catchall.git
    cd libnss-catchall
    dpkg-buildpackage or make

  • Install the resulting libnss-catchall package or shared library:
  • dpkg -i ../libnss-catchall*.deb

  • create /etc/passwd_nss_catchall as follows:
  • grep gitolite /etc/passwd >/etc/passwd_nss_catchall

  • Change the passwd line in /etc/nsswitch.conf as follows:
  • passwd: compat catchall

  • Append the following lines to your sshd_config [2]:
  • PermitUserEnvironment yes
    Match User !root,*
    ForceCommand /usr/local/bin/gitolite_wrapper_script

  • Create the gitolite_wrapper_script as follows:
  • echo -e '#!/bin/bash\n\n/usr/local/bin/gitolite-shell $LOGNAME\n' >/usr/local/bin/gitolite_wrapper_script

  • su to user gitolite and clone the gitoline code into this users home directory:
  • git clone git://github.com/sitaramc/gitolite.git gitolite.clone

  • Loosely follow the Installation instructions in README.txt which will boil down to the following commands [3]:

  • cd gitolite.clone
    mkdir -p $HOME/bin
    ./install -to $HOME/bin
    $HOME/bin/gitolite setup -a <adminid>

  • Make shure you have gitolite and gitolite-shell available in your PATH, I did this by adding symlinks to /usr/local/bin
  • That’s it! You should have a working gitolite server now

Public-key usage is a little bit different from the gitolite documentation. The lines in the file authorized_keys need to look like this:
environment="LOGNAME=your_username" ssh-rsa AAA

A command Option might be present, but is ignored because of the ForceCommand Option in sshd_config.

As with my old setup, Windows users will need to use plink.exe and point the environment variable GIT_SSH to this executable, openssh on Unix will work out of the box if gssapi authentication has been enabled.

[1] The whole stuff works because of libnss-catchall, a NSS (Name Service Switch) module written by me. It will always return a given single uid/gid/home combination for any user who managed to login somehow. This way we always end up being logged in as the gitolite user regardless of the username provided. The login username will however be present in the LOGNAME environment variable in case of gssapi or password authentication and must be set manually when using ssh public keys.
[2]If you have local users on your machine which should be able to use interactive logins adjust the “Match User” line. On a multi-purpose machine one should IMO consider using the chroot feature of ssh and a separate IP-address for gitolite anyway.
[3]The string I call <adminid> here is most likely the login-name (local part of the kerberos realm) of the one installing this stuff (you!).

Using gitolite with Kerberos Authentication

Geschrieben am 12. Juli 2012 in FOSS, kerberos, Linux von giggls || Kommentare deaktiviert

This article is obsolete now! There is a new article now which does describe a slightly different and better solution.

Once you have been succeeded in taming the three-headed beast called Kerberos, this powerful beast will prove handy for quite a lot of stuff!

I have been fiddling with Linux AD Integration and thus Kerberos at my workplace for quite some time now. Recently I needed to setup a gitolite server for software development, as more and more people tend to migrate from SVN to GIT now and using file based git repositories are a pain in the ass, especially when talking about file permissions and platform independent access.

So thinking about gitolite deployment, I wondered why the hell we should use ssh-keys for authentication if all our users (Windows or Linux) are already authenticated on their system using Kerberos anyway.

It turned out, that gitolite, openssh and an AD-integrated Linux machine (acting as gitolite server) will be 99% sufficient to get this stuff running. All I found on the web about this particular issue was this short discussion which does not offer a proper solution. The 1% missing to the solution I’m presenting here is a 3-line patch for openssh, but read on…

So here comes the gitolite+kerberos mini HOWTO:

  • Start from a kerberized Linux server system with a working kerberized ssh setup
  • Install a version of openssh with this patch (I also have packages build for debian squeeze, just drop me a line if you like to get them)
  • Download my gitolite_wrapper_script and copy it into /usr/local/bin/
  • Add the following two lines to your sshd_config:
  • Match User gitolite
    ForceCommand /usr/local/bin/gitolite_wrapper_script

  • Add a local (non AD) gitolite user without password to this system
    (I used /usr/share/gitolite as its home)
  • su to this user and clone the gitoline repository into this users home
    (git clone git://github.com/sitaramc/gitolite.git gitolite.clone)
  • Follow the Installation instructions in README.txt, but use -a <adminid> instead of -pk for setup
  • Make shure you have gitolite and gitolite-shell available in your PATH, I did this by adding symlinks to /usr/local/bin
  • create a file .k5login in the homedirectory of the gitolite user and add the kerberos realm of your admin <adminid>@<REALM>
  • now run git clone gitolite@/<yourserver>:gitolite-admin.git from a client (already using kerberos authentication)
  • create a file k5login inside this clone and again add the kerberos realm of your admin <adminid>@<REALM>
  • commit and push this file
  • on the server replace the .k5login file with a symbolic link to .gitolite/k5login
  • You are now running a pure kerberos5 based gitolite server
  • The only thing which is different from an ordinary gitolite now is that we don’t manage ssh-keys but kerberos realms using the file k5login

Nice stuff you might think, but why the hell will we need to patch the secure shell daemon? The answer is simple: Once your login has succeeded the Unix shell running with gitolite userid does not know about the kerberos prinzipal used for authentication and there is no way (at least none, that I know of) to figure it out.
The username part of this prinzipal is however needed for gitolite. The only thing my patch does now is adding an environment variable called GSS_AUTH_KRB5_PRINC which can be evaluated by gitolite.
BTW, using the perl-script provided in the usenet discussion linked above was not an option because especially windows machines will not forward tickets by default and forwarding ticket is unnecessary for this purpose anyway.

As far as clients are concerned this has currently been tested using Linux git with openssh as well as Windows git and eclipse EGIT in conjunction with plink.exe provided by putty.

Why I would never swap my FOOS OS to a proprietary one

Geschrieben am 5. Juni 2011 in FOSS, Linux von giggls || Kommentare deaktiviert

At my workplace we started the policy that devices leaving the campus must be encrypted. For this reason I created a USB pen-drive based boot scenario for our Debian based Laptops which allows for a completely encrypted drive using LUKS a couple of years ago.

However, the encryption of the pen-drives themselves has been a problem ever since we started to follow this policy. Devices like Kingston DataTraveler Vault are Windows only and alternative solutions are rare.

Fortunately I recently discovered the Corsair Padlock devices which might be a solution for people who need platform independent encrypted USB storage.

When testing one of those devices I thought about securing my laptop kernels with such a thing, because this would remove or at least reduce the attack option (unencrypted kernel and initrd) left in my encrypted laptop drive setup.

Well this is where the problem started and where one oft the best software support in the world (this of the linux kernel) was again
able to solve it by just sending a few mails around the globe.

So here comes the whole story:

Quite a lot of BIOS are unable to boot from a so called Superfloppy device (no partition at all). For this reason I just added one to my Padlock2 device. Unfortunately this device could not be detected by Linux but worked fine in M$-Windows.

Sending an error report and some debug output to the usb-storage mailing-list revealed some strange bug (or just a tricky windows only workaround to enable hardware encryption?) in the Padlock device not showing the correct partition Information on the first read by the OS. Windows reads a bunch of stuff before looking at the
partition data so the problem does not arise there.

And here comes the real beauty of FOOS. I got a patch which worked around the problem just a couple of hours later from somebody a thousand kilometeres away (credits go to Alan Stern) and fortunately this patch will go into Linux 3.0.1.

Problem solved :)

This is not the first time something like this happened. I’m a 100% certain that something like this would never be possible in a closed source world which is all about workarounds in application Software because it is just impossible to send an email to the maintainer of a particular piece of OS code.

A WMS-server in about 100 lines of code…

Geschrieben am 6. November 2010 in FOSSGIS, Linux, Openstreetmap, Python von giggls || Kommentare deaktiviert

or how to use geoimage.at and others in josm

A few weeks ago a few austrian mappers contacted me because we are now allowed to us the WMS server at http://geoimage.at/ for mapping.

Unfortunately the data is currently only available in an austrian koordinate system (EPSG:31287). With EPSG:4326 beeing unavailable this is in fact a violation of the WMS spec :(

This could however be easily fixed using UMN-Mapserver as WMS-proxy, but unfortunately we are not allowed to do this at wms.openstreetmap.debecause we are not permitted to set up a cascading WMS based on their rules.

Anyway, with my wms.openstreetmap.de setup already using the python wsgi-interface (apache mod_wsgi) I thought that a standalone UMN-Mapserver based WMS-server should be very easy to hack, given the fact, that all the difficult stuff is already available in python and mapscript. Well, my presumption proved to be true :)

So here is my standalone WMS-server written in roughly 100 lines of python code.

As far as geoimage.at is concerned, there are already other solutions at the OSM-Wiki page, but this was fun to hack and might be useful for other purposes as well. An advantage of my solution is that it is possible to use the Austrian GIS-grid file for reprojection to achieve the highest possible accuracy. For this purpose the proj4-definition for EPSG:31287 (defined in /usr/share/proj/epsg on Linux) must look like this:

<31287> +proj=lcc +lat_1=49 +lat_2=46 +lat_0=47.5 +lon_0=13.33333333333333 +x_0=400000 +y_0=400000 +ellps=bessel +units=m +nadgrids=/path/to/GIS_GRID_austria.gsb +no_defs

I would be interested in feedback on how to get this to work on Windows as well. Talking about Linux this has only been a matter of typing apt-get install python-mapscript and adjusting the proj4 definition file to use the GIS-grid.

When running kernel.org kernels you are on your own :(

Geschrieben am 19. September 2010 in FOSS, Linux von giggls || Kommentare deaktiviert

A few days ago a local root exploit for the Linux Kernel has been posted and is available from all over the Internet now.

As usual in recent years the kernel.org people just fixed it silently without mentioning in the commit log that this is a mayor security fix.

People like me which are still running kernel.org kernels and which are actually the people reporting bugs in recent kernels are actually drawing the shortest straw here now :(

IMO it is a shame that there is still no 2.6.35.5 Kernel available which fixes this flaw.

So what I will need to do now is trying to repair this by merging the fix from git.kernel.org to 2.6.35.4…

Best of breed CAN device driver hitting mainline Linux Kernel 2.6.31

Geschrieben am 30. Juni 2009 in Controller Area Network, FOSS, Linux von giggls || Kommentare deaktiviert

Starting with the upcoming 2.6.31 Linux Kernel (already available as 2.6.31-rc1) a functional version of the socketcan device driver found its way into mainline.

While socketcan itself has been in mainline for quite a while, there where no drivers for actual real-world hardware. This has now changed and I will therefore be able to run a Kernel which is one step closer to mainline.

Unfortunately however some kind of unioning filesystem is still missing.

Anyway, back to Controller–area network!

Since I wrote my own CAN device driver as a character device a couple of years ago I started thinking about a CAN-bus driver beeing implemented as a Network device. I wrote my own driver in the first place, because I needed concurrent access which is (needless to say) also possible with socketcan.

Fortunately two guys at Volkswagen Research finally started doing exactly this. Thank you to the socketcan people for providing such a nice piece of software!

So here is a mini HOWTO for getting it up and running.

What you need:

  • A GNU/Linux machine
  • A decent Kernel (2.6.31-rc1 or higher)
  • git and subversion
  • A supported CAN-bus card. I’m using one sold by EMS Wünsche, but a lot of other SJA1000 based cards may also work.
  • The simple userland utilities from svn://svn.berlios.de/socketcan/trunk/can-utils
  • A decent Version of iproute2 (not yet available in current GNU/Linux distributions at the time of writing)

How to get it up and running:

First of all compile and boot a Kernel where all CAN-related support is enabled either as a module or build into the Kernel.
Afterwords check out the userland utilities from subversion (svn checkout svn://svn.berlios.de/socketcan/trunk/can-utils) and compile them by just typing make.
Same goes for iproute2 (git clone git://git.kernel.org/pub/scm/linux/kernel/git/shemminger/iproute2.git); just type make as well. Copy all the resulting binaries to an appropriate place (e.g. /usr/local/bin). Do not use make install at least in iproute2 sourcetree, because this will replace your systems ip command.

Now all you need to do ist to enable you can0 interface by means of the following command:

ip link set can0 up txqueuelen 1000 type can bitrate 1000000

Use another value than 1000000 for different bitrates.

The txqueuelen might not be needed in your case if you don’t have peak traffic like I do.

OK, that`s it! You can now connect your bus to some device and use candump for watching your raw CAN pakets on the bus.

For integration into Debian GNU/Linux you may use something like the following in /etc/network/interfaces for automatically starting up your can0 device on system boot:


--cut--
auto can0
iface can0 inet manual
# bitrate 1MBps
# increase TX queue length because of the pulsed nature of
# the traffic we generate
up /usr/local/bin/ip link set can0 up txqueuelen 1000 type can bitrate 1000000
down /sbin/ifconfig $IFACE down
--cut--

See Documentation/networking/can.txt inside your Kernel tree for further documentation.

Effektiver Arbeiten auf der GNU bash

Geschrieben am 19. Juni 2009 in Linux, Tipp/hint von giggls || 3 Kommentare

Wer mich kennt weiß, dass ich immer noch einen für manche vergleichsweise archaischen Desktop benutze. Streng nach dem alten Motto: Das X-Window System ist dafür da, dass man mehrere Konsolen nebeneinander benutzen kann.

Langer Rede kurzer Sinn, die aktuelle Version 4.0.x der Linux Standardshell hat nun zwei nette neue features, die ich natürlich gleich in meine .bashrc eingebaut habe.

Unterm Strich erspart man sich das häufige tippen der Kommandos cd und ssh.

Findet der Rechner einen unbekannten Befehl wird geschaut ob es ein Verzeichnis mit passendem Namen gibt, wenn ja wird in dieses Verzeichnis gewechselt. Wenn Nein wird nachgeschaut ob es einen passenden Rechnernamen gibt und wenn das der Fall ist erfolgt ein Remote Login über ssh auf diesem Rechner.

Das ganze sieht so aus:

# Auto-cd feature
shopt -s autocd

function command_not_found_handle() {
    host=$(echo $1 |sed -e 's/^.*@//g')
    # wenn hostname in .ssh/config, dann ssh dorthin
    if grep -q "Host $host" $HOME/.ssh/config; then
      ssh $*
    else
      if ! getent hosts $host >/dev/null; then
        echo "bash: $*: Kommando nicht gefunden."
      else
        ssh $*
      fi
    fi
}