BTRFS, EXTLINUX and i3.

0
Posted by mike on September 2, 2010 – 3:24 pm
Filed under Arch, Linux

Lately I have been experimenting with some of the netbook specific Linux distros on my Dell Mini 9. I looked at Moblin, ChromeOS, Ubuntu Netbook Remix and Jolicloud. They were all fairly impressive though I did miss the simplicity and versatility of Arch Linux and decided to reinstall Arch Linux. I installed Arch Linux to a single ext4 partition with a 512M swap partition. I had been having some trouble with the Broadcom wireless adapter so I replaced it with a spare Intel wireless adapter I had on hand. No more out of tree kernel modules as the Broadcom card required the wl driver from AUR which made kernel updates interesting if I didn’t have an ethernet port handy.

I recently watched a webcast by Chris Mason the lead developer of btrfs and one of the things covered was the ability to convert extX to btrfs via btrfs-convert. I decided to give this a try and on a 16GB SSD, with 2GB used, it only took a minute or so to convert. One nifty feature is that the btrfs is created in the empty part of the extX filesystem with the extX filesystem being saved as a sparse file inside a dedicated subvolume:

[mike@mini|~] $ sudo btrfs subvolume list /
ID 256 top level 5 path ext2_saved
[mike@mini|~] $ ls -lh /ext2_saved/
total 1.9G
-r-------- 1 root root 14G Jan 1 1970 image

This makes it possible to revert to the extX filesystem at any point if btrfs does not live up to expectations. The process is explained much more clearly on the btrfs wiki.

As usual of late I installed extlinux as a boot loader. Due to the COW nature of btrfs extlinux installs in a slightly different manner to extX. Instead of installing as a file inside the filesystem it installs in the slack space inside the partition directly following the boot sector:

/* btrfs has to install the ldlinux.sys in the first 64K blank area, which
is not managered by btrfs tree, so actually this is not installed as files.
since the cow feature of btrfs will move the ldlinux.sys every where */

I normally use Xmonad as my WM however I wanted something a little slimmer dependency wise for my netbook. I looked at scrotwm for a while and eventually ended up with i3. So far I am impressed with it. It works pretty much out of the box though I am intending to customize it a little more. i3 is written in C, has great documentation and features a simple IPC interface for control/status integration with external apps. It also has good multi-monitor support though I haven’t tried it yet. I may give it a trial on my main workstation.

Easy Debian/Ubuntu chroots.

0
Posted by mike on August 10, 2010 – 8:56 pm
Filed under Arch, Debian, Linux

Until recently I had been using Adobe’s 64 bit Flash Player plugin. Unfortunately the 64 bit Flash Player has been removed from Adobe’s site with no clear indication of exactly when it will return. For a while I continued to use the last released version however this version contains major security vulnerabilities and I eventually removed it. I don’t use Flash that much but I did miss it every now and then so decided to do something about it. I’m not a fan of multi-lib setups or nspluginwrapper so went with a 32 bit chroot. At first I tried an Arch Linux 32 bit chroot however I experienced issues with some GUI applications not working correctly and segfaulting. While I look into this I installed a Ubuntu Lucid 32 bit chroot using debootstrap.

Debootstrap is a handy way to install a Debian/Ubuntu base system into a directory. It is available in the AUR. Installing a base Lucid 32 bit system is as easy as:

[mike@mercury|~] $ sudo mkdir /opt/lucid32
[mike@mercury|~] $ sudo debootstrap --arch=i386 lucid /opt/lucid32/

To manage entering the chroot I installed schroot. Schroot allows a normal user to enter a chroot. It also handles copying files from the host into the chroot as well as mounting any required filesystems inside the chroot. This comes in handy for keeping /etc/{resolv.conf,hosts,passwd,shadow} and so on in sync as well as mounting /{proc,dev,tmp} inside the chroot. I also mount my /usr/share/fonts directory this way so Chromium has access to my fonts. I still need to do something similar with ny mouse cursor icons. Once configured schroot allows me to launch my 32 bit web browser via:

[mike@mercury|~] $ schroot -c lucid32 -- chromium http://www.archlinux.org

I now have flash working again.

Native resolution console with the binary nVidia blob.

0
Posted by mike on July 26, 2010 – 9:42 pm
Filed under Arch, Hardware, Linux

My first serious graphics card was a Canopus Pure3D which was based on 3dfx’s Voodoo chipset. It was a strange beast as it installed in addition to the 2D graphics card and interfaced via a pass-through VGA cable. I purchased it solely to play id’s GLQuake. When it came time to upgrade I went for a Diamond Viper V770 which was based on the nVidia RIVA TNT2 chipset. Since that day I have always used nVidia graphic cards, except for laptops where nVidia was not an option, and am currently using a Geforce 8600GT.

While I have always been happy with nVidia there are some downsides to using nVidia cards in Linux. If you want decent 3D performance you need to use the propriety nVidia binary blob which means you miss out on KMS and have to deal with an out of tree kernel module and a tainted kernel. For the moment I am willing to do this though when it comes time to update again it may be time to look at ATI or perhaps the nouveau project will have attained decent 3D performance by then.

Having seen KMS in action on my Intel based notebook I do miss having a native resolution console on my main workstation and, until recently, thought it was impossible without switching to the nouveau driver. The solution was to install uvesafb. This is a more recent frame buffer than the standard vesafb frame buffer. I followed this Arch Linux wiki article and was up and running in no time with a native 1680×1050 console.

While rebooting to test out my new console settings I had a play with HDT (Hardware Detection Tool) which I had just recently added to my boot menu. While using HDT to look at the VESA information I noticed that there was a 1680×1050 graphics mode listed though I didn’t make a note of the mode number. I can’t help but wonder if I would have been able to get a native resolution console by just using a vga=xxx kernel switch? Uvesa is working fine so I will probably just stick with it.

extlinux

0
Posted by mike on July 18, 2010 – 1:59 pm
Filed under Arch, Linux

I had been thinking about switching from grub to grub2 on my main workstation for a while now. Grub did everything I required however I felt like trying something new. Recently one of the Arch Linux devs mentioned extlinux on one of the Arch Linux mailing lists as an alternative to the more well known grub and lilo.

I was vaguely aware of extlinux as it is part of the syslinux bundle developed by H. Peter Anvin and I have used pxelinux in the past. After reading up on extlinux I decided to give it a go. Install was a breeze with the syslinux wiki giving instructions that covered my RAID setup that would allow the system to boot regardless of which disks were present in the case of a disk failure. The config file uses a slightly different format to grub. The following is what I am currently using:

[mike@mercury|~] $ cat /boot/extlinux/extlinux.conf
ui vesamenu.c32
timeout 100

menu title Mercury Boot Menu

label 1
    menu label Arch Linux
    kernel /vmlinuz26
    append initrd=/kernel26.img root=/dev/mapper/vg0-root ro vga=0x307

label 2
    menu label Arch Linux Fallback
    kernel /vmlinuz26
    append initrd=/kernel26-fallback.img root=/dev/mapper/vg0-root ro vga=0x307

This gives me a VGA resolution boot menu with support for customizable colors and background images though I have not made use of these features yet.

While looking into extlinux I came across a Google Tech Talk by Marty Connor, H. Peter Anvin and Michael Brown. It mainly covers networking booting though it is very entertaining being mostly freeform.

Exams over for another semester.

0
Posted by mike on June 23, 2010 – 10:20 pm
Filed under Arch, Linux

I haven’t had a chance to post much in the last month as study has kept me fairly busy. My last exam was last Friday and I am now waiting for my results due out early next month. In the mean time I am enjoying having some time to relax.

The Arch Linux devs have been busy of late with a couple of significant updates coming down the pipe, most notably kernel 2.6.34 and Xorg 1.8. As usual the upgrade was painless thanks to the developers and [testing] users. I have been looking forward to 2.6.34 due to it’s new btrfs capabilities. I also took the opportunity with the Xorg 1.8 update to disable hal and reduce my Xorg.conf down to the minimum required to work with my nVidia 8600GT using the nVidia binary blob with twinview enabled. I managed to get it down to a single “Screen” section of 15 lines and may be able to reduce it a bit more.

In the little spare time I have had I have been playing with pyglet, a Python based OpenGL framework. I had not heard of pyglet until recently and am impressed by it. The documentation, made up of the programming guide and api reference are top quality. While pyglet is aimed at game development I am using it to make a prototype OpenGL based image viewer. This is coming along nicely though it is a little sluggish when loading an image, especially when I am used to feh which displays images very quickly. It appears that if I want to take this beyond the prototype stages I will need to switch to C/C++ using GLX to improve performance.

Pyglet makes displaying text very easy and it is one area of OpenGL that I don’t have to much experience in. To this end I have also been playing with freetype and was surprised at how simple it was to load a font and render it to a bitmap. At the moment I am rendering single glyphs but am working up to rendering strings with and without kerning. The freetype tutorials have been a big help.

linux.conf.au presentation videos

0
Posted by mike on April 10, 2010 – 4:51 pm
Filed under Linux

I recently came across some videos for 2010′s linux.conf.au presentations. I haven’t watched all of them however a couple of them look interesting. One that I have watched is Going mad with MDADM. I learnt a few things from this presentation about software raid, most of which are covered in the md.txt file distributed with the kernel tree. The most useful piece of info was how to request a raid array check:
[root@mercury ~]# echo check > /sys/block/md1/md/sync_action

You can then check the status of the check via /proc/mdstat as normal:

[mike@mercury|~] $ cat /proc/mdstat
Personalities : [raid1] [raid10]
md1 : active raid10 sda3[0] sdd3[3] sdc3[2] sdb3[1]
      484231040 blocks 64K chunks 2 near-copies [4/4] [UUUU]
      [=>...................]  check =  6.3% (30890816/484231040) finish=73.7min...

md0 : active raid1 sda1[0] sdd1[3] sdc1[2] sdb1[1]
      128384 blocks [4/4] [UUUU]

unused devices: <none>

Checking a modestly sized array can take some time. In the case of a large array and a RAID level that uses parity such as 5 or 6 it is reported that it can take a very long time to do a check. Resyncs are even worse as they involve writing as well as reading.

Check the log file for the status of the completed check:
[mike@mercury|~/data/scans] $ grep 'kernel: md' /var/log/messages.log | tail -n 5
Apr 10 16:27:58 mercury kernel: md: data-check of RAID array md1
Apr 10 16:27:58 mercury kernel: md: minimum _guaranteed_ speed: 1000 KB/sec/disk.
Apr 10 16:27:58 mercury kernel: md: using maximum available idle IO bandwidth (but not more than 200000 KB/sec) for data-check.
Apr 10 16:27:58 mercury kernel: md: using 128k window, over a total of 484231040 blocks.
Apr 10 18:07:10 mercury kernel: md: md1: data-check done.

BTRFS/RSYNC based backups.

3
Posted by mike on April 5, 2010 – 1:15 pm
Filed under Uncategorized

As mentioned previously I have recently converted my file server to Linux from Open Solaris and ZFS. I was using ZFS’s snapshot capability extensively. My main storage zpool used snapshots to provide read only previous versions of files and my backup zpool used snapshots to provide space efficient read only backups. I wanted something similar in Linux using either rsync with hard links or rsync with btrfs. I decided to try rsync with btrfs.

Btrfs is still far from complete though progress is being made quickly. The btrfs wiki does however state:

Btrfs is under heavy development, but every effort is being made to keep the filesystem stable and fast. As of 2.6.31, we only plan to make forward compatible disk format changes, and many users have been experimenting with Btrfs on their systems with good results. Please email the Btrfs mailing list if you have any problems or questions while using Btrfs.

I setup a btrfs file system on a 1TB external USB drive.

With error checking removed my backup script is simply:

#!/bin/bash

EXCLUDE_FILE=/home/mike/backup-script/rsync-exclude

rsync -axv --stats --exclude-from=${EXCLUDE_FILE} \
       --delete-after /home/mike/ /mnt/BTRFS/mike

btrfsctl -s /mnt/BTRFS/snapshots/mike-$( date +%Y%m%d-%H%M ) /mnt/BTRFS/mike

This results in the following file system layout on the external drive:


[mike@mercury|~] $ tree /mnt/BTRFS -L 1
/mnt/BTRFS
|-- mike
`-- snapshots

2 directories, 0 files
[mike@mercury|~] $ tree /mnt/BTRFS/snapshots -L 1
/mnt/BTRFS/snapshots
|-- mike-20100314-1827
|-- mike-20100314-2337
|-- mike-20100315-1727
|-- mike-20100316-0405
|-- mike-20100317-0621
|-- mike-20100318-0405
|-- mike-20100319-0415
`-- mike-20100320-0411

The top level mike directory contains a mirror of my home directory with the snapshots directory containing daily snapshots performed after that days rsync completes. While there are several days worth of backups on the drive, total file system usage is roughly the same as the home directory.

[mike@mercury|~] $ pydf /home /mnt/BTRFS
Filesystem           Size Used Avail Use%                 Mounted on
/dev/mapper/vg0-home 350G 177G  173G 50.7 [#######......] /home
/dev/sde1            932G 178G  754G 19.1 [##...........] /mnt/BTRFS

While btrfs will eventually support read only snapshots through per snapshot block quotas it has not been implemented as of yet. This was one of ZFS’s features that I really liked as it was comforting to know that a daily snapshot could not be modified intentionally or unintentionally. Due to btrfs lacking read only snapshots at this point in time it could be argued that using rsync with hardlinks on a proven filesystem such as xfs would be a better way to go.

Goodbye Open Solaris and ZFS.

0
Posted by mike on March 11, 2010 – 12:42 am
Filed under Arch, Hardware, Linux, OpenSolaris, Uncategorized

As the title of the post states I have recently decommissioned my Open Solaris ZFS based file server. I have now combined my file server with my workstation. There are several reasons why I have done this:

  • I’m trying to be a little bit greener in regards to my energy usage. I really only connected to my file server from my workstation and lately I had been leaving my workstation on for one reason or another often racking up uptime measured in days. Having two full sized tower machines running seemed wasteful. This was the main reason I decided to combine my file server and workstation.
  • OpenSolaris is a great product however I never really felt comfortable in it. Open Solaris and ZFS make for a rock solid combination. It has been sitting in the corner of my spare bedroom closet and I haven’t needed to really touch it. ZFS makes filesystem management and backups ridiculously easy. My backup system consisted of a 12 line bash script that backed up my data to an external usb based zpool which was snapshotted after every run giving me 6 months of daily backups in very little disk space. Due to the stability of this system along with it’s low maintenance I slowly forgot all my Open Solaris knowledge, what little there was, that I had gleaned from setting the machine up.

As I run Arch Linux on my workstation and did not want to resort to using ZFS through fuse I had to say good bye to ZFS. I will definitely miss ZFS. Managing storage is really a no brainer with ZFS compared to setting up a comparable Linux based layered software raid > LVM > file systems type setup. I salvaged the disks from my file server, after backing up the data, and along with the disk from my workstation and a spare I had on hand, this gave me 4 x 250G drives. I partitioned each drive with a 128M boot partition, 2G swap partition and a 248G main partition.

[mike@mercury|~] $ sudo parted /dev/sda print
Model: ATA ST3250820AS (scsi)
Disk /dev/sda: 250GB
Sector size (logical/physical): 512B/512B
Partition Table: msdos

Number  Start   End     Size    Type     File system     Flags
 1      32.3kB  132MB   132MB   primary  ext2            boot, raid
 2      132MB   2130MB  1999MB  primary  linux-swap(v1)
 3      2130MB  250GB   248GB   primary                  raid

The four boot partitions were combined into a RAID1 array while the 4 large partitions were combined into a RAID10 array.

[mike@mercury|~] $ cat /proc/mdstat
Personalities : [raid1] [raid10]
md1 : active raid10 sda3[0] sdd3[3] sdc3[2] sdb3[1]
      484231040 blocks 64K chunks 2 near-copies [4/4] [UUUU]

md0 : active raid1 sda1[0] sdd1[3] sdc1[2] sdb1[1]
      128384 blocks [4/4] [UUUU]

unused devices:

I was tossing up between RAID5 or RAID10 and finally came down on the side of RAID10. With RAID10 you sacrifice a drive, storage wise, though you do get the benefit of no parity calculations and no RAID5 write hole. Having said that I’m not sure how much of an issue the parity calculations would have been after seeing ZFS perform fine on some very modest hardware. Linux can perform RAID10 as a single layer which gives you a few more options when setting things up and also allows you to do things like setup a RAID10 array on 3 drives.

On top of the RAID10 array I setup LVM with a separate home logical volume. The final file system layout looks like:

[mike@mercury|~] $ pydf / /boot /home
Filesystem           Size Used Avail Use%                            Mounted on
/dev/mapper/vg0-root  63G  24G   36G 38.3 [#########...............] /
/dev/md0             121M  16M   99M 13.2 [###.....................] /boot
/dev/mapper/vg0-home 350G 176G  173G 50.4 [############............] /home

I chose the following file systems:

[mike@mercury|~] $ mount | grep 'root\|home\|boot' | cut -d ' ' -f 1,3,5  | column -t
/dev/mapper/vg0-root  /      ext4
/dev/md0              /boot  ext2
/dev/mapper/vg0-home  /home  xfs

There was no real reasoning behind the selection of ext4 and xfs for root and home respectively. I have used both before and found them to be solid. I left around 50G spare in my volume group which I will probably end up adding to my home file system. Xfs will make this easy as it allows online resizing.

I didn’t want to have to reinstall Arch Linux so I used my usual technique of transfering a Linux install using the Arch Linux live CD:

  1. Backing up my root file system to external storage using cp -axv /mnt/src/* /mnt/dest
  2. Adding new drives, partition drives, setup raid arays, setup lvm and create filesystems.
  3. Restore my root file system using another cp -axv /mnt/src/* /mnt/dst

I wanted things to go smoothly so before I backed up my root files system I made sure I had added mdadm to my mkinitcpio hooks array, I already had lvm in there, and recreated my initrd. I also changed my /etc/fstab and /boot/grub/menu.lst to reflect the new filesystem layout. The last step was to chroot into my Arch Linux install and reinstall GRUB and generate a /etc/mdadm.conf. I hadn’t realised I needed a /etc/mdadm.conf until trying to boot. I had been roughly following the Arch Linux wiki article Installing with Software RAID or LVM and it set me straight. It also included a neat trick with sfdisk to make partitioning multiple disks easy.

All in all it went well. I managed to get the whole thing done during the Oscar’s broadcast. I did catch bits and pieces though. Steve Martin and Alec Baldwin did a fantastic job hosting and it was great to see an underdog get up in the form of Hurt Locker.

Next I need to work out a backup system. I would really like something similar to the rsync/ZFS snapshot system I used in Open Solaris. I will probably end up going with something similar using rsync and hard links or I might try btrfs and rsync. Btrfs seems to be coming along and it may be stable enough for use as a backup file system.

Weird lvm2 issue.

0
Posted by mike on February 26, 2010 – 6:47 pm
Filed under Uncategorized

Booted my machine today and it failed to find my LVM filesystems. Took a while to track down the issue. I didn’t have lvm2 in my HOOKS array.

Pacman’s logs show that the lvm2 hook was not processed when I upgraded to kernel26 (2.6.32.9-1) on 2010-02-25, however it was when kernel26 (2.6.32.8-1) was installed on the 2010-02-13.

I’m not sure what happened in between. I may have deleted the wrong file when cleaning up pacnew files perhaps.

Postfix, reject_rbl_client and Google DNS

0
Posted by mike on February 12, 2010 – 10:01 pm
Filed under Uncategorized

Just spent 30 minutes trying to work out why Postfix was not rejecting mail using the zen.spamhaus.org DNSBL. I finally found the answer in my postfix-users mailing list archives.

I was using Google Public DNS. Spamhaus.org simply returns NXDOMAIN for all queries from Google’s Public DNS servers. I quickly setup pdns-recursor and the test message was correctly blocked.

I wasn’t familiar with pdns-recursor, it was mentioned in the same postfix-users mailing list thread, however it was easy to install and worked out of the box. I was using dnsmasq for resolving local machine names so I will need to fix that at some point. It looks like pdns can do that as well through pdns-server or I might just bite the bullet and install BIND. I did like the ease of configuration of dnsmasq compared to BIND though. Hopefully pdns-server is somewhere in between the two.