Category Archives: Hardware

BTRFS Fileserver

2
Posted by mike on June 12, 2012 – 8:37 pm
Filed under btrfs, Hardware, Linux

Introduction

For the last 9 months or so I have been using a HP N36L Microserver as a home Xen server running a mail relay for my local network. It wasn’t doing much else and I figured it was time to rectify that. I wanted to turn it into a backup file server as well as be able to use it as a test bench for things like lxc. The server comes standard with one sata drive with room for three more and a gig of ram. I upgraded the RAM to the maximum of 8GB and filled the drive bays with four 1TB drives I scrounged up. Two of the drives were canabalized from a couple of cheap USB 2 external drives. The server also has two PCIe half height slots so I filled them with an extra gigabit NIC and a USB 3 card.

I decided I wanted to use btrfs as the main filesystem. For anyone else thinking of using btrfs at this stage keep in mind that btrfs is still under heavy development. The btrfs wiki states:

Btrfs is under heavy development, but every effort is being made to keep the filesystem stable and fast. Because of the speed of development, you should run the latest kernel you can (either the latest release kernel from kernel.org, or the latest -rc kernel.

For this reason I will be using the latest rc kernel and the latest btrfs-progs. For something different I went with Funtoo Linux as the operating system. I had installed and used Gentoo several years ago and had always wanted to check out Funtoo. I used sysrescuecd as my install medium booted from an external USB flash drive.

Read More »

GPT RAID1 setup.

0
Posted by mike on February 9, 2011 – 7:33 pm
Filed under Hardware, Linux

My home partition hit 90% recently and I decided it was time to upgrade my storage. I had been running 4x250GB hard disks in a RAID1/RAID10 setup and switched to a 2x1TB RAID1 setup. The drives were Western Digital Advanced Format drives with 4kB physical sectors and 512B logical sectors. It is important when partitioning these drives to ensure that partitions start on a logical sector number that is divisible by 8 otherwise performance will suffer as file system clusters will not be aligned to the underlying 4kB physical sector. If the partition is not aligned then writes turn into a read-modify-write. I also decided to use a GPT instead of an MBR partition table. I used gdisk to create boot, swap, root and home partitions:

$ sudo sgdisk -p /dev/sda
Disk /dev/sda: 1953525168 sectors, 931.5 GiB
Logical sector size: 512 bytes
Disk identifier (GUID): 0658FE0C-AE06-4FD9-8CCC-7DDB178BB0CA
Partition table holds up to 128 entries
First usable sector is 34, last usable sector is 1953525134
Partitions will be aligned on 2048-sector boundaries
Total free space is 3437 sectors (1.7 MiB)

Number  Start (sector)    End (sector)  Size       Code  Name
   1            2048          264191   128.0 MiB   FD00  Linux RAID
   2          264192        12847103   6.0 GiB     8200  Linux swap
   3        12847104       147064831   64.0 GiB    FD00  Linux RAID
   4       147064832      1953523711   861.4 GiB   FD00  Linux RAID

I actually aligned the start of all my partitions to 1MiB (2048 sectors) which seems to be somewhat of an industry standard and was suggested by gdisk as well.

I had done some research into how to exactly boot Linux on a GPT disk and was happy to find that my boot loader, extlinux, works out of the box. Installing extlinux was as simple as:

# extlinux --raid --install /boot/extlinux
# sgdisk /dev/sda --attributes=1:set:2
# sgdisk /dev/sdb --attributes=1:set:2
# cat /usr/lib/syslinux/gptmbr.bin > /dev/sda
# cat /usr/lib/syslinux/gptmbr.bin > /dev/sdb

These commands install ldlinux.sys, set the bootable attribute for partition 1 of both disks and install the gptmbr.bin boot code into the first sector of both disks.

Once my file systems were copied onto my new disk setup I recreated my initrd images with a new /etc/mdadm.conf and edited /etc/fstab and /boot/extlinux/extlinux.conf. The Arch Linux Wiki article Installing with Software RAID or LVM proved handy for assembling the arrays.

I now have plenty of disk space again:

$ pydf / /boot /home
Filesystem Size Used Avail Use%                               Mounted on
/dev/md1    63G  31G   29G 48.6 [#############..............] /         
/dev/md0   124M  17M  101M 13.6 [####.......................] /boot     
/dev/md2   861G 373G  488G 43.3 [############...............] /home

A quick check with Bonnie++ showed that disk performance was as expected which confirmed that I hadn’t made any mistakes with partition alignment.

USB 3.0

1
Posted by mike on September 5, 2010 – 8:43 pm
Filed under Hardware, Linux

I’m currently in the process of encrypting my USB 1TB backup drives with LUKS. Part of this process involves filling the disk with random data. For USB2 this takes about 10 hours for a 1TB drive. Admittedly this is a one off process though I often move large chunks of data around for one reason or another. I decided to have a look at USB3. I picked up a Western Digital My Book 3.0 along with a Gigabit PCI-e 1x USB 3 card. I had a little difficulty getting Linux to recognize the drive and found I had to reload the xhci kernel module:

sudo rmmod xhci_hcd
sudo modprobe xhci_hcd

Not sure if this issue was a one off or not. I may need to look at this later. Perhaps this module needs to be loaded before/after some of the other USB related modules? Once it was detected I reformatted it with XFS and ran the bonnie++ benchmark on it:

Version 1.03e       ------Sequential Output------ --Sequential Input- --Random-
                    -Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- --Seeks--
Machine        Size K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP  /sec %CP
mercury         12G 88824  99 100197  31 43076  18 55770  94 137263  23 273.3   1
                    ------Sequential Create------ --------Random Create--------
                    -Create-- --Read--- -Delete-- -Create-- --Read--- -Delete--
              files  /sec %CP  /sec %CP  /sec %CP  /sec %CP  /sec %CP  /sec %CP
                 16  7882  67 +++++ +++  5057  45  7693  69 +++++ +++  5516  50

Performance was impressive. Looking at block transfers showed writes just below 100MiB/s and reads above 130MiB/s. I didn’t test a USB2 drive as I know from experience it would be no where near that speed. I did test my internal storage which is XFS on LVM on RAID10 (4 x 250GB SATA):

Version 1.03e       ------Sequential Output------ --Sequential Input- --Random-
                    -Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- --Seeks--
Machine        Size K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP  /sec %CP
mercury         12G 89567  97 115355  38 58405  32 65973  88 123363  33 536.0   3
                    ------Sequential Create------ --------Random Create--------
                    -Create-- --Read--- -Delete-- -Create-- --Read--- -Delete--
              files  /sec %CP  /sec %CP  /sec %CP  /sec %CP  /sec %CP  /sec %CP
                 16   188   2 +++++ +++   185   2   189   2 +++++ +++   173   2

My internal storage performed slightly better than the USB3 drive though it couldn’t match the 130MiB/s block read speed. I’m not sure why there was such a difference with the metadata tests. The USB3 device was an order of magnitude faster. It may have been connected with the fact that the USB3 test was a new file system and the internal XFS file system has been in use for quite a while. Not sure though.

Bottom line is that next time I need to transfer my main workstation’s file systems to external storage for hardware maintenance it will be a lot faster!

Native resolution console with the binary nVidia blob.

0
Posted by mike on July 26, 2010 – 9:42 pm
Filed under Arch, Hardware, Linux

My first serious graphics card was a Canopus Pure3D which was based on 3dfx’s Voodoo chipset. It was a strange beast as it installed in addition to the 2D graphics card and interfaced via a pass-through VGA cable. I purchased it solely to play id’s GLQuake. When it came time to upgrade I went for a Diamond Viper V770 which was based on the nVidia RIVA TNT2 chipset. Since that day I have always used nVidia graphic cards, except for laptops where nVidia was not an option, and am currently using a Geforce 8600GT.

While I have always been happy with nVidia there are some downsides to using nVidia cards in Linux. If you want decent 3D performance you need to use the propriety nVidia binary blob which means you miss out on KMS and have to deal with an out of tree kernel module and a tainted kernel. For the moment I am willing to do this though when it comes time to update again it may be time to look at ATI or perhaps the nouveau project will have attained decent 3D performance by then.

Having seen KMS in action on my Intel based notebook I do miss having a native resolution console on my main workstation and, until recently, thought it was impossible without switching to the nouveau driver. The solution was to install uvesafb. This is a more recent frame buffer than the standard vesafb frame buffer. I followed this Arch Linux wiki article and was up and running in no time with a native 1680×1050 console.

While rebooting to test out my new console settings I had a play with HDT (Hardware Detection Tool) which I had just recently added to my boot menu. While using HDT to look at the VESA information I noticed that there was a 1680×1050 graphics mode listed though I didn’t make a note of the mode number. I can’t help but wonder if I would have been able to get a native resolution console by just using a vga=xxx kernel switch? Uvesa is working fine so I will probably just stick with it.

Goodbye Open Solaris and ZFS.

4
Posted by mike on March 11, 2010 – 12:42 am
Filed under Arch, Hardware, Linux, OpenSolaris, Uncategorized

As the title of the post states I have recently decommissioned my Open Solaris ZFS based file server. I have now combined my file server with my workstation. There are several reasons why I have done this:

  • I’m trying to be a little bit greener in regards to my energy usage. I really only connected to my file server from my workstation and lately I had been leaving my workstation on for one reason or another often racking up uptime measured in days. Having two full sized tower machines running seemed wasteful. This was the main reason I decided to combine my file server and workstation.
  • OpenSolaris is a great product however I never really felt comfortable in it. Open Solaris and ZFS make for a rock solid combination. It has been sitting in the corner of my spare bedroom closet and I haven’t needed to really touch it. ZFS makes filesystem management and backups ridiculously easy. My backup system consisted of a 12 line bash script that backed up my data to an external usb based zpool which was snapshotted after every run giving me 6 months of daily backups in very little disk space. Due to the stability of this system along with it’s low maintenance I slowly forgot all my Open Solaris knowledge, what little there was, that I had gleaned from setting the machine up.

As I run Arch Linux on my workstation and did not want to resort to using ZFS through fuse I had to say good bye to ZFS. I will definitely miss ZFS. Managing storage is really a no brainer with ZFS compared to setting up a comparable Linux based layered software raid > LVM > file systems type setup. I salvaged the disks from my file server, after backing up the data, and along with the disk from my workstation and a spare I had on hand, this gave me 4 x 250G drives. I partitioned each drive with a 128M boot partition, 2G swap partition and a 248G main partition.

[mike@mercury|~] $ sudo parted /dev/sda print
Model: ATA ST3250820AS (scsi)
Disk /dev/sda: 250GB
Sector size (logical/physical): 512B/512B
Partition Table: msdos

Number  Start   End     Size    Type     File system     Flags
 1      32.3kB  132MB   132MB   primary  ext2            boot, raid
 2      132MB   2130MB  1999MB  primary  linux-swap(v1)
 3      2130MB  250GB   248GB   primary                  raid

The four boot partitions were combined into a RAID1 array while the 4 large partitions were combined into a RAID10 array.

[mike@mercury|~] $ cat /proc/mdstat
Personalities : [raid1] [raid10] 
md1 : active raid10 sda3[0] sdd3[3] sdc3[2] sdb3[1]
      484231040 blocks 64K chunks 2 near-copies [4/4] [UUUU]
      
md0 : active raid1 sda1[0] sdd1[3] sdc1[2] sdb1[1]
      128384 blocks [4/4] [UUUU]
      
unused devices:

I was tossing up between RAID5 or RAID10 and finally came down on the side of RAID10. With RAID10 you sacrifice a drive, storage wise, though you do get the benefit of no parity calculations and no RAID5 write hole. Having said that I’m not sure how much of an issue the parity calculations would have been after seeing ZFS perform fine on some very modest hardware. Linux can perform RAID10 as a single layer which gives you a few more options when setting things up and also allows you to do things like setup a RAID10 array on 3 drives.

On top of the RAID10 array I setup LVM with a separate home logical volume. The final file system layout looks like:

[mike@mercury|~] $ pydf / /boot /home
Filesystem           Size Used Avail Use%                            Mounted on
/dev/mapper/vg0-root  63G  24G   36G 38.3 [#########...............] /         
/dev/md0             121M  16M   99M 13.2 [###.....................] /boot     
/dev/mapper/vg0-home 350G 176G  173G 50.4 [############............] /home

I chose the following file systems:

[mike@mercury|~] $ mount | grep 'root\|home\|boot' | cut -d ' ' -f 1,3,5  | column -t
/dev/mapper/vg0-root  /      ext4
/dev/md0              /boot  ext2
/dev/mapper/vg0-home  /home  xfs

There was no real reasoning behind the selection of ext4 and xfs for root and home respectively. I have used both before and found them to be solid. I left around 50G spare in my volume group which I will probably end up adding to my home file system. Xfs will make this easy as it allows online resizing.

I didn’t want to have to reinstall Arch Linux so I used my usual technique of transfering a Linux install using the Arch Linux live CD:

  1. Backing up my root file system to external storage using cp -axv /mnt/src/* /mnt/dest
  2. Adding new drives, partition drives, setup raid arays, setup lvm and create filesystems.
  3. Restore my root file system using another cp -axv /mnt/src/* /mnt/dst

I wanted things to go smoothly so before I backed up my root files system I made sure I had added mdadm to my mkinitcpio hooks array, I already had lvm in there, and recreated my initrd. I also changed my /etc/fstab and /boot/grub/menu.lst to reflect the new filesystem layout. The last step was to chroot into my Arch Linux install and reinstall GRUB and generate a /etc/mdadm.conf. I hadn’t realised I needed a /etc/mdadm.conf until trying to boot. I had been roughly following the Arch Linux wiki article Installing with Software RAID or LVM and it set me straight. It also included a neat trick with sfdisk to make partitioning multiple disks easy.

All in all it went well. I managed to get the whole thing done during the Oscar’s broadcast. I did catch bits and pieces though. Steve Martin and Alec Baldwin did a fantastic job hosting and it was great to see an underdog get up in the form of Hurt Locker.

Next I need to work out a backup system. I would really like something similar to the rsync/ZFS snapshot system I used in Open Solaris. I will probably end up going with something similar using rsync and hard links or I might try btrfs and rsync. Btrfs seems to be coming along and it may be stable enough for use as a backup file system.

Nokia N95 on Linux

1
Posted by mike on January 17, 2010 – 8:12 pm
Filed under Hardware, Linux

I’m currently using an Nokia N95 mobile phone. One thing that always bugged me was that when plugging it in to a Linux machine via USB to copy files, it would always complain when it was disconnected about the possibility that data may have been lost, even though the file system was unmounted. I have just put up with it however today I found a solution:

# eject /dev/sdb

I don’t know why I didn’t try that earlier.

Macbook, EFI, GPT and Grub2

0
Posted by mike on November 7, 2009 – 7:01 pm
Filed under Arch, Hardware, Linux

As mentioned earlier I am currently using a MacBook (3.1) as my main laptop. Everything is working fine except there is a noticeable delay when the machine is turned on until the boot loader, grub 0.97, appears. It is not a huge hassle but it is a little irritating. I wanted to setup lvm on my laptop so I took the opportunity to play around to try to remove this delay. Unfortunately I was not successful.

I found an interesting blog post describing the Mac EFI boot process. Using this as a guide I used parted to repartition sda as a GPT disk. I then created an EFI System Partition formatted as fat32. Inside this partition I created the required /efi/boot directory structure. I downloaded the latest Grub2 and compiled it as an EFI executable and installed it and all it’s required modules along with a linux kernel into the /efi/boot directory.

The result was the machine successfully booted into Grub 2 and loaded the linux kernel however there was still a significant delay between power on and Grub 2 appearing.

I’m guessing that this delay is caused by one of two things:

  • Apple uses a hybrid, non standard, MBR/GPT partitioning scheme. Maybe this is causing an issue for the Apple firmware?
  • The EFI System Partition on a MacBook is actually empty. Apple’s firmware can boot OS X directly from the hfs+ system partition. Perhaps the firmware is looking for a hfs+ OS X system partition first?

Both of these are guesses and could both be incorrect. I don’t understand why either of these would take as long as the delay I am seeing. For the moment I am putting up with the delay as there doesn’t seem to be an easy way around it without installing OS X and/or rEFIt alongside Linux. I might have another go using an external hard drive at some point. I am at least now using lvm. I haven’t used lvm before and am looking forward to playing around with it. As I understand it, lvm’s snapshots have major performance issues which is a shame as this is something I would have used.

XMonad 0.9

0
Posted by mike on October 26, 2009 – 7:35 pm
Filed under Hardware, Haskell, xmonad

XMonad 0.9 has been released. While I have been using the Darcs version for a while there looks to be a few features in 0.9 that I was not aware of. The changes since 0.8 page makes interesting reading and any XMonad users should check it out.

I’m still configuring Arch Linux x86_64 on my MacBook and everything seems to be working well except for the iSight. It appears to be possible to get it working using firmware extracted from a file in OSX. I never really used it and don’t miss it enough to bother doing that. The brightness controls didn’t initially work until I installed a daemon called pommed which handles the special Apple keys on the keyboard. Screen brightness is now easily adjustable.

I ordered 4Gb of RAM for it today so that should make it nice and snappy. It only came with 1 Gb with 144M of this being used by the integrated Intel graphics. 4Gb is probably overkill for my use but RAM is so cheap these days.

All in all it has been pretty easy installing Arch Linux on a Mac Book. I was expecting it to be more difficult. There is a pause during boot which I take is something to do with the machine looking for a GPT disk before it falls back to a MBR disk. It is not a huge hassle but if I find a way to prevent that without having to resort to using OSX as a second OS that would be cool.

MacBook

0
Posted by mike on October 23, 2009 – 7:00 pm
Filed under Arch, Hardware, Linux

As mentioned previously my main laptop died. I’m currently writing this on my MacBook runing Arch Linux x86_64. I mainly used this machine for watching movies and websurfing with it running OSX. Installing Arch Linux went surprisingly well. Most hardware worked out of the box and unlike my HP this machine has Intel graphics. I’m glad to leave ATI behind. Intel and KMS are much more fun to play with.

Have been too busy to post much. Too many uni assignments to do. One due at midnight tonight. Better go and make sure it builds cleanly on uni’s server.

RIP HP nx8420.

0
Posted by mike on September 26, 2009 – 11:23 am
Filed under Hardware

Bugger! Shut my laptop down last night and when I booted it up this morning it seems to be dead. It’s locking up, usually in the BIOS section though sometimes it gets as far as GRUB and once it made it to the console before locking up. This has been a great machine (Core 2 Duo) and I am sorry to see it go. It was happily running Arch Linux x86_64. The only downside to it was the ATI graphics. Not sure if I will replace it or maybe start using my netbook more. I’ll have a better look at it when I get home. Curently traveling for work. Will arrive home later today.