Adding a new hard disk to a Linux server

19 April 2007 » Linux, System administration

Last week I added a second hard disk to an IBM eServer xSeries 226 server running CentOS 4.4.

I picked up a large hard drive in January, but I didn’t have the proper hardware to install it right away.

Besides that, I had to settle on how to partition the drive, create the filesystems, and decide on the mount points.

Since the overall process did take me some time, I figured I’d share some notes on the steps I took for folks interested in doing the same. I’ve also provided links to resources that helped me out along the way.

Planning is the lion’s share of the work. Once you’ve decided what to do, the actual addition of the drive and the execution of the commands is fairly straightforward.

  • Determine why you need another hard drive
    It’s always nice to have more space, but beyond that I had some more practical concerns.

    The server came with a single 80GB disk. When I installed CentOS, I was quite lazy and settled on a small swap partition with a large root partition taking up the rest of the disk.

    So, there were at least four reasons guiding my decision to add another drive:

    • To prevent the single root filesystem from filling up and rendering the machine unusable. This could happen since I have nightly cron jobs that sync backups (large media files) from my workstation to the server. It might also happen if my machine was suddenly hit with a burst of traffic or a DOS attack.
    • I didn’t want to resize the existing disk partition or reinstall the operating system. I felt more comfortable creating overflow filesystems on a blank disk.
    • I wanted to provide a measure of redundancy in case the first hard drive failed. Based on rumors from colleagues and other unscientific anecdotes, Maxtor hard drives are more prone to failure. There’s also the small matter of my server sitting on the floor of a dusty basement next to the litter box.
    • To limit different types of disk activity to different partitions. For example, logging (writes to large files) is a different usage pattern than Web serving (reading lots of little files). I wanted a measure of control about how to optimize I/O.
  • Find the right drive for your server
    Given that my machine had only one free SATA bay, my strategy was to get the largest drive possible for around $100.

    A great tool for analyzing your current system to determine its current configuration and potential for expansion is Hardware Lister (lshw). This will give you specific model information and insight into where you might expand your system.

    Pipe the output of this command to a text file, so that you’ll have the information readily available. This will guide your upgrade decisions and serve to verify installation later.

    [root@192.168.1.1]# /usr/sbin/lshw > /root/lshw-before.txt

    Based on this information, I was able to Google compatible drives for the machine, and found a solid price on a Western Digital 400MB SATA drive at Newegg.

  • Make sure you have the right hardware to install the drive
    Hard drives are often sold in Retail or OEM packaging. Retail comes in a pretty box and will include the essential hardware bits, such as cables and fasteners. OEM is the way to save some cash if you’re replacing an existing drive or have extra hardware for your box.

    I had plenty of screws from the carcasses of some E450s, but I mistakenly thought I could reuse the brackets too. Instead, I had to pay $25 for a new plastic bracket in which to mount the drive. Arg.

    Along with an anti-static wrist strap, a Torx T10 screwdriver came in quite handy.

  • Install the drive and make sure it’s seen by the kernel

    DISCLAIMER: If you intend to add a hard drive to your system, please take all due precautions before starting, as there is a real possibility that you will lose some data.

    I shut down the machine, removed the power cable, slid the drive in, closed it up, then booted back up.

    I then ran lshw again. Comparing this with the prior snapshot of the system, I could verify that the drive was recognized as /dev/sdb.

    [root@192.168.1.1]# /usr/sbin/lshw > /root/lshw-after.txt
    [root@192.168.1.1]# diff /root/lshw-before.txt /root/lshw-after.txt
  • Create the partitions
    Through trial and error I had settled on a reasonable filesystem layout for my previous Sun systems. Solaris has very impractical defaults for my purposes, so I needed to do some leg work in the past to make sure I had enough space in the right places.

    This is what my latest Solaris 10 partitioning scheme looks like:

    c0t0d0 (~20GB)
            0 /             (~8GB)          [c0t0d0s0]
            1
            3
            4 /swap         (~1GB)          [c0t0d0s4]
            5
            6
            7 /export/home  (~10GB)         [c0t0d0s7]
    
    c0t2d0 (~80GB)
            0 /var          (~25GB)         [c0t2d0s0]
            1
            3
            4 /opt          (~25GB)         [c0t2d0s4]
            5
            6
            7 /usr          (~25GB)         [c0t2d0s7]
    

    If I was doing a fresh Linux operating system install today, I might choose something similar. But given that I was upgrading an existing system, I decided only to offload what was in the /var and /home directories.

    Using fdisk, I decided to slice up the 400GB hard drive like so:

    • New primary partition of 50GB at /dev/sdb1 (for /var/www)
    • New primary partition of 50GB at /dev/sdb2 (for /var/log)
    • New primary partition of 50GB at /dev/sdb3 (for /var/mail)
    • New primary partition of 200GB at /dev/sdb4 (for /home)

    Partitioning with fdisk” and “Creating Linux partitions” are helpful resources for working with fdisk.

  • Create the filesystems
    There are several options for filesystems, but ext3 seems to be the sweet spot for most Linux usage scenarios.

    IBM developerWorks has a good overview of ext3. There’s more information in chapter 1 of the new “Linux Performance and Tuning Guidelines” Redbook as well.

    I formatted each of the partitions as ext3 using the following command. (From this point on I’ll only show the commands for one of the four new partitions).

    [root@192.168.1.1]# /sbin/mkfs -t ext3 /dev/sdb1
  • Mount the filesystems
    To make the new filesystems accessible, I mounted each to a location under /mnt.

    [root@192.168.1.1]# mount -t ext3 /dev/sdb1 /mnt/var/www

    At this point, all I saw in each new mounted directory was a “lost+found” folder.

  • Copy over data to the new filesystem
    Now that the new filesystem is ready for use, I needed to move the existing directories to it.

    To make sure nothing is being written to the source filesystem, I dropped to single user mode.

    [root@192.168.1.1]# init 1

    I then copied data from the source filesystem recursively while preserving file metadata, such as owner and last modified time.

    [root@192.168.1.1]# cd /var/www
    [root@192.168.1.1]# cp -ax * /mnt/var/www

    I verified quickly to make sure everything was copied. You might want to check things more thoroughly than this however.

    [root@192.168.1.1]# find /var/www | wc -l
    [root@192.168.1.1]# find /mnt/var/www | wc -l

    Moved the source directory so I could mount the new filesystem in its place.

    [root@192.168.1.1]# mv /var/www /var/www.old

    Mounted the new filesystem and returned to multi-user mode.

    [root@192.168.1.1]# mkdir -p /var/www
    [root@192.168.1.1]# umount /mnt/var/www
    [root@192.168.1.1]# mount -t ext3 /dev/sdb1 /var/www
    [root@192.168.1.1]# ctrl-D

    System Administration Toolkit: Migrating and moving UNIX filesystems,”
    Partitioning in action: Moving /home” and
    Partitioning in action: Consolidating data” are good resources which document these steps.

  • Add the drive to fstab
    Everything looked good, so I mapped the new filesystems permanently and rebooted the box.

    [root@192.168.1.1]# vi fstab
            /dev/sdb1   /var/www  ext3    defaults     0 0

    In a few days, once I’m satisfied that everything is working as planned, I’ll archive the old directory and remove it to save space.

    [root@192.168.1.1]# rm -fr /var/www.old

And that’s it. Once I had a plan, everything went very quickly. The recursive copy from the source directory to the new destination was probably the longest command timewise, mainly because I had 40GB already in my home directory, but even that step took less than half an hour.

Here are some more resources I consulted when adding my new hard disk.

I also owe thanks to Martin Corona for doing a sanity check on my setup.

  

7 Responses to 'Adding a new hard disk to a Linux server'

Subscribe to comments with RSS

  1. Martin said,

    24 May 2007 at 1:08 am

    Dahhhhh stop hosting at home. TOS violation!!

  2. Daniel Krook said,

    24 May 2007 at 8:40 am

    Dah. Optimum Online Boost lets you run your own servers. Regular OO doesn’t. Time to get off that crappy FiOS and away from Verizon, Martin.

  3. Bob Dobbs said,

    16 April 2008 at 2:21 pm

    “cp -ax *” will not copy all files. ‘Hidden’ files (start with a dot) do not get copied.

  4. Daniel Krook said,

    17 April 2008 at 11:08 am

    Hi Bob,

    Interesting, I’ll research that more but I didn’t lose my .bash_profile, .htaccess or those infernal .DS_Store files in the move.

    Everything I expect to be copied over, was.

  5. Peter said,

    12 May 2008 at 3:36 pm

    I used your method to move my whole /var directory to a different partition, everything seems to be ok. Is it safe to use same method to move my /home to a different partition as well? Thanks.

  6. Wyatt said,

    16 February 2010 at 2:47 am

    Wow, this is what i’ve been looking for. I’m playing with the ClearOS. i’m running a old dual PIII server gig ram with 4- 8gig scsi’s. Woo hoo. it was free.
    i have reinstall’d , 3 times. last i made partitions for my /var and my /home.
    but OS still used /. why i thought the OS would re-direct on it on is my duh.
    I have not been this deep in linux sense the Maximum Linux mag and forum days!

    Thanks!!

  7. Praveen said,

    02 May 2011 at 8:04 am

    Very good article. Very useful.

    Thanks a lot.

Leave a Reply