FreeBSD-ZFS

Inhaltsverzeichnis

    * 1 Migrate UFS to ZFS
          o 1.1 Install FreeBSD 8.0 with ZFS Root
          o 1.2 Copy Old System to ZFS
          o 1.3 Install Software
          o 1.4 Attach all Disk and Restore them

Migrate UFS to ZFS

The system is ported in two steps, first step is to generate the first disk and copy the data to this disk, the second step is to add the second disk and start the restore of the raid 1.
Install FreeBSD 8.0 with ZFS Root

To really use ZFS it is recommended to install a AMD64 environment, so boot from USB Memstick and choose Fixit:

Zero your new drive to destroy any existing container:

dd if=/dev/zero of=/dev/ad4
*cancel it after some seconds*

We will use GPT to boot so we create at first these volumes:

gpart create -s gpt ad4
gpart add -b 34 -s 128 -t freebsd-boot ad4
gpart add -b 162 -s 8388608 -t freebsd-swap -l swap0 ad4
gpart add -t freebsd-zfs -l disk0 ad4

Install proteced MBR:

gpart bootcode -b /mnt2/boot/pmbr -p /mnt2/boot/gptzfsboot -i 1 ad4

Load ZFS:

kldload /mnt2/boot/kernel/opensolaris.ko
kldload /mnt2/boot/kernel/zfs.ko
kldload /mnt2/boot/kernel/geom_mirror.ko

Create ZFS pool:

mkdir /boot/zfs
zpool create zroot /dev/gpt/disk0
zpool set bootfs=zroot zroot

Create the ZFS filessystem hierarchy:

zfs set checksum=fletcher4 zroot

zfs create -o compression=on    -o exec=on      -o setuid=off   zroot/tmp
chmod 1777 /zroot/tmp

zfs create                                                      zroot/usr
zfs create                                                      zroot/usr/home
cd zroot ; ln -s /usr/home home

zfs create -o compression=lzjb                  -o setuid=off   zroot/usr/ports
zfs create -o compression=off   -o exec=off     -o setuid=off   zroot/usr/ports/distfiles
zfs create -o compression=off   -o exec=off     -o setuid=off   zroot/usr/ports/packages

zfs create -o compression=lzjb  -o exec=off     -o setuid=off   zroot/usr/src

zfs create                                                      zroot/var
zfs create -o compression=lzjb  -o exec=off     -o setuid=off   zroot/var/crash
zfs create                      -o exec=off     -o setuid=off   zroot/var/db
zfs create -o compression=lzjb  -o exec=on      -o setuid=off   zroot/var/db/pkg
zfs create                      -o exec=off     -o setuid=off   zroot/var/empty
zfs create -o compression=lzjb  -o exec=off     -o setuid=off   zroot/var/log
zfs create -o compression=gzip  -o exec=off     -o setuid=off   zroot/var/mail
zfs create                      -o exec=off     -o setuid=off   zroot/var/run
zfs create -o compression=lzjb  -o exec=on      -o setuid=off   zroot/var/tmp
chmod 1777 /zroot/var/tmp

Install the FreeBSD to the zroot:

cd /dist/8.0-*
export DESTDIR=/zroot
for dir in base catpages dict doc games info lib32 manpages ports; \
    do (cd $dir ; ./install.sh) ; done
cd src ; ./install.sh all
cd ../kernels ; ./install.sh generic
cd /zroot/boot ; cp -Rlp GENERIC/* /zroot/boot/kernel/

Make /var/empty readonly

zfs set readonly=on zroot/var/empty

chroot into /zroot

cd /
chroot /zroot

echo 'zfs_enable="YES"' > /etc/rc.conf
echo 'hostname="freebsdtest.mydomain.local"' >> /etc/rc.conf
echo 'ifconfig_em0="DHCP"' >> /etc/rc.conf

Setup the bootloader:

echo 'zfs_load="YES"' > /boot/loader.conf
echo 'vfs.root.mountfrom="zfs:zroot"' >> /boot/loader.conf
echo 'geom_mirror_load="YES"' >> /boot/loader.conf

Install ZFS aware boor loader

echo 'LOADER_ZFS_SUPPORT=YES' > /etc/src.conf
mount -t devfs devfs /dev
export DESTDIR=""
cd /usr/src/sys/boot/
make obj
make depend
make
cd i386/loader
make install

Change the root password:

passwd

Set local time zone:

tzsetup

Create /etc/mail/aliases.db

cd /etc/mail
make aliases

Exit from the /zroot

umount /dev
exit

Install zpool.cache to the ZFS filesystem

cp /boot/zfs/zpool.cache /zroot/boot/zfs/zpool.cache

Create the swap partition:

gmirror label -b prefer swap gpt/swap0

Create the /zroot/etc/fstab

# Device                       Mountpoint              FStype  Options         Dump    Pass#
/dev/mirror/swap               none                    swap    sw              0       0

export LD_LIBRARY_PATH=/mnt2/lib

Unmount all zfs filesystems:

zfs unmount -a

Change mount points for the zroot pool

zfs set mountpoint=legacy zroot
zfs set mountpoint=/tmp zroot/tmp
zfs set mountpoint=/usr zroot/usr
zfs set mountpoint=/var zroot/var

Exit Fixit mode and the sysinstall. The system should now boot the ZFS environment.

After the system is up again it should say something like this:

zpool status

  pool: zroot
 state: ONLINE
 scrub: none requested
config:

        NAME         STATE     READ WRITE CKSUM
        zroot        ONLINE       0     0     0
          gpt/disk0  ONLINE       0     0     0

errors: No known data errors

gmirror status

       Name    Status  Components
mirror/swap  COMPLETE  ad4p2

Copy Old System to ZFS

cd /zroot
rsync -av /etc /zroot/
rsync -av /usr/local/etc /zroot/usr/local/
rsync -av /var/amavis /zroot/var/
rsync -av /var/db/DAV /var/db/clamav /var/db/dhcpd.* /var/db/mysql /var/db/openldap-data /var/db/openldap-data.backup /zroot/var/db/
rsync -av /var/log /zroot/var/
rsync -av /var/spool /var/named /zroot/var/
rsync -av /usr/home /zroot/usr/
rsync -av /root /zroot/
rsync -av /usr/src/sys/i386/conf /zroot/usr/src/sys/i386/
rsync -av /usr/local/backup /usr/local/backup_rsync /usr/local/cvs /usr/local/dbdump /usr/local/faxscripts /zroot/usr/local/
rsync -av /usr/local/firewall /usr/local/pgsql /usr/local/cvs /usr/local/psybnc /usr/local/router /zroot/usr/local/
rsync -av /usr/local/squirrelmail_data /usr/local/src /usr/local/ssl /usr/local/svn /usr/local/tftp /zroot/usr/local/
rsync -av /usr/local/var /usr/local/video /usr/local/www /usr/local/idisk /zroot/usr/local/
rsync -av /usr/local/bin/printfax.pl /usr/local/bin/grepm /usr/local/bin/block_ssh_bruteforce /usr/local/bin/learn.sh /zroot/usr/local/bin/
mkdir -p /zroot/usr/local/libexec/cups/
rsync -av /usr/local/libexec/cups/backend /zroot/usr/local/libexec/cups/
rsync -av /usr/local/share/asterisk /zroot/usr/local/share/
rsync -av /usr/local/libexec/mutt_ldap_query /zroot/usr/local/libexec/
rsync -av /usr/local/lib/fax /zroot/usr/local/lib/
mkdir -p /zroot/usr/local/libexec/nagios/
rsync -av /usr/local/libexec/nagios/check_zfs /usr/local/libexec/nagios/check_gmirror.pl /zroot/usr/local/libexec/nagios/

Check your /etc/fstab, /etc/src.conf and /boot/loader.conf after this and adapt it like described above.
Install Software

portsnap fetch
portsnap extract
cd /usr/ports/lang/perl5.10 && make install && make clean
cd /usr/ports/ports-mgmt/portupgrade && make install && make clean
portinstall bash zsh screen sudo radvd
portinstall sixxs-aiccu security/openvpn quagga isc-dhcp30-server
portinstall cyrus-sasl2 mail/postfix clamav amavisd-new fetchmail dovecot-sieve imapfilter p5-Mail-SPF p5-Mail-SpamAssassin procmail
portinstall databases/mysql51-server net/openldap24-server databases/postgresql84-server mysql++-mysql51
portinstall asterisk asterisk-addons asterisk-app-ldap
portinstall www/apache22 phpMyAdmin phppgadmin mod_perl2 mod_security www/mediawiki smarty
portinstall php5-extensions squirrelmail squirrelmail-avelsieve-plugin

portinstall munin-main munin-node net-mgmt/nagios nagios-check_ports nagios-plugins nagios-spamd-plugin logcheck nrpe
portinstall portmaster portaudit portdowngrade smartmontools
portinstall awstats webalizer
portinstall bazaar-ng subversion git
portinstall rsync ipcalc doxygen john security/gnupg nmap unison wol mutt-devel wget miniupnpd

portinstall editors/emacs jed
portinstall www/tomcat6 hudson
portinstall cups
portinstall squid adzap
portinstall samba
portinstall net-snmp
portinstall teamspeak_server
portinstall scponly


Attach all Disk and Restore them

Insert now the second disk (in my case ad6). We use GPT on the second disk too:

gpart create -s gpt ad6
gpart add -b 34 -s 128 -t freebsd-boot ad6
gpart add -b 162 -s 8388608 -t freebsd-swap -l swap1 ad6
gpart add -t freebsd-zfs -l disk1 ad6

Install MBR:

gpart bootcode -b /boot/pmbr -p /boot/gptzfsboot -i 1 ad6

Create swap:

gmirror insert swap gpt/swap1

While rebuilding it will show:

gmirror status
       Name    Status  Components
mirror/swap  DEGRADED  ad4p2
                       ad6p2 (48%)

After it is finished:

 gmirror status
       Name    Status  Components
mirror/swap  COMPLETE  ad4p2
                       ad6p2

Create the zfs mirror:

zpool attach zroot /dev/gpt/disk0 /dev/gpt/disk1

It will resilver now the data:

zpool status
  pool: zroot
 state: ONLINE
status: One or more devices is currently being resilvered.  The pool will
        continue to function, possibly in a degraded state.
action: Wait for the resilver to complete.
 scrub: resilver in progress for 0h1m, 0.49% done, 4h1m to go
config:

        NAME           STATE     READ WRITE CKSUM
        zroot          ONLINE       0     0     0
          mirror       ONLINE       0     0     0
            gpt/disk0  ONLINE       0     0     0  12.4M resilvered
            gpt/disk1  ONLINE       0     0     0  768M resilvered

errors: No known data errors