I’ve been a fan of the ZFS filesystem for many years now. Its attention to reliability, data integrity, ease of volume expansion and unlimited snapshots make many tasks so much easier. ZFS was originally developed on Solaris and was then ported to FreeBSD, among other systems, when Sun Microsystems open-sourced ZFS. FreeBSD is a rock solid operating system I have used since its inception in the early 1990s.  The FreeNAS distribution combines the FreeBSD operating system with a great web based management interface for a storage appliance. FreeNAS supports Windows (CIFS/SMB), Macintosh (AFP) and Unix (NFS) file sharing along with ISCSI block access for applications like VMWare’s ESXi virtualization platform.

I’ve been wanting to put a FreeNAS system into my home rack for a while now but hadn’t quite found the right platform on which to do this.  My goal was to have a system with low energy consumption but with a reasonable amount of power. I wanted to have at least 4 drive bays, ideally hot swappable, at least 2 gigabit ethernet ports and a rack mountable chassis. I believe I’ve found a great solution in the Supermicro 5018A-MHN4 system. It boasts an 2.4GHz Intel Atom C2758 8-core processor, up to 64GB of error correcting (ECC) memory, 4 hot-swappable 3.5″ drive bays, 4 gigabit ethernet ports, and an integrated lights-out management port, all in a 1U rack  mount case.

Next came the hard drive selection process. I debated whether to go with the Western Digital Red/Seagate NAS drives or higher performing 7200RPM enterprise class drives but decided on the Red drives because of their much lower power consumption of 4.5W active/3.3W idle vs 10.9W active/8.6W idle per drive. This is a home server, so performance isn’t as critical as it would be in a data center and the power savings and noise reduction is welcome. One thing to note is that the Red drive specification lists read errors at 1 per 10¹⁴ bits vs a much lower error rate of  1 per 10¹⁶ bits for the enterprise class drives.

Parts List

Note: Prices are in Canadian Dollars and suppliers are based in Canada.

The total cost for a server with 16TB of raw storage capability was $1924CDN + taxes.

The server chassis arrives with the motherboard already installed. A rail kit is also included, both for 2 port racks as well as 4 post racks. The rails are designed for round hole racks so cage nuts are required to mount it in a square hole rack.

(click on any image for a larger view)

Setup

The first step is to install the memory modules. I used four 8GB modules for a total of 32GB of ECC memory.

The motherboard supports 6 SATA ports – two 6Gbps ports (SATA 0/1) and four 3Gbps ports (SATA 2/3/4/5). The chassis is wired such that SATA 0/1 are connected to the first two drive bays and SATA 2/3 are connected to the second two drive bays.  Hard drives typically can sustain no more than 120-240MBps so I chose to change the cabling to map the 3Gbps ports to the four drive bays on the front of the unit and keep the two 6Gbps ports for internal 2.5″ SSD drives.

Cabling as shipped (above) and after reassignment (below)

I did add in an 120GB Kingston SSD drive I had laying around and tucked it under the cables from the power supply. This drive is plugged into SATA port 0 and can be used as a ZIL or L2ARC drive.

The final system image is shown below.

FreeNAS installation

One of the nice features of the Supermicro system is the integrated management feature. A dedicated gigabit ethernet port is used to provide a web based interface with a remote console and virtual media. I installed the FreeNAS software from the ISO distribution image to a 2GB USB flash drive by booting the system using the virtual CDROM media redirection to reference the ISO image on my laptop. Another possibility is to use the FreeNAS USB image and load it directly onto the 2GB USB flash drive with another computer. In either case, the 64-bit version of FreeNAS is recommended. Once loaded, ensure that the system BIOS tries to boot from the USB flash drive before trying the hard drives.

Once the FreeNAS operating system is installed it is time to configure it. This process is the same as it would be for any FreeNAS installation. I chose to create a RAID 0+1 style drive layout for optimal read and write performance from the drives. With 4 drives, the reads are split across all four drives and each write needs to go to the two drives in each mirror pair. This layout scheme gives me 7.27TB of usable space. LZ4 compression is enabled and further increases the effective storage space.

Performance

Performance testing was done after loading 1.32TB of data on the system. The performance numbers shown below are therefore slightly lower than on a fresh system as the fastest parts of the disk will already have been allocated.

The numbers are nonetheless quite good considering the slower WD Red drives used. 7200RPM drives should perform even better. The tests with compression disabled give an idea of the raw IO performance of the system. Tests with compression enabled had bonnie++ at 95-100% CPU utilization on one core, which may suggest that the Intel Atom is maxing out doing the compression. Tests with ZFS compression disabled had a much lower CPU utilization, in the 40-50% area.

This system and drive configuration provides approximately 200 MBps write and 400 MBps read of raw sequential performance depending on block size. Compressible data will give even better performance numbers.

I repeated the tests with iozone and they showed slightly higher numbers in the ZFS compression disabled tests, possibly related to different block sizes chosen by the two benchmarks. It appears that iozone uses highly compressible data, so those tests with ZFS compression enabled report very high IO speeds.

Note: Higher bars are better. The SSD L2ARC is large enough to hold the entire test file size, so those numbers may be skewed.

Conclusion

The Supermicro 5018A-MHN4, with its low power consumption of approximately 50W while doing a ZFS scrub and overall performance, makes it an ideal FreeNAS platform.