Building a NAS, Part 1: Requirements

A friend would like a new NAS, with a combo of features and performance that would normally place them in the $10k+ range for enterprise grade storage. I offered to see what I could do with hardware from Ebay, and I figured I’d write down my thought process for this kind of thing, in case it’s of use to other homelab shoppers.

I should note that I’m writing this as I go, so I have no idea what the conclusion is going to be. We might end up with something boring like “just buy this Western Digital MyCloud thing”, or it might be a more elaborate hardware adventure. Who knows!

This post is part of a series:


When I start on a new machine build for someone, my first questions are always aimed at teasing out requirements. Often the conversation starts like “I need a new NAS, any recommendations?” With only that to go on, I could suggest anything from a single USB external hard drive, all the way up to a hundred-drive NVMe grid computing monster in the 6-figure range.

The true need is probably somewhere in the middle. So, for a NAS, here’s some conversation starters I used to try and tease out requirements.

After that chat, for this build we ended up with the following:

Exploring the solution space

Okay, that’s a lot to take in. Let’s sit with those requirements for a moment, and let some deductions fall out…

What kind of box?

Rackmount and ECC RAM eliminates almost all the consumer off the shelf options. We’re looking at entry level enterprise gear and up.

Good news is, at our price point there are enterprise “off the shelf” options. Synology, ixSystems, QNAP, Buffalo all make boxes that fit comfortably in budget. For example, you could get a Synology RackStation RS2418RP+ for $2400, leaving $2600 for drives.

However, at this price point most appliances are anemic on RAM and CPU. The RS2418RP+ for example comes with 4GiB installed, and a wimpy Celeron processor. This is no good in several dimensions (running VMs, in-RAM disk cache for performance). Adding RAM is possible at a cost, but we’re stuck with the CPU unless we go up to the next tier of hardware… which blows out the budget.

My friend also has a love/meh relationship with appliances. They tend to enforce being too hands-off. That’s great if you’re a business, but it gets in the way of tinkering. ixSystems and FreeNAS is a reasonable compromise on that axis, but our requirement to run Linux eliminates that option.

So, this is pushing us in one of two directions: used enterprise gear, to get a higher tier of hardware at budget prices; or a “whitebox” build, in which we pick out individual components and assemble our own system.

What kind of storage?

We have to use traditional “spinning rust” hard drives as the main storage pool. For the number of bytes we need to store, they’re the only thing that hits a $/TiB that makes sense for the budget.

For reference, at time of writing you can get a 6TB NAS drive for $230. The equivalent storage in SATA SSDs costs $780, and $2400 if you want NVMe SSDs. And that’s before any kind of redundancy, or space constraints on our SATA and PCIe busses.

Does this square with our requirement of saturating 1Gbps in and out of the box? Quick napkin math: a basic NAS drive is going to ballpark 100-150MiB/sec in either direction, or 800-1200Mbps. So, turns out, we don’t have to worry too much for raw throughput! We’ll need a little attention to performance when we pick out specific drives and storage layouts, but the order of magnitude is right.

This also tells us that SSDs as primary storage would be wasteful, as we can’t utilize anywhere near their full bandwidth with 1Gbps networking.

One more thing to consider is fsync-heavy loads. These will fall short of theoretical throughput numbers, because we’re forcing the OS to wait for specific bytes to commit to drives before continuing. This means we’re bounded by the round-trip latency of our drives, which is relatively poor for spinning rust.

At this stage it’s not clear to me that the write load we’re dealing with is going to have that problem. If it does, and we’re using ZFS for storage, we can throw in dedicated ZIL (ZFS Intent Log) devices to give our server a hefty write buffer that hides the spinning rust latency.

To be continued

That’s all for this post. We’ve gone from requirements to a very vague idea of what we’re looking for: a rackmount server, either used enterprise appliance or custom whitebox, with traditional HDD storage, supplemented by SSDs.

There are still a lot of details to work out, and we’ll be returning to the requirements to guide us through that. In part 2, we’ll do a bit more napkin math to work out minimum hardware requirements.