Converting LVM array to ZFS to migrate to FreeNAS

The Something Awful Forums > Discussion > Serious Hardware/Software Crap > Haus of Tech Support > Converting LVM array to ZFS to migrate to FreeNAS

Tiger.Bomb: Jan 22, 2012

I have two computers. One was dying (hanging randomly) for a reason no one could diagnose, so I bought a new computer. The new computer doesn't have any drives yet, but has FreeNAS on a usb drive.

The first computer is a Linux machine with 3 HDDs combined with LVM to create a mirrored 3TB drive. The physical drives are 3, 2, 1 TB.

What I want to do is convert the LVM to ZFS without losing my data.

1. How do I remove the 3TB from my LVM setup? I need to remove it so that the 2 + 1 contains the original data.
2. What would I format it as before copying over the data from the 2 +1 TB? I understand FreeNAS uses features that aren't available to Linux on the ZFS partitions.

Thanks

# ? May 14, 2015 00:14

Adbot: ADBOT LOVES YOU

# ? May 3, 2024 15:09

prak: Jan 3, 2006; ---------; Nap Ghost

Tiger.Bomb posted:

I have two computers. One was dying (hanging randomly) for a reason no one could diagnose, so I bought a new computer. The new computer doesn't have any drives yet, but has FreeNAS on a usb drive.

The first computer is a Linux machine with 3 HDDs combined with LVM to create a mirrored 3TB drive. The physical drives are 3, 2, 1 TB.

What I want to do is convert the LVM to ZFS without losing my data.

1. How do I remove the 3TB from my LVM setup? I need to remove it so that the 2 + 1 contains the original data.
2. What would I format it as before copying over the data from the 2 +1 TB? I understand FreeNAS uses features that aren't available to Linux on the ZFS partitions.

Something like: (test it on something you don't care about first and/or back it up)

code:

lvs -a -o name,devices <VolumeGroup>            # to get the device of your 3TB drive
lvconvert -m 0 <LogicalVolume> <DevicePath>     # to stop mirroring  and drop the 3TB drive

Don't use ZFS. :eng101:

Anything you heard about the features of ZFS assumes you know exactly what you are doing; it can be extremely unforgiving if you don't. It requires regular monitoring and maintenance, hardware support for ECC (motherboard, processor and RAM), and that you accept, in the case of catastrophic failure, you will lose all of your data with no hope of recovery from anything but a backup. Not some or most, all. The recovery tools and options that usually allow at least partial recovery of data with other file systems do not work in any way with ZFS.

Drive Failure

quote:

if multiple devices have failed and there is not enough redundancy to compensate, the pool enters the faulted state. If a sufficient number of devices cannot be reconnected to the pool, the pool becomes inoperative and data must be restored from backups.

The minimum number of drives required to have any drive redundancy is three. Your usable space will be constrained by the size of your smallest drive and you cannot expand it by adding drives later. (You can replace them later with larger drives however.)

Bad RAM

quote:

ZFS is operating on a system without ECC RAM and a bit flip triggers undefined behavior that damages the disk format beyond ZFS� ability to recover.

Then there is the ECC hardware which is going to cost more. While memory errors might be fairly rare ZFS' design makes them particularly bad. You will corrupt data on both reads and writes, the regular scrubs to prevent data corruptions will, instead, corrupt all of it, and if it is the file system data itself you lose the entire pool.

You didn't really say much about your hardware though. It is possible (since you bought a machine with FreeNAS on a thumbdrive) you do have ECC support; I still would recommend you go with something else.

ZFS is for people (companies) with very large storage requirements, who care deeply that every single last bit stay in place, and have the time and money to devote to it; for everyone else there are just too many pitfalls. Connecting your drives through hardware RAID? No. Which SATA controller are you using? Oh, you say made sure it is one of the recommended models? Are you sure it has the right firmware? You flashed it to the latest available? Oh no, sorry, that doesn't match the drivers. Did you ignore the warnings about deduplication because 32GB is a lot of RAM and you think that it will save you a bunch of disk space? Yeah, the recommended is anywhere from 2GB to 10GB per TB of disk space and if that table ever grows large enough to not fit in RAM you lose all access to your data until you get more memory. Yeah, turning it off isn't an option; create a whole new pool and restore from backup. :emo:

Oh, almost forgot my favorite: bought a processor that is documented to support ECC? Oops, no.

quote:

I know that many of you bought i3s expecting ECC support. Sorry, but you'll have to get a new CPU if you want to use ECC as it appears you probably never were.

prak fucked around with this message at 05:54 on May 18, 2015

# ? May 18, 2015 03:11

Tiger.Bomb: Jan 22, 2012

prak posted:

Something like: (test it on something you don't care about first and/or back it up)
code:
lvs -a -o name,devices <VolumeGroup>            # to get the device of your 3TB drive
lvconvert -m 0 <LogicalVolume> <DevicePath>     # to stop mirroring  and drop the 3TB drive
Don't use ZFS.

Anything you heard about the features of ZFS assumes you know exactly what you are doing; it can be extremely unforgiving if you don't. It requires regular monitoring and maintenance, hardware support for ECC (motherboard, processor and RAM), and that you accept, in the case of catastrophic failure, you will lose all of your data with no hope of recovery from anything but a backup. Not some or most, all. The recovery tools and options that usually allow at least partial recovery of data with other file systems do not work in any way with ZFS.

Drive Failure

The minimum number of drives required to have any drive redundancy is three. Your usable space will be constrained by the size of your smallest drive and you cannot expand it by adding drives later. (You can replace them later with larger drives however.)

Bad RAM

Then there is the ECC hardware which is going to cost more. While memory errors might be fairly rare ZFS' design makes them particularly bad. You will corrupt data on both reads and writes, the regular scrubs to prevent data corruptions will, instead, corrupt all of it, and if it is the file system data itself you lose the entire pool.

You didn't really say much about your hardware though. It is possible (since you bought a machine with FreeNAS on a thumbdrive) you do have ECC support; I still would recommend you go with something else.

ZFS is for people (companies) with very large storage requirements, who care deeply that every single last bit stay in place, and have the time and money to devote to it; for everyone else there are just too many pitfalls. Connecting your drives through hardware RAID? No. Which SATA controller are you using? Oh, you say made sure it is one of the recommended models? Are you sure it has the right firmware? You flashed it to the latest available? Oh no, sorry, that doesn't match the drivers. Did you ignore the warnings about deduplication because 32GB is a lot of RAM and you think that it will save you a bunch of disk space? Yeah, the recommended is anywhere from 2GB to 10GB per TB of disk space and if that table ever grows large enough to not fit in RAM you lose all access to your data until you get more memory. Yeah, turning it off isn't an option; create a whole new pool and restore from backup.

Oh, almost forgot my favorite: bought a processor that is documented to support ECC? Oops, no.

Thanks for the help. What I did was I converted to 0 mirrors and removed the 3TB from the LV, VG, and PV. I then formatted it as ext4 and copied everything from the lv to the ext4 drive.

I then rebooted in FreeNAS, and was planning on formatting the 1 + 2 as ZFS to copy back from the EXT4. Problem is that it couldn't read the GPT partition table on the 3TB. Played with it for a bit... and it turns out the drive failed!

Now I am in the tough spot of having only one copy of my data.

I have this computer, which does have ECC but is an i3. Why would they sell a server with ram that the processor doesn't support? According to this I should be OK.

What do you suggest I run, then? I was running a ubuntu desktop before, but like i said it started dying.

I could put ubuntu server or something on the new one, but ideally I'd like to boot from a usb key. Mostly because there are only three drive bays and I don't want to need to dedicate one for the OS.

Tiger.Bomb fucked around with this message at 03:25 on May 19, 2015

# ? May 19, 2015 03:22

prak: Jan 3, 2006; ---------; Nap Ghost

Tiger.Bomb posted:

I then rebooted in FreeNAS, and was planning on formatting the 1 + 2 as ZFS to copy back from the EXT4. Problem is that it couldn't read the GPT partition table on the 3TB. Played with it for a bit... and it turns out the drive failed!

Now I am in the tough spot of having only one copy of my data.

I have this computer, which does have ECC but is an i3. Why would they sell a server with ram that the processor doesn't support? According to this I should be OK.

What do you suggest I run, then? I was running a ubuntu desktop before, but like i said it started dying.

I could put ubuntu server or something on the new one, but ideally I'd like to boot from a usb key. Mostly because there are only three drive bays and I don't want to need to dedicate one for the OS.

There are some i3s that support it and since you bought a pre-built server chances are pretty good you do, in fact, have ECC support. :confuoot:

The i3 problem was more for people building their machines in parts and went for the cheaper version of the processor thinking they were just giving up speed and Intel having their documentation wrong.

If you are looking for a NAS setup: Packrats unite! The consumer NAS/storage megathread
The first post is almost three years out of date :eng99:

and the latest discussion is mostly about which hard drives to buy :homebrew:

however that would be where to start for advice getting a basic RAID setup going . Pretty much everything has a live disk option these days; you should be able to run your choice of OS from a thumb drive.

Without knowing more about what you plan on doing with the machine, what sort of data you have, how you value data integrity, and your general comfort level setting up and running this sort of thing it is difficult to make recommendations. There are a lot of options and all have major advantages and disadvantages. Additionally, you haven't mentioned new or replacement drives; those can heavily influence what options you have. If you are replacing the 3TB you could just go back to RAID 1 or three 3TB drives allows for RAID 5 (this might not be any better than RAID-Z1 for the risk of catastrophic data loss).

FreeNAS is still not a good option both because it only supports ZFS (and, barring more information, I still don't think you gain anything using it). It also looks like you have 4GB of memory and the minimum is 8GB.
NAS4Free supports UFS and software RAID with memory requirements of 1GB and under. It forked from FreeNAS a while back and is not focused on the enterprise market. While it also supports ZFS the same complications and downsides remain; ignore the warnings on that page.
unRAID is also something I've heard people using; that will run you $60.
NASLite for $30 and now I'm really just pulling things from the NAS thread.
Windows Home Server but those instructions almost certainly need updating.
LVM from whatever Linux distribution live disk you want. You were using it before, why change?

And, because you expressed concern about being down to a single copy of your data, RAID is not backup! There are a decent number of affordable online :yaycloud:

backup solutions. Three terabytes is small enough to be uploaded in not insane amounts of time. Unless you are in Australia or something. :australia:

# ? May 19, 2015 09:16

Tiger.Bomb: Jan 22, 2012

prak posted:

There are some i3s that support it and since you bought a pre-built server chances are pretty good you do, in fact, have ECC support. The i3 problem was more for people building their machines in parts and went for the cheaper version of the processor thinking they were just giving up speed and Intel having their documentation wrong.

If you are looking for a NAS setup: Packrats unite! The consumer NAS/storage megathread
The first post is almost three years out of date and the latest discussion is mostly about which hard drives to buy however that would be where to start for advice getting a basic RAID setup going . Pretty much everything has a live disk option these days; you should be able to run your choice of OS from a thumb drive.

Without knowing more about what you plan on doing with the machine, what sort of data you have, how you value data integrity, and your general comfort level setting up and running this sort of thing it is difficult to make recommendations. There are a lot of options and all have major advantages and disadvantages. Additionally, you haven't mentioned new or replacement drives; those can heavily influence what options you have. If you are replacing the 3TB you could just go back to RAID 1 or three 3TB drives allows for RAID 5 (this might not be any better than RAID-Z1 for the risk of catastrophic data loss).

FreeNAS is still not a good option both because it only supports ZFS (and, barring more information, I still don't think you gain anything using it). It also looks like you have 4GB of memory and the minimum is 8GB.

NAS4Free supports UFS and software RAID with memory requirements of 1GB and under. It forked from FreeNAS a while back and is not focused on the enterprise market. While it also supports ZFS the same complications and downsides remain; ignore the warnings on that page.

unRAID is also something I've heard people using; that will run you $60.

NASLite for $30 and now I'm really just pulling things from the NAS thread.

Windows Home Server but those instructions almost certainly need updating.

LVM from whatever Linux distribution live disk you want. You were using it before, why change?

And, because you expressed concern about being down to a single copy of your data, RAID is not backup! There are a decent number of affordable online backup solutions. Three terabytes is small enough to be uploaded in not insane amounts of time. Unless you are in Australia or something.

Thanks a lot for the help! I have also read through (and posted in) the packrats thread. I wish the OP was up-to-date.

I was considering nas4free, but they are pretty explicit about it not being for anything other than NAS, while FreeNAS has plugins for things like sickbeard, plex, sabnzbd, etc. I also think (but cannot find at the moment) that nas4free said that it required minimum 8GB as well.

My needs are pretty simple: I have under 3TB of media. TV, Movies, photos, and music. I was sharing it with SMB before to various Kodi/Plex front ends, but I also had the PC connected directly to a TV for another Kodi front end. I could ditch the last part, but it's important that this machine can do the downloading for me.

Since ZFS doesn't seem appropriate for my needs, I think I am still leading towards figuring out how to get ubuntu running from USB (My only concern is that I'd probably want to cut down on a lot of the logging and such to extend the life of the usb drive).

# ? May 19, 2015 20:50

prak: Jan 3, 2006; ---------; Nap Ghost

Tiger.Bomb posted:

My needs are pretty simple: I have under 3TB of media. TV, Movies, photos, and music. I was sharing it with SMB before to various Kodi/Plex front ends, but I also had the PC connected directly to a TV for another Kodi front end. I could ditch the last part, but it's important that this machine can do the downloading for me.

Since ZFS doesn't seem appropriate for my needs, I think I am still leading towards figuring out how to get ubuntu running from USB (My only concern is that I'd probably want to cut down on a lot of the logging and such to extend the life of the usb drive).

Kodi has an Ubuntu live disk: http://kodi.wiki/view/kodibuntu. The Inspect Your Gadgets forum appears to be where HTPC setups are discussed. Not sure if you have been in there but there are a few threads.

Wear leveling should put the point at which the drive fails from overuse well past something else failing or being replaced (so long as you have a relatively modern USB drive). You are much more likely to have a mechanical failure in a hard drive before hitting flash write limits.

Here is a guy in 2008 that set out to kill a 1GB drive. You would have to go out of you way to turn on excessive logging to replicate that and current drives are much larger and have higher write limits. Even then it looks like it just the worst that would happen is your logs wouldn't be updated anymore.

# ? May 19, 2015 22:58

Adbot: ADBOT LOVES YOU

# ? May 3, 2024 15:09

Tiger.Bomb: Jan 22, 2012

prak posted:

Kodi has an Ubuntu live disk: http://kodi.wiki/view/kodibuntu. The Inspect Your Gadgets forum appears to be where HTPC setups are discussed. Not sure if you have been in there but there are a few threads.

Wear leveling should put the point at which the drive fails from overuse well past something else failing or being replaced (so long as you have a relatively modern USB drive). You are much more likely to have a mechanical failure in a hard drive before hitting flash write limits.

Here is a guy in 2008 that set out to kill a 1GB drive. You would have to go out of you way to turn on excessive logging to replicate that and current drives are much larger and have higher write limits. Even then it looks like it just the worst that would happen is your logs wouldn't be updated anymore.

That's awesome. I expected the life to be a lot shorter. I am making a bootable usb using the pendrivelinux installer as we speak. It's a brand new usb 3 16GB, so I am thinking/hoping I'll have a decent lifetime on it.

Thanks again for the help.

# ? May 20, 2015 03:06

The Something Awful Forums > Discussion > Serious Hardware/Software Crap > Haus of Tech Support > Converting LVM array to ZFS to migrate to FreeNAS