NAS/Storage Megathread: What is this "File Deletion" You Speak of

The Something Awful Forums > Discussion > Serious Hardware/Software Crap > NAS/Storage Megathread: What is this "File Deletion" You Speak of

«‹›847 »

Combat Pretzel: Jun 23, 2004; No, seriously... what kurds?!

I'm using it on TrueNAS Scale, because they initially decided on it (altho it'll go away with something called overlayfs, which incidentally did work for ZFS support, or the other way around). I don't see what the problem is. Everything is zippy.

# ? Jul 7, 2023 17:43

Adbot: ADBOT LOVES YOU

# ? May 29, 2024 11:42

H2SO4: Sep 11, 2001; put your money in a log cabin; Buglord

The docker zfs storage driver stores every layer of every image as its own dataset which would probably be really elegant if you designed your stack with that in mind and really lovely if not (the latter of course being the default).

OpenZFS has overlay2 support now so the defaults work fine even when pointed at zfs storage, even though you aren't taking advantage of all the native zfs fun that you could be. That said, some have reported that the changes made to support overlay2 also helped out the native zfs storage driver - long story short a lot of docker/zfs content out there seems to be stale.

H2SO4 fucked around with this message at 18:06 on Jul 7, 2023

# ? Jul 7, 2023 17:43

ILikeVoltron: May 17, 2003; I <3 spyderbyte!

I had some experience using the zfs docker driver and it was a complete shitshow. No documentation about what it expects to do, hardly anything was explained but you'd see dataset after dataset being used with snapshots and all sorts of cruft. It would eventually get so massive that the whole thing would fall over and never start. Imagine 100 docker images with all their layers each representing a item found in `zfs list pool -d 2 -t snap`

# ? Jul 7, 2023 17:58

Hughlander: May 11, 2005

All that. However using the zfs drivers for named volumes is easy and good.

# ? Jul 7, 2023 18:18

Wibla: Feb 16, 2011

Fileserver power consumption update - I got some more SATA cables and managed to ditch my SAS HBA ( LSI 2008 ) and also replaced an ancient, noisy Corsair TX750 with a 2021 Corsair RM750x at the same time. I'm pretty sure the fan in that TX750 is toast, all the vibrations from the machine are now gone, and I can only hear a very slight whirr from the drives if the room is otherwise quiet.

Idle power consumption decreased from 132-134 W to 110-114W, that's with an E5-2670 v3, 128GB DDR4 ECC, Nvidia P400, intel X520 10 gbit NIC (w/10gbit DAC), 8x14TB shucked WD drives and 2 SATA SSDs.

I'm fairly happy with that result. I (still) don't use the P400 for anything, so I could probably just yank it out, but I have been meaning to get plex running again, so it stays for now.

# ? Jul 7, 2023 18:27

KKKLIP ART: Sep 3, 2004

The damndest thing happened to my TrueNAS Scale box... and it feels like a foot-in-mouth situation after saying I had good experience with TrueCharts.

We went out of town for a few days and the power must have tripped, my NAS restarted itself but none of the apps were working. I restarted it manually and told the apps to refresh and we were back in business except not. For some reason, they all got new IP addresses. Before they were 192.168.x.120 (same as the TrueNAS box) but now they are .124. This motherboard has 4 NICS, which are all plugged and one of them is assigned to .124, but it is just so bizzare that they decided to get a new IP. I have done manual resets in the past and never saw that happen.

I bonded the NICs and it seems the hard reset unbonded them. Once I did that, it popped to the correct .120 IP address and all is working :iiam:

KKKLIP ART fucked around with this message at 17:14 on Jul 8, 2023

# ? Jul 8, 2023 17:06

Takes No Damage: Nov 20, 2004; The most merciful thing in the world, I think, is the inability of the human mind to correlate all its contents. We live on a placid island of ignorance in the midst of black seas of infinity, and it was not meant that we should voyage far.; Grimey Drawer

Is there any sort of network config or bandwidth limit that can be configured in TrueNAS? I'm syncing things up to Backblaze and no matter what I set the BW limit in the Task to, I can't get an upload speed above 1.1 meg/sec. Speedtests from my PC peg my upload at 9.8 meg, expected since my ISP is claiming 10. Could it be just a B2 thing or is something else choking it down?

# ? Jul 8, 2023 23:23

withoutclass: Nov 6, 2007; Resist the siren call of rhinocerosness; College Slice

Takes No Damage posted:

Is there any sort of network config or bandwidth limit that can be configured in TrueNAS? I'm syncing things up to Backblaze and no matter what I set the BW limit in the Task to, I can't get an upload speed above 1.1 meg/sec. Speedtests from my PC peg my upload at 9.8 meg, expected since my ISP is claiming 10. Could it be just a B2 thing or is something else choking it down?

IIRC backblaze heavily limits bandwidth.

# ? Jul 8, 2023 23:37

Paul MaudDib: May 3, 2006; TEAM NVIDIA:
FORUM POLICE

try using rclone and tuning the max chunks parameter (I think - see "rclone help flags"). It may be that backblaze limits per-thread bandwidth but you can use multiple chunks in a single file/upload multiple files.

# ? Jul 8, 2023 23:40

BlankSystemDaemon: Mar 13, 2009

On FreeBSD 14-CURRENT (I don't know if it's been MFC'd yet), it's not just ipfw(4) that's capable of using dummynet(4) - pf(4) is also capable of it.

This lets you do not schedule packets (ie. implement weighted fair queueing, as well as flowcontrol-codel and flowcontrol-pie, with the latter to being known for being good at dealing with bufferbloat), it also lets you set cumulative probabilistic curves for delay (so that you can emulate anything from fiber networks to satellite and even earth-to-moon communication with or without packet-loss), as well as implement arbitrary bandwidth limitations based on source-/destination- ip/port/protocol.

Whether TrueNAS can do this, I've no idea - doesn't seem likely, since it's a filesharing appliance.

# ? Jul 9, 2023 00:57

Windows 98: Nov 13, 2005; HTTP 400: Bad post

ok so my fancy shelves and disks came in and I am trying to get everything connected, and of course nothing ever works! I am trying to get 2 NetAPP DS4246 shelves connected to a PMC-Sierra Inc. PM8072 Tachyon SPCv HBA controller. I have verified the the HBA card is being detected by TrueNAS, I can see it in the CLI. I have also verified the HDDs are not being detected (sudo fdisk -l). My cabling is as follows

code:

IOMA Shelf 1 Circle -> IOMA Shelf 2 Square
IOMB Shelf 1 Circle -> IOMB Shelf 2 Square
IOMA Shelf 1 Square -> HBA Card
IOMB Shelf 2 Circle -> HBA Card

No error lights. Everything green. Each shelf has a unique ID assigned to it on the little LED tab. Not really sure what to do. I have never connected shelves before

EDIT: Trying to troubleshoot I have tried to connect just a single shelf. To simply if further I have no redundant connections. I have both In/Out connections direct to the HBA on IOMA. Drives still not showing up.

Windows 98 fucked around with this message at 04:30 on Jul 9, 2023

# ? Jul 9, 2023 03:40

Takes No Damage: Nov 20, 2004; The most merciful thing in the world, I think, is the inability of the human mind to correlate all its contents. We live on a placid island of ignorance in the midst of black seas of infinity, and it was not meant that we should voyage far.; Grimey Drawer

withoutclass posted:

IIRC backblaze heavily limits bandwidth.

I did a search for B2 upload speeds and at least in 2019 people were seeing 100s to at least 30-40 Mbps.

Backblaze's own speedtest shows me significantly less upload speed, but still over 3 meg:

Paul MaudDib posted:

try using rclone and tuning the max chunks parameter (I think - see "rclone help flags"). It may be that backblaze limits per-thread bandwidth but you can use multiple chunks in a single file/upload multiple files.

rclone is what is built into TrueNAS under its Cloud Sync Task ability:

My chunk size is 96meg, the default setting. Right now I'm trying to get my music collection backed up, so almost every file is going to be smaller than 96. Transfers are currently set to 1 but I did have them at 2 before and I'm not noticing a difference in speeds. Suppose I can test it with 3 or 4 just to be sure, since as I'm reading further into it that should mean the server will try to upload 4 files that are all less than 96meg (or split a larger file into sections of 96 each).

# ? Jul 9, 2023 04:09

BlankSystemDaemon: Mar 13, 2009

Windows 98 posted:

ok so my fancy shelves and disks came in and I am trying to get everything connected, and of course nothing ever works! I am trying to get 2 NetAPP DS4246 shelves connected to a PMC-Sierra Inc. PM8072 Tachyon SPCv HBA controller. I have verified the the HBA card is being detected by TrueNAS, I can see it in the CLI. I have also verified the HDDs are not being detected (sudo fdisk -l). My cabling is as follows
code:
IOMA Shelf 1 Circle -> IOMA Shelf 2 Square
IOMB Shelf 1 Circle -> IOMB Shelf 2 Square
IOMA Shelf 1 Square -> HBA Card
IOMB Shelf 2 Circle -> HBA Card
No error lights. Everything green. Each shelf has a unique ID assigned to it on the little LED tab. Not really sure what to do. I have never connected shelves before

EDIT: Trying to troubleshoot I have tried to connect just a single shelf. To simply if further I have no redundant connections. I have both In/Out connections direct to the HBA on IOMA. Drives still not showing up.

I'm not familiar with Microsemi beyond the fact that they're the ODM that HPE uses for their RAID controllers.

As for your cabling, I'm not sure that amkes sense to me. None of the SFF cables involved in SAS should be circular.

The recommendation du jour has always been LSI/Avago/Broadcom HBAs flashed with IT mode firmware, as they're both highly available and cheap on the used market.

# ? Jul 9, 2023 11:11

Computer viking: May 30, 2011; Now with less breakage.

BlankSystemDaemon posted:

On FreeBSD 14-CURRENT (I don't know if it's been MFC'd yet), it's not just ipfw(4) that's capable of using dummynet(4) - pf(4) is also capable of it.

This lets you do not schedule packets (ie. implement weighted fair queueing, as well as flowcontrol-codel and flowcontrol-pie, with the latter to being known for being good at dealing with bufferbloat), it also lets you set cumulative probabilistic curves for delay (so that you can emulate anything from fiber networks to satellite and even earth-to-moon communication with or without packet-loss), as well as implement arbitrary bandwidth limitations based on source-/destination- ip/port/protocol.

Whether TrueNAS can do this, I've no idea - doesn't seem likely, since it's a filesharing appliance.

I think their question is in the other direction - "this is strangely slow, is there a throttling setting I may accidentally have enabled".
(But otherwise I just heard the BSD Now guys mention this - cool, and I'm glad pf keeps getting new features.)

And IIRC TrueNAS deliberately makes it hard to even enable a firewall, their stance is something like "we make a file server appliance, if you want a firewall then put it behind a firewall appliance".

# ? Jul 9, 2023 13:36

Wibla: Feb 16, 2011

Figured I would setup proxmox backup server on a VM on my Truenas Scale server since I have two proxmox servers running now, and I want better backups than just straight up vzdump.

Apparently you have to do a bunch of dumb poo poo to get guest to host connectivity on Truenas Scale via bridge. This poo poo is by NO means thought through properly by iX.

Time to downscale the specs of that machine, I don't need 12 cores and 128GB RAM to serve files, because that's all that machine is good for.

# ? Jul 9, 2023 14:32

Windows 98: Nov 13, 2005; HTTP 400: Bad post

BlankSystemDaemon posted:

I'm not familiar with Microsemi beyond the fact that they're the ODM that HPE uses for their RAID controllers.

As for your cabling, I'm not sure that amkes sense to me. None of the SFF cables involved in SAS should be circular.

The recommendation du jour has always been LSI/Avago/Broadcom HBAs flashed with IT mode firmware, as they're both highly available and cheap on the used market.

According to their docs and ChatGPT there should be a full loop happening

https://library.netapp.com/ecm/ecm_download_file/ECMP1115547

# ? Jul 9, 2023 14:41

BlankSystemDaemon: Mar 13, 2009

Windows 98 posted:

According to their docs and ChatGPT there should be a full loop happening

https://library.netapp.com/ecm/ecm_download_file/ECMP1115547

Pretty sure looping won't work if you're not using their controller software.

# ? Jul 9, 2023 14:43

Nitrousoxide: May 30, 2011; do not buy a oneplus phone

Wibla posted:

Figured I would setup proxmox backup server on a VM on my Truenas Scale server since I have two proxmox servers running now, and I want better backups than just straight up vzdump.

Apparently you have to do a bunch of dumb poo poo to get guest to host connectivity on Truenas Scale via bridge. This poo poo is by NO means thought through properly by iX.

Time to downscale the specs of that machine, I don't need 12 cores and 128GB RAM to serve files, because that's all that machine is good for.

You could install proxmox on the machine you currently have Scale on, pass through the drives to a Scale vm, and then you can scale (heh) the compute/memory resources you provision to it up and down. Adding another proxmox server node would also let you reach 3 for HA shenanigans if you ever want to gently caress around with that.

Edit: You could also setup a NFS share on your Scale NAS, mount that share on a Proxmox Backup VM and then direct your backups to that share. Your vm backups will then be managed by the Backup Server, but will sit on your NAS without the networking headache of Scale.

Plus you can send your VM backups offsite to some object storage solution for your 1-2-3 backup more easily.

Nitrousoxide fucked around with this message at 14:54 on Jul 9, 2023

# ? Jul 9, 2023 14:48

BlankSystemDaemon: Mar 13, 2009

You could also install single OS that does everything, instead of running multiple guest OS'.

# ? Jul 9, 2023 15:01

Wibla: Feb 16, 2011

Nitrousoxide posted:

You could install proxmox on the machine you currently have Scale on, pass through the drives to a Scale vm, and then you can scale (heh) the compute/memory resources you provision to it up and down. Adding another proxmox server node would also let you reach 3 for HA shenanigans if you ever want to gently caress around with that.

I've tried to do pass-through before, it did not end particularly well, so I'd rather not risk my data to a setup like that.

I've already tried to gently caress around with Proxmox HA and I am not touching that again until I have 3-4 identical nodes so I can do it right :v:

Nitrousoxide posted:

Edit: You could also setup a NFS share on your Scale NAS, mount that share on a Proxmox Backup VM and then direct your backups to that share. Your vm backups will then be managed by the Backup Server, but will sit on your NAS without the networking headache of Scale.

Plus you can send your VM backups offsite to some object storage solution for your 1-2-3 backup more easily.

Yeah, that's probably what I'll end up doing - both of my proxmox hosts have CPU and RAM to burn at any rate, so one of them can host PBS and just dump backups to the NAS via NFS.

BlankSystemDaemon posted:

You could also install single OS that does everything, instead of running multiple guest OS'.

Like?

# ? Jul 9, 2023 15:05

BlankSystemDaemon: Mar 13, 2009

Wibla posted:

Like?

Pick a Linux distro, or FreeBSD. :shrug:

# ? Jul 9, 2023 15:09

Combat Pretzel: Jun 23, 2004; No, seriously... what kurds?!

Proxmox does ZFS out of the box nowadays apparently?

# ? Jul 9, 2023 15:12

Nitrousoxide: May 30, 2011; do not buy a oneplus phone

BlankSystemDaemon posted:

You could also install single OS that does everything, instead of running multiple guest OS'.

I mean, I guess? But they would access to neato stuff like being able to easily perform daily image backups of both Truenas VM (I don't believe that proxmox will, by default, backup passed through drives, just the virtual drives so it shouldn't bloat the vm backup) and even backup the Proxmox Backup Server VM to a seperate data store. That way they can easily recover the Backup Server and recover the other VM backups from that with a couple of clicks in the event of a catastrophic failure on the node.

Wibla posted:

I've tried to do pass-through before, it did not end particularly well, so I'd rather not risk my data to a setup like that.

Hmm, unfortunate. Fair enough in wanting to simplify your setup though.

Wibla posted:

Yeah, that's probably what I'll end up doing - both of my proxmox hosts have CPU and RAM to burn at any rate, so one of them can host PBS and just dump backups to the NAS via NFS.

Just make sure to take backups of you PBS and keep those backups outside of the PBS server itself (like on a share directly mounted to the Proxmox nodes) it'll make recovering your backup server much easier if you don't have to manually reinstall it to regain access to its backups)

Nitrousoxide fucked around with this message at 15:16 on Jul 9, 2023

# ? Jul 9, 2023 15:13

Eletriarnation: Apr 6, 2005; People don't appreciate the substance of things...
objects in space.; Oven Wrangler

Wibla posted:

Apparently you have to do a bunch of dumb poo poo to get guest to host connectivity on Truenas Scale via bridge. This poo poo is by NO means thought through properly by iX.

Could you elaborate? I am running Scale and have a Fedora torrent VM using a bridge for host connectivity, and I don't recall doing anything especially complicated. The NAS side just looks like this:

I think on the VM side I just gave it two VirtIO interfaces from the jump, one for the internal bridge and one for a uplink to the web, and configured them both in initial installation. From shell history, it looks like the first thing I did on initial login was ping the NAS side of the bridge to make sure it worked already.

# ? Jul 9, 2023 15:47

Wibla: Feb 16, 2011

Eletriarnation posted:

Could you elaborate? I am running Scale and have a Fedora torrent VM using a bridge for host connectivity, and I don't recall doing anything especially complicated. The NAS side just looks like this:

I think on the VM side I just gave it two VirtIO interfaces from the jump, one for the internal bridge and one for a uplink to the web, and configured them both in initial installation. From shell history, it looks like the first thing I did on initial login was ping the NAS side of the bridge to make sure it worked already.

I'm not getting that option, but I am still on 22.12.0. I guess I should upgrade to the newest, though I am dreading having to deal with the dumb app poo poo that they tend to break on every upgrade because they have pants on head opinions about how apps should access storage.

Nitrousoxide posted:

Just make sure to take backups of you PBS and keep those backups outside of the PBS server itself (like on a share directly mounted to the Proxmox nodes) it'll make recovering your backup server much easier if you don't have to manually reinstall it to regain access to its backups)

That would be a given, yeah.

BlankSystemDaemon posted:

Pick a Linux distro, or FreeBSD.

To consolidate all the things, then? That's not something I want to do, for multiple reasons.

Proxomx 1 runs pfsense, an IRC vm, some other misc stuff that needs to be online (screenshot webhost etc). This is a pretty low-powered uSFF lenovo tiny with 4 cores, 16GB RAM and 256GB SSD
Proxmox 2 runs some linux VMs hosting game-related stuff that runs isolated on its own VLAN, behind haproxy on pfsense. This is a 3700x with 32GB ram and all SSD/NVMe storage. I could probably consolidate everything on host 1 onto this without much issue, but for now I want to keep them separated because one of them needs to work, the other one is more "nice to have".

Then there's the NAS, with an E5-2670 v3, 128GB ram and 8x14TB drives in raidz2 and some SSDs. For what should be obvious reasons I want to keep this separate from all the rest, and preferably only run things on it that touches storage, like a torrent client and PBS. I'm not opposed to yanking out 64GB ram and replacing the E5 v3 cpu with a lower core count v4 CPU though.

Wibla fucked around with this message at 16:05 on Jul 9, 2023

# ? Jul 9, 2023 16:03

Windows 98: Nov 13, 2005; HTTP 400: Bad post

Windows 98 posted:

ok so my fancy shelves and disks came in and I am trying to get everything connected, and of course nothing ever works! I am trying to get 2 NetAPP DS4246 shelves connected to a PMC-Sierra Inc. PM8072 Tachyon SPCv HBA controller. I have verified the the HBA card is being detected by TrueNAS, I can see it in the CLI. I have also verified the HDDs are not being detected (sudo fdisk -l). My cabling is as follows
code:
IOMA Shelf 1 Circle -> IOMA Shelf 2 Square
IOMB Shelf 1 Circle -> IOMB Shelf 2 Square
IOMA Shelf 1 Square -> HBA Card
IOMB Shelf 2 Circle -> HBA Card
No error lights. Everything green. Each shelf has a unique ID assigned to it on the little LED tab. Not really sure what to do. I have never connected shelves before

EDIT: Trying to troubleshoot I have tried to connect just a single shelf. To simply if further I have no redundant connections. I have both In/Out connections direct to the HBA on IOMA. Drives still not showing up.

Ok I have done a little more digging around testing. I can definitely confirm with 100% certainty that the HBA card (which is actually a NetApp X2069-R6) is visible in lspci, looks pretty normal with lspci -v -s 81:00.0, and is throwing no errors in dmesg. I can confirm that the DS4246 has full green lights on all the drive bays, no warning light, and has a unique ID on the LED. Neither the activity light for shelf or the LED for the port on the HBA are lighting up. Which has got me thinking that the HBA card is probably fine, and something is going wrong between my shelf and HBA card. I have tried just about every cabling configuration I can think of, including a pretty surefire Square IOMA -> HBA card (shelf output / hba input). Thoughts?

# ? Jul 9, 2023 16:34

Windows 98: Nov 13, 2005; HTTP 400: Bad post

BlankSystemDaemon posted:

The recommendation du jour has always been LSI/Avago/Broadcom HBAs flashed with IT mode firmware, as they're both highly available and cheap on the used market.

Circling back to this. I think this may be my issue? I went back to the documentation for the HBA card I got and noticed that there is a "Version of software supported" that is listed as "Data ONTAP 8.2.3 and later". Asked ChatGPT if that card is able to be used in any system outside of NetApp ONTAP and it seems to think it's a toss up and that the HBA card I have is technically not designed for random systems. I think in my effort to try and get everything to be NetApp/compatible I may have actually done the exact opposite. I feel like the only thing I haven't eliminated is card or firmware incompatibility.

So I did some more asking for a known-to-be-working HBA card that is compatible with the DS4246 shelf and TrueNAS. Picked up an LSI SAS9305-16e and the cables for it. Hopefully I haven't wasted $300 for nothing and this makes it work! Pray for me.

# ? Jul 10, 2023 01:52

IOwnCalculus: Apr 2, 2003

DS4246: What shows up on lsscsi output? Do you even see anything tagged with "enclosure" like this:

code:

$ lsscsi
[0:0:0:0]    disk    HGST     HUH728080AL4200  AT11  /dev/sde
...
[0:0:21:0]   disk    ATA      WDC WD100EMAZ-00 0A83  /dev/sdz
[0:0:22:0]   enclosu NETAPP   DS424-E6EBD      221a  -

I never tried to get mine working with the actual Netapp IOM6. I swapped that out for the Dell/Compellent 0952913-07 controller and have had it hooked up to two different LSI HBAs over the years. I also made no attempt at multi-pathing and have just a single 8088-8088 cable from my current 9217-4i4e to the "A" port on the Compellent controller. I also just have the single Compellent controller actually installed in the system; I think when I tried to boot it with two Compellent controllers it refused to work.

I did get a bug up my rear end reading the ServeTheHome forums and tried to update the firmware on the Compellent controller, and while it appears to have worked fine, it no longer sees any disks when using that controller. So I swapped it out for my spare and it went right back to normal operation. Not making that mistake again. I'd attempt reflashing it to whatever it had on it but I haven't been able to find that, and these things are at the rock bottom of the depreciation curve.

*It's not a controller, it's a glorified SAS expander, but Xyratex calls them controllers. Also I have no actual affiliation with this eBay seller but after my above-mentioned firmware fiasco I bought a spare from them and a couple of power supplies and have them sitting in my car for the next time I go visit my server.

# ? Jul 10, 2023 02:33

Hughlander: May 11, 2005

Combat Pretzel posted:

Proxmox does ZFS out of the box nowadays apparently?

For many many years. I started my NAS as ESXI with FreeNAS passthrough and in 2015(?) or so just popped a new USB in with Proxmox and imported the ZFS vdevs from freenas.

# ? Jul 10, 2023 03:55

BlankSystemDaemon: Mar 13, 2009

Windows 98 posted:

Circling back to this. I think this may be my issue? I went back to the documentation for the HBA card I got and noticed that there is a "Version of software supported" that is listed as "Data ONTAP 8.2.3 and later". Asked ChatGPT if that card is able to be used in any system outside of NetApp ONTAP and it seems to think it's a toss up and that the HBA card I have is technically not designed for random systems. I think in my effort to try and get everything to be NetApp/compatible I may have actually done the exact opposite. I feel like the only thing I haven't eliminated is card or firmware incompatibility.

So I did some more asking for a known-to-be-working HBA card that is compatible with the DS4246 shelf and TrueNAS. Picked up an LSI SAS9305-16e and the cables for it. Hopefully I haven't wasted $300 for nothing and this makes it work! Pray for me.

That sounds quite likely to be the case; a real NetApp SAN would have racks full of everything explicitly NetApp.

Hughlander posted:

For many many years. I started my NAS as ESXI with FreeNAS passthrough and in 2015(?) or so just popped a new USB in with Proxmox and imported the ZFS vdevs from freenas.

That's more to the credit of OpenZFS' commitment to portability, as it's heading to both macOS and Windows, and should hopefully land on NetBSD and Illumos-derivatives at some point too.

# ? Jul 10, 2023 16:07

RoboBoogie: Sep 18, 2008

A year ago I bought 3 x 14 tb easy stores and now I just ordered 3x18 tb. Once the 18TBs arrive I�m going to decommission my desktop and turn it into a unraid box.

Goal is to move all the content from my google drive (48TB). I would like to find a cloud storage solution that I can back up the NAS on. Would it be possible to use the $7 black blaze option?

# ? Jul 12, 2023 04:18

yoloer420: May 19, 2006

RoboBoogie posted:

A year ago I bought 3 x 14 tb easy stores and now I just ordered 3x18 tb. Once the 18TBs arrive I’m going to decommission my desktop and turn it into a unraid box.

Goal is to move all the content from my google drive (48TB). I would like to find a cloud storage solution that I can back up the NAS on. Would it be possible to use the $7 black blaze option?

The Backblaze unlimited options require use of their client and it will refuse to run on "server" or "NAS" operating systems. It will also refuse to back up any network shares etc (this being the obvious work around).

Crashplan small business backup will work, *but* you will face memory issues as the client software you're forced to use is garbage and will consume huge amounts of memory. The guideline at the time was 1GB of memory per 1TB in the backup set. They have no interest in fixing this as it is a convenient way to remain unlimited while making large backup sets impractical.

Here (warning: Reddit) is a customer reporting that Crashplan has an undisclosed 10TB limit.

yoloer420 fucked around with this message at 04:42 on Jul 12, 2023

# ? Jul 12, 2023 04:40

IOwnCalculus: Apr 2, 2003

I definitely had more than 10TB in Crashplan at one point but holy gently caress their backup client was rear end. I don't think I ever tried more than a partial restore and it loving sucked. Reading that post I wonder if it may well have been impossible to properly restore it.

There is no sane solution out there to back up an entire array of Linux ISOs. Recognize that the ease with which you can re-acquire said ISOs is the real backup for the vast majority of that.

If you truly have dozens of terabytes of irreplaceable family photos, open that wallet.

# ? Jul 12, 2023 05:20

Splinter: Jul 4, 2003; Cowabunga!

RoboBoogie posted:

A year ago I bought 3 x 14 tb easy stores and now I just ordered 3x18 tb. Once the 18TBs arrive I’m going to decommission my desktop and turn it into a unraid box.

Goal is to move all the content from my google drive (48TB). I would like to find a cloud storage solution that I can back up the NAS on. Would it be possible to use the $7 black blaze option?

I believe you could do that on the unraid server via a Windows VM (to run the unlimited client) if you mirrored everything you wanted to backup to external drives (or more realistically USB JBOD enclosures w multiple drives). Now buying another ~$1k+ in drives to make that work just to pay unlimited pricing over more expensive plans might be hard to justify, but it does at least give you another local copy of your data (which some recommend).

# ? Jul 12, 2023 07:23

Don Dongington: Sep 27, 2005; #ideasboom; College Slice

Any OS recommendations for a small form factor desktop pc with a single large capacity drive/maybe an ssd for OS, to store a bunch of media and host a handful of docker containers to manage it + plex?

Was running scale previously but no longer see the need for redundancy, and understand that ZFS is bad for shingled hdds like the one I'm planning on using.

# ? Jul 12, 2023 08:10

BlankSystemDaemon: Mar 13, 2009

You could also just not use shingled drives.

# ? Jul 12, 2023 08:50

HalloKitty: Sep 30, 2005; Adjust the bass and let the Alpine blast

BlankSystemDaemon posted:

You could also just not use shingled drives.

Excellent advice.

# ? Jul 12, 2023 08:56

Theophany: Jul 22, 2014; SUCCHIAMI IL MIO CAZZO DA DIETRO, RANA RAGAZZO

2022 FIA Formula 1 WDC

Splinter posted:

I believe you could do that on the unraid server via a Windows VM (to run the unlimited client) if you mirrored everything you wanted to backup to external drives (or more realistically USB JBOD enclosures w multiple drives). Now buying another ~$1k+ in drives to make that work just to pay unlimited pricing over more expensive plans might be hard to justify, but it does at least give you another local copy of your data (which some recommend).

Could you not use virtiofs to emulate a network share as a physical disk within the VM, saving on buying a bunch of additional disks?

# ? Jul 12, 2023 09:40

That Works: Jul 22, 2006; Every revolution evaporates and leaves behind only the slime of a new bureaucracy

Whats a shingled drive? New term for me.

# ? Jul 12, 2023 14:28

Adbot: ADBOT LOVES YOU

# ? May 29, 2024 11:42

Less Fat Luke: May 23, 2003; Exciting Lemon

It's a type of drive technology known as shingled magnetic resonance drives (SMR) where instead of regular streams of liner data things are written in gigantic shingles on the drive. This means that if you're writing a stream of linear data at drive speed and the drive is empty things are pretty fast, but when you're doing random IO or things like rebuilding a RAID device the drive has to first read a shingle and then write the whole thing to update like 1% of it.

There was a lot of drama because manufacturers started making this to save money but you know, didn't tell anyone their drives were SMR and they weren't cheaper in anyway LOL.

# ? Jul 12, 2023 14:31

The Something Awful Forums > Discussion > Serious Hardware/Software Crap > NAS/Storage Megathread: What is this "File Deletion" You Speak of

«‹›847 »