Register a SA Forums Account here!
JOINING THE SA FORUMS WILL REMOVE THIS BIG AD, THE ANNOYING UNDERLINED ADS, AND STUPID INTERSTITIAL ADS!!!

You can: log in, read the tech support FAQ, or request your lost password. This dumb message (and those ads) will appear on every screen until you register! Get rid of this crap by registering your own SA Forums Account and joining roughly 150,000 Goons, for the one-time price of $9.95! We charge money because it costs us money per month for bills, and since we don't believe in showing ads to our users, we try to make the money back through forum registrations.
 
  • Post
  • Reply
Theophany
Jul 22, 2014

SUCCHIAMI IL MIO CAZZO DA DIETRO, RANA RAGAZZO



2022 FIA Formula 1 WDC

Chimp_On_Stilts posted:

Do y'all do cloud backups for large amounts of data from your NAS? (Like, 30TB+)

I've been pricing options, even building my own thing by just buying raw storage on Google Cloud or whatever, and it's all pretty expensive (which isn't a surprise).

I'm guessing most NAS owners probably don't have an offsite backup and are thus vulnerable to their house burning down or whatever. Are there good, affordable options for very large amounts of data?

Depends on your definition of 'cheap.' I've set up synology boxes at my parents house and my brother's house. The deal being that they get access to my Plex library on a NAS they can use and in return, the stuff I genuinely cannot afford to lose is backed up on the synos.

Naturally that will probably be a big outlay for 30TB of data unless you're happy to use ex-datacentre drives, but the 'upside' of the big initial outlay is no monthly fees and, depending upon physical distance/internet speed you're assured of a swift data recovery if you need it.

Adbot
ADBOT LOVES YOU

Thanks Ants
May 21, 2004

#essereFerrari


Are they all on beefy fibre lines or did you do the initial sync with the box locally

HalloKitty
Sep 30, 2005

Adjust the bass and let the Alpine blast
Anyone have experience with on-prem object storage?
I'm thinking Dell ECS, Scality Artesca/Ring

Ideally a hardware appliance I can just deploy and scale. Interested in hearing quotes people have had, just out of curiosity. It'll be for backup.

Edit: oh, and I should include Cohesity here too

evil_bunnY
Apr 2, 2003

Chimp_On_Stilts posted:

Do y'all do cloud backups for large amounts of data from your NAS? (Like, 30TB+)
Nah, backup to a friend's house, and vice-versa

evil_bunnY
Apr 2, 2003

HalloKitty posted:

Anyone have experience with on-prem object storage?
I'm thinking Dell ECS, Scality Artesca/Ring

Ideally a hardware appliance I can just deploy and scale. Interested in hearing quotes people have had, just out of curiosity. It'll be for backup.

Edit: oh, and I should include Cohesity here too

https://forums.somethingawful.com/showthread.php?threadid=2943669

HalloKitty
Sep 30, 2005

Adjust the bass and let the Alpine blast

Whoops, I ended up in the wrong thread. Cheers.

Theophany
Jul 22, 2014

SUCCHIAMI IL MIO CAZZO DA DIETRO, RANA RAGAZZO



2022 FIA Formula 1 WDC

Thanks Ants posted:

Are they all on beefy fibre lines or did you do the initial sync with the box locally

The initial backup was all over LAN, but their internet connections are fine for the regular sync work. One is a 300/10Mbps Starlink and the other is garden variety copper at 50/5Mbps so they can fully saturate my gigabit line's upload speed on the odd occasion something big is added to my 'cannot lose these' files like importing a bunch of raw photographs.

Flyndre
Sep 6, 2009
Is there some way to spin up a windows VM on my Synology and connect my TV to it (maybe using my Apple TV)? It would be nice to have a desktop environment to stream things from a regular browser (for instance YouTube with the SponsorBlock extension).

BlankSystemDaemon
Mar 13, 2009



Does anyone have experience with QuTS Hero? It's apparently a ZFS based appliance OS for QNAP systems with 8GB RAM or more.

Computer viking
May 30, 2011
Now with less breakage.

A linux VM may be easier just because they tend to have the right drivers out of the box and no license requirements. Beyond that, it looks like Synology has a VM manager, so give it a shot?

The hard part is going to be the streaming - we solved it here by getting one of those mini-PCs and hooking it up instead of finding out how you'd do that.

Flyndre
Sep 6, 2009
Yeah it seems like it’s mainly the streaming part that’s the issue. I’d love to not have to get another box to connect to the TV since I already have the Apple TV and the NAS right next to it though

The Gunslinger
Jul 24, 2004

Do not forget the face of your father.
Fun Shoe

Nitrousoxide posted:

I don't really see why anyone would use Unraid if you have even a modest level of technical acumen. TrueNAS is free, doesn't have any of that license bullshit, has a better file system, and if you really want to run containers on the same device you're using as a NAS you can use its built-in container engine (with scale) or spin up a vm (both core and scale) and use any server OS you want.

I've worked with FreeBSD for a good chunk of my life and I just don't want to deal with it at home. ZFS is insane overkill for Linux ISOs, I don't care about data integrity for a media server. I just run a cron job that dumps a directory listing of my shares once a month and emails it. I work nutty hours and want minimal computer janitoring which unRAID is fantastic for. If I need the CLI it's there but most of the time I never need it, the GUI and community support is excellent. I have a bunch of mixed 10-14TB drives and parity calcs take hilariously long but they're fine. I think I paid like $80 for it several years ago, the cost is a handwave.

Thanks Ants
May 21, 2004

#essereFerrari


Flyndre posted:

Yeah it seems like it’s mainly the streaming part that’s the issue. I’d love to not have to get another box to connect to the TV since I already have the Apple TV and the NAS right next to it though

Could you pass a USB DisplayLink device through to the VM? I don't know if a Synology device supports that.

hogofwar
Jun 25, 2011

'We've strayed into a zone with a high magical index,' he said. 'Don't ask me how. Once upon a time a really powerful magic field must have been generated here, and we're feeling the after-effects.'
'Precisely,' said a passing bush.

Flyndre posted:

Is there some way to spin up a windows VM on my Synology and connect my TV to it (maybe using my Apple TV)? It would be nice to have a desktop environment to stream things from a regular browser (for instance YouTube with the SponsorBlock extension).

For YouTube it might be better to get an android TV box and use SmartTube? I guess there might possibly be an equivalent app for the apple TV, but I have nearly no knowledge about that platform.

Nitrousoxide
May 30, 2011

do not buy a oneplus phone



My personal recommendation is to get an Amazon fire stick. They are super cheap. Grab the "Downloader" app off the appstore on it and then use that to side load SmartTube. SmartTube has its own built-in updater so you don't need to worry about it getting out of date or something.

Moey
Oct 22, 2010

I LIKE TO MOVE IT

hogofwar posted:

For YouTube it might be better to get an android TV box and use SmartTube? I guess there might possibly be an equivalent app for the apple TV, but I have nearly no knowledge about that platform.

This.

Apple TV if that is your ecosystem.

Nvidia Shield TV if you need bitstreaming audio (Dolby TrueHD or DTS HD-MA), or prefer android.

Roku for dead simple.

withoutclass
Nov 6, 2007

Resist the siren call of rhinocerosness

College Slice
Someone pointed out that you can sign up for a YouTube family plan at $4/month if you VPN to Ukraine if you want to save some money/hastle. I did it last week and it works perfectly.

Keito
Jul 21, 2005

WHAT DO I CHOOSE ?
Paying for YouTube still doesn't get you SponsorBlock, and IMO using SmartTube/yt-dlp is honestly nicer and more ethical than abusing regional pricing.

HalloKitty
Sep 30, 2005

Adjust the bass and let the Alpine blast

Keito posted:

Paying for YouTube still doesn't get you SponsorBlock, and IMO using SmartTube/yt-dlp is honestly nicer and more ethical than abusing regional pricing.

I don't know about smarttube as I'd not consider using smart tv functions, but sponsorblock is boss. Blocking all the ads is free and gets you a better experience than paying.

mawarannahr
May 21, 2019

I have been tasked with coming up with a strategy to back up ~ 50 tb from one server to another (on the same intranet).

  • source is one directory and everything under it, stored on a Linux server in a GPFS filesystem
  • backup target is a SMB mount to a Windows machine (potential filesystem attribute/permission loss/symlink issues?)
  • preserving symlinks (or having a way to represent and restore them) is important
  • a full restore is extremely unlikely
  • ability to restore a particular file or directory is very likely
  • it's not crucial that the exact directory structure and individual files are mirrored to the SMB mount (i.e., it's ok if the backup solution stores blobs that are opaque to the target filesystem)
  • probably want this to run at least every 2 weeks
  • want this to be incremental because 44 TiB is a lot

we also want to prune the files kept on the server by access timestamp so we no longer have to keep 44 TiB in the directory. this might be a little separate from backing up, cause we don't want to remove these files from the backups, unless there is an append-only solution?

also, for pruning little-used files, is there a way to keep around a "ghost" of the file (kind of like in the Dropbox selective sync ui) in the source filesystem that indicates it was there but is now backed up?

the backup will be backed up regularly offsite by the admin of the windows server, so i am off the hook there.

i need to find some software and write some scripts to get this going. i've used rsync, borg, and timeshift before, but never on such a scale.

thank you for any pointers/symlinks/junctions.

Falcon2001
Oct 10, 2004

Eat your hamburgers, Apollo.
Pillbug

Chimp_On_Stilts posted:

Do y'all do cloud backups for large amounts of data from your NAS? (Like, 30TB+)

I've been pricing options, even building my own thing by just buying raw storage on Google Cloud or whatever, and it's all pretty expensive (which isn't a surprise).

I'm guessing most NAS owners probably don't have an offsite backup and are thus vulnerable to their house burning down or whatever. Are there good, affordable options for very large amounts of data?

Surprised this didn't get an answer, but Amazon S3 Glacier is basically dirt cheap offsite storage, and is basically what I'm probably going to use. Their deep storage (which is hard to retrieve but for most personal backups that shouldn't be a regular concern) would be around 30 bucks a month for 30 TB, give or take. https://aws.amazon.com/s3/pricing/. I'm not sure if anyone else can match that price, but I'll look around myself. Whenever I finally get around to setting up my offsite setup I'll post in here with details.

That being said, the answers you did get (don't bother backing up poo poo you can get again like movies/etc) are also good, because 30 bucks a month is still a few hundred a year and why bother backing something up you can either stream or download again.

Alternatives to a cloud storage provider would be things like 'having a set of drives in a safety deposit box/etc' but honestly like...none of those are super great options.

Wiggly Wayne DDS
Sep 11, 2010



BlankSystemDaemon posted:

Does anyone have experience with QuTS Hero? It's apparently a ZFS based appliance OS for QNAP systems with 8GB RAM or more.
yeah works for my use case

Twerk from Home
Jan 17, 2009

This avatar brought to you by the 'save our dead gay forums' foundation.
I just can't take A NAP seriously after seeing that their marketing made a Btrfs hate page: https://www.qnap.com/solution/qnap-ext4/en/

Twerk from Home
Jan 17, 2009

This avatar brought to you by the 'save our dead gay forums' foundation.

Twerk from Home posted:

I just can't take QNAP seriously after seeing that their marketing made a Btrfs hate page: https://www.qnap.com/solution/qnap-ext4/en/

Edit: gently caress, quote isn't edit. I'm leaving the original now. I guess I needed A NAP yesterday.

Thanks Ants
May 21, 2004

#essereFerrari


Their file system speed comparison is an 8 core Ryzen vs. a Synology dual core Pentium. If the argument being made is "QNAP gives you more performance for the same budget" then maybe that's a fair comparison, otherwise lol.

Clark Nova
Jul 18, 2004

they're cowards and bullies for picking on poor little btrfs instead of going after zfs

BlankSystemDaemon
Mar 13, 2009



Clark Nova posted:

they're cowards and bullies for picking on poor little btrfs instead of going after zfs
It's especially funny because it talks about the benefits of ext4, meanwhile it's been having a bit of a fun time recently too.

Salt Fish
Sep 11, 2003

Cybernetic Crumb
I'm on Fedora and I know I'm going to want to reinstall the OS every few years. I have four 8tb platter drives and my plan right now is to build a RAID10 with md. I'm pausing to check though, because when I move to the next Fedora I want to be absolutely sure that I can rebuild the array afterwards. Is there anything I should know before I start?

Kibner
Oct 21, 2008

Acguy Supremacy
It's only been a couple weeks, but I am liking TrueNAS Scale, so far. I still need to learn how to create my own TrueChart via K8/Kubernetes or whatever, but I think I have a handle on most every other feature that I am likely to use.

BlankSystemDaemon
Mar 13, 2009



Salt Fish posted:

I'm on Fedora and I know I'm going to want to reinstall the OS every few years. I have four 8tb platter drives and my plan right now is to build a RAID10 with md. I'm pausing to check though, because when I move to the next Fedora I want to be absolutely sure that I can rebuild the array afterwards. Is there anything I should know before I start?
Why would you need to reinstall the OS every couple of years?
The installation on my laptop started out life as FreeBSD 6.0-RELEASE on a desktop with a Intel Pentium 4 Prescott E0, and it's been on at least two completely distinct systems in between then.

I know it used to be a thing with Windows, where you'd need to reinstall every 6 months to avoid some slow-down, because that's the reason why I switched to FreeBSD in 2000 to begin with - but I find it hard to believe that it's true for anything not-Windows, and especially in the year 2023.

First of May
May 1, 2017
🎵 Bring your favorite lady, or at least your favorite lay! 🎵


My Synology 1515+ has recently started rebooting itself randomly, and after sending the logs to support they said "motherboard problem, you're out of warranty, soz."

I'd rather not spend $900 on a new bay right now. With "automatically restart after power supply issue" turned off, it does not turn it self back on which indicates a power supply issue, so I'm going to attempt to replace it. Has anyone done this before and have any advise?

Volguus
Mar 3, 2009

Salt Fish posted:

I'm on Fedora and I know I'm going to want to reinstall the OS every few years. I have four 8tb platter drives and my plan right now is to build a RAID10 with md. I'm pausing to check though, because when I move to the next Fedora I want to be absolutely sure that I can rebuild the array afterwards. Is there anything I should know before I start?

On my personal computer I had fedora going for more than a decade, just upgrading from release to release until I tried out Arch to just come back to fedora in few months. The current installation has been going for 4 years now. Again, just upgrading. You shouldn't need to reinstall, pretty much never, baring any critical, OS-killing bugs, of which I haven't seen yet in Fedora at all.

Falcon2001
Oct 10, 2004

Eat your hamburgers, Apollo.
Pillbug
Yeah I'd say if you're going to bother reinstalling your OS regularly you should be looking at something like Ansible / Docker / etc to make sure you're forcing yourself to be ready, but I haven't actually done any hardware RAID support for something like that.

Quixzlizx
Jan 7, 2007
With Synology's settings, if I want to prioritize the longevity of the drives, should I be trying to keep the drives active whenever possible, or spun down whenever possible?

Also, the new 18TB Easystore I'm using to back up the array sounds like grinding gears when it spins up. I was pretty disturbed when I first set it up until I googled about it and found videos of other people asking on Reddit about identical noises, haha.

Nitrousoxide
May 30, 2011

do not buy a oneplus phone



Other the the initial distrohopping I did when I first picked up Linux I've stuck with Fedora Workstation since -- for the last 3ish years.

If I were to need to reinstall now though it'd be to install silverblue.

Combat Pretzel
Jun 23, 2004

No, seriously... what kurds?!
There's sure some BRT related fixes going into OpenZFS right now.

Harik
Sep 9, 2001

From the hard streets of Moscow
First dog to touch the stars


Plaster Town Cop
Some of the 25TB seagate EXOS drives have been leaking out of the hyperscaler market to the secondary. They're host-managed SMR, not available for consumers and don't work with pretty much everything except some specific highend SAS chips.

... and if you use a normal filesystem on them they melt down instantly. 2 IOPS for random writes. Not a typo. 2. Then the storage stack crashes.

Kinda tempted get some to throw logfs on. They have incredible random read IOPS for spinning rust - nearly 10x what a 20tb CMR drive can do. For something that gets laid down once and not edited (linux ISOs, pictures, blob-based backup systems, etc) they'd probably work pretty well.

El Mero Mero
Oct 13, 2001

So on my synology I have 1 bay reserved for a SSD, which is where I run my apps and plex off of. That drive is starting to get full and I want to upgrade it, but because it's not a pool I'm going to have to copy everything to the new drive.

Is there a good/straightforward way to do this? Right now I think my plan is to manually move the volume storage location of the app data to my HDD volume, do a backup, and then restore from backup to the new drive and re-point everything over to the new SSD.

Twerk from Home
Jan 17, 2009

This avatar brought to you by the 'save our dead gay forums' foundation.

Harik posted:

Some of the 25TB seagate EXOS drives have been leaking out of the hyperscaler market to the secondary. They're host-managed SMR, not available for consumers and don't work with pretty much everything except some specific highend SAS chips.

... and if you use a normal filesystem on them they melt down instantly. 2 IOPS for random writes. Not a typo. 2. Then the storage stack crashes.

Kinda tempted get some to throw logfs on. They have incredible random read IOPS for spinning rust - nearly 10x what a 20tb CMR drive can do. For something that gets laid down once and not edited (linux ISOs, pictures, blob-based backup systems, etc) they'd probably work pretty well.

Apparently if you want to play with these, btrfs supports them too: https://www.chia.net/2022/02/22/zoned-storage-using-smr-drives-for-chia-farming/

Adbot
ADBOT LOVES YOU

Harik
Sep 9, 2001

From the hard streets of Moscow
First dog to touch the stars


Plaster Town Cop

Twerk from Home posted:

Apparently if you want to play with these, btrfs supports them too: https://www.chia.net/2022/02/22/zoned-storage-using-smr-drives-for-chia-farming/

I'd prefer to move away from btrfs, especially for multi-drive applications. But yes, COW filesystems are in a good place to take advantage of HM-SMR as long as their allocation rules are done correctly.

chia, aside from being a shitcoin scam, is also a WORM system that recommends you mount your filesystem read-only. I wouldn't expect it to get great IOPS on HM-SMR because it's Q1 random reads - chia is an enormous game of pointer chasing, so every read depends on the result of the read before it. I suppose you could be chasing 32 pointer chains at a time to keep the queue depth up but I've already blown way past the amount of fucks I give for scams to turn money into wasted hardware and energy.

That doesn't translate well into real-world use where you've got a lot of metadata that needs to be updated for every write. In a filesystem where I trust multi-drive redundancy (i.e. not btrfs) you could put the metadata on flash or optane and the data on hm-smr.

e: interposing dm-zoned on host-managed or host-aware smr apparently solves a lot of the issues. Does nothing for stealth SMR drives but those are just ewaste.

Harik fucked around with this message at 19:23 on Dec 15, 2023

  • 1
  • 2
  • 3
  • 4
  • 5
  • Post
  • Reply