Register a SA Forums Account here!
JOINING THE SA FORUMS WILL REMOVE THIS BIG AD, THE ANNOYING UNDERLINED ADS, AND STUPID INTERSTITIAL ADS!!!

You can: log in, read the tech support FAQ, or request your lost password. This dumb message (and those ads) will appear on every screen until you register! Get rid of this crap by registering your own SA Forums Account and joining roughly 150,000 Goons, for the one-time price of $9.95! We charge money because it costs us money per month for bills, and since we don't believe in showing ads to our users, we try to make the money back through forum registrations.
 
  • Post
  • Reply
Matt Zerella
Oct 7, 2002

Norris'es are back baby. It's good again. Awoouu (fox Howl)
I would not be mixing ZFS into the Fuse based array. I'd stick with XFS.

Adbot
ADBOT LOVES YOU

e.pilot
Nov 20, 2011

sometimes maybe good
sometimes maybe shit

Mr. Crow posted:

Man look like google is finally cracking down on google workspace and they're gonna restrict my unlimited storage for $20/month to 5TB (for $20, 5 more for every $20 iirc) :(

Whats the next cheapest offsite storage option for dozen or two TB of storage? DropBox Advanced and try to share it woth some people? What are you people with large collections doing, just ZFS with a lot of disks and hoping for the best? A separate backup machine thats mostly offline?


Related to that last point whats the lifetime of HDDs that are mostly powered off? Wondering feasibility of buying a second enclosure or something and just spinning it up once a month, I know standard hard drives have some inherent shelf life which is why tape drives are still a thing but im not sure how long that is.

how quickly do you need access to it? backblaze is unlimited

Combat Pretzel
Jun 23, 2004

No, seriously... what kurds?!

Matt Zerella posted:

I would not be mixing ZFS into the Fuse based array. I'd stick with XFS.
IIRC you can tell it not to do that (now?) for select devices and ZFS pools.

Mr. Crow
May 22, 2008

Snap City mayor for life

e.pilot posted:

how quickly do you need access to it? backblaze is unlimited

Its not super urgent cause it just goes read-only in two months, the unlimited is for windows only though right? Does it work if I e.g. have a windows guest with a read-only nfs share? Or does it have to be physically mapped drives on a windows host?

e.pilot
Nov 20, 2011

sometimes maybe good
sometimes maybe shit

Mr. Crow posted:

Its not super urgent cause it just goes read-only in two months, the unlimited is for windows only though right? Does it work if I e.g. have a windows guest with a read-only nfs share? Or does it have to be physically mapped drives on a windows host?

duplicati can back up to it

https://www.backblaze.com/blog/duplicati-backups-cloud-storage/

Hughlander
May 11, 2005


That's using B2 to their object store, not the unlimited.

e.pilot
Nov 20, 2011

sometimes maybe good
sometimes maybe shit

Hughlander posted:

That's using B2 to their object store, not the unlimited.

ah poo poo I didn’t know that, I haven’t looked that deeply into it

I’m just using googles annual 2tb plan, was planning on backblaze if that ever got full but welp

wolrah
May 8, 2006
what?

withoutclass posted:

Still not sure what a ZFS pool of 1disk does or why anyone would bother.
Single drive ZFS (or btrfs) allows you to know if errors occur that the drive doesn't detect, and more importantly lets you validate the entire drive's contents both to proactively detect failures and to know what you've actually lost when a failure occurs.

Years ago I had a LVM JBOD volume across a bunch of drives running ext4 and I lost one drive. Some files were damaged, some files were destroyed, and a lot were fine. The catch was I had no idea which ones were which, so I spent a lot of time writing shell scripts to have ffmpeg check the media, opening other files manually, and eventually just saying "gently caress it" and deleting entire subdirectories.

If I had for whatever nonsensical reason been running ZFS on that volume I'd still have lost files but I'd have known for sure which ones I lost within however much time it took a scrub to run, and realistically regular scrubs would have likely revealed the failing drive before it entirely gave up.

Computer viking
May 30, 2011
Now with less breakage.

Single-drive ZFS also lets you use compression - which is probably pointless but harmless if you're storing compressed video files, but can give you a decent space and performance boost if your data is compressible.

Wibla
Feb 16, 2011

ZFS compression saved a fair bit of space for me, on documents and misc crap that wasn't already compressed.

Computer viking
May 30, 2011
Now with less breakage.

Wibla posted:

ZFS compression saved a fair bit of space for me, on documents and misc crap that wasn't already compressed.

It's also hilariously effective on uncompressed DNA and RNA sequencing data, but I accept that this may be a niche use.

Wibla
Feb 16, 2011

Computer viking posted:

It's also hilariously effective on uncompressed DNA and RNA sequencing data, but I accept that this may be a niche use.

Considering the amount of junk data there's in DNA, that's not a surprise :downsrim:

That Works
Jul 22, 2006

Every revolution evaporates and leaves behind only the slime of a new bureaucracy


Computer viking posted:

It's also hilariously effective on uncompressed DNA and RNA sequencing data, but I accept that this may be a niche use.

Oh?? That changes things

Computer viking
May 30, 2011
Now with less breakage.

Wibla posted:

Considering the amount of junk data there's in DNA, that's not a surprise :downsrim:

Ha, you're not wrong. At least the RNAseq is like a sparknotes version.

Nitrousoxide
May 30, 2011

do not buy a oneplus phone



Yeah, I have a single drive pool on my OpenMediaVault server, and it saves a bit of space through the compression.

code:
tank  refcompressratio      1.08x                                 -

tank  written               183G                                  -

tank  logicalused           198G                                  -

tank  logicalreferenced     198G                                  -

calandryll
Apr 25, 2003

Ask me where I do my best drinking!



Pillbug

That Works posted:

Oh?? That changes things

For real, I mean its basically text files so should easily compress. I do wonder what the compression values are like because that would be awesome to save a ton of space.

necrobobsledder
Mar 21, 2005
Lay down your soul to the gods rock 'n roll
Nap Ghost
Not just any kind of text files but ones that have a very darn small plaintext symbol space so encodings can be very, very efficient. In fact, I'm kind of surprised that people don't by default use a common format that lets researchers avoid using tons of data for no good reason but given these are likely academicians I'm somewhat not surprised that they're more interested in doing stuff that works for their own purposes than trying to use industry standards. NIH is such a big problem in academic spaces it was frustrating enough to keep me from ever bothering with continuing studies (on top of other factors, sure).

Twerk from Home
Jan 17, 2009

This avatar brought to you by the 'save our dead gay forums' foundation.

necrobobsledder posted:

Not just any kind of text files but ones that have a very darn small plaintext symbol space so encodings can be very, very efficient. In fact, I'm kind of surprised that people don't by default use a common format that lets researchers avoid using tons of data for no good reason but given these are likely academicians I'm somewhat not surprised that they're more interested in doing stuff that works for their own purposes than trying to use industry standards. NIH is such a big problem in academic spaces it was frustrating enough to keep me from ever bothering with continuing studies (on top of other factors, sure).

These files are typically always handled in a compressed form though, which effectively takes advantage of the small symbol space. They get insanely good compression ratios with plain old gzip, and there's also a whole host of other better options including things like bgzip that allows reasonably performance random reads in compressed files.

It's only a very rare tool that can't read/write these files in compressed form, and even if you get unlucky there you can probably just feed it stdin/stdout and not store anything uncompressed.

Combat Pretzel
Jun 23, 2004

No, seriously... what kurds?!

Nitrousoxide posted:

Yeah, I have a single drive pool on my OpenMediaVault server, and it saves a bit of space through the compression.
Compressratio on my ZVOL hosting the Flight Simulator data files is 1.40x currently. And still 1.11x for the ZVOL hosting a bunch of Steam games.

H110Hawk
Dec 28, 2006
Shucking easystores still the way to fill up a synology? Like this:

https://www.bestbuy.com/site/wd-easystore-8tb-external-usb-3-0-hard-drive-black/6425302.p?skuId=6425302 $150
https://www.bestbuy.com/site/wd-easystore-12tb-external-usb-3-0-hard-drive-black/6425301.p?skuId=6425301 $205

Little annoying I have several 10TB ones and want to replace a 3TB to grab some extra space, and yet no 10TB. What's the current pricing situation look like?

Tiny Timbs
Sep 6, 2008

H110Hawk posted:

Shucking easystores still the way to fill up a synology? Like this:

https://www.bestbuy.com/site/wd-easystore-8tb-external-usb-3-0-hard-drive-black/6425302.p?skuId=6425302 $150
https://www.bestbuy.com/site/wd-easystore-12tb-external-usb-3-0-hard-drive-black/6425301.p?skuId=6425301 $205

Little annoying I have several 10TB ones and want to replace a 3TB to grab some extra space, and yet no 10TB. What's the current pricing situation look like?

I bought 2x 18 TB easystores at Best Buy for $285 each a couple months ago

I dunno about synologies but I had to tape over a couple of the power pins to get my desktop to recognize them

FunOne
Aug 20, 2000
I am a slimey vat of concentrated stupidity

Fun Shoe

Mr. Crow posted:

Man look like google is finally cracking down on google workspace and they're gonna restrict my unlimited storage for $20/month to 5TB (for $20, 5 more for every $20 iirc) :(



Back of the envelope has Google cloud cold at about 30/mo for 24TB. I have a few TB up there with CloudBerry and I've been happy, but I only do files, not images.

THF13
Sep 26, 2007

Keep an adversary in the dark about what you're capable of, and he has to assume the worst.
https://shucks.top/ keeps track of current/lowest/recent prices on the drives worth shucking. They're still a good way to get new, good drives for cheap but only when they're on sale. That happens quite frequently at least.

Mr. Crow
May 22, 2008

Snap City mayor for life

FunOne posted:

Back of the envelope has Google cloud cold at about 30/mo for 24TB. I have a few TB up there with CloudBerry and I've been happy, but I only do files, not images.

Never looked into it but assume egress is insane? Glacier is like $130 a TB

FunOne
Aug 20, 2000
I am a slimey vat of concentrated stupidity

Fun Shoe

Mr. Crow posted:

Never looked into it but assume egress is insane? Glacier is like $130 a TB

Oh yeah, close to $50/TB in retrieval alone, before network.

H110Hawk
Dec 28, 2006

THF13 posted:

https://shucks.top/ keeps track of current/lowest/recent prices on the drives worth shucking. They're still a good way to get new, good drives for cheap but only when they're on sale. That happens quite frequently at least.

Awesome thanks.

CancerCakes
Jan 10, 2006

What is the absolute easiest way to backup photos with some redundancy? I don't want to spend a huge amount just to back up family photos, but my current system is very not good: Currently they are just on 1TB drive in the PC, with periodic manual backup to an external drive, which is then stored at a family member's house. Total storage demand currently is 1TB, but obviously likely to go up.

First concern: drive failure during backup - this 1TB drive these photos are on is seriously long in the tooth and I now treat it with caution. Everything needs to come off the drive and onto new storage but I don't know how to do this in the most gentle manner.

Second concern: what do I move to? My preference would be local redundancy to cover disk failure paired with a low cost online storage which would only be used if there was a fire or something managed to wipe all the local.

My current thoughts: off the shelf 2 port PCI RAID card (£60) dropped into the PC with 2x 6TB (2x£150) with cloud cold storage (Glacier? I'm not clear on how to actually accomplish this).

I don't use plex or store any other media to speak of, and I don't really want the upfront or ongoing expense of running a NAS (especially as there is no intention to access the photos remotely).

Clearly anything is better than my current system, but working out what to actually do is very confusing!

rufius
Feb 27, 2011

Clear alcohols are for rich women on diets.

CancerCakes posted:

What is the absolute easiest way to backup photos with some redundancy? I don't want to spend a huge amount just to back up family photos, but my current system is very not good: Currently they are just on 1TB drive in the PC, with periodic manual backup to an external drive, which is then stored at a family member's house. Total storage demand currently is 1TB, but obviously likely to go up.

First concern: drive failure during backup - this 1TB drive these photos are on is seriously long in the tooth and I now treat it with caution. Everything needs to come off the drive and onto new storage but I don't know how to do this in the most gentle manner.

Second concern: what do I move to? My preference would be local redundancy to cover disk failure paired with a low cost online storage which would only be used if there was a fire or something managed to wipe all the local.

My current thoughts: off the shelf 2 port PCI RAID card (£60) dropped into the PC with 2x 6TB (2x£150) with cloud cold storage (Glacier? I'm not clear on how to actually accomplish this).

I don't use plex or store any other media to speak of, and I don't really want the upfront or ongoing expense of running a NAS (especially as there is no intention to access the photos remotely).

Clearly anything is better than my current system, but working out what to actually do is very confusing!

Honestly - first step I would recommend is pony up for a paid sub of OneDrive or GDrive and move the files there.

If you want to get more extravagant later, great.

As to “the most gentle way” - that’s hard to control so I’d probably start with just doing the upload to OneDrive/GDrive.

If the drive fails, as long as the platters are in tact, you can find an expert that could recover the data. Think computer forensics person or even some photography shops may have a computer nerd that does it.

Saukkis
May 16, 2003

Unless I'm on the inside curve pointing straight at oncoming traffic the high beams stay on and I laugh at your puny protest flashes.
I am Most Important Man. Most Important Man in the World.
The most gentle way is probably to get another 1+TB drive and then disk clone the drive with Macrium Reflect, Acronis or similar. It will read the drive from start to end linearly, that should minimize moving the read head. Unless there is a lot of free space, then a normal copy might be gentler overall.

Fanged Lawn Wormy
Jan 4, 2008

SQUEAK! SQUEAK! SQUEAK!
I got an old NAS from work, a Thecus N7710-G, and I'm trying to set it up as a time machine backup for our home stuff.

I'm running OSX Monterey on my old laptop to try and get everything setup. As far as I can tell, I should simply be able to go to the network services, turn on AFP, and then set a folder as useable for time machine. I do this, but when I try and tell Time Machine to backup to that directory, I get a message (after an unusually long wait):

"There was a problem connecting [...] The server may not exist or is not connected at this time, check the server name or ip address [yadda yadda]"

I did some basic looking around, and suggestions of making sure the disk isn't mounted already, and also to disable SMB, which I have also done. However, these were tips from many years ago, so IDK if any of this is still relevant. Any further suggestions?

SlowBloke
Aug 14, 2017

Fanged Lawn Wormy posted:

I got an old NAS from work, a Thecus N7710-G, and I'm trying to set it up as a time machine backup for our home stuff.

I'm running OSX Monterey on my old laptop to try and get everything setup. As far as I can tell, I should simply be able to go to the network services, turn on AFP, and then set a folder as useable for time machine. I do this, but when I try and tell Time Machine to backup to that directory, I get a message (after an unusually long wait):

"There was a problem connecting [...] The server may not exist or is not connected at this time, check the server name or ip address [yadda yadda]"

I did some basic looking around, and suggestions of making sure the disk isn't mounted already, and also to disable SMB, which I have also done. However, these were tips from many years ago, so IDK if any of this is still relevant. Any further suggestions?

Time machine expects smb3 on current macos, that thecus only does smb1. Try enabling AFP on that nas and see if monterey likes it, otherwise i would e-waste it.

Fanged Lawn Wormy
Jan 4, 2008

SQUEAK! SQUEAK! SQUEAK!
ahhh shucks. I've got AFP on, I figured it may be a this-poo poo-is-too-old problem.

shame on an IGA
Apr 8, 2005

CancerCakes posted:

What is the absolute easiest way to backup photos with some redundancy? I don't want to spend a huge amount just to back up family photos, but my current system is very not good: Currently they are just on 1TB drive in the PC, with periodic manual backup to an external drive, which is then stored at a family member's house. Total storage demand currently is 1TB, but obviously likely to go up.

First concern: drive failure during backup - this 1TB drive these photos are on is seriously long in the tooth and I now treat it with caution. Everything needs to come off the drive and onto new storage but I don't know how to do this in the most gentle manner.

Second concern: what do I move to? My preference would be local redundancy to cover disk failure paired with a low cost online storage which would only be used if there was a fire or something managed to wipe all the local.

My current thoughts: off the shelf 2 port PCI RAID card (£60) dropped into the PC with 2x 6TB (2x£150) with cloud cold storage (Glacier? I'm not clear on how to actually accomplish this).

I don't use plex or store any other media to speak of, and I don't really want the upfront or ongoing expense of running a NAS (especially as there is no intention to access the photos remotely).

Clearly anything is better than my current system, but working out what to actually do is very confusing!

replace your external drive with two external drives and also throw everything into backblaze. I would love to use it for the same reason but uploading my photo collection is just unfeasible on my dogshit 25/3 cable internet

Tamba
Apr 5, 2010

Mr. Crow posted:

Man look like google is finally cracking down on google workspace and they're gonna restrict my unlimited storage for $20/month to 5TB (for $20, 5 more for every $20 iirc) :(

Whats the next cheapest offsite storage option for dozen or two TB of storage? DropBox Advanced and try to share it woth some people? What are you people with large collections doing, just ZFS with a lot of disks and hoping for the best? A separate backup machine thats mostly offline?


I haven't used it myself, but ArsTechnica recently mentioned https://www.idrive.com/ which seems to be quite cheap ($12.5 per month for 20TB or $31.25 for 50TB, if you pay for 2 years at once).

https://arstechnica.com/information-technology/2023/02/ars-archivum-top-cloud-backup-services-worth-your-money/

DJ Burette
Jan 6, 2010
I use idrive and it works well for me. I've backed up 3+tb over 3 computers and have restored things from it multiple times in the past with no issue.

As far as plug and play online backup solutions it works perfectly for me.

Dyscrasia
Jun 23, 2003
Give Me Hamms Premium Draft or Give Me DEATH!!!!
I use idrive as well. Watch for sales, they often do like 90% off. Something around $10 for a year for 2tb of resold s3 storage

I do consider it a risk due to price, but Ive used them for at least 3 years now without a problem.

priznat
Jul 7, 2009

Let's get drunk and kiss each other all night.
Anyone know for Unraid when a docker shows up like this:


does that mean that the docker image is no longer being maintained and should switch to the other one (if there is one)?

Matt Zerella
Oct 7, 2002

Norris'es are back baby. It's good again. Awoouu (fox Howl)

priznat posted:

Anyone know for Unraid when a docker shows up like this:


does that mean that the docker image is no longer being maintained and should switch to the other one (if there is one)?

Find the docker patch plugin and install it. This is fixed in the new version apparently.

priznat
Jul 7, 2009

Let's get drunk and kiss each other all night.

Matt Zerella posted:

Find the docker patch plugin and install it. This is fixed in the new version apparently.

Nice, thanks that did it!

Adbot
ADBOT LOVES YOU

Aware
Nov 18, 2003
Also install the auto update plugin. It does your plugins and docker, very useful. And yes you can pick a d choose if you have some that you don't want auto updated.

  • 1
  • 2
  • 3
  • 4
  • 5
  • Post
  • Reply