Register a SA Forums Account here!
JOINING THE SA FORUMS WILL REMOVE THIS BIG AD, THE ANNOYING UNDERLINED ADS, AND STUPID INTERSTITIAL ADS!!!

You can: log in, read the tech support FAQ, or request your lost password. This dumb message (and those ads) will appear on every screen until you register! Get rid of this crap by registering your own SA Forums Account and joining roughly 150,000 Goons, for the one-time price of $9.95! We charge money because it costs us money per month for bills, and since we don't believe in showing ads to our users, we try to make the money back through forum registrations.
 
  • Post
  • Reply
BlankSystemDaemon
Mar 13, 2009




Sniep posted:

As far as how I understand it, on a Synology the OS partition is mirrored among each and every single disk in the array regardless of size pool volume what have you - where you'd have to have a 100% disk failure for it to not actually boot
That's how all root-on-RAID should be set up, and how a surprisingly large amount of them aren't set up.

Adbot
ADBOT LOVES YOU

H110Hawk
Dec 28, 2006

Incessant Excess posted:

I have a Synology running DSM 6.2.2, is there a way to backup the NAS OS itself? I have set up a few docker containers that I would like to be able to restore as they are if something were to happen to my NAS or if I migrated to a new device.

Control panel, update and restore, configuration backup tab, backup. The NAS OS will self-install on the first disk you install in a new NAS box. The Synology does it correctly. If you have made a bunch of non-control panel modifications those will not backup because the synology has no knowledge of them. I believe you also have to do this for major apps such as their surveillance app, etc.

KS
Jun 10, 2003
Outrageous Lumpwad

I had no idea, thanks.

I've been trying to figure out what to target if I want a new box and not sure I gain much --- is it correct that the current xeon E3 is 3 years old? Pondering just buying a pair of xeon E5640s on Ebay for $10 as they're apparently still on the ESXi 6.7 HCL and work in my current motherboard.

Thermopyle
Jul 1, 2003

...the stupid are cocksure while the intelligent are full of doubt. —Bertrand Russell

Incessant Excess posted:

I have a Synology running DSM 6.2.2, is there a way to backup the NAS OS itself? I have set up a few docker containers that I would like to be able to restore as they are if something were to happen to my NAS or if I migrated to a new device.

I don't know how Synology works, but part of the whole point of docker containers is that you don't have to back them up.

If you have data generated or used by the container you should back that up of course.

Then, when your NAS burns to the ground you just spin up a container on whatever system you use next and point it at your backed up data.

DrDork
Dec 29, 2003
commanding officer of the Army of Dorkness

D. Ebdrup posted:

That's how all root-on-RAID should be set up, and how a surprisingly large amount of them aren't set up.

The other question is whether that applies to the docker/containers. A lot of NAS OSes will happily replicate the core OS multiple times while leaving any containers/jails/etc to the user to figure out how to back up. Free/TrueNAS, for example, only backs up the fact that a jail exists when you do an OS export--it doesn't bring the actual jail or any of its data along with it, since that data is stored elsewhere.

DrDork
Dec 29, 2003
commanding officer of the Army of Dorkness

Thermopyle posted:

I don't know how Synology works, but part of the whole point of docker containers is that you don't have to back them up.

This sometimes works better in theory than in practice, unfortunately, especially if you have several dockers that are interconnected. It should work like that, but I've seen more than one setup where it just doesn't for one reason or another.

BlankSystemDaemon
Mar 13, 2009




DrDork posted:

The other question is whether that applies to the docker/containers. A lot of NAS OSes will happily replicate the core OS multiple times while leaving any containers/jails/etc to the user to figure out how to back up. Free/TrueNAS, for example, only backs up the fact that a jail exists when you do an OS export--it doesn't bring the actual jail or any of its data along with it, since that data is stored elsewhere.
It doesn't surprise me that jails aren't totally backed up, unlike the old plugin system using the warden jail manager where jail configuration was stored as part of the plugin.
Then again, Free/TrueNAS isn't made to be using root-on-ZRAID, but a turn-key appliance on a separate USB disk, which is another part of the reason why I never liked it - although I understand why it is that way.

FreeNAS started life out as a fork of m0n0wall, ie. using NanoBSD, a way of building FreeBSD to fit onto a floppy drive and then a small CF drive once the kernel became too big around 7.0.
Current minimum that FreeBSD 13 runs on is 16MB, which basically just leaves room for a kernel and a statically-compiled version of wpa_supplicant and ifconfig, to work as an access point.

BlankSystemDaemon fucked around with this message at 17:34 on Apr 12, 2020

Thermopyle
Jul 1, 2003

...the stupid are cocksure while the intelligent are full of doubt. —Bertrand Russell

DrDork posted:

This sometimes works better in theory than in practice, unfortunately, especially if you have several dockers that are interconnected. It should work like that, but I've seen more than one setup where it just doesn't for one reason or another.

You can set up a pile of poo poo with anything.

If you can't blow away your container and then restart it with your backed up data then you messed up by using the wrong tool for the job or you messed up by not using docker correctly.

H110Hawk
Dec 28, 2006

Thermopyle posted:

You can set up a pile of poo poo with anything.

The synology OS and control panel is not there to backup user generated content. Use the backup program to make sure the registry is backed up. (I have admittedly not used the docker stuff.) If you have configuration files you need to make sure are backed up, put them in git, hosted on your synology in a regular fileshare, and back that up to B2 or something.

When you press backup, it is very clear what it is backing up, it's a complete list. Read it carefully. It's enough to get your synology back up and sharing files again and that's it.

Warbird
May 23, 2012

America's Favorite Dumbass

Question regarding doing things beyond simple storage on a NAS. I know that doing things like running a Plex server on a NAS is fairly popular. What are the advantages there vs simply storing the library in the NAS and having a dedicated machine handle the heavy lifting? For the sake of simplicity let's assume everything is wired into a decent switch. I assume you cut down on redundant network traffic that way if nothing else.

If you do go down that road it almost seems that you'd want to just eschew a dedicated NAS and build a spec'd up server box and throw FreeNAS or whatever into a Docker container with volumes pointed at the mount points of your drives or the like. I totally get running stuff like Rsync for data intake, but I'm not quite grocking the breakdown at the point of having Plex or Apache living there by default.

H110Hawk
Dec 28, 2006

Warbird posted:

Question regarding doing things beyond simple storage on a NAS. I know that doing things like running a Plex server on a NAS is fairly popular. What are the advantages there vs simply storing the library in the NAS and having a dedicated machine handle the heavy lifting? For the sake of simplicity let's assume everything is wired into a decent switch. I assume you cut down on redundant network traffic that way if nothing else.

If you do go down that road it almost seems that you'd want to just eschew a dedicated NAS and build a spec'd up server box and throw FreeNAS or whatever into a Docker container with volumes pointed at the mount points of your drives or the like. I totally get running stuff like Rsync for data intake, but I'm not quite grocking the breakdown at the point of having Plex or Apache living there by default.

If you have one of the ones with a GPU and you're happy with the performance, it's a one stop shop. If you aren't, it makes it so your storage is isolated from your server and easily accessible from other computers. I never want to play storage janitor. It's the least fun computer janitor task to me, and the one with the highest stakes if I make a mistake. So my underpowered NUC runs plex, and everything else for that matter, and my synology does the storage and backups. Its all over gig ethernet on a 16 port netgear switch. Yes copying a 40GB file takes a while, but I don't do that synchronously generally so who cares if the computer spends more time doing whatever?

I used to run plex on synology, but it was a pain anytime something tried to transcode and plex hates freedom so they don't let you refuse that. Thankfully for now my new roku has a "force direct play" option.

Warbird
May 23, 2012

America's Favorite Dumbass

H110Hawk posted:

If you have one of the ones with a GPU and you're happy with the performance, it's a one stop shop. If you aren't, it makes it so your storage is isolated from your server and easily accessible from other computers. I never want to play storage janitor. It's the least fun computer janitor task to me, and the one with the highest stakes if I make a mistake. So my underpowered NUC runs plex, and everything else for that matter, and my synology does the storage and backups. Its all over gig ethernet on a 16 port netgear switch. Yes copying a 40GB file takes a while, but I don't do that synchronously generally so who cares if the computer spends more time doing whatever?

I used to run plex on synology, but it was a pain anytime something tried to transcode and plex hates freedom so they don't let you refuse that. Thankfully for now my new roku has a "force direct play" option.

Ok that's about what I suspected. I already have most of my flows set up in Docker so it doesn't sound like there's too much reason to lift and shift somewhere else; though lord knows a Synology box probably has more oomph than the old rear end laptop/server I have doing work right now.

So new question, let's assume I go with building my own setup. I don't have an issue throwing parts together, I do it all the time. Assuming I go do some PCPart Picker guide or whatever, is there a preferred OS/Software solution out there? I know that FreeNas/OpenNAS/UnRaid/NoRaid/ReRaid/etc etc etc are out there, but damned if know the ins and outs. Assuming I want something with a decent GUI so the Mrs. can go get stuff she's stored and that's fairly idiot proof so I don't end up breaking something and get yelled at, what would be a decent place to start? Something with the ability to plug in varying sizes of drives and have them pooled up would be nice. Doubly so if it'll notify me well in advance of a drive dying as well. I think some of that goes into Raid configuration, but I imagine most solutions will let you set things up however you need.

H110Hawk
Dec 28, 2006

Warbird posted:

Ok that's about what I suspected. I already have most of my flows set up in Docker so it doesn't sound like there's too much reason to lift and shift somewhere else; though lord knows a Synology box probably has more oomph than the old rear end laptop/server I have doing work right now.

So new question, let's assume I go with building my own setup. I don't have an issue throwing parts together, I do it all the time. Assuming I go do some PCPart Picker guide or whatever, is there a preferred OS/Software solution out there? I know that FreeNas/OpenNAS/UnRaid/NoRaid/ReRaid/etc etc etc are out there, but damned if know the ins and outs. Assuming I want something with a decent GUI so the Mrs. can go get stuff she's stored and that's fairly idiot proof so I don't end up breaking something and get yelled at, what would be a decent place to start? Something with the ability to plug in varying sizes of drives and have them pooled up would be nice. Doubly so if it'll notify me well in advance of a drive dying as well. I think some of that goes into Raid configuration, but I imagine most solutions will let you set things up however you need.

Hang on, what are your goals with this thing you're assembling? Do you want her to do management tasks like creating shares and managing backups? Or do you just want "stuff your files here" ? Because... why not use the synology? What is her desktop OS?

Unraid is the answer if you are making a second NAS, but why?

DrDork
Dec 29, 2003
commanding officer of the Army of Dorkness

Warbird posted:

Question regarding doing things beyond simple storage on a NAS. I know that doing things like running a Plex server on a NAS is fairly popular. What are the advantages there vs simply storing the library in the NAS and having a dedicated machine handle the heavy lifting? For the sake of simplicity let's assume everything is wired into a decent switch. I assume you cut down on redundant network traffic that way if nothing else.

It's mostly to address two scenarios:

(1) You rolled a DIY solution and thus have a ton of spare power left over. Throwing Plex et al on there as well means you can then access it anytime you want to without having to have another machine on. Since everything is handled all on one box, you can often leverage it to get the advantages of no intranet traffic and higher performance, since your Plex Media Server doesn't have to reach out over the network to touch its own database. Alternately you get the advantage of seamless backup/redundancy by having the PMS database on the NAS, instead of on some other device.

(2) You grabbed a Synology or whatever and it's good enough to stream to your one single device at a time because you're not trying to handle 15 different streams for all your friends and family, and you don't want to have to worry about the cost and hassle of another device. Even a NUC or Shield or whatever is $150+, after all.

Honestly, (1) + (2) covers the majority of people. The only time you really see people straying away from it is when they bought a Synology or similar that isn't powerful enough to meet their stream/transcoding needs. In that case the only options are either replace/upgrade the Synology for $$$, buy yourself a NUC, shield, etc., if you want Plex to be available when your computer is off, or simply accept that you can only do Plex when your computer is on.

DrDork
Dec 29, 2003
commanding officer of the Army of Dorkness

Warbird posted:

So new question, let's assume I go with building my own setup. I don't have an issue throwing parts together, I do it all the time. Assuming I go do some PCPart Picker guide or whatever, is there a preferred OS/Software solution out there? I know that FreeNas/OpenNAS/UnRaid/NoRaid/ReRaid/etc etc etc are out there, but damned if know the ins and outs. Assuming I want something with a decent GUI so the Mrs. can go get stuff she's stored and that's fairly idiot proof so I don't end up breaking something and get yelled at, what would be a decent place to start? Something with the ability to plug in varying sizes of drives and have them pooled up would be nice. Doubly so if it'll notify me well in advance of a drive dying as well. I think some of that goes into Raid configuration, but I imagine most solutions will let you set things up however you need.

Agree that UnRaid / Xpenology are probably good options if you want to use variously sized drives--ZFS isn't gonna give you what you want there, and thus there's not much point looking at Free/True/OpenNAS.

GUI...yeah, I'm confused here, too. You should be just exporting the datastores via NSF/CIFS/SMB so they show up as normal network drives on whatever system she's using, so a GUI never comes into the question. Same with managing Plex, your torrent client of choice, etc.--they should all be accessed via their own web-based GUI, which is independent of your OS choice (mostly).

The back-end stuff is also largely a wash, since Ubuntu/CentOS/whatever are all pretty similar, and once you get it working all you need to do is occasionally upgrade Plex if it has new features you want, which takes like 2 commands on the CLI.

Matt Zerella
Oct 7, 2002

Norris'es are back baby. It's good again. Awoouu (fox Howl)
Backend doesn't even matter if you use docker. You can run those containers on Windows if you want. Though I don't suggest it since docker on anything other than Linux blows.

Thermopyle
Jul 1, 2003

...the stupid are cocksure while the intelligent are full of doubt. —Bertrand Russell

Matt Zerella posted:

Backend doesn't even matter if you use docker. You can run those containers on Windows if you want. Though I don't suggest it since docker on anything other than Linux blows.

In what way?

I've deployed many apps on many Windows docker machines (not my choice of course), I also do a lot of development across linux and windows machines and use docker to maintain a common build environment. In other words, I'm constantly running and creating containers across Linux and Windows machines.

And...Docker on windows seems fine.

Warbird
May 23, 2012

America's Favorite Dumbass

It's way better now than it used to be, particularly if you have Pro/Enterprise with HyperV. There's also WSL2, but that's another thread.

Matt Zerella
Oct 7, 2002

Norris'es are back baby. It's good again. Awoouu (fox Howl)

Thermopyle posted:

In what way?

I've deployed many apps on many Windows docker machines (not my choice of course), I also do a lot of development across linux and windows machines and use docker to maintain a common build environment. In other words, I'm constantly running and creating containers across Linux and Windows machines.

And...Docker on windows seems fine.

Persistent storage on macOS and Windows is not great in my experience. Especially on macOS but windows hasn't been great for me either.

E: yeah, I haven't tested this with WSL2 and I've read support for that is much better.

Thermopyle
Jul 1, 2003

...the stupid are cocksure while the intelligent are full of doubt. —Bertrand Russell

Matt Zerella posted:

Persistent storage on macOS and Windows is not great in my experience. Especially on macOS but windows hasn't been great for me either.

It just works nowadays.

At least on Windows. Dunno about mac.

Hughlander
May 11, 2005

Warbird posted:

It's way better now than it used to be, particularly if you have Pro/Enterprise with HyperV. There's also WSL2, but that's another thread.

What other thread if I may ask?

Incessant Excess
Aug 15, 2005

Cause of glitch:
Pretentiousness

Thermopyle posted:

If you have data generated or used by the container you should back that up of course.

Then, when your NAS burns to the ground you just spin up a container on whatever system you use next and point it at your backed up data.

On DSM, when first creating a Docker container, I have to enter some parameters (path mounts, PUID and PGID) that stuff is what I'd like to backup. The configuration of the container itself, Sonarr for example, I can backup by making a copy of the Docker config folder.

spincube
Jan 31, 2006

I spent :10bux: so I could say that I finally figured out what this god damned cube is doing. Get well Lowtax.
Grimey Drawer

Incessant Excess posted:

On DSM, when first creating a Docker container, I have to enter some parameters (path mounts, PUID and PGID) that stuff is what I'd like to backup. The configuration of the container itself, Sonarr for example, I can backup by making a copy of the Docker config folder.

From the Container menu, right-click on the container, Settings -> Export:



You can then use the exported settings file to import into a replacement (but you have to create the replacement first, then import the settings over whatever you set it up with). This is of course separate from backing up /config or whatever, that should be handled by something else.

Thermopyle
Jul 1, 2003

...the stupid are cocksure while the intelligent are full of doubt. —Bertrand Russell

Incessant Excess posted:

On DSM, when first creating a Docker container, I have to enter some parameters (path mounts, PUID and PGID) that stuff is what I'd like to backup. The configuration of the container itself, Sonarr for example, I can backup by making a copy of the Docker config folder.

I see.

On a "regular" OS, I'd have all that stuff in a docker-compose file which you can just backup like you'd backup any file.

What scenario are you trying to protect against here?

If you're worried about your NAS blowing up, can you be confident that you're mounts, user/group ids, etc are going to remain the same on the replacement system?

Generally, I'd expect any scenario where you're restoring the runtime configuration of a docker container to require different values for the stuff you're talking about.

In other words, the point of docker is specifically so you're not tied to those values you're talking about.

Note that I'm not saying that it's useless to backup those values, but you also shouldn't expect to back them up and then everything is just all A-OK.

Former Human
Oct 15, 2001

Saukkis posted:

Could you drill new holes in correct spots?

I could possibly drill holes through the plastic sides of the tray, but then I would need my own rubber grommets/washers and longer screws. It's better to mount it in the 5.25" bay.

Here's a photo of two drives, an old Samsung on the left and new HGST on the right.

Incessant Excess
Aug 15, 2005

Cause of glitch:
Pretentiousness

spincube posted:

From the Container menu, right-click on the container, Settings -> Export:



You can then use the exported settings file to import into a replacement (but you have to create the replacement first, then import the settings over whatever you set it up with). This is of course separate from backing up /config or whatever, that should be handled by something else.

Thank you.

Thermopyle posted:

What scenario are you trying to protect against here?

If you're worried about your NAS blowing up, can you be confident that you're mounts, user/group ids, etc are going to remain the same on the replacement system?

Generally, I'd expect any scenario where you're restoring the runtime configuration of a docker container to require different values for the stuff you're talking about.

In other words, the point of docker is specifically so you're not tied to those values you're talking about.

Note that I'm not saying that it's useless to backup those values, but you also shouldn't expect to back them up and then everything is just all A-OK.

That's a good point, I thought that if I could restore the entire setup as is on a new Synology (OS + Docker and it's configuration) that would save me some potential hassle but it looks like that's not really possible.

Volguus
Mar 3, 2009

Former Human posted:

I could possibly drill holes through the plastic sides of the tray, but then I would need my own rubber grommets/washers and longer screws. It's better to mount it in the 5.25" bay.

Here's a photo of two drives, an old Samsung on the left and new HGST on the right.



You said 2 holes do line up. Isn't this enough? Do you live in an earthquake-prone area? If so, add some tape i suppose. If not, why worry? It's not gonna run from the case.

Warbird
May 23, 2012

America's Favorite Dumbass

Hughlander posted:

What other thread if I may ask?

I meant more in the “we’re veering off topic somewhat”, but I don’t actually know of there’s a WSL specific thread. Given how WSL2 is literally just Linux I’m sure one of the *nix YOSPOS threads would be adequate. I’d be down for a thread though. I may make one once 2004 releases to the public in the next month or so.

Axe-man
Apr 16, 2005

The product of hundreds of hours of scientific investigation and research.

The perfect meatball.
Clapping Larry

Sniep posted:

As far as how I understand it, on a Synology the OS partition is mirrored among each and every single disk in the array regardless of size pool volume what have you - where you'd have to have a 100% disk failure for it to not actually boot

It is set as a raid 1 on all disks for about 2 gigs.

It will always attempt to read disk in slot 1 first (left to right top to bottom) there are times when that has been corrupted and it will ask to do a migration. What that is it is reading disk 2 then 3 etc and reinstalling that mirror.

Synology hyper backup even configurstion backup is the best way to keep the most from the os.

This is because the actual partition itself links to the volume where various databases are to do the programs. So if you recover from nothing and just clone your old the symlinks might not work and break everything.

Granite Octopus
Jun 24, 2008

I'm in need of some replacement drives for my 4-bay Synology. I originally bought some NAS-specific drives for it. Money is tight right now, and external hard drives are fully half the price of the cheapest NAS-specific drives for the same capacity.

Is it a terrible idea to shuck drives these days? While the data is important to me, it is raided, plus I have a local backup in the form of another external disk, and remote backup via Backblaze, so even if it lasts half as long I don't really mind. Speed isn't necessarily important either, since its mostly streaming large movie/tv files.

Raymond T. Racing
Jun 11, 2019

Granite Octopus posted:

I'm in need of some replacement drives for my 4-bay Synology. I originally bought some NAS-specific drives for it. Money is tight right now, and external hard drives are fully half the price of the cheapest NAS-specific drives for the same capacity.

Is it a terrible idea to shuck drives these days? While the data is important to me, it is raided, plus I have a local backup in the form of another external disk, and remote backup via Backblaze, so even if it lasts half as long I don't really mind. Speed isn't necessarily important either, since its mostly streaming large movie/tv files.

shuck is the way to go

Sniep
Mar 28, 2004

All I needed was that fatty blunt...



King of Breakfast

Granite Octopus posted:

I'm in need of some replacement drives for my 4-bay Synology. I originally bought some NAS-specific drives for it. Money is tight right now, and external hard drives are fully half the price of the cheapest NAS-specific drives for the same capacity.

Is it a terrible idea to shuck drives these days? While the data is important to me, it is raided, plus I have a local backup in the form of another external disk, and remote backup via Backblaze, so even if it lasts half as long I don't really mind. Speed isn't necessarily important either, since its mostly streaming large movie/tv files.

get you some best buy easystores stat, it's the pro strat

H110Hawk
Dec 28, 2006

Granite Octopus posted:

I'm in need of some replacement drives for my 4-bay Synology. I originally bought some NAS-specific drives for it. Money is tight right now, and external hard drives are fully half the price of the cheapest NAS-specific drives for the same capacity.

Is it a terrible idea to shuck drives these days?

Go back a page or two or three and you are basically guaranteed to see people suggesting just that!

DrDork
Dec 29, 2003
commanding officer of the Army of Dorkness

Granite Octopus posted:

I'm in need of some replacement drives for my 4-bay Synology. I originally bought some NAS-specific drives for it. Money is tight right now, and external hard drives are fully half the price of the cheapest NAS-specific drives for the same capacity.

Is it a terrible idea to shuck drives these days? While the data is important to me, it is raided, plus I have a local backup in the form of another external disk, and remote backup via Backblaze, so even if it lasts half as long I don't really mind. Speed isn't necessarily important either, since its mostly streaming large movie/tv files.

The only people buying retail drives now are the ones made out of money. Or businesses / people who can expense them. There's effectively no reason to buy full retail drives at this point.

Granite Octopus
Jun 24, 2008

Great, thanks!! Sorry I only went one page back and checked the first post. Drive ordered :D

IOwnCalculus
Apr 2, 2003





Shuck away. I was only paranoid about them until they stopped allowing "but you opened it" as a reason to void warranties.

Raymond T. Racing
Jun 11, 2019

IOwnCalculus posted:

Shuck away. I was only paranoid about them until they stopped allowing "but you opened it" as a reason to void warranties.

Which is the very important note:

Either keep all your shells (and note which one went with which drive), or don't bother. WD apparently gets touchy but will accept a bare shucked drive, but if you screw up and put the drive back in the wrong shell, it turns into a massive back and forth to get them to RMA it as they'll claim you're trying to defraud them.

SlowBloke
Aug 14, 2017
It seems like QNAP wants to get some of that whitebox action, they added a link to a novel QuTSCloud page whose links do not work in the last mail blast:

https://www.qnap.com/solution/qutscloud/en-us/

They are going to license the QTS OS sorta like a CHR, you pay by the storage volume

BurgerQuest
Mar 17, 2009

by Jeffrey of YOSPOS
I just added an old nVidia GTX970 I had spare to my unraid server and with a simple driver patch to overcome the 2x stream limitation it's now doing hardware transcoding for my plex users. My poor old i5 wasn't keeping up with demand (mostly chromecast users which seems to require transcoding 90% of the time).

Adbot
ADBOT LOVES YOU

Generic Monk
Oct 31, 2011

Sniep posted:

get you some best buy easystores stat, it's the pro strat

are these available in the uk? i note the ‘elements’ drives have amazon reviews of people doing this stuff

  • 1
  • 2
  • 3
  • 4
  • 5
  • Post
  • Reply