Register a SA Forums Account here!
JOINING THE SA FORUMS WILL REMOVE THIS BIG AD, THE ANNOYING UNDERLINED ADS, AND STUPID INTERSTITIAL ADS!!!

You can: log in, read the tech support FAQ, or request your lost password. This dumb message (and those ads) will appear on every screen until you register! Get rid of this crap by registering your own SA Forums Account and joining roughly 150,000 Goons, for the one-time price of $9.95! We charge money because it costs us money per month for bills, and since we don't believe in showing ads to our users, we try to make the money back through forum registrations.
 
  • Post
  • Reply
Axe-man
Apr 16, 2005

The product of hundreds of hours of scientific investigation and research.

The perfect meatball.
Clapping Larry
Go to DSM Control Panel > Network Interface > Manage > Service order

Then all you have to do is make sure that the Bond (I assume you are using bonding if not, you should) is above the wireless interface and boom, it will prefer the ethernet over wireless.

Adbot
ADBOT LOVES YOU

Hadlock
Nov 9, 2004

Ok great, very clear instructions

It's not currently set to bond, I'll give that a go too as soon as this transfer wraps up

Thanks!

Axe-man
Apr 16, 2005

The product of hundreds of hours of scientific investigation and research.

The perfect meatball.
Clapping Larry
https://www.synology.com/en-us/knowledgebase/DSM/help/DSM/AdminCenter/connection_network_linkaggr

Gives a good overview what it does and how it might best work for you, honestly if you are transferring a lot of data, that extra gig connection is worth it. If you don't have the right switch/router config to allow the full protocol it will still do adaptive load balancing.

There are few drawbacks in my opinion for what you get out of it.

Bob Morales
Aug 18, 2006


Just wear the fucking mask, Bob

I don't care how many people I probably infected with COVID-19 while refusing to wear a mask, my comfort is far more important than the health and safety of everyone around me!

Hadlock posted:

So I have a synology d418 or whatever with a couple TB of disk in it

I have both gig-e ethernet cables plugged into it and a name brand gig-e router.

I also have the great asus usb-ac56 plugged in to the synology unit, because why not, it's supported and occasionally I want to do direct wifi and the NAS is sometimes far away from a wired router.

Went to go transfer 200GB of tiny tiny files from my PC server which also has a decent gig-e network adapter. Was getting ~88KB/s. All the cables are cat 5e, etc. Seemed extremely slow given the networking capabilities of the system. 1Mbps minimum, preferably at least 100Mbps. Was telling me that I was expecting 22+ hours to complete.

Finally dug in to the performance pane on the synology, was showing all the traffic going over the wireless usb adapter instead of the 2x 1Gbps wired connections that feed in to the same gig-e router as my PC server.

Unplugged the usb wireless adapter, restarted the transfer, and now I'm getting closer to the expected 200Mbps. Looking at closer to 4 hours to complete now.

TL;DR is there a way I can setup the wireless adapter to connect, but always prefer the gig-e 1 and 2 ports over the much slower usb wifi? Sometimes I need direct wifi access but most of the time I am going to want wired.

Thanks

Sounds like a client-side issue

Put a wireless bridge/AP on the switch instead of the DS. What’s the network metric set on the WiFi interface?

Hadlock
Nov 9, 2004

Is there a preferred/standard way to organize files on a NAS? I have 2TB+ of files spread across probably five to twenty thousand folders, many of them photos but I have a day job and can't be arsed to fine grain move everything as it would take days if not weeks.

I guess the upside is that for the last almost 25 years I intuitively know where all my important files are based on sort of filesystem muscle memory

possibly amusing backstory, feel free to share your own

Since I was 16 in the mid 1990s with a $100 Zip Disk I've just sort of been sequentially nesting all my files inside of other nested folders and now I'm starting to run in to windows file name length problems. I just shed probably 200 packrat ebooks i'll never read because their total name length exceeded 256 characters including folder structure. I think windows technically has a workaround now but the 256 limit is probably going to bite me elsewhere in the future.

Sort of looks like this except with slightly more clever names
\\server\share\2007_backup\2002_backup\1999_backup\1996_backup


\\admont\test\neogoliath\Goliath\E Drive\backups\backup 02-26-08\120gb drive (hadlock)\Toast's World\backup\tempbackup\backup

BobHoward
Feb 13, 2012

The only thing white people deserve is a bullet to their empty skull
don’t use a bad os to hoard your files maybe???

Schadenboner
Aug 15, 2011

by Shine

BobHoward posted:

don’t use a bad os to hoard your files maybe???

Except that every OS is a POS?

:shrug:

Hadlock
Nov 9, 2004

The problem isn't server side, it's doing copy operations on the client os. You have to have admin access and then modify the registry to HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\FileSystem\LongPathsEnabled = 1

Synology (my NAS) offers > 260 char support out of the box so that's a moot point

What organizational method do you use?

Bob Morales
Aug 18, 2006


Just wear the fucking mask, Bob

I don't care how many people I probably infected with COVID-19 while refusing to wear a mask, my comfort is far more important than the health and safety of everyone around me!

store all your files in mysql database

Atomizer
Jun 24, 2007



Hadlock posted:

Is there a preferred/standard way to organize files on a NAS? I have 2TB+ of files spread across probably five to twenty thousand folders, many of them photos but I have a day job and can't be arsed to fine grain move everything as it would take days if not weeks.

I guess the upside is that for the last almost 25 years I intuitively know where all my important files are based on sort of filesystem muscle memory

possibly amusing backstory, feel free to share your own

Since I was 16 in the mid 1990s with a $100 Zip Disk I've just sort of been sequentially nesting all my files inside of other nested folders and now I'm starting to run in to windows file name length problems. I just shed probably 200 packrat ebooks i'll never read because their total name length exceeded 256 characters including folder structure. I think windows technically has a workaround now but the 256 limit is probably going to bite me elsewhere in the future.

Sort of looks like this except with slightly more clever names
\\server\share\2007_backup\2002_backup\1999_backup\1996_backup


\\admont\test\neogoliath\Goliath\E Drive\backups\backup 02-26-08\120gb drive (hadlock)\Toast's World\backup\tempbackup\backup

If you're talking just about photos, it's generally wise to organize them by a combination of subject and date. So, you could do something like:
> Family
> Dogs
> Cars
> Computers
> Vacations
> Sports

...or:
>2019
>2018
...and so on. Either way you're going to have to reorganize everything, but you could start by pulling up all of those yearly directories. Ideally you'd have details in the metadata of each file so you can easily search for them later but let's take it one step at a time.

Also, if you really do tackle this project in long sessions, installing a file manager like Teracopy can help; it'll let you queue up file movements, so you don't have to deal with Windows just trying to start a bunch of them simultaneously as you organize.

Chris Knight
Jun 5, 2002

me @ ur posts


Fun Shoe

Hadlock posted:

Since I was 16 in the mid 1990s with a $100 Zip Disk I've just sort of been sequentially nesting all my files inside of other nested folders and now I'm starting to run in to windows file name length problems. I just shed probably 200 packrat ebooks i'll never read because their total name length exceeded 256 characters including folder structure. I think windows technically has a workaround now but the 256 limit is probably going to bite me elsewhere in the future.
1) Invent time machine
2) Go back in time and slap self repeatedly

Ursine Catastrophe
Nov 9, 2009

It's a lovely morning in the void and you are a horrible lady-in-waiting.



don't ask how i know

Dinosaur Gum
Just to check, is there a difference between the WD Books vs Elements when it comes to shucking? Everything I saw ITT seems to prefer the former but I wasn’t sure if that was just based on sale price or if there’s a distinct hardware difference

Atomizer
Jun 24, 2007



Ursine Catastrophe posted:

Just to check, is there a difference between the WD Books vs Elements when it comes to shucking? Everything I saw ITT seems to prefer the former but I wasn’t sure if that was just based on sale price or if there’s a distinct hardware difference

The MyBooks tend to have that blocky chassis, whereas the Elements & Easystore have more or less the same one, that chassis that's rounded on the front. The latter should open the same way, and the preferences for certain models is based on the drives that tend to come inside, which differs between capacities.

Moey
Oct 22, 2010

I LIKE TO MOVE IT

Atomizer posted:

The MyBooks tend to have that blocky chassis, whereas the Elements & Easystore have more or less the same one, that chassis that's rounded on the front. The latter should open the same way, and the preferences for certain models is based on the drives that tend to come inside, which differs between capacities.

^^ this guy shucks

Paul MaudDib
May 3, 2006

TEAM NVIDIA:
FORUM POLICE

Atomizer posted:

If you're talking just about photos, it's generally wise to organize them by a combination of subject and date. So, you could do something like:
> Family
> Dogs
> Cars
> Computers
> Vacations
> Sports

...or:
>2019
>2018
...and so on. Either way you're going to have to reorganize everything, but you could start by pulling up all of those yearly directories. Ideally you'd have details in the metadata of each file so you can easily search for them later but let's take it one step at a time.

Also, if you really do tackle this project in long sessions, installing a file manager like Teracopy can help; it'll let you queue up file movements, so you don't have to deal with Windows just trying to start a bunch of them simultaneously as you organize.

If you have a generally haphazard mess of a filesystem this is the best approach. Think of it as bucket sort - you can pre-sort each individual drive into categories, shove them together into a single filesystem, then manually deduplicate things as appropriate.

Rsync is nice for this since it won't re-copy files that already exist on the destination filesystem.

Also, ZFS snapshots are nice for this too, because if you mess up you can pull any lost files back from a previous version of the filesystem.

Hadlock
Nov 9, 2004

Your post got me thinking,

How good is Synology deduplication?

Like, could I just leave the haystack of files in a cryptic sounding folder like \old\ and then copy files as needed into my new Goliath folder with less than 5% inefficient?

Microsoft's deduplication is block level so I think it's less than 10% inefficient which is pretty good.

Schadenboner
Aug 15, 2011

by Shine
So if I get a 4-disk Synology but I’m poor and lovely and only put two disks in at first, can I later convert that RAID 1 to a RAID 10 (by adding two more matching disks)?

E: same question about adding a second M.2 SSD after the first is already in-place, actually?

E2: Or could I maybe expand the 2 disk RAID1 into a 3 disk RAID5 and put in a 2.5” SSD for caching?

Schadenboner fucked around with this message at 17:28 on May 10, 2019

IOwnCalculus
Apr 2, 2003





Hadlock posted:

Your post got me thinking,

How good is Synology deduplication?

Like, could I just leave the haystack of files in a cryptic sounding folder like \old\ and then copy files as needed into my new Goliath folder with less than 5% inefficient?

Microsoft's deduplication is block level so I think it's less than 10% inefficient which is pretty good.

Dedupe in general is very RAM intensive. ZFS dedupe is so RAM hungry that you should pretty much never ever ever ever enable it. No idea on those specific implementations though.

Crunchy Black
Oct 24, 2017

by Athanatos

Hadlock posted:


Since I was 16 in the mid 1990s with a $100 Zip Disk I've just sort of been sequentially nesting all my files inside of other nested folders and now I'm starting to run in to windows file name length problems. I just shed probably 200 packrat ebooks i'll never read because their total name length exceeded 256 characters including folder structure. I think windows technically has a workaround now but the 256 limit is probably going to bite me elsewhere in the future.

Jesus goddamn christ on the loving cross.

Paul MaudDib
May 3, 2006

TEAM NVIDIA:
FORUM POLICE
C:\backups\new\new\2012\keep\maindrive\backup\1\temp\temp1\

IOwnCalculus
Apr 2, 2003





Z:\photos\New Folder (4)\New Folder (3)\

dexefiend
Apr 25, 2003

THE GOGGLES DO NOTHING!
D:\downloads\porn\hot ones\2019\unsorted\keep\creepy\gonzo\filthy\

Schadenboner
Aug 15, 2011

by Shine

dexefiend posted:

D:\downloads\porn\hot ones\2019\unsorted\keep\creepy\gonzo\filthy\taxforms\

MagusDraco
Nov 11, 2011

even speedwagon was trolled
Z:\<name>\<name> backup\C\Users\<user name>\documents\my documents\docs\<photos folder from when windows xp had photos in in my documents>\family\screensaver\

then you right click, view folder detail and laugh as it enumerates files and folders and the file size of the folder suddenly balloons into 50 TB of data on a 8 TB hard-drive because when folder names get this long a weird windows 10 bug shows up.

Though I think that got fixed in 1809? Haven't checked in awhile

MagusDraco fucked around with this message at 18:57 on May 10, 2019

Krailor
Nov 2, 2001
I'm only pretending to care
Taco Defender

Schadenboner posted:

So if I get a 4-disk Synology but I’m poor and lovely and only put two disks in at first, can I later convert that RAID 1 to a RAID 10 (by adding two more matching disks)?

E: same question about adding a second M.2 SSD after the first is already in-place, actually?

E2: Or could I maybe expand the 2 disk RAID1 into a 3 disk RAID5 and put in a 2.5” SSD for caching?

If you create it as a SHR volume you'll be able to add disks to it later.

FORUMS USER 1135
Jan 14, 2004

Atomizer posted:

The MyBooks tend to have that blocky chassis, whereas the Elements & Easystore have more or less the same one, that chassis that's rounded on the front. The latter should open the same way, and the preferences for certain models is based on the drives that tend to come inside, which differs between capacities.
Is there a quick rule of thumb for this? I just bought a 6TB Elements and got a Blue (literally, has the blue labeling on it). Its fine, but I am going to put together a Plex server later this year and want to get better drives for that.

Ika
Dec 30, 2004
Pure insanity

Occasionally I have attempted to master the mess:

E:\UNSORTED-TODO\F\unsort\Game stuff semisorted

(who am I kidding?)


All my digital photos are archived properly though.

Crunchy Black
Oct 24, 2017

by Athanatos
Okay, finally figured it out. I had NFS shares trying to be the same pathname as my SMB shares. BSD does not like this! But, yay! Found the data. Glad I didn't give up, the only reason I didn't is that when trying to create a share, the web interface for freeNAS could still enumerate the file structure, which was my clue that the data was there but also that I am an idiot.

Axe-man
Apr 16, 2005

The product of hundreds of hours of scientific investigation and research.

The perfect meatball.
Clapping Larry

Schadenboner posted:

So if I get a 4-disk Synology but I’m poor and lovely and only put two disks in at first, can I later convert that RAID 1 to a RAID 10 (by adding two more matching disks)?

E: same question about adding a second M.2 SSD after the first is already in-place, actually?

E2: Or could I maybe expand the 2 disk RAID1 into a 3 disk RAID5 and put in a 2.5” SSD for caching?

i would use SHR to auto make a raid 1 for you and then when you get 2 other disks it will be expanded to Raid 5. Unless you got a really good reason for RAID 10, it isn't the best for like general use.

Schadenboner
Aug 15, 2011

by Shine
Given that (like I said) I’m poor and lovely I’m leaning more and more towards starting with a RAID1 then expanding to the a 3-disk R5 with a hot swap, that way the NAS can keep NASing while a dead disk is getting returned to WD? And I guess speed it up with a R/W SSD cache made from the two M.2s?

Atomizer
Jun 24, 2007



FORUMS USER 1135 posted:

Is there a quick rule of thumb for this? I just bought a 6TB Elements and got a Blue (literally, has the blue labeling on it). Its fine, but I am going to put together a Plex server later this year and want to get better drives for that.

Rule of thumb for how to shuck or for which are the preferred drives? If it's the latter, no, but the best you can do is to just monitor the deal sites and see what drives others are saying they got in theirs. First, you can take a look to see what the retail drives are, and you'll see that for given capacities, only certain drive types are available (e.g. WD doesn't make Green and Blue drives above 6 or 8 TB, Seagate uses SMR which isn't a dealbreaker but just something to note, the 8+ TB capacities may be Reds or white-labels of the same, may be only Helium-filled, etc.) Then, while you might have to settle for receiving one of 3 or 4 different white-labels, just keep in mind that they're all going to be more than acceptable, especially for the price (considering they're so much cheaper than retail internal drives.) For WD's drives, basically anything 8+ TB will be NAS-grade Reds or equivalent, so just go with the best deal.

Paul MaudDib
May 3, 2006

TEAM NVIDIA:
FORUM POLICE
WD EasyStore 8TB or 10TB are the desired drives. 8 TB drives will be either Reds or rebranded HGST Heliums (the latter need a pin taped over, or to be powered from a Molex adapter), the 10TB will be all Heliums.

Ideally you want the "Made in Thailand" version as these have 256MB cache, but tbh it really doesn't matter anyway, your plex server isn't going to run noticeably different on the 128MB drives.

MyBooks usually have inferior drives inside, Seagate usually have SMR drives and also are made by Seagate, the worst hard drive company on the planet.

BlankSystemDaemon
Mar 13, 2009



For software RAID that does a mix of asyncronous and syncronous writes, the cache on spinning rust means gently caress-all except in the rare instances where the cache is not backed by enough power capasitance to keep the drive spinning for long enough that the cache is written to disk before the disk shuts down.

Also, be aware that SMR is not "just" SMR; there's at least three different forms and only one of them requires the host to be aware of the SMR state of the disks and modify its behaviour accordingly - and most importantly, consumer drives are basically never the host-aware type because vendors know that customers can't be trusted to tell the difference, or have an OS that knows how to write to SMR drives for all its supported filesystems.

susan b buffering
Nov 14, 2016

Hadlock posted:

Your post got me thinking,

How good is Synology deduplication?

Like, could I just leave the haystack of files in a cryptic sounding folder like \old\ and then copy files as needed into my new Goliath folder with less than 5% inefficient?

Microsoft's deduplication is block level so I think it's less than 10% inefficient which is pretty good.

DSM doesn't do any deduplication on its own. If you're ok with getting your hands dirty you can try these instructions for running the btrfs dedup tool on DSM.

You do still get all the benefits of a COW filesystem like snapshots, and DSM automate those.



Re-read your post and realized you weren't really talking about deduplication in the technical sense. In your scenario the efficiency would be basically 100% due to btrfs being a copy-on-write filesystem. I think you need to enable an option in the DSM control panel to make it work that way over SMB but I'm not sure.

Basically when you copy a file on a btrfs volume, no extra space is consumed until you change one of the files.

susan b buffering fucked around with this message at 00:33 on May 12, 2019

Axe-man
Apr 16, 2005

The product of hundreds of hours of scientific investigation and research.

The perfect meatball.
Clapping Larry
I was wondering if any of you had been using VMM and trying to install arch.

I have the weirdest bug. When I install it on hardware it does fine. When I install it on VMM it will randomly stop the dchpcd process and kill the virtual switch.

ProjektorBoy
Jun 18, 2002

I FUCK LINEN IN MY SPARE TIME!
Grimey Drawer

Moey posted:

^^ this guy shucks

"This Guy Shucks" - The Consumer NAS/storage megathread

Crunchy Black
Oct 24, 2017

by Athanatos

ProjektorBoy posted:

"This Guy Shucks" - The Consumer NAS/storage megathread

Inshallah.

Everything back up and running on the stable tree:


More RAM is sitting on the bench, just need to bring it down to install. Probably get to that this morning.

Enos Cabell
Nov 3, 2004


The biggest downside of building a nice server and automating more stuff, is that it makes it way too easy to fill up your drives. Bring on the next big sale of 10tb EasyStores!

Hadlock
Nov 9, 2004

ProjektorBoy posted:

"This Guy Shucks" - The Consumer NAS/storage megathread

Seconding

Adbot
ADBOT LOVES YOU

Schadenboner
Aug 15, 2011

by Shine

ProjektorBoy posted:

"This Guy Shucks" - The Consumer NAS/storage megathread

  • 1
  • 2
  • 3
  • 4
  • 5
  • Post
  • Reply