Register a SA Forums Account here!
JOINING THE SA FORUMS WILL REMOVE THIS BIG AD, THE ANNOYING UNDERLINED ADS, AND STUPID INTERSTITIAL ADS!!!

You can: log in, read the tech support FAQ, or request your lost password. This dumb message (and those ads) will appear on every screen until you register! Get rid of this crap by registering your own SA Forums Account and joining roughly 150,000 Goons, for the one-time price of $9.95! We charge money because it costs us money per month for bills, and since we don't believe in showing ads to our users, we try to make the money back through forum registrations.
 
  • Post
  • Reply
evil_bunnY
Apr 2, 2003

Does anyone make a synology-style desktop enclosure NAS that takes SAS drives?

Adbot
ADBOT LOVES YOU

Aware
Nov 18, 2003
Uhhh

Some of them have pcie slots I think so maybe you could DIY it

evil_bunnY
Apr 2, 2003

Aware posted:

Uhhh

Some of them have pcie slots I think so maybe you could DIY it
I dont think the drives will work in a SATA backplane?

BlankSystemDaemon
Mar 13, 2009



ASRock (well, their Rack department) are at it again:

Only registered members can see post attachments!

That Works
Jul 22, 2006

Every revolution evaporates and leaves behind only the slime of a new bureaucracy


BlankSystemDaemon posted:

ASRock (well, their Rack department) are at it again:



:swoon:

Wibla
Feb 16, 2011

I like that case. Meh on the arm poo poo though.

BlankSystemDaemon
Mar 13, 2009



Wibla posted:

I like that case. Meh on the arm poo poo though.
Neoverse N2 is a pretty stellar IP core from ARM - so I'm hoping they offer the LCC chips as options as well.

BlankSystemDaemon fucked around with this message at 17:56 on Oct 31, 2023

Perplx
Jun 26, 2004


Best viewed on Orgasma Plasma
Lipstick Apathy

Wibla posted:

I like that case. Meh on the arm poo poo though.

It's a nas, it going to move your bits just as fast and use less power.

HalloKitty
Sep 30, 2005

Adjust the bass and let the Alpine blast

Wibla posted:

I like that case. Meh on the arm poo poo though.

I had the same thought, to be honest

Wibla
Feb 16, 2011

"Meh" is not "gently caress this poo poo, gimme amd64", it's more "I don't know what this gives me that's cool over a 5000-series or 7000-series Ryzen with ECC ram". Or an intel CPU with QSV :haw:

M_Gargantua
Oct 16, 2006

STOMP'N ON INTO THE POWERLINES

Exciting Lemon
Anyone have a good recommendation for used rackmount gear that can work as both as a NAS and home media server/docker box? Or two systems, I've got the rack space for it to just have a 10g link between an encoding box and a dumb drive bay.

I've got a spare T4 to slot in for transcoding, and probably would initially load in 7x20TB drives with room to grow. Ideally something that I can throw a lot of RAM + a couple of NVMe SSDs into as a faster caching levels.

Since its going to be ~$3k in storage i'd rather not spend another $1k just on the chassis.

BlankSystemDaemon
Mar 13, 2009



There's no doubt that the case is by far the most important thing to come out of that.

Still it's neat to see not-underpowered ARM boards finally being somewhat of an option - because aside from the spinning rust, the CPU is easily what uses the most power in any system.
Well, except if it's an Altera Max with 2TB of memory....

Wibla posted:

"Meh" is not "gently caress this poo poo, gimme amd64", it's more "I don't know what this gives me that's cool over a 5000-series or 7000-series Ryzen with ECC ram". Or an intel CPU with QSV :haw:
Well, here you're almost-guaranteed to get a board that'll inform the OS about ECC errors, because it conforms to the RAS specifications set out by ARM - whereas with the average consumer board, there are no such promises and you'll have to hope that it works, or do a lot of research.

QSV/NVENC/whatever is poo poo for archival video - you want software encoding with CRF, and with 192 cores, you can do h265 in multithreaded mode :science:

BlankSystemDaemon fucked around with this message at 18:44 on Oct 31, 2023

Spectracide
May 27, 2004
IT'S ARGH, BABY!
Western Digital is running a deal on the 20TB Red Pro, two for $600 ($15/TB). 5% cash back if you use Rakuten, too. Not amazing but I picked up 4 for a new NAS I'm building.

evil_bunnY
Apr 2, 2003

Apparently I'm not crazy and people make the kind of 8-bay SAS enclosures I want (something like a J0812T or Silverstone Tek RL-TS821S), but no one in euroland seems interested in selling those for reasonable prices :(

Combat Pretzel
Jun 23, 2004

No, seriously... what kurds?!
Anyone happens to know how ZFS will behave in regards to prefetching if I set primarycache=metadata on a dataset? I guess I'll finally set my big file datasets like the movie ones to that. But it's always been nice of ZFS to prefetch data in big chunks.

--edit: There seems to be some kind of minor prefetching to be done. Playing a movie starts with 2MB of IO every 1 second or so to eventually space out to 6MB every 3-4 seconds. But nothing special beyond that.
--edit: Now 8MB every 5-6 seconds. Probably gonna go up some more. That's nice. Keeps disk IO down despite no data caching.

Combat Pretzel fucked around with this message at 23:31 on Nov 2, 2023

BlankSystemDaemon
Mar 13, 2009



evil_bunnY posted:

Apparently I'm not crazy and people make the kind of 8-bay SAS enclosures I want (something like a J0812T or Silverstone Tek RL-TS821S), but no one in euroland seems interested in selling those for reasonable prices :(
I know that feeling all to well :mood:

Combat Pretzel posted:

Anyone happens to know how ZFS will behave in regards to prefetching if I set primarycache=metadata on a dataset? I guess I'll finally set my big file datasets like the movie ones to that. But it's always been nice of ZFS to prefetch data in big chunks.

--edit: There seems to be some kind of minor prefetching to be done. Playing a movie starts with 2MB of IO every 1 second or so to eventually space out to 6MB every 3-4 seconds. But nothing special beyond that.
--edit: Now 8MB every 5-6 seconds. Probably gonna go up some more. That's nice. Keeps disk IO down despite no data caching.
I assume it works like anything else, because of these OIDs:
pre:
kstat.zfs.misc.arcstats.prefetch_metadata_misses: 307871
kstat.zfs.misc.arcstats.prefetch_metadata_iohits: 383904
kstat.zfs.misc.arcstats.prefetch_metadata_hits: 8610473

Computer viking
May 30, 2011
Now with less breakage.

evil_bunnY posted:

Apparently I'm not crazy and people make the kind of 8-bay SAS enclosures I want (something like a J0812T or Silverstone Tek RL-TS821S), but no one in euroland seems interested in selling those for reasonable prices :(

Oh those would have been so neat compared to the MD1200 I use at work. But yeah, not exactly easily available in Norway.

Wibla
Feb 16, 2011

Jetcarrier exists :v:

Amazon also has reasonable shipping to Norway these days.

Mr Shiny Pants
Nov 12, 2012

Smashing Link posted:

VT-x and VT-d settings are enabled in BIOS. I double checked the GPU is in the x16 PCIE lane. Processor is a Xeon 2650L V4 so should support virtualization.

nvm: unraid.

Computer viking
May 30, 2011
Now with less breakage.

Wibla posted:

Jetcarrier exists :v:

Amazon also has reasonable shipping to Norway these days.

True, but for work it's way easier if we can buy from a local supplier - preferably one the hospital has a contract with. Privately I'd definitely consider it. :)

evil_bunnY
Apr 2, 2003

There's this on amazon.de but of loving course no sellers :X

https://www.amazon.de/dp/B0771S45X3/

Combat Pretzel
Jun 23, 2004

No, seriously... what kurds?!

BlankSystemDaemon posted:

I know that feeling all to well :mood:

I assume it works like anything else, because of these OIDs:
pre:
kstat.zfs.misc.arcstats.prefetch_metadata_misses: 307871
kstat.zfs.misc.arcstats.prefetch_metadata_iohits: 383904
kstat.zfs.misc.arcstats.prefetch_metadata_hits: 8610473
Oh, I don't doubt that metadata will still be prefetched. I was more concerned about the data itself. I'd rather not have any kind of read amplification, because I have movies stored in a dataset with 1MB record size, while MPC-BE keeps reading in 256KB blocks and causing ZFS to dump the block out of memory as soon the 256KB request has been served. Or some poo poo like that.

Computer viking
May 30, 2011
Now with less breakage.

evil_bunnY posted:

There's this on amazon.de but of loving course no sellers :X

https://www.amazon.de/dp/B0771S45X3/

Ha, another fan of German Amazon. Even though it'd be twenty times better if all the sellers realized that Norway counts as inside the EU for their purposes.

BlankSystemDaemon
Mar 13, 2009



Combat Pretzel posted:

Oh, I don't doubt that metadata will still be prefetched. I was more concerned about the data itself. I'd rather not have any kind of read amplification, because I have movies stored in a dataset with 1MB record size, while MPC-BE keeps reading in 256KB blocks and causing ZFS to dump the block out of memory as soon the 256KB request has been served. Or some poo poo like that.
A single stream of 256KB reads shouldn't invalidate what's been cached by using a combination of MFU and MRU - so if that's what you're seeing, I might suspect there's been some tuning done that's made things "better".

evil_bunnY
Apr 2, 2003

Computer viking posted:

it'd be twenty times better if all the sellers realized that Norway counts as inside the EU for their purposes.
Why do I have to fill in export paperwork when returning poo poo there then goddamnit

In any case, I think I'm just going to repurpose an old gaming machine to NAS duty with an LSI9k 8i and a pair of of miniSAS to 4*SFF-8482 cables. It would cost hundreds to add hotswap functionality it's just going to be a jellyfin datastore anyway.

Tankakern
Jul 25, 2007

Computer viking posted:

Ha, another fan of German Amazon. Even though it'd be twenty times better if all the sellers realized that Norway counts as inside the EU for their purposes.

why does norway count as inside eu? that's only true if the seller use amazon's own shipping solution, because they handle vat and stuff

Wibla
Feb 16, 2011

Tankakern posted:

why does norway count as inside eu? that's only true if the seller use amazon's own shipping solution, because they handle vat and stuff

That's my understanding of how it works too.

Computer viking
May 30, 2011
Now with less breakage.

evil_bunnY posted:

Why do I have to fill in export paperwork when returning poo poo there then goddamnit

In any case, I think I'm just going to repurpose an old gaming machine to NAS duty with an LSI9k 8i and a pair of of miniSAS to 4*SFF-8482 cables. It would cost hundreds to add hotswap functionality it's just going to be a jellyfin datastore anyway.

Honestly a good question, and I cannot honestly say I understand enough of the relevant law and treaties to even guess at an answer.

Joyboy
Mar 18, 2021

I'm sure everyone feels the same.
I realized I have a spare cpu/mobo/ram combo (r5 5600x, b550 mobo and 32GB) sitting unused in a closet and decided I want to setup babys first nas to replace my hodgepodge of storage and servers running on my main pc. Main usage would be running Jellyfin/Plex/Sonarr/Radar/Komga/etc along with usage as a general network drive for various hobby projects.

I don't really have any experience in running a nas or home server so I don't know what os would be recommended or whether it'd be smarter for me to run the programs in docker containers or not. Also not sure how much storage to order, what RAID type to use (if any) or whether it would be better to order many smaller drives vs a few larger ones. I was looking at ordering 8x12TB but I don't know if that'd be an insane amount or not for my use case.

If anyone has any recommendations I'd love to hear them.

Aware
Nov 18, 2003

Joyboy posted:

I realized I have a spare cpu/mobo/ram combo (r5 5600x, b550 mobo and 32GB) sitting unused in a closet and decided I want to setup babys first nas to replace my hodgepodge of storage and servers running on my main pc. Main usage would be running Jellyfin/Plex/Sonarr/Radar/Komga/etc along with usage as a general network drive for various hobby projects.

I don't really have any experience in running a nas or home server so I don't know what os would be recommended or whether it'd be smarter for me to run the programs in docker containers or not. Also not sure how much storage to order, what RAID type to use (if any) or whether it would be better to order many smaller drives vs a few larger ones. I was looking at ordering 8x12TB but I don't know if that'd be an insane amount or not for my use case.

If anyone has any recommendations I'd love to hear them.

Get Unraid, follow trashguides. https://trash-guides.info/Hardlinks/How-to-setup-for/Unraid/

Computer viking
May 30, 2011
Now with less breakage.

Joyboy posted:

I realized I have a spare cpu/mobo/ram combo (r5 5600x, b550 mobo and 32GB) sitting unused in a closet and decided I want to setup babys first nas to replace my hodgepodge of storage and servers running on my main pc. Main usage would be running Jellyfin/Plex/Sonarr/Radar/Komga/etc along with usage as a general network drive for various hobby projects.

I don't really have any experience in running a nas or home server so I don't know what os would be recommended or whether it'd be smarter for me to run the programs in docker containers or not. Also not sure how much storage to order, what RAID type to use (if any) or whether it would be better to order many smaller drives vs a few larger ones. I was looking at ordering 8x12TB but I don't know if that'd be an insane amount or not for my use case.

If anyone has any recommendations I'd love to hear them.

8x12TB in raidz2/raid6 would give you roughly 72TB of usable space. That's way more than I use, and about enough for the unsorted/incoming buffer for some of the hoarders archivists here. Consider how much you use today, and how you expect it to grow the next, idk, five years?

Twerk from Home
Jan 17, 2009

This avatar brought to you by the 'save our dead gay forums' foundation.
What's most likely to successfully read all the data back after being stored in a closet for 30 years? Optical media, external or internal hard disk, or tape?

Just curious, I'm not planning to act on it right now, but given that hard disks seem the cheapest storage, even cheaper than disc now, makes me wonder if they last well sitting unused. I know that flash memory doesn't.

Harik
Sep 9, 2001

From the hard streets of Moscow
First dog to touch the stars


Plaster Town Cop

Twerk from Home posted:

What's most likely to successfully read all the data back after being stored in a closet for 30 years? Optical media, external or internal hard disk, or tape?

Just curious, I'm not planning to act on it right now, but given that hard disks seem the cheapest storage, even cheaper than disc now, makes me wonder if they last well sitting unused. I know that flash memory doesn't.

None of it will work in 30 years. Do you have a SCSI-1 HBA? RLE controller? Does your BIOS let you specify heads, cylinders and sectors? Got an iomega ZIP drive? That was the tech 30 years ago. In 30 you won't have a SATA controller or bluray drive.

Nitrousoxide
May 30, 2011

do not buy a oneplus phone



I have to imagine that hard disks will still be used in 30 years and tapes will still be used for cold storage data.

Now, whether the file formats you'd write to those media now will still be readable, or whether we'll still use something compatible with SATA or SAS to interface with them I'm less sure.

Harik
Sep 9, 2001

From the hard streets of Moscow
First dog to touch the stars


Plaster Town Cop

Nitrousoxide posted:

I have to imagine that hard disks will still be used in 30 years and tapes will still be used for cold storage data.

Now, whether the file formats you'd write to those media now will still be readable, or whether we'll still use something compatible with SATA or SAS to interface with them I'm less sure.

Media from 30 years ago is the realm of specialists and retrocomputing fans. There's nothing an average person can just throw in a closet for three decades and pull back out with useful information. If you're not actively maintaining it it doesn't exist.

Combat Pretzel
Jun 23, 2004

No, seriously... what kurds?!
It'll work just fine in 30 years.

Sub Rosa
Jun 9, 2010




Building a new NAS. Decided to try Unraid after being pretty happy with XPEnology for over a decade I think.

Started first array with 2x 20TB and 1x 14TB drive. They had all been precleared, but I didn't understand to click the button that parity was already valid I guess? So if proceeds to a parity build.

The 20TB drivers were direct to SATA. The 14TB drive was to a IT mode HBA which I've learned runs way way hotter than I expected. During the parity build, the disk on the HBA drops.

Somehow the parity build proceeds without it. End result is

code:
Parity is valid
Last check completed on Sun 5 Nov 2023 03:46:47 AM PST (today)
Duration: 19 hours, 2 minutes, 18 seconds. Average speed: 291.8 MB/s
Finding 2188194704 errors
I pull the HBA and connect the data disk also straight to a SATA port. Reboot.

Unraid sees the disk again. I expect when I start the array that it will need to rebuild parity? But no, it still claims valid parity.

I feel it is impossible for parity to actually be valid as a massive amount of bits on the data drive weren't ever read. But yet it claims Parity is valid.

If I'm understanding things correctly, that means that Unraid is claiming valid parity even though parity isn't valid at all. I don't know how I can trust Unraid with my data now, if it fails to recognize the need to recover from a disk dropping during a parity build?

I'm letting it run a parity check now to see what happens, but I'm so grateful this has all happened before I moved any data over.

Am I misunderstanding something?

BlankSystemDaemon
Mar 13, 2009



Twerk from Home posted:

What's most likely to successfully read all the data back after being stored in a closet for 30 years? Optical media, external or internal hard disk, or tape?

Just curious, I'm not planning to act on it right now, but given that hard disks seem the cheapest storage, even cheaper than disc now, makes me wonder if they last well sitting unused. I know that flash memory doesn't.
Anything that's not magnetic like external/internal harddisks/flashdisks won't be affected by cosmic rays et al, but are likely to be affected by other types of degration - for example, UV light will do a number on optical disks (especially if they're +30 years old and you take them out of the closet where they've been stored).
The best way to keep data is to store it on an online array using a hash-tree of checksums with its own mirroring/striping+distributed parity with periodic scrub and periodically replacing disks as they age out.

In theory, a technology such as that used by Microsoft Project Silica might be a better solution - it works by treating a very thin layer of quartz glass on a substrate as a punchcard using a laser (not unlike CDs/DVDs/BluRays, though at a much higher density and as WORM-only, ie. you'll never get rewritable versions of it).
In practice, I doubt prosumers will ever see something like that, and even the other company who's touting a similar technology only appears interested in advertising to the hyperscalers.

BlankSystemDaemon fucked around with this message at 17:15 on Nov 5, 2023

Twerk from Home
Jan 17, 2009

This avatar brought to you by the 'save our dead gay forums' foundation.

Harik posted:

None of it will work in 30 years. Do you have a SCSI-1 HBA? RLE controller? Does your BIOS let you specify heads, cylinders and sectors? Got an iomega ZIP drive? That was the tech 30 years ago. In 30 you won't have a SATA controller or bluray drive.

Are you telling me that if I put a Myst CD into my computer it won't read? Optical media seems like it still might be the best bet, CD-ROMs are more than 30 years old and still cheaply, easily readable. I have a BD-ROM drive in my desktop, everything works except for SecuROM games because Windows no longer supports CD-based DRM.

I have 28 year old Sega Saturn CDs that still worked as of last year, but I don't think I still have any PC games because my family was entirely MacOS in the 90s and Mac backwards compatibility is WAY worse than Windows.

Twerk from Home fucked around with this message at 17:23 on Nov 5, 2023

Adbot
ADBOT LOVES YOU

BlankSystemDaemon
Mar 13, 2009



Twerk from Home posted:

Are you telling me that if I put a Myst CD into my computer it won't read? Optical media seems like it still might be the best bet, CD-ROMs are more than 30 years old and still cheaply, easily readable. I have a BD-ROM drive in my desktop, everything works except for SecuROM games because Windows no longer supports CD-based DRM.

I have 28 year old Sega Saturn CDs that still worked as of last year, but I don't think I still have any PC games because my family was entirely MacOS in the 90s and Mac backwards compatibility is WAY worse than Windows.
Optical media is good in so far as it's easy to store without risking degradation while it's in-storage - but the trouble is that it has very low density and incredibly low IOPS, so in practice you're limited to what you can store.
A BluRay maxes out at 128GB - and with a limit of 576Mbps write speed assuming you can get a 16x double-layer drive (I've not seen many, and the few that do rarely achieve those speeds in practice), you're looking at over almost 12 minutes for a single disk in the best case scenario.

And it still doesn't address the issue of how quickly optical media can degrade once you expose it to any source of UV light.
EDIT: To add to this, consider how often optical media can be exposed to UV light - I have disks in my closet containing stuff I burned when I still had a optical writer in my system, but I couldn't make a guarantee that they weren't exposed to UV light at any point since they were manufactured, if you held a gun to my head.

BlankSystemDaemon fucked around with this message at 17:47 on Nov 5, 2023

  • 1
  • 2
  • 3
  • 4
  • 5
  • Post
  • Reply