|
Harik posted:Media from 30 years ago is the realm of specialists and retrocomputing fans. There's nothing an average person can just throw in a closet for three decades and pull back out with useful information. If you're not actively maintaining it it doesn't exist. An ide drive fomatted with fat can be thrown in a cheap usb dock/adapter from amazon/ebay/whatever and will read fine on any os Old cds can be read without issue excluding physical damage
|
#
?
Nov 5, 2023 18:10
|
|
|
|
| # ? May 15, 2024 10:26 |
|
|
Optical media probably has the best change to still work and have working compatible hardware without too much effort. Tape may last that long, but the standards evolve too quickly. We are at LTO-9 currently and those drives can only support down to LTO-7, so you would need to upgrade the tapes every five years or so. With harddrives the risk of seized bearings is probably too high. But this is the wrong way to think about this. You can hardly trust any media for 30 years so you would have to test them every five years or so, so might as well transfer the data to newer media while doing it. And keeping the data on active storage is of course a good option. More interesting question would be what is the cheapest method of long term storage for different amounts of data.
|
|
#
?
Nov 5, 2023 18:31
|
|
|
Saukkis posted:Optical media probably has the best change to still work and have working compatible hardware without too much effort. Tape may last that long, but the standards evolve too quickly. We are at LTO-9 currently and those drives can only support down to LTO-7, so you would need to upgrade the tapes every five years or so. With harddrives the risk of seized bearings is probably too high. It was a great analogy-demonstration of why you shouldn't rely on hardware RAID though, because hardware RAID HBAs can do the exact same change, except there's usually not multiple vendors involved as each vendor have their own proprietary standards. The issue with all archival storage is that you need to be able to check it to confirm its consistency.
|
|
#
?
Nov 5, 2023 18:48
|
|
|
Saukkis posted:More interesting question would be what is the cheapest method of long term storage for different amounts of data. I'm under the impression that you need an insane amount of data for tape to be worth it now, because disk has gotten so much cheaper than it used to be while the LTO group spent all of LTO-8 fighting each other over patents or licensing or some poo poo and the tape capacity size increases across generations have been slowing down, even before and after LTO-8. Disk seems to be king right now. Edit: I guess you could go IBM proprietary tape instead of LTO, I've heard good things. Twerk from Home fucked around with this message at 19:08 on Nov 5, 2023 |
|
#
?
Nov 5, 2023 19:06
|
|
|
Microsoft have Project Silica which is super cool https://www.youtube.com/watch?v=-rfEYd4NGQg
|
|
#
?
Nov 5, 2023 19:38
|
|
|
Twerk from Home posted:What's most likely to successfully read all the data back after being stored in a closet for 30 years? Optical media, external or internal hard disk, or tape? base64 encoded fanfold dot matrix printouts
|
|
#
?
Nov 5, 2023 19:43
|
|
|
Thanks Ants posted:Microsoft have Project Silica which is super cool https://memory-alpha.fandom.com/wiki/Isolinear_chip
|
|
#
?
Nov 5, 2023 20:08
|
|
|
HalloKitty posted:An ide drive fomatted with fat can be thrown in a cheap usb dock/adapter from amazon/ebay/whatever and will read fine on any os Yeah I was about to say. A nice large desktop disk in 1993 would probably be ATA-1, which should be forwards compatible with all the ATA standards up to the point where SATA took over. The first ATA to SATA converter I found on amazon is $8, and in theory this should allow you to use that 1993 ATA drive as long as your desktop has a SATA port. Even beyond that, you should be able to jam it into one of those external SATA-to-USB boxes and keep going as long as USB backwards compatibility lasts. Which I expect to be decades into the future. Of course, that's kind of the best case. If you went SCSI, you'd probably have to dig up an old SCSI controller and a PCIe to PCI adapter. While SCSI to SAS translation seems like it should be fairly easy in theory, there does not seem to be anything available. I do have a working USB zip drive at work, though. Formatted up a 250MB disk in BTRFS last week just to test - and it Just Works. About 1 MB/sec reads and writes, according to the gnome disk tool.
|
|
#
?
Nov 5, 2023 22:27
|
|
|
I was going to make that same point, but: that falls into the category of "isn't it neat this still works" and not "I depend on this for data archival". It also doesn't hurt that the amount of data stored on a hard drive in 1993 is trivial to store on your smartphone or any of the free storage accounts you can get from the various cloud providers. And while I wouldn't trust my life to Google Drive or Dropbox, I'd trust them more than I trust a 30 year old hard drive.
|
|
#
?
Nov 5, 2023 22:46
|
|
|
I also think it's interesting to say that in 30 years you won't have a SATA controller or Blu-Ray drive since those are both clearly technologies that have reached a plateau. SATA has been capable of 6Gb/s for more than a decade, but that's still a lot faster than hard drives are today or are likely to be for a while. Hard drives aren't going away so I would bet motherboards are going to continue to have SATA ports for a while yet. Even if we come out with a new and faster version of the standard I would expect backward compatibility to be a priority, and regardless USB-SATA dongles will be a thing for a while yet too. Blu-Ray is in more danger since interest is waning for optical media in general. Most computers don't have optical drives at all now and Blu-Ray never reached majority uptake, unlike DVD drives 15 years ago or certainly CD drives 20 years ago. The supporting hardware like the optics and spindle are way more specialized than a SATA controller, so OEMs need to sell a lot more units to justify continued production - but at least Blu-Ray also probably doesn't have to worry about being replaced by a successor without backward compatibility. The small segment of the population which cares about keeping a physical movie library in an era of ubiquitous streaming and no-optical-drive laptops (and the console game factor, also dwindling) may not be enough to keep optical media as a market alive for 30 more years but it seems reasonable to expect that you could still find a working drive with a bit of effort, like a VCR today. Eletriarnation fucked around with this message at 23:25 on Nov 5, 2023 |
|
#
?
Nov 5, 2023 23:10
|
|
|
I actually wouldn't be surprised if SATA eventually got dropped from most chipsets. I'd wager we're already at or near the point where more computers are sold out of the box only with NVMe storage installed than with SATA-only or SATA+NVMe configurations.
|
|
#
?
Nov 5, 2023 23:26
|
|
|
Eletriarnation posted:I also think it's interesting to say that in 30 years you won't have a SATA controller or Blu-Ray drive since those are both clearly technologies that have reached a plateau. SATA has been capable of 6Gb/s for more than a decade, It's always bothered me that SATA stopped advancing but SAS didn't, there's 24Gbit and higher speeds are in development, even if it seems kind of pointless in the light of newer PCIe standards
|
|
#
?
Nov 5, 2023 23:27
|
|
|
IOwnCalculus posted:I actually wouldn't be surprised if SATA eventually got dropped from most chipsets. I'd wager we're already at or near the point where more computers are sold out of the box only with NVMe storage installed than with SATA-only or SATA+NVMe configurations. Yeah, I am confident that most new systems are NVMe only at this point - certainly if you consider laptops - so Intel and AMD are probably having internal conversations about when they should drop SATA support from chipsets. I think that will be at least a few years away for desktops because consumer HDDs are still a thing and most people aren't buying servers to plug them into, but even once the chipset support ends you'll probably have several years of premium motherboards offering a couple SATA ports via an ASMedia chip or whatever. If you allow for enterprise SAS hardware then I'm not even sure that's declining - HDDs are continuing to get bigger, even if they're not getting much faster - so I think it will be a long, long time, definitely more than 30 years (assuming civilization continues to thrive etc.) until finding any device at all that can read a SATA disk is a real challenge. HalloKitty posted:It's always bothered me that SATA stopped advancing but SAS didn't, there's 24Gbit and higher speeds are in development, even if it seems kind of pointless in the light of newer PCIe standards I get that but with all the standards out there it doesn't seem like such a bad thing when you have a perfect justification to stop and say "good enough" for a while. I think a lot of Bluetooth controllers still use 12Mbps USB 1.1 as an interface, because BT's so bandwidth throttled that it doesn't matter. Sure, someone could do the work to uplift those to support a 3.0 standard, but they could also work on an improvement that actually expands the system's capabilities. e: Continuing my aside, I would love for Intel to do the work to have their BT controllers share the PCIe interface with their WiFi controllers so that you don't have to run a cable to an internal USB header if you're using an M.2 WiFi module in a riser card for an x1 slot. Seriously, how in the hell is that still a thing? Eletriarnation fucked around with this message at 23:57 on Nov 5, 2023 |
|
#
?
Nov 5, 2023 23:43
|
|
|
There was an evolution of sata past 6gbit, it was called sata express, and it died basically immediately due to nvme
|
|
#
?
Nov 6, 2023 00:10
|
|
|
It's way off in a tangent, but I thought it'd be interesting for this conversation. My grandfather dictated some history of his life onto tape in the mid 1980s and my family was just recently able to digitize it. I was amazed it still worked. Really was a special moment for my family to be able to listen to it together.
|
|
#
?
Nov 6, 2023 02:30
|
|
|
i think the discussion hangs mostly on whether you have a resourceful custodian
|
|
#
?
Nov 6, 2023 02:54
|
|
|
Sony are single-handedly making sure that optical drives stay in more households by hiking the price on the disc-less version of their 3 year old console.
|
|
#
?
Nov 6, 2023 03:22
|
|
|
Sub Rosa posted:Building a new NAS. Decided to try Unraid after being pretty happy with XPEnology for over a decade I think. Just a heads up, if you're going to be ingesting a lot of data once your array is setup, don't bother with parity until you're done with that or you're going to have a miserable time of it. You're effectively writing the ingested data multiple times (once to disk, once to parity) which massively slows down the throughput. Additionally, you can pick up a small noctua fan and zip tie it to your HBA's heatsink, which is a big help with how hot they get.
|
|
#
?
Nov 6, 2023 10:24
|
|
|
Theophany posted:Just a heads up, if you're going to be ingesting a lot of data once your array is setup, don't bother with parity until you're done with that or you're going to have a miserable time of it. You're effectively writing the ingested data multiple times (once to disk, once to parity) which massively slows down the throughput. Theophany posted:Additionally, you can pick up a small noctua fan and zip tie it to your HBA's heatsink, which is a big help with how hot they get.
|
|
#
?
Nov 6, 2023 12:37
|
|
|
evil_bunnY posted:lol what is this amateur hour poo poo Unraid  I made the same mistake with my first build with it, moving 20+ TB to a new array with parity on which took over a week. When I checked their forums, the common advice for a new build is not to enable your parity disk(s) until you've done your first ingest/data migration then set up and build parity once all your data is on the array.
|
|
#
?
Nov 6, 2023 12:43
|
|
|
Theophany posted:Just a heads up, if you're going to be ingesting a lot of data once your array is setup, don't bother with parity until you're done with that or you're going to have a miserable time of it. You're effectively writing the ingested data multiple times (once to disk, once to parity) which massively slows down the throughput. Is reconstruct write not the default yet? Guess not, to reduce power consumption. Make sure it's enabled, as it will almost certainly increase write speeds.
|
|
#
?
Nov 6, 2023 12:46
|
|
|
So following my earlier post my company went ahead and bought a Synology DiskStation. I’ve been trying to set it up and have discovered that none of the network ports in our office are actually connected; every single machine is networked over WiFi and the only physical network ports are the ones connecting the switch and WiFi hardware. Going to have to do some ratchet bridged network poo poo on a desktop machine to use this thing, lmao
|
|
#
?
Nov 6, 2023 13:39
|
|
|
Theophany posted:Unraid
|
|
#
?
Nov 6, 2023 14:19
|
|
|
Welcome to unraid
|
|
#
?
Nov 6, 2023 14:24
|
|
|
TACD posted:So following my earlier post my company went ahead and bought a Synology DiskStation. I’ve been trying to set it up and have discovered that none of the network ports in our office are actually connected; every single machine is networked over WiFi and the only physical network ports are the ones connecting the switch and WiFi hardware.
|
|
#
?
Nov 6, 2023 14:29
|
|
|
evil_bunnY posted:This is so funny because moving 1) your parity should be loving distributed to equalize wear anyway 2) 20TB at SATA 5.4/7.2k speed takes like 2+ days. Yeah, it's a bit of a curiosity. I think its popularity is that once you're up and running, even the greenest of hobbyists can have it functioning as a NAS with a couple of VMs and a bunch of docker containers with minimal effort or prior *nix experience, whilst still having to ability to go beyond what a commodity grade Synology or QNAP box will allow you to do once you get to grips with it.
|
|
#
?
Nov 6, 2023 14:38
|
|
|
evil_bunnY posted:This is so funny because moving 1) your parity should be loving distributed to equalize wear anyway 2) 20TB at SATA 5.4/7.2k speed takes like 2+ days. The design is based around dedicated parity disks, it's designed for flexibilty for the home user (which it achieves), low power consumption (without reconstruct write enabled, only the parity and written drive need to be awake). being able to mix disk sizes at will, and having non-striped data are hallmarks of unraid. If you want to write files bigger than your drives, crave speed, or have a beautifully built box from day 1 that's filled with equal size drives, unraid is a poor fit; but just because it does some core things differently doesn't mean it's useless. For those who have a random pile of drives and don't care about high write speed, which describes many linux iso hoarders, it's pretty great HalloKitty fucked around with this message at 14:55 on Nov 6, 2023 |
|
#
?
Nov 6, 2023 14:53
|
|
|
I have previously used XPenology which also can handle mismatched disks, and I basically already miss it. I guess it isn't too late to reconsider Unraid entirely.
|
|
#
?
Nov 6, 2023 15:07
|
|
|
Pablo Bluth posted:Can you not locate it so it's next to the physical switch? Or get a physical port connected? Bodging a solution over wifi is just asking for a lifetime of problems and/or bad performance, just to save a bit of work now. I’m sure it’ll be resolved and work properly sooner or later but I’d like to at least have the system largely set up and ready to go before it’s a panic situation.
|
|
#
?
Nov 6, 2023 15:09
|
|
|
Theophany posted:Just a heads up, if you're going to be ingesting a lot of data once your array is setup, don't bother with parity until you're done with that or you're going to have a miserable time of it. You're effectively writing the ingested data multiple times (once to disk, once to parity) which massively slows down the throughput. Wait, why would you want your data sitting at rest on your array without any parity? That seems... dumb?
|
|
#
?
Nov 6, 2023 15:20
|
|
|
Nitrousoxide posted:Wait, why would you want your data sitting at rest on your array without any parity? That seems... dumb? It is, but the argument is that it is only for a very short period of time (or at all if you're not migrating from another location). Once all your data is on the array, you assign your parity disks and it will build it. Of course, if you're doing a big migration, you've already got a copy of your data on the source, so parity is less important at that point in time.
|
|
#
?
Nov 6, 2023 16:37
|
|
|
I bought an Intel ARC GPU yesterday from Micro Center for my home NAS because it is slot powered, supports AV1 encoding, single slot (I think), and was only $110. Might be a couple more weeks before I do anything with it as I need to do some case modification to let my HDD drive bays fit properly (it has some little tabs that stick out that I either need to bend flat or remove).
|
|
#
?
Nov 6, 2023 19:33
|
|
|
Kibner posted:I need to do some case modification to let my HDD drive bays fit properly (it has some little tabs that stick out that I either need to bend flat or remove). As someone who as delt with this before a couple times, I'm going to suggest bending the tabs. Even on a good quality case it was less work overall vs a Dremel and you don't have to deal with metal shaving cleanup in your case nor filing down the sharp edges and the blood sacrifice that will eventually be extracted if you don't. There are probably better ways to do it, but a harbor freight c-clamp and elbow grease did the trick on my last one with minimal fuss.
|
|
#
?
Nov 6, 2023 21:45
|
|
|
Fancy_Lad posted:As someone who as delt with this before a couple times, I'm going to suggest bending the tabs. Even on a good quality case it was less work overall vs a Dremel and you don't have to deal with metal shaving cleanup in your case nor filing down the sharp edges and the blood sacrifice that will eventually be extracted if you don't. I tried yesterday with a heat gun and some pliars and it just wouldn't budge.  I could always try it again, I guess.
|
|
#
?
Nov 6, 2023 21:59
|
|
|
Kibner posted:I bought an Intel ARC GPU yesterday from Micro Center for my home NAS because it is slot powered, supports AV1 encoding, single slot (I think), and was only $110. Might be a couple more weeks before I do anything with it as I need to do some case modification to let my HDD drive bays fit properly (it has some little tabs that stick out that I either need to bend flat or remove). Definitely keen to hear a trip report!
|
|
#
?
Nov 6, 2023 23:18
|
|
|
Theophany posted:It is, but the argument is that it is only for a very short period of time (or at all if you're not migrating from another location). Once all your data is on the array, you assign your parity disks and it will build it. The ability to turn on parity after the fact is cool, but intentionally doing that just to gain a bit of performance feels so bad.
|
|
#
?
Nov 7, 2023 01:04
|
|
|
I'm getting a very steady 80MB/s copying data over now that the parity check completed with zero errors, and that's not so slow as to bother me. Thanks for the tip about turning reconstruct write on.
|
|
#
?
Nov 7, 2023 16:49
|
|
|
wolrah posted:But then you have a period of time where the data could potentially have been silently corrupted and your parity will be built from that corrupted data. With UnRaid's standard arrays, if data were silently corrupted on a data disk, I'm pretty sure that even if the parity has the 'uncorrupted' information, it would simply get rewritten anyway when the next parity check occurs. UnRaid's standard arrays don't protect from bit rot. If that's a concern, you'll probably want ZFS. For noncritical data like linux ISOs, not sure it's a big deal. It isn't for me at least.
|
|
#
?
Nov 7, 2023 18:21
|
|
|
Unraid seems like a good fit for storing stuff that you can just download again if some of it gets corrupted, with the ability to run Plex servers, the various arrs etc. It wouldn't be first pick as a home server that people were backing their PCs up to and storing photos that they wanted to keep forever.
|
|
#
?
Nov 7, 2023 18:31
|
|
|
|
| # ? May 15, 2024 10:26 |
|
|
IMO just don't use a NAS OS and file system that can suffer silent bitrot. Synology (BTRFS) and TrueNAS (ZFS) both have hash checks and can self-repair as long as the damage is within the limits allowed by the parity for the array.
|
|
#
?
Nov 7, 2023 18:39
|
|