|
ZFS fun moment. NZBGet had a beta build that created monstrous log files due to a bug, like drive-consuming sizes. My log was going to my array with lz4 compression turned on ... a 1.1TB log file was 28GB on disk when I found it and deleted it.
|
# ? Jun 21, 2018 17:12 |
|
|
# ? May 23, 2024 15:43 |
|
Just went and bought another easystore, this time for USB use with my synology unit. Just out of curiosity, I checked what kind of drive it was with DriveDX...EFAX, again! I seem to win the drive lottery every time.
|
# ? Jun 21, 2018 20:52 |
|
More of a Transmission on Linux question, please point me in the right direction if this is the wrong place. Trying to set it up so that I can download torrents into 2 different folders, Folder A and Folder B. When I do this, the torrent file itself still goes into config/torrents/, but the torrent data gets moved to the correct folder after the download is complete. I assume this uses syslinks, but where is the data stored that links that torrent to the file in either Folder A or Folder B? I want to make sure that is backed up or accounted for (using Docker, so it's important) so that if I need to re-deploy I don't have to go through that giant folder of torrents and point them at the data in the correct folder. If that doesn't make sense I can re-state. Anyone have any clue? Thank you.
|
# ? Jun 22, 2018 02:40 |
|
Internet Explorer posted:I assume this uses syslinks, but where is the data stored that links that torrent to the file in either Folder A or Folder B? I want to make sure that is backed up or accounted for (using Docker, so it's important) so that if I need to re-deploy I don't have to go through that giant folder of torrents and point them at the data in the correct folder. If you think it's a symlink you can follow it with "readlink -f /path/to/symlink".
|
# ? Jun 22, 2018 02:49 |
|
Paul MaudDib posted:If you think it's a symlink you can follow it with "readlink -f /path/to/symlink". I guess my question is more, will that information get destroyed if/when I update the Docker container from the latest image. I have the config files stored outside the container, the torrent files, the data, etc., but I just don't know about the mechanism that points the torrent to the data. If it's a symlink (thanks), I assume that's in the OS and that would be problematic?
|
# ? Jun 22, 2018 03:00 |
|
What's your log rotation tool/policy on unraid? Recently I had a bug spamming syslog and was wondering what's a good strategy, I didn't find any plugins to deal with this. The "Fix Common Problems" plugin is great to find out about this sort of thing btw.
|
# ? Jun 22, 2018 03:09 |
|
You should be able to update the container without problems. All of the containers I use store literally everything outside of themselves on the main filesystems, so when it's time to update you just stop the container, rm the old instance, pull the latest, and redeploy with the same command / settings. Picks up right where it left off.
|
# ? Jun 22, 2018 03:10 |
|
Internet Explorer posted:I guess my question is more, will that information get destroyed if/when I update the Docker container from the latest image. I have the config files stored outside the container, the torrent files, the data, etc., but I just don't know about the mechanism that points the torrent to the data. If it's a symlink (thanks), I assume that's in the OS and that would be problematic? purely talking out my rear end here but I assume there is a directory somewhere in the real OS and Docker has a symlink to that directory. That would be the sensible way to do it, and you would just need to re-create that symlink with the new container via whatever mechanism docker normally uses. If it's actually a mess of individual symlinks to individual files then you have a problem and you should start thinking about something like rsync to help back that up.
|
# ? Jun 22, 2018 03:12 |
|
I currently have a Ubuntu file server with 3x 3TB WD REDs in RAIDZ. Disk are just over 5 years old and the array is full. Looking to replace them all. Data will be copied from old array to new array. Would you rather have more drives in a higher RAIDZ mode or fewer larger drives in a lower RAIDZ mode? For example: 5x 6TB WD Reds in RAIDZ2 for 18TB of usable storage with 2 parity or 3x 10TB WD Reds in RAIDZ for 20TB of usable storage with 1 parity Cost for each option is close enough that it's not factor for me. The extra 2TB isn't a big concern for me either. I am more concerned about reliability and data loss issues.
|
# ? Jun 22, 2018 15:21 |
|
Lowen SoDium posted:I currently have a Ubuntu file server with 3x 3TB WD REDs in RAIDZ. Disk are just over 5 years old and the array is full. Looking to replace them all. Data will be copied from old array to new array. You want dual parity in both scenarios, thus the first option.
|
# ? Jun 22, 2018 15:32 |
|
IOwnCalculus posted:You should be able to update the container without problems. All of the containers I use store literally everything outside of themselves on the main filesystems, so when it's time to update you just stop the container, rm the old instance, pull the latest, and redeploy with the same command / settings. Picks up right where it left off. Paul MaudDib posted:purely talking out my rear end here but I assume there is a directory somewhere in the real OS and Docker has a symlink to that directory. That would be the sensible way to do it, and you would just need to re-create that symlink with the new container via whatever mechanism docker normally uses. It turns out that the part that says "this torrent file corresponds with this torrent data" is stored in a .resume file located in \config\resume. So, as long as the config folder is stored outside of the container, you are good to go. https://github.com/transmission/transmission/wiki/Transmission-Resume-Files Not sure why I had trouble finding that info, but in my defense when you try to click on the "Configuration File" link (or many of the other links) on Transmission's wiki it is a dead link, so finding documentation was awkward. Thanks all for the help. And hopefully that little bit of information helps someone at some point. Internet Explorer fucked around with this message at 16:39 on Jun 22, 2018 |
# ? Jun 22, 2018 16:16 |
|
Lowen SoDium posted:I currently have a Ubuntu file server with 3x 3TB WD REDs in RAIDZ. Disk are just over 5 years old and the array is full. Looking to replace them all. Data will be copied from old array to new array. You have to consider that the rebuilding requires lots of writes and reads across all drives. If you have 1 parity, then you need to hope to hell or high water no other drive goes offline/goes bad while rebuilding. This happens (seeingly) more often if you raid is made of hard drives from the same batch. If you have 2 parity, one other drive can go bad during the heavy write/reads and it will not stop the rebuild process.
|
# ? Jun 22, 2018 16:29 |
|
3x10 is going to be the more reliable option there, but it also has lower throughput.
|
# ? Jun 22, 2018 16:42 |
|
Paul MaudDib posted:3x10 is going to be the more reliable option there, but it also has lower throughput. How could single parity with 10TB disks be more reliable than double parity on 6TB ones?
|
# ? Jun 22, 2018 16:57 |
|
HalloKitty posted:How could single parity with 10TB disks be more reliable than double parity on 6TB ones? it isn't...
|
# ? Jun 22, 2018 17:31 |
|
Paul MaudDib posted:3x10 is going to be the more reliable option there, but it also has lower throughput. I am not expecting throughput to be a concern. It hasn't been with the current 3x 3TB array. I am not hosting VMs or anything on it that needs real high throughput. HalloKitty posted:How could single parity with 10TB disks be more reliable than double parity on 6TB ones? More drives means more points for failure means a lower MTBF. But I am not sure it works out to be better for data integrity in these specific cases. I mean, I am actually not sure and I am trying to find out, which is why I asked. I found this calculator and if I am using it right, it does look like 5x 6TB should have a much lower chance of data loss due to drive failure. edit: forgot to paste the link http://wintelguy.com/raidmttdl.pl Lowen SoDium fucked around with this message at 17:58 on Jun 22, 2018 |
# ? Jun 22, 2018 17:33 |
|
EVIL Gibson posted:You have to consider that the rebuilding requires lots of writes and reads across all drives. Counterpoint: ZFS does not poo poo the bed when this happens. I had this occur once during a rebuild and it marked a few files as corrupt, which let me easily restore them from backup.
|
# ? Jun 22, 2018 18:00 |
|
Lowen SoDium posted:I found this calculator and if I am using it right, it does look like 5x 6TB should have a much lower chance of data loss due to drive failure. Correct.
|
# ? Jun 22, 2018 18:12 |
|
IOwnCalculus posted:Counterpoint: ZFS does not poo poo the bed when this happens. I had this occur once during a rebuild and it marked a few files as corrupt, which let me easily restore them from backup. Really, it's more safety concious where I build in case the worst case possible which always seems to happen lol
|
# ? Jun 22, 2018 18:30 |
|
EVIL Gibson posted:Really, it's more safety concious where I build in case the worst case possible which always seems to happen lol Oh yeah, I agree it's absolutely worth considering (and a reminder that RAID IS NOT BACKUP). It's just so very nice that ZFS will do its best to work around the issue instead of rolling over and dying because of one block with a URE.
|
# ? Jun 22, 2018 19:05 |
|
Lowen SoDium posted:I found this calculator and if I am using it right, it does look like 5x 6TB should have a much lower chance of data loss due to drive failure. If I'm reading that calculator correctly you are 10,000 times more likely to lose data with a 3x10tb raid 5 compared to a 5x6tb raid 6
|
# ? Jun 22, 2018 19:52 |
Droo posted:If I'm reading that calculator correctly you are 10,000 times more likely to lose data with a 3x10tb raid 5 compared to a 5x6tb raid 6 Remember, ZFS comes to us from Big Iron, where Reliability, Availability and Servicability are enshrined among the highest of principles to such a degree that ZFS was designed without any kind of memory checksumming (except as a debug feature) because ECC and lockstepping+parity RAM is practically the default. In the case of ZFS, it's arguably completely tied to the availability section of the RAS trifecta, because you always want to have online backup and/or offline backup of the data.
|
|
# ? Jun 22, 2018 21:20 |
|
Don't use RAID 5. Who cares about the stats on drives failing. RAID 5 is old lovely technology.
|
# ? Jun 23, 2018 01:01 |
|
Thoughts on using a PowerEdge T410 as a NAS? I could get one for extremely cheap, but I want to see if there's anything I should worry about with hardware like that considering how old it is. Desktop HDDs and PSUs and everything are fairly standardized throughout generations, I assume the same isn't necessarily true for servers, especially servers from '09.
|
# ? Jun 23, 2018 03:13 |
|
It should be *very* cheap since those E5500 generation procs are both ancient and power hungry. They can be damned powerful too, but they love to suck down juice. I have a couple of X5647s in my home server, that I'm about to swap out for L5640s at about half the TDP because the box here at home just never has anything that heavy to do. I'd drop to a single socket board if I could get one cheap enough that also had a LSI controller onboard like mine Doubtful that the PSU is standardized, because Dell. It looks like it might be new enough to have at least a LSI2xxx generation HBA, if it has one at all, so that's good.
|
# ? Jun 23, 2018 07:20 |
|
Are the poweredge towers as insanely loud as the rack mounted ones? I am guessing not because those 2U fans have got to really scream.
|
# ? Jun 23, 2018 07:29 |
IOwnCalculus posted:It should be *very* cheap since those E5500 generation procs are both ancient and power hungry. They can be damned powerful too, but they love to suck down juice.
|
|
# ? Jun 23, 2018 15:29 |
|
priznat posted:Are the poweredge towers as insanely loud as the rack mounted ones? I am guessing not because those 2U fans have got to really scream. Not generally, but if the fans need to turn up then they can drat sure get that way.
|
# ? Jun 23, 2018 22:13 |
|
priznat posted:Are the poweredge towers as insanely loud as the rack mounted ones? I am guessing not because those 2U fans have got to really scream. No, there's a massive difference. I have a dual socket rackmount that makes the noise you'd expect from all those small fans, and I have that in a storage room in the office so as not to annoy everyone. We have a couple of towers in the main office for the main server and replication. The UPS for the main server has a louder fan and it's no trouble in a small office.
|
# ? Jun 24, 2018 01:04 |
|
D. Ebdrup posted:All of this is true, but unfortunately they all lack AES-NI which makes them pretty much useless if you're wanting to do GELI encryption on your array and/or external drives. Not sure what exactly GELI is, it looks like a BSD thing? How essential would that be? I really just want this thing for backups of my media and downloads and documents and my wife’s art and poo poo. I could even keep it on a separate VLAN from the WiFi and set up ssh access in case I needed to get on with my laptop without plugging in on Ethernet. Not sure why I would though, I could just keep it isolated on the Ethernet only VLAN. I would probably be keeping it in the basement so I don’t have to give a poo poo about noise.
|
# ? Jun 24, 2018 01:13 |
|
It's worth noting that the E5600 chips should be a drop in replacement, are dirt cheap, and do support AES-NI. I will have a couple of X5647s available soon if anyone wants them. I would expect a tower setup like that to be louder than a typical desktop, but far quieter than a 1U or 2U screamer. You can probably swap the fans with standard ones too if that becomes an issue.
|
# ? Jun 24, 2018 04:01 |
|
Would going from a W3565 to an L5640 help my idle power at all? Doing the math, I'm pulling about 120W at idle spinning 4 drives, measured at the wall. I typically figure about 5W per drive, so 100W idle system power. Or, maybe one of the low-power quads? Or if it's all the same, would going to a X5680 hurt me if it's mostly idle? From what I've read, most of the power savings has been in faster SpeedStep, so if you are either full-idle or full-load then maybe it doesn't hurt as much? The one thing I can't help is this poo poo HP power supply in my Z400... I don't even think it's 80+. But I'd also need to find one of those adapters that mates its non-standard 24-pin connector to a normal ATX.
|
# ? Jun 24, 2018 04:32 |
|
I'm currently running a P212 in an old Microserver N54L (Gen7) since it was crazy cheap a few years ago and had no issues saturating the SATA--II bus. It still runs Server 2012 and is only a 2x3TB Red setup. What is the next upgrade path to get SATA 6Gb capability that the drives have? I'd hope for dual internal SAS ports so I can use the 4x256GB SSD array I got crazy cheap of SA-Mart (currently its on the crappy onboard AMD FakeRAID). I looked up the P222 but I don't think its new enough. This thing is going to be run until it dies, probably shortly since the drive have 40K hours on them.
|
# ? Jun 24, 2018 06:05 |
|
DJ Commie posted:I'm currently running a P212 in an old Microserver N54L (Gen7) since it was crazy cheap a few years ago and had no issues saturating the SATA--II bus. It still runs Server 2012 and is only a 2x3TB Red setup. What is the next upgrade path to get SATA 6Gb capability that the drives have? There are almost no modern mobos with onboard SAS. I won't say zero, but very few, and they are probably restricted to rackmount server hardware. There are some that use mini-SAS ports for their SATA controllers, you can get up to roughly 12 SATA-III ports on a mobo, in a format as small as mini-ITX (with compromises).
|
# ? Jun 24, 2018 06:16 |
22 Eargesplitten posted:Not sure what exactly GELI is, it looks like a BSD thing? How essential would that be? I really just want this thing for backups of my media and downloads and documents and my wife’s art and poo poo. I could even keep it on a separate VLAN from the WiFi and set up ssh access in case I needed to get on with my laptop without plugging in on Ethernet. Not sure why I would though, I could just keep it isolated on the Ethernet only VLAN. Nowadays, I'd say it's pretty drat important to ensure that your backups are encrypted, and having your array be encrypted too means that it's very very easy to decommision, as you simply recycle it in a bin without worrying about anyone being able to get to whatever you've stored there (financial or otherwise personal documents, for example). BlankSystemDaemon fucked around with this message at 12:25 on Jun 24, 2018 |
|
# ? Jun 24, 2018 12:11 |
|
Paul MaudDib posted:There are almost no modern mobos with onboard SAS. I won't say zero, but very few, and they are probably restricted to rackmount server hardware. I don't know where you're getting I'm using native SAS, its just the standard connector since it has 1 cable 4 drive support, the HBA uses it, as well as the motherboard's original controller. The P212 is currently going onto the N54L's drive cage via a SAS to SATA breakout cable, and the onboard port going through a similar breakout cable to the SSDs. I'm just asking about what low profile 4/8x PCIe HBA would give me 6Gb SATA, and probably surplus or system pulls from eBay, and utilize the internal SAS connector. I use hardware RAID-1 on the P212, and FakeRAID on the SSDs.
|
# ? Jun 24, 2018 23:31 |
|
Need validation of plan and/or advice. Budget is < $1000 prefer to buy through Amazon Random background Ok quick sanity check. I have like, twenty....two? years of files I have been slowly accumulating since I was a nerd in middle school. Since college mostly just backups of digital camera photos etc and more recently cell phone photos/videos. I have 4-ish TB of data, no video no Plex, no Kodi/XBMC bullshit, no streaming security cameras, ...I guess I have ~15GB of MP3s from the MP3 craze of the early 2000s. At some point I will probably be adding baby photos/videos and want to back those up as well. Anyways, no hard performance requirements, modern technology vastly outstrips my needs. My existing setup is this massive home built 4U (rackmount) core i5 haswell (2013?) era device (also doubled as a VM lab) with 4x 2-4TB drives, it is massive, noisy, and helped me learn windows server data stuff. Anyways it's huge and noisy and 5 years old at this point. I need something smaller and quieter for my 4TB of data. Living in California my house is substantially smaller than the old one in Texas. Here is my plan: Initial setup (phase I ) - Buy a Synology DiskStation DS418 for like, $370 - Buy 2 x brand new 6TB drives, one each from different manufacturers @ $165 ea = $330 - Setup the disks in SHR (Synology Hybrid RAID) array - Rsync(?) 4TB from the old machine to the new synology diskstation Phase II - Format the old 2 and/or 4TB disks still in my old server - Add them to the Synology DS418 as part of the above SHR (Synology Hybrid RAID) array - triple zero out the old unused disks drill a hole in them - Throw away/craigslist the old core i5, old now-dead disks Total cost: $700-ish I plan on connecting this thing primarily to a windows 10 laptop a couple times a week via gig-e and very periodically to a 2017-era mac laptop Hadlock fucked around with this message at 05:32 on Jun 25, 2018 |
# ? Jun 25, 2018 05:30 |
|
Hadlock posted:Need validation of plan and/or advice. Budget is < $1000 prefer to buy through Amazon This seems like a great idea to me. I'd do the same.
|
# ? Jun 25, 2018 13:26 |
|
"- Format the old 2 and/or 4TB disks still in my old server - Add them to the Synology DS418 as part of the above SHR (Synology Hybrid RAID) array" Sell the 2 and 4tbs and add 6tbs when you have more money. Always keep a cold spare 6tb disk around. Aside from that, your plan looks good.
|
# ? Jun 25, 2018 13:46 |
|
|
# ? May 23, 2024 15:43 |
|
How often should I consider proactively replacing disks? My NAS runs 24x7 but doesn't see a lot of day to day use. I'm running 4x Western Digital WD40EFRX 4 TB WD Red, all purchased almost exactly 4 years ago.
|
# ? Jun 25, 2018 19:21 |