|
Less Fat Luke posted:Well two things, first we're talking about home usage, and second "citation needed". Recommended by whom? The ZFS on Linux team or docs? Because at this point ECC and RAM sizing seems to be very much tribal knowledge at best or an old wives tale at worst. Officially, the recommendation for ZFS is 1GB of memory and 64MB of diskspace (ie. you can't run it on floppies). Second one is easy; the deduplication table takes up ~330 bytes per LBA on Illumos-derived installations, and ~70 bytes on what is now OpenZFS. If it seems familiar, it's because it's the same for L2ARC. EDIT: Oh, and I found the pdf presentation of things Matt Ahrens wants to do to the dedup feature to make it not suck. So far as I know, he's still looking for a company who's using dedup to fund development. BlankSystemDaemon fucked around with this message at 10:26 on Dec 14, 2020 |
#
?
Dec 14, 2020 10:21
|
|
|
|
| # ? May 16, 2024 21:38 |
|
|
Most dedupe on a robust cow-snapshot capable filesystem is a engineering and implementation mistake on the user side. Change my mind. For a home user where it's mostly media files I simply can't see a huge benefit. You aren't shipping storage to 1000 users who then make 1000 copies of the same file because they download it to their desktop which is then mapped to a remote file share.
|
|
#
?
Dec 14, 2020 16:20
|
|
|
IOwnCalculus posted:Speaking of ZFS and resilvering, is there any way to tell ZFS to do multiple drive replacements simultaneously? I'm doing some drive replacements in four-drive raidz vdevs and even if I stick them in immediately following commands: Quoting myself but it *seems* after digging through the documentation that the best way to handle this is to simply issue a "zpool resilver <pool>" after the last zpool replace, which restarts the resilver with all pending drives included. Guess I'll find out in the next ~48hr if this drive replacement gets finished with a single resilver pass.
|
|
#
?
Dec 14, 2020 17:39
|
|
|
I did a zfs send from an old drive to new one, by snapshotting and then zfs send -Rv oldpool@20201213 | zfs receive newpool/data Whats the best way to verify that the files are all the same and everything?
|
|
#
?
Dec 14, 2020 17:56
|
|
|
Wild EEPROM posted:I did a zfs send from an old drive to new one, by snapshotting and then zfs send -Rv oldpool@20201213 | zfs receive newpool/data Probably snapshots and zfs diff? I'm not sure if that'll work though
|
|
#
?
Dec 14, 2020 18:10
|
|
|
Wild EEPROM posted:I did a zfs send from an old drive to new one, by snapshotting and then zfs send -Rv oldpool@20201213 | zfs receive newpool/data E: loving beaten. BlankSystemDaemon fucked around with this message at 18:35 on Dec 14, 2020 |
|
#
?
Dec 14, 2020 18:33
|
|
|
1. This last resilver is going much, much faster and is already only a few minutes from completion. Yes, issuing another zpool resilver after a batch of zpool replace commands is your friend. 2. Yet again, zfs is saving my array from multiple UREs during a rebuild. My last two 3TB Reds both randomly hosed off during this last rebuild, which is good because they're actively being replaced. code:IOwnCalculus fucked around with this message at 00:14 on Dec 16, 2020 |
|
#
?
Dec 15, 2020 23:41
|
|
|
12TB My Book for $200 as an amazon daily deal: https://smile.amazon.com/12TB-Desktop-External-Drive-WDBBGB0120HBK-NESN/dp/B07X3RBTQT
|
|
#
?
Dec 17, 2020 16:27
|
|
|
Finally got the final parts for my replacement NAS and got Debian installed on it. Imported the old ZFS pool with zero issues, no data loss or anything resembling a hiccup. Got plex going well enough, although I discovered that the wizard won't run if you access via FQDN rather than IP so I had a bit of a pain there until I found someone on the forums mentioning that and it put me on the right track. Unifi stunk. Those dicks still don't support anything remotely modern Debian-wise (or Ubuntu or really anything.) They still use Java 8, an ancient version of MongoDB, and just in general their poo poo doesn't work on buster without some obnoxious steps, and even then a few things don't work. I got it going with This Guy's Blog Post if anyone else needs help, there's some other instructions but they're doing more than they need to. So far the only downside I've found here is that you can't outright download backups as you get a java error, but it still creates them in /var/lib/unifi/backup so you can scp them somewhere safe. I unfortunately didn't have a saved backup so I had to hard reset and re-adopt all my APs as well as recreate my wireless networks and my MAC whitelist for the IoT network (wemo switches and the like don't support AES/CCMP only so they get to go on a slower and somewhat segregated SSID). That whole unifi thing was way more trouble than it needed to be, and unfortunately the previous drive was a completely dead SSD so I wasn't able to get anything off of it, and ironically I'd never backed the NAS itself up to the ZFS pool  Got a couple cron jobs to set up and need to get mailx working through gmail again so I can send myself zpool status reports again, plus a couple odds and ends. Including setting Samba and NFS up again, but that'll be quick. Honestly outside of the unifi pain, this was a pretty painless experience. Importing the zpool took about 6 seconds, it was great. Fractal 304 is a nice but small case and I ended up having my tiny-handed wife do cable management in it because my dumb clumsy fingers were tired, plus she does a better job. ssb fucked around with this message at 14:31 on Dec 19, 2020 |
|
#
?
Dec 19, 2020 05:43
|
|
|
shortspecialbus posted:Unifi stunk. Those dicks still don't support anything remotely modern Debian-wise (or Ubuntu or really anything.) They still use Java 8, an ancient version of MongoDB, and just in general their poo poo doesn't work on buster without some obnoxious steps, and even then a few things don't work. I got it going with This Guy's Blog Post if anyone else needs help, there's some other instructions but they're doing more than they need to. So far the only downside I've found here is that you can't outright download backups as you get a java error, but it still creates them in /var/lib/unifi/data/backup so you can scp them somewhere safe. I unfortunately didn't have a saved backup so I had to hard reset and re-adopt all my APs as well as recreate my wireless networks and my MAC whitelist for the IoT network (wemo switches and the like don't support AES/CCMP only so they get to go on a slower and somewhat segregated SSID). That sounds kinda fragile, and will probably be a pain in the rear end to maintain with upgrades and such as dependencies change. I wonder if running a docker container of the unifi controller would have been less hassle? https://hub.docker.com/r/linuxserver/unifi-controller I wanted the painless setup experience and paid the unifi tax on the cloud key gen 2. It's a nice little device but god drat why does it have to be so expensive.
|
|
#
?
Dec 19, 2020 06:58
|
|
|
Yeah that should have went in a docker image or LXC. Keep your OS clean and docker all the things.
|
|
#
?
Dec 19, 2020 07:48
|
|
|
It's something to think about, I've been meaning to screw around with docker. I wish unifi would just update their poo poo. Either way I can undo it easy enough if/when I have to, and I have a backup of the unifi config and adopted APs now which is really the main thing I need for it. Edit: Actually 2 of the APs didn't import properly but should be fixable. ssb fucked around with this message at 15:18 on Dec 19, 2020 |
|
#
?
Dec 19, 2020 14:14
|
|
|
fletcher posted:paid the unifi tax I find this faintly amusing, seeing as far as I know, there's nothing better for the price. I remember being quoted for wifi in a school building, and all the quotes were sky high (and some with centralised controllers, or "cloud" dependencies) compared to just buying some unifi gear, with clear prices on each piece upfront
|
|
#
?
Dec 19, 2020 14:45
|
|
|
Looked at the container, had it up and running in about 10 minutes. That was pretty easy. Thanks for the suggestion! It's interesting that I'm willing to do stuff on my own linux stuff that I'd never consider doing in a million years at work in an enterprise environment, and I think it stems from a monumental laziness combined with not caring if I have downtime and have to reinstall stuff. Either way docker was something that's been on the fringe of what I've been doing for a long time so it was nice to actually give it a go and learn a little about it, just need to spend some time understanding how it works now. Edit: Actually neither UAC-AP-Pro will adopt even after forgetting and resetting them, this is strange. Wonder if I need to downgrade the firmware on them. Edit 2: The container documentation covers this if I could be bothered to read. ssb fucked around with this message at 15:34 on Dec 19, 2020 |
|
#
?
Dec 19, 2020 15:05
|
|
|
HalloKitty posted:I find this faintly amusing, seeing as far as I know, there's nothing better for the price. I remember being quoted for wifi in a school building, and all the quotes were sky high (and some with centralised controllers, or "cloud" dependencies) compared to just buying some unifi gear, with clear prices on each piece upfront Wendell over at Level 1 has been exploring Engenius products as an alternative. I say, if a guy like him is willing to install it into his house over Ubiquiti, it's probably got something going for it. Looking over pricing, they seem comparable. SwissArmyDruid fucked around with this message at 15:46 on Dec 19, 2020 |
|
#
?
Dec 19, 2020 15:43
|
|
|
SwissArmyDruid posted:Wendell over at Level 1 has been exploring Engenius products as an alternative. I say, if a guy like him is willing to install it into his house over Ubiquiti, it's probably got something going for it. Looking over pricing, they seem comparable. Ubiquiti has a lot going for it, but they make a lot of dumb decisions, have firmwares that break things get released (although they've been better), and their cameras are a joke. Plus the aforementioned BS with not updating the controller software to support OS's that aren't end-of-lifed. If I didn't already have 3 access points that worked really well I'd probably be looking at alternatives myself.
|
|
#
?
Dec 19, 2020 15:55
|
|
|
shortspecialbus posted:Looked at the container, had it up and running in about 10 minutes. That was pretty easy. Thanks for the suggestion! A) next, look into docker compose and the ZFS driver for docker so your images use a ZFS slice instead of a sparse file. B) check out linuxserver.io images. Try to stick to them as the docker images use the same base layer which saves you space in the long run Docker is not perfect (it just got kicked out of Kubernetes) and some people prefer LXC which is also very good. But being able to write a docker compose file, have your full stack of apps up in a minute and a centralized config storage directory is pretty sweet. Just remember, anything inside of the container that is not mounted to persistent storage is ephemeral and will be lost the next time you pull the image for an update.
|
|
#
?
Dec 19, 2020 16:22
|
|
|
I did use docker-compose for it, I didn't bother putting it on the ZFS since a backup is plenty in case of catastrophe (honestly, discounting the install nonsense, it took me maybe 20 minutes to fully set up my networks and adopt the 3 APs in the unifi controller from scratch so even if I lose the backup I'm not really hurting too bad). Beyond that, I think my terminal laziness comes into play here. It works, I'm not excessively concerned about the rest. The container was a good idea though for sure. What do you mean in this context about "persistent storage"? I used the sample docker-compose.yml and changed the config location and the UID/GID and that's about it. It's just using the /var/lib/docker LVM volume I made I would imagine.
|
|
#
?
Dec 19, 2020 17:28
|
|
|
shortspecialbus posted:I did use docker-compose for it, I didn't bother putting it on the ZFS since a backup is plenty in case of catastrophe (honestly, discounting the install nonsense, it took me maybe 20 minutes to fully set up my networks and adopt the 3 APs in the unifi controller from scratch so even if I lose the backup I'm not really hurting too bad). Beyond that, I think my terminal laziness comes into play here. It works, I'm not excessively concerned about the rest. The container was a good idea though for sure. You want to mount the config directory to a folder on your host OS because otherwise you're gonna lose it when you update the container.
|
|
#
?
Dec 19, 2020 18:19
|
|
|
HalloKitty posted:I find this faintly amusing, seeing as far as I know, there's nothing better for the price. I remember being quoted for wifi in a school building, and all the quotes were sky high (and some with centralised controllers, or "cloud" dependencies) compared to just buying some unifi gear, with clear prices on each piece upfront Yeah it is quality gear (I haven't had any software/firmware issues many others have mentioned) but it's hard to justify spending so much for residential use. But drat it is nice having a stable network with great Wifi everywhere, and never having having to worry about performance issues. Xfinity Gigabit Pro is becoming available in my area. Apparently you can get a 2 Gbps connection through them now. I've only just finished getting everything setup for 1 Gbps...I think I will probably put off any upgrades beyond that for a few years. Too drat expensive to upgrade all these components now!
|
|
#
?
Dec 19, 2020 19:43
|
|
|
Matt Zerella posted:You want to mount the config directory to a folder on your host OS because otherwise you're gonna lose it when you update the container. OK, then I'm fine. Thanks.
|
|
#
?
Dec 19, 2020 19:54
|
|
|
shortspecialbus posted:Ubiquiti has a lot going for it, but they make a lot of dumb decisions, have firmwares that break things get released (although they've been better), and their cameras are a joke. Plus the aforementioned BS with not updating the controller software to support OS's that aren't end-of-lifed. If I didn't already have 3 access points that worked really well I'd probably be looking at alternatives myself. What's wrong with their cameras?
|
|
#
?
Dec 20, 2020 00:52
|
|
|
SwissArmyDruid posted:Wendell over at Level 1 has been exploring Engenius products as an alternative. I say, if a guy like him is willing to install it into his house over Ubiquiti, it's probably got something going for it. Looking over pricing, they seem comparable. I installed some Engenius 2.4ghz stuff in a hotel about 8 years ago and they are still working flawlessly. Ancedotal but something. MIKROTIK needs some loving wifi 6 stuff. Their poo poo is rock solid for easy home and soho stuff.
|
|
#
?
Dec 20, 2020 00:58
|
|
|
VostokProgram posted:What's wrong with their cameras? They're massively overpriced, and locked down. There's also the issue of them randomly dropping support for stuff. Cameras shouldn't be like that. Once you go with unifi protect, you're locked into their ecosystem. The unifi cameras don't offer the same features as better brands.
|
|
#
?
Dec 20, 2020 00:59
|
|
|
sharkytm posted:They're massively overpriced, and locked down. There's also the issue of them randomly dropping support for stuff. Cameras shouldn't be like that. Once you go with unifi protect, you're locked into their ecosystem. The unifi cameras don't offer the same features as better brands. What other brands in the same price range are usable or provide all their features without the need for usage of cloud services? Not rhetorical I actually want to know this.
|
|
#
?
Dec 20, 2020 01:08
|
|
|
SCheeseman posted:What other brands in the same price range are usable or provide all their features without the need for usage of cloud services? Not rhetorical I actually want to know this. I was using Hikvision cameras until recently. Still the rebrands are decent values and solid.
|
|
#
?
Dec 20, 2020 01:19
|
|
|
SCheeseman posted:What other brands in the same price range are usable or provide all their features without the need for usage of cloud services? Not rhetorical I actually want to know this. Dahua. I run many of their cameras. Great sensors, adequate firmware, great prices. They never see the internet, but I wouldn't let unifi cameras see the internet either. VLAN them onto their own network, and Blue Iris handles the triggering and recording.
|
|
#
?
Dec 20, 2020 02:59
|
|
|
SCheeseman posted:What other brands in the same price range are usable or provide all their features without the need for usage of cloud services? Not rhetorical I actually want to know this. Hikvision is unbeatable in terms of hardware, but they are heavily subsidized by the Chinese government, and their firmware is an absolute black box, with a bootloader that's got more obscured spaghetti code in it than there is actual spaghetti in the entire Asian continent. Security types and crazy people like me view that with a great deal of suspicion.
|
|
#
?
Dec 20, 2020 03:40
|
|
|
SwissArmyDruid posted:Hikvision is unbeatable in terms of hardware, but they are heavily subsidized by the Chinese government, and their firmware is an absolute black box, with a bootloader that's got more obscured spaghetti code in it than there is actual spaghetti in the entire Asian continent. Security types and crazy people like me view that with a great deal of suspicion. I would disagree. Dahua and Hik are comparable on features and hardware. There's a huge community on ipcamtalk, and Dahua has more users and a vendor who provides support for the cameras. English firmware updates are a big one. I've bought over 3 dozen cameras from Andy and never had a problem with the hardware or his service. I wouldn't trust any of the cameras other than maybe Axis with internet access. That's not crazy, that's how you avoid being hacked or part of a botnet. sharkytm fucked around with this message at 13:52 on Dec 20, 2020 |
|
#
?
Dec 20, 2020 13:49
|
|
|
shortspecialbus posted:Ubiquiti has a lot going for it, but they make a lot of dumb decisions
|
|
#
?
Dec 20, 2020 17:11
|
|
|
CopperHound posted:Do they use actual power over ethernet standards yet? They're getting there, I think all or most of the current equipment does. Of course I think they also changed some models over from proprietary to standard without a new model number or anything so good luck on the used market  On my own storage saga - I don't know why this has taken me so long to sort out but I think I've figured out what's going on with these "dead" WD30EFRX drives. They just really, really don't like being in my SAS enclosure. If I run any sort of heavy workload on them (badblocks, nwipe, whatever) then not only do they randomly drop, it seems like they're causing other drives to drop off. If I have nwipe targeting those 3TB drives and I run a scrub on my pool that has no 3TB drives in it, other drives in the pool will drop off and throw read/checksum errors. With the 3TB drives idle, scrub comes back 100% clean.
|
|
#
?
Dec 20, 2020 17:43
|
|
|
Crossposting from the HTPC thread: Looking at purchasing an entry level NAS to update and streamline my home media setup. Current setup is an Nvidia Shield TV hosting a Plex server going to the TV over HDMI, with media storage on a single 7-8 year old 2TB platter drive inside a external enclosure direct to router via USB. All devices are wired to the wall through a custom installed CAT6 network. Use case is streaming 1080p video at the most to a single TV locally. Only streaming to just the TV, no other devices or any simultaneous streaming. I'd like to also use the hard drive for general backup of photos, videos, documents, etc. Biggest concerns: The HDD is way past it's lifespan, and it's just a matter of time until device failure and data loss, it takes ages to copy files over USB, and I'm full up to capacity on 2TB and could really use more storage. My current rough idea plan is to get a minimum 2 bay NAS to wire to the network, throw in a pair of 4+ TB platter drives in a RAID 1 setup for backup, and decide whether to continue running Plex off the shield, or switch to running the Plex server off the NAS. I'd like to keep the price point at or below around $200 for the NAS, but I can be flexible here. Questions: 1) Are there any benefits to running the Plex server off a NAS instead of the Shield TV? The Shield seems to work fine, don't know if I'd need to spend the money on processing power in the NAS if its main purpose is just going to be media/general storage. I'm only ever going to be streaming to 1 device (the TV). 2) Is it worth it to continue to consider platter drives in a RAID setup in 2020, or should I just look at SSDs? Just a cursory glance around the internet, it looks like at a rough $200 price point, you can get SSDs at 2TB, or platter drives at 8TB. 3) Assuming platter drives, 5400RPM drives would probably be fine here for this use case right? 4) What are some recommendations for NASs that would fit my profile here? I've scrolled back the past several pages, and the Synology NASs seem to come up much. I'm not sure I'd need the processing power of a DS220+ versus just going with a DS200j. I don't know if I'm off the mark here. Are there any better/comparative options? Thanks guys!
|
|
#
?
Dec 20, 2020 18:17
|
|
|
sharkytm posted:I would disagree. Dahua and Hik are comparable on features and hardware. There's a huge community on ipcamtalk, and Dahua has more users and a vendor who provides support for the cameras. English firmware updates are a big one. I've bought over 3 dozen cameras from Andy and never had a problem with the hardware or his service. I wouldn't trust any of the cameras other than maybe Axis with internet access. That's not crazy, that's how you avoid being hacked or part of a botnet. Obligatory gently caress ipcamtalk and gently caress fenderman. Do NOT go to that website.
|
|
#
?
Dec 20, 2020 21:35
|
|
|
Constellation I posted:Obligatory gently caress ipcamtalk and gently caress fenderman. Do NOT go to that website. Well I mean this just makes me want to go, care to elaborate?
|
|
#
?
Dec 20, 2020 21:38
|
|
|
Just google the 2 words together and you'd get a very clear picture lol
|
|
#
?
Dec 20, 2020 21:43
|
|
|
Constellation I posted:Just google the 2 words together and you'd get a very clear picture lol I don't know why you're being so cagey and weird instead of just saying "the guy's an rear end in a top hat who gives bad/dangerous advice."
|
|
#
?
Dec 20, 2020 23:43
|
|
|
Ubiquiti it is then 
|
|
#
?
Dec 21, 2020 01:06
|
|
|
H2SO4 posted:I don't know why you're being so cagey and weird instead of just saying "the guy's an rear end in a top hat who gives bad/dangerous advice." The guy is a piece of work that I can't really do him justice. Googling for yourself is probably more informative/entertaining. Off the top of my head: - Massive rear end in a top hat - Gives terrible advice - When signing up for his forum, he uses any and all info that he gets as site admin to doxx, harass and intimidate you for "crossing" him Crossing him can mean as little as someone saying "I think this is bad advice" to "Hey, Blue Iris runs fine in a VM for me"
|
|
#
?
Dec 21, 2020 01:40
|
|
|
I saw some company that managed to get Synology Surveillance Station to work with Uquititi cameras and it was not pretty, it basically was just diverting the steam and almost going "gently caress you, i'm not using your environment just cause!" It was one of the more janky things I've seen. I have to admit though, that Axis has been relatively dependable and all cameras should be on their own vlan not accessible to the internet cause seriously, why on earth would you want them too be.  thats what the controller is for!
|
|
#
?
Dec 21, 2020 09:17
|
|
|
|
| # ? May 16, 2024 21:38 |
|
|
Praxis Prion posted:Crossposting from the HTPC thread: First things first: RAID is NOT backup. It is redundancy for reducing downtime in the event a drive fails. If you delete a file in a RAID1 array that data is lost on both disks. You need a proper versioned backup schema in mind for true backups. To answer your questions: 1) If your shield is doing the job then keep it that way. No need to spend the extra money on a DS2xx+ model if you aren't going to run plex on it. One of the advantages of the + series (I think this is exclusive to them, so don't take my word for it) is BTRFS snapshots. So if you nuke a directory or some files, you can browse the previous snapshot and get the data back. I think that's restricted to the 64bit Intel processor line of Synology devices. Something to consider when making your purchase. While this is no substitute for an off device/off site backup, it is a handy feature to have available. 2) Platter drives are fine for NAS systems as a single 5400 RPM drive can saturate a gigabit ethernet connection. 3) See number 2. 4) A DS220 would probably do fine for your needs. Avoid the 'j' models as they tend to be a bit anemic when it comes to transfer speeds. I don't know if that has changed much since I last used one though. Synology seems to be the go-to for NAS solutions as they have great support and will really help you if something shits the bed. There are other cheaper models on the market from different vendors, and the only other one I have used is a Q-Nap. It worked okay, but it seemed to be a tad slow. This was a TS-423 I think, it's been several years now. I returned a different Q-Nap for performance reasons as well as even with bonded ethernet I couldn't get over 45Mbps speed. My synology pushes over 120Mbps easily on the same network architecture. And again: RAID is NOT backup.
|
|
#
?
Dec 21, 2020 11:54
|
|
