|
is your PC crawling through all of the data on the NAS to index it or something dumb like that?
|
#
?
Sep 29, 2018 17:45
|
|
|
|
| # ? May 12, 2024 00:26 |
|
|
Or have you accidentally clicked something involving offline access? I legitimately don't even know what the point of that function is, if you want proper syncing offline files aren't the way to go at all.
|
|
#
?
Sep 29, 2018 18:25
|
|
|
Thanks Ants posted:The data is going somewhere, if the only connections are your PC then it's something on that PC. You might need to use the all users task manager to see what it is if it's a system process. This is what I thought as first but I can't figure it out. Nothing shows up in the task manager even through numerous restarts of both devices. I tried using WinDirStat to see if there was some huge hidden file or directory somewhere that was filling up but couldn't find anything that route either. I have a 3 TB HD and a 250GB SSD in my actual PC, the 3 TB drive has ~500GB of data on it and the SSD only has about 50GB used vs the 13+TB of data on my NAS. redeyes posted:Backups?! That looks like backups. adorai posted:is your PC crawling through all of the data on the NAS to index it or something dumb like that? D. Ebdrup posted:Or have you accidentally clicked something involving offline access? I'm think it has to be something along these lines. In the midst of changing settings trying to get the SMB stuff setup better for the faster transfer speeds I might have turned something on that wasn't previously on. I don't think it's a cloud backup thing because I'd imagine I would have had to setup an account and paid for something for that to happen. I run PLEX server on my computer and it hosts/uses my media libraries that are on the NAS. Could that be the culprit? Maybe there was a plex update or something and it's going back through scrubbing the files generating Metadata and thumbnails and such? I can't really figure out why it'd be doing it at this time since I haven't changed or added any large amount of files to the NAS anytime recently and everything that's on there has been indexed/scrubbed already.
|
|
#
?
Sep 29, 2018 20:16
|
|
|
100% Dundee posted:The thing I cant figure out is that the unit won't stop transferring data, I can't figure out what its doing or where this data is going. The network resource monitor is literally always fluctuating from 70-140MB/s or so, and it's been doing it 24/hr a day for multiple days in a row. I'm not transferring any actual data to the device and I'm not transferring any data off of the device, so I don't understand where this connection is going or whats being sent or to where. Under the "Connected Users" section it only has 2 users and both of them are me/my home PC so I don't think someone has hacked into my poo poo and is stealing my data remotely or something. Test this theory. Unplug your internet access cord from your router and leave your computers on. If it stays going turn off one computer at a time until you have a culprit. Otherwise someone hacked you. If you have a user with root privileges on your device they are probably smart enough to not show up on connected users. Do you have things like UPnP enabled on your router? If so turn that poo poo off.
|
|
#
?
Sep 29, 2018 21:08
|
|
|
100% Dundee posted:I run PLEX server on my computer and it hosts/uses my media libraries that are on the NAS. Could that be the culprit? Yes, absolutely. Go to the Library settings on the Plex app and you can control when library updates and other functions are done.
|
|
#
?
Sep 29, 2018 22:13
|
|
|
H2SO4 posted:Yes, absolutely. Go to the Library settings on the Plex app and you can control when library updates and other functions are done. This is what I'm thinking is happening, I didn't suspect that as the culprit at first because nothing on the NAS changed so I don't know why it would re-scrub through my entire library. Maybe it was some combination of a plex update, a DSM update and me loving with the SMB protocol settings that tripped it up thinking that all the directories changed or moved or something? I've always had it set to update/do maintenance/etc from 5am-9am in morning when I'm least likely to be using either plex or the computer and this new issue was happening 24 hours a day for 2-3 days before I got annoyed enough to start investigating and posting for help. It seems crazy that it would take that long to scrub through 13TB of media.
|
|
#
?
Sep 29, 2018 22:52
|
|
|
100% Dundee posted:This is what I'm thinking is happening, I didn't suspect that as the culprit at first because nothing on the NAS changed so I don't know why it would re-scrub through my entire library. Maybe it was some combination of a plex update, a DSM update and me loving with the SMB protocol settings that tripped it up thinking that all the directories changed or moved or something? If it was a rescrub, it would not be doing network traffic.
|
|
#
?
Sep 29, 2018 23:48
|
|
|
Open Task Manager -> Open Resource Manager -> Network tab , pick Network Activity, sort by Total (B/sec). Should tell you very quickly if the computer is transfering anything.
|
|
#
?
Sep 29, 2018 23:59
|
|
|
And the Disk tab to identify what process is accessing what files.
|
|
#
?
Sep 30, 2018 00:03
|
|
|
redeyes posted:Open Task Manager -> Open Resource Manager -> Network tab , pick Network Activity, sort by Total (B/sec). Should tell you very quickly if the computer is transfering anything. Combat Pretzel posted:And the Disk tab to identify what process is accessing what files. Does this help at all? I can't really decipher all this. Maybe its something to do with windows defender? That seems to be the process using all the network on the PC side but the NAS is still going haywire sending out data.  Edit: Disabled windows defender entirely and now all of that data is being used by Plex Media Scanner, so I think that might answer the question? 
100% Dundee fucked around with this message at 01:56 on Sep 30, 2018 |
|
#
?
Sep 30, 2018 01:42
|
|
|
What the gently caress is Windows Defender doing there?! I'd go into its settings and put an exception for your network drive for starters.
|
|
#
?
Sep 30, 2018 01:54
|
|
|
At a guess, Plex is accessing the data and calling it via Defender which is doing a real-time scan
|
|
#
?
Sep 30, 2018 02:01
|
|
|
redeyes posted:What the gently caress is Windows Defender doing there?! I'd go into its settings and put an exception for your network drive for starters. I'm not really sure how to do that but I'll look into it and see if I can figure out something. Once I disabled windows defender entirely and then closed plex entirely everything is 100% peachy now. Still quite confused about how this has been all working perfectly fine for a few months now and then I went in to mess with SMB protocols and all this poo poo started happening.
|
|
#
?
Sep 30, 2018 02:04
|
|
|
In Win10, how do we exclude from Defender scans network shares? Not mapped drives, but shares. In MSE it was a reg entry to exclude, say: \\server How do we do that again? Same reg key, I wonder... will have to test when I get home.
|
|
#
?
Sep 30, 2018 02:12
|
|
|
100% Dundee posted:I'm not really sure how to do that but I'll look into it and see if I can figure out something. Once I disabled windows defender entirely and then closed plex entirely everything is 100% peachy now. It isn't generating thumbnails, is it? Plex by default will run a periodic background task that looks for new media, and generates thumnails by transcoding/downscaling which is very CPU intensive (not to mention requires it to hit the disks pretty hard). If you aren't using a Plex client that supports them, or don't want to incur the time/space costs of generating them, you can turn them off in the Settings->Library screen.
|
|
#
?
Sep 30, 2018 02:12
|
|
|
EpicCodeMonkey posted:It isn't generating thumbnails, is it? Plex by default will run a periodic background task that looks for new media, and generates thumnails by transcoding/downscaling which is very CPU intensive (not to mention requires it to hit the disks pretty hard). If you aren't using a Plex client that supports them, or don't want to incur the time/space costs of generating them, you can turn them off in the Settings->Library screen. Thumbnails not so much. Now if he turned on the Plex option to automatically transcode and store a smaller version of the video so mobile devices can direct play that version instead of on the fly transcoding then I can see that taking a day or two if all videos need to be first transferred to the Windows machine.
|
|
#
?
Sep 30, 2018 02:29
|
|
|
Thanks Ants posted:At a guess, Plex is accessing the data and calling it via Defender which is doing a real-time scan EpicCodeMonkey posted:It isn't generating thumbnails, is it? Plex by default will run a periodic background task that looks for new media, and generates thumnails by transcoding/downscaling which is very CPU intensive (not to mention requires it to hit the disks pretty hard). If you aren't using a Plex client that supports them, or don't want to incur the time/space costs of generating them, you can turn them off in the Settings->Library screen. Think it was some combination of both of these activities. Plex Scanner seems to have been going through and completely re-scanning everything and creating thumbnails/metadata/etc for some reason that I cannot figure out and at times which it was not set to do it at. Then all of this data was being filtered through windows defender for some reason. Effectively choking up like 90% of my network usage. I've managed to set an exclusion for my NAS via network defender as suggested(doing this https://www.tekrevue.com/tip/windows-defender-nas/). Then I went into my plex settings and re-set everything, made sure it was never set to sync or transcode anything, never generate thumbnails, never automatically scan and I reset the "scheduled maintenance" timeframe from 5AM-9AM as it was previously. Everything seems to be back to normal for now. For now.
|
|
#
?
Sep 30, 2018 02:55
|
|
|
Defender is a piece of crap. Everyone says it has the best detection rates nowaways, but there's always something. I'm still using NOD32, and for some reason, it doesn't require me to set up any exclusions to let my system operate normally.
|
|
#
?
Sep 30, 2018 13:32
|
|
|
Windows Defender doesn't normally scan UNC paths.
|
|
#
?
Sep 30, 2018 14:21
|
|
|
Combat Pretzel posted:Defender is a piece of crap. Everyone says it has the best detection rates nowaways, but there's always something. I'm still using NOD32, and for some reason, it doesn't require me to set up any exclusions to let my system operate normally. All AV is crap. ALL!
|
|
#
?
Sep 30, 2018 14:56
|
|
|
D. Ebdrup posted:Windows Defender doesn't normally scan UNC paths. This is my experience as well. I have both mapped drives and unc paths I use regularly. Maybe it's a windows 7 thing?
|
|
#
?
Sep 30, 2018 15:17
|
|
|
D. Ebdrup posted:Windows Defender doesn't normally scan UNC paths. Maybe not for periodic full scans, but I've definitely had the realtime engine pick up on things I'm accessing via UNC and take action.
|
|
#
?
Sep 30, 2018 17:53
|
|
|
redeyes posted:All AV is crap. ALL!
|
|
#
?
Sep 30, 2018 18:30
|
|
|
IOwnCalculus posted:Might be a 15% coupon on eBay tomorrow too. RAM trip report: RIP experiment with 2Rx4GB. I now have 2 8GB DDR3R-1333 ECC sticks to put back up on ebay. Not sure why hynix 1rx4 works when it's not supposed to, but I'll fill my slots up with that and deal with "only" 24GB. intel. 
|
|
#
?
Sep 30, 2018 22:08
|
|
|
H2SO4 posted:Maybe not for periodic full scans, but I've definitely had the realtime engine pick up on things I'm accessing via UNC and take action.
|
|
#
?
Sep 30, 2018 22:52
|
|
|
Harik posted:I keep missing these. How do you guess that there might be a code upcoming? Do they do it sometime around quarterly or what? Yeah they pretty reliably do it at the end of the quarter, probably to goose their sales figures.
|
|
#
?
Sep 30, 2018 23:51
|
|
|
Someone else had posted it elsewhere as a "very likely". Can't recall where I saw it.
|
|
#
?
Oct 1, 2018 00:20
|
|
|
I no longer need my old living room media pc for anything other than network storage at this point, so I've decided to donate its guts to a friend and put together a dedicated NAS box. Now that I'm not tied to Win10, I'm guessing unRAID is probably the route to go? I'll be recycling some of the drives and huge full tower case, but will need to buy everything else. Leaning towards an i3-8100 w/ 8gb ddr4 2400 but I'm not sure how much is really necessary for an unRAID box? As for drives, I've currently got 2 4tb wd blues that are a year or so old, and 3 2tb drives that are old as heck and I'm not planning to reuse. Was thinking of grabbing an 8tb wd mybook external to rip the drive out of. It's my understanding I'll need two of those, since the parity drive will also need to be 8tb? And last question for now, does unRAID work ok booting off a usb stick or should I drop an SSD in for it to run from?
|
|
#
?
Oct 2, 2018 03:08
|
|
|
I still haven't set up an unraid box yet (a christmas holiday project for next year). You need a flash drive to get started and at least two hard drives, but as I haven't set one up before refer to the link. https://lime-technology.com/getting-started/
|
|
#
?
Oct 2, 2018 05:56
|
|
|
I've been rolling my own for awhile now (and I'm also an idiot) but thread consensus seems to be unraid. Why that over FreeNAS?
|
|
#
?
Oct 2, 2018 09:23
|
|
|
Crunchy Black posted:I've been rolling my own for awhile now (and I'm also an idiot) but thread consensus seems to be unraid. Why that over FreeNAS? Cheaper to expand and ZFS is overkill for a media server which is what most people make a server for in this thread. With Unraid you can just toss in whatever drives you have while RAIDZ1/2/3 is restrictive and if you want to increase storage you have to upgrade all of your drives to see the increase happen. Unraid also has a decent out of box experience that even novice users can setup. I also like that Unraid is hardware agnostic for the most part. You can literally take your flashdrive and your drives and plug it into an entirely new system and it will work just like did on the old hardware. 8-bit Miniboss fucked around with this message at 09:35 on Oct 2, 2018 |
|
#
?
Oct 2, 2018 09:32
|
|
|
I found this site that someone posted upthread https://www.serverbuilds.net/anniversary Thinking maybe I'll price out a build like that, seems like it wouldn't be too much more than the i3 build I was considering.
|
|
#
?
Oct 2, 2018 14:15
|
|
|
8-bit Miniboss posted:Cheaper to expand and ZFS is overkill for a media server which is what most people make a server for in this thread. With Unraid you can just toss in whatever drives you have while RAIDZ1/2/3 is restrictive and if you want to increase storage you have to upgrade all of your drives to see the increase happen. Unraid also has a decent out of box experience that even novice users can setup. I also like that Unraid is hardware agnostic for the most part. You can literally take your flashdrive and your drives and plug it into an entirely new system and it will work just like did on the old hardware. Cool. I like ZFS. I even went through the frugal mode awhile back with Windows File Server that was the predecessor to Unraid and found out that oh god, you can't just use any hard drive for a NAS.
|
|
#
?
Oct 2, 2018 14:46
|
|
|
8-bit Miniboss posted:Cheaper to expand and ZFS is overkill for a media server which is what most people make a server for in this thread. With Unraid you can just toss in whatever drives you have while RAIDZ1/2/3 is restrictive and if you want to increase storage you have to upgrade all of your drives to see the increase happen. Unraid also has a decent out of box experience that even novice users can setup. I also like that Unraid is hardware agnostic for the most part. You can literally take your flashdrive and your drives and plug it into an entirely new system and it will work just like did on the old hardware. How it works now, and has worked since autoexpand got added quite a long time ago, is something like this: Let's say you start with a pool of 5 drives with autoexpand enabled in raidz2, all of varying size - your pool will roughly be <size of smallest-drive>*3 and every time you replace the smallest drive with a bigger drive (ie. if you're buying drives peacemeal on a budget) your pool will grow to the size of the new smallest-drive automatically. In addition to that, vdev expansion is getting worked on by Matt Ahrens as we speak, and it will of course be upstreamed as soon as it's ready. Also, media servers typically end up working as document backup servers, so it makes sense to use ZFS for that. *: Holy poo poo has it really been that long 
|
|
#
?
Oct 2, 2018 14:56
|
|
|
With unRAID, you get all the capacity minus your parity disks, regardless of their size - the only "caveat" being of course that the parity disks need to be as big as or larger than the largest disk in the system. There are other positives too; even if your parity disk(s)+1 fail, you only lose the data on the affected disk. The disks are simply accessible as normal drives, without any special "RAID" structuring on them, making data recovery absolutely trivial. It might be not be quite as well featured in terms of error checking and checksumming, but unRAID really is an excellent solution for the home user, and I would build an unRAID box before touching a QNAP or Synology box, where you end up locked in to hardware that can't be easily upgraded or replaced if faults occur. HalloKitty fucked around with this message at 15:22 on Oct 2, 2018 |
|
#
?
Oct 2, 2018 15:16
|
|
|
Crunchy Black posted:I've been rolling my own for awhile now (and I'm also an idiot) but thread consensus seems to be unraid. Why that over FreeNAS? I think more people still use FreeNAS in this thread but I swapped over to unRAID despite using FreeBSD/ZFS at work: + Easier to expand, pretty much key for home media servers + Ease of use, far better UI + CLI is still there if you need it too, it's Linux under the hood + I can still do all of my nerdy homelab/VM crap in it + Dockers are perfect for home use and the community for unRAID dockers/plugins is stellar - Slow array write speeds outside of cache - No built in bitrot type protection - not free Basically I wanted my server to be relatively hands off and after initial setup you do very little janitoring with unRAID. Mine runs all of the usual Plex/Sonarr/etc nonsense, I do hardware passthru and run some VMs off it, etc. It's very well supported by Lime-Tech and I can just pickup cheap hard drives whenever they're on sale and slap them in. ZFS is amazing but its complete overkill for my needs at home.
|
|
#
?
Oct 2, 2018 16:12
|
|
|
This is the packrat thread, there is no such thing as overkill. 
|
|
#
?
Oct 2, 2018 16:25
|
|
|
HalloKitty posted:There are other positives too; even if your parity disk(s)+1 fail, you only lose the data on the affected disk. The disks are simply accessible as normal drives, without any special "RAID" structuring on them, making data recovery absolutely trivial. That right there is the big difference. I don't want to lose files. End sentence. You put it on a different lower level of risk but to people like me that means a personal failure. How they can suggest this software as enterprise quality is now even more wtf.
|
|
#
?
Oct 2, 2018 17:28
|
|
|
EVIL Gibson posted:That right there is the big difference. I don't want to lose files. End sentence. You put it on a different lower level of risk but to people like me that means a personal failure. I don't really know what you're trying to get at, and I don't think you understand what I meant. In traditional RAID scenarios, like RAID 5 (or RAID Z) or RAID 6 (or RAID Z2), your parity (and data) is striped across the drives, single parity for RAID 5, double for RAID 6. unRAID also allows you to run with single or double parity, but the parity is stored on dedicated disks - the contents of the data disks is not touched in any way. If, in your traditional RAID 5 scenario, one drive dies, no big deal, you replace the drive and the array begins to rebuild. As long as only one drive dies (in RAID 5), you don't lose any files. Then, if a second drive dies whilst rebuilding, you're up poo poo creek without a paddle. The data was striped across those drives, and now chunks of files are missing. Recovery is now going to be hellish for ANY of the files, if not impossible. In the unRAID (or comparable software, SnapRaid, Flexraid etc.) scenario, a drive fails when you have single parity, again, no big deal, replace it.. rebuild.. yada, yada... again, as long as only one drive dies (with single parity), you don't lose any files. OK, then another drive dies whilst rebuilding - you lose the data on that dead drive, but the other drives are utterly unaffected, because their files were not striped across the dead disks, making recovery trivial from the drives that haven't failed. Also, NONE of these data availability strategies are a replacement for backups!! You don't just have one copy of your data, no matter how much fault tolerance you think there's built in to it. Even with snapshotting, you can easily lose data for reasons other than hardware failure... HalloKitty fucked around with this message at 17:51 on Oct 2, 2018 |
|
#
?
Oct 2, 2018 17:31
|
|
|
|
| # ? May 12, 2024 00:26 |
|
|
D. Ebdrup posted:Well, that was sort of true on a very old version of ZFS (the one you could manually add to FreeBSD 6.3 with a bit of backporting from 7-CURRENT if you were brave enough a decade ago in 2008*) I just used Solaris 10 and then Open Indiana because I started my first build in 07, I'm lazy and just wanted to copy settings files to the other one and have them (mostly) just work. Solaris' role based permission system was nice. Seriously thinking about switching to BSD with the next one, though.
|
|
#
?
Oct 2, 2018 17:52
|
|
