Register a SA Forums Account here!
JOINING THE SA FORUMS WILL REMOVE THIS BIG AD, THE ANNOYING UNDERLINED ADS, AND STUPID INTERSTITIAL ADS!!!

You can: log in, read the tech support FAQ, or request your lost password. This dumb message (and those ads) will appear on every screen until you register! Get rid of this crap by registering your own SA Forums Account and joining roughly 150,000 Goons, for the one-time price of $9.95! We charge money because it costs us money per month for bills, and since we don't believe in showing ads to our users, we try to make the money back through forum registrations.
 
  • Post
  • Reply
BlankSystemDaemon
Mar 13, 2009



It isn't really as impressive as some sensationalist headlines have made it out to be, because they did it by making going to a 3.5" form factor compared to 2.5" that the 16TB Samsung drive uses. What is interesting, however, is that it offers a whole new option for people with enough money to do all-flash mass-storage with much higher IOps than previously possible, potentially removing the need for read-ahead caching and seperate syncronous write caches.


I'm kind of in a pickle because I can't decide whether to go with a mATX form-factor with a E3-1260L-v5 based Supermicro option (and pool extentability that it offers in the form of L2ARC and SLOG devices, which I can see a potential use-case for, if I want to boot my next workstation via iSCSI-target network bot) or go with the embedded D-1521 Supermicro option (that mostly just has the potential to be slightly more power efficient).

BlankSystemDaemon fucked around with this message at 22:06 on Aug 10, 2016

Adbot
ADBOT LOVES YOU

Azhais
Feb 5, 2007
Switchblade Switcharoo

Welp, already obsolete

http://www.theregister.co.uk/2016/08/10/toshiba_100tb_qlc_ssd/

Leng
May 13, 2006

One song / Glory
One song before I go / Glory
One song to leave behind


No other road
No other way
No day but today
Welp, got the NAS build finished, Plex installed, mirrored vdev set up and most of the data migrated over. Now I've run into more of an annoyance rather than an issue per se - I can't seem to find an easy solution for centralising and managing loads of home videos/photos. This is only for big events like weddings and vacations where we're taking thousands of photos/videos in HD (the unimportant day to day food pics/selfies/whatevers can just stay as crappy compressed versions in Google Photos).

I'm pretty sure my use case is fairly basic - while there are both Macs and PCs on the network, I'd only have a couple of users (just immediate family) total. No one is a pro or even an amateur photographer (so no RAW) and while multiple users might - on occasion - be viewing files at the same time, we don't really do much photo/video editing except for adding/correction EXIF description/location information/fix timestamps and then using the EXIF metadata to find/show stuff.

Based on the googling, I've only found a couple of options and all of them seem less than ideal:

1. Use the Plex Photos and Home Videos libraries functionality - would probably be the path of least resistance since Plex is already set up; just highly, highly annoying that photos and videos from the same event can't be kept in the same folder.

2. Install the now defunct Picasa program on every device that needs access to the photos and videos, set them all up to point to a Picasa database on the NAS

3. Repeat #2 above, except with the iPhoto library, although skimming a couple of google results gives me the impression that this is even worse than the Picasa solution

4. Lychee - but would require setting up a PHP5.5 server and MySQL on the NAS (which I can do but seems like a waste as this would literally be the only thing running on it)

What is everyone here using? Any recommendations?

Anime Schoolgirl
Nov 28, 2002

we'll get to 1PB in a 3.5" form factor

it'll have about 10 write cycles but that's fine

Samuel L. ACKSYN
Feb 29, 2008


I don't know if this is the right thread or not but it says storage in the title so lol


is there such a thing as an actual good USB3 hard drive dock? I had a Rosewill one that worked fine for a while but it was USB2 and after a while it died anyway.

Now I have some Orico thing and it works fine for a few minutes but then the USB connection disconnects and reconnects, and it does this poo poo all the time.


so uhhhh is there such a thing as one of these that doesnt suck

BlankSystemDaemon
Mar 13, 2009



I use this. It uses generic USB Attached SCSI/SAS (as opposed to USB Mass Storage Bulk-only Transport, which means its both faster and presents disks as seperate logical units) and apparently has very few (or no) hardware quirks that the drivers need to take account for given that I've had it plugged in to four different OS', and none of them note any hardware quirks. The disk cloning fascility is also kinda neat, even if you probably won't use it much.

BlankSystemDaemon fucked around with this message at 08:59 on Aug 13, 2016

Cowman
Feb 14, 2006

Beware the Cow





I'm pretty sure my HDDs are about to die, at the very least they're just very old and slow. Have SSDs gotten cheap enough that it's worthwhile to replace my HDDs and keep the same amount of storage space? Currently my HDDs are 2TB and I've got two of them, one for steam game and the other is a digital backup of my bluray/dvd collection. The HDDs are a couple years or more old and have had some serious wear and tear so I'm not surprised or mad that they're acting up every now and then. How much am I looking at to get 2 good 2TB SSDs? There's so many different brands and I don't know poo poo about what's a good brand and what isn't good. I took a look and was seeing about $400 or so for a 1TB so I'm assuming I'm going to be spending a lot if I wanted a big SSD so are there any modern HDDs that are pretty fast and a good replacement if it does turn out that I'm going to have to spend $1000+ on SSDs? Same thing goes for the HDDs. However, if it turns out that I can mix it up and get a SSD and HDD (SSD for some/most steam games and HDD for the rest of it) then that would be good too. I don't want to spend a ton because I don't have a lot of money on hand. I'm probably going to wait until the drives are on their last leg like I usually do.

Skandranon
Sep 6, 2008
fucking stupid, dont listen to me

Cowman posted:

I'm pretty sure my HDDs are about to die, at the very least they're just very old and slow. Have SSDs gotten cheap enough that it's worthwhile to replace my HDDs and keep the same amount of storage space? Currently my HDDs are 2TB and I've got two of them, one for steam game and the other is a digital backup of my bluray/dvd collection. The HDDs are a couple years or more old and have had some serious wear and tear so I'm not surprised or mad that they're acting up every now and then. How much am I looking at to get 2 good 2TB SSDs? There's so many different brands and I don't know poo poo about what's a good brand and what isn't good. I took a look and was seeing about $400 or so for a 1TB so I'm assuming I'm going to be spending a lot if I wanted a big SSD so are there any modern HDDs that are pretty fast and a good replacement if it does turn out that I'm going to have to spend $1000+ on SSDs? Same thing goes for the HDDs. However, if it turns out that I can mix it up and get a SSD and HDD (SSD for some/most steam games and HDD for the rest of it) then that would be good too. I don't want to spend a ton because I don't have a lot of money on hand. I'm probably going to wait until the drives are on their last leg like I usually do.

2x2tb is going to set you back by quite a lot, but 2x1tb might be doable, I have this in a RAID-0 for my Steam games, works well. I got some Samsung 850s on sale, but the Sandisk equivalent is even cheaper and often goes on sale as well. I've yet to come close to filling up 2TB of Steam games, maybe you could save a lot of money just by being a little more judicious how many games you have installed?

Cowman
Feb 14, 2006

Beware the Cow





Skandranon posted:

2x2tb is going to set you back by quite a lot, but 2x1tb might be doable, I have this in a RAID-0 for my Steam games, works well. I got some Samsung 850s on sale, but the Sandisk equivalent is even cheaper and often goes on sale as well. I've yet to come close to filling up 2TB of Steam games, maybe you could save a lot of money just by being a little more judicious how many games you have installed?

Yea I can easily cut down on the installed Steam games, I probably made that sound more important than it actually is. I'm mainly just forgetful about uninstalling games and with the recent size of modern games that means that the install folder grows by a bunch. I just checked and my steam folder is just 706GB and I can easily cut that down so a 1TB SSD would work perfectly fine. I took a look at my steam drive and there's some GoG and Origin games on there though not nearly as much. I've also put some of my backups on there so I can easily just clean it up and keep the whole thing under 1TB without too much work. I just looked at the total size and assumed I was good about organizing. I could probably get a big HDD for my backups since I don't need the fastest speeds to watch videos and listen to music. I'll keep an eye on that Samsung 850 and see when it gets to a good price.

Anime Schoolgirl
Nov 28, 2002

Is there a decent or at least not-poo poo non-raid 8-port SAS/SATA HBA card? I'm not using this in a dedicated FreeNAS/etc server (actually it's my gaming/emulation/workstation box :downsgun:) which means only shockingly few things will work.

Skandranon
Sep 6, 2008
fucking stupid, dont listen to me

Anime Schoolgirl posted:

Is there a decent or at least not-poo poo non-raid 8-port SAS/SATA HBA card? I'm not using this in a dedicated FreeNAS/etc server (actually it's my gaming/emulation/workstation box :downsgun:) which means only shockingly few things will work.

This is pretty decent. I've got one in my server now, been running for 1.5 years so far, no issues. I've got another that I've been meaning to install for awhile now. Only trick is to make sure to get the right breakout cables, it doesn't come with them. Can find them on eBay, Amazon, or Newegg.

And to clarify, my 'server' was once my desktop, so doesn't need a special motherboard, should work in any PCI Express 4x slot.

http://www.supermicro.com/products/accessories/addon/aoc-sas2lp-mv8.cfm

necrobobsledder
Mar 21, 2005
Lay down your soul to the gods rock 'n roll
Nap Ghost
You're looking at the IBM M1015 and variants or the Dell H200 (I think that was the one that's the M1015 equivalent) that have LSI controllers. You need to be careful that the cheapo card you grab off Ebay actually supports hard drives above 2 TB. Almost all the ones you can easily snag for < $30 are not capable of greater than 2 TB addressing on a drive. Additional to the card, you'll need a SAS forward breakout cable, too (presuming you're going from the controller directly to the SATA drives).

Anime Schoolgirl
Nov 28, 2002

necrobobsledder posted:

You're looking at the IBM M1015 and variants or the Dell H200 (I think that was the one that's the M1015 equivalent) that have LSI controllers. You need to be careful that the cheapo card you grab off Ebay actually supports hard drives above 2 TB. Almost all the ones you can easily snag for < $30 are not capable of greater than 2 TB addressing on a drive. Additional to the card, you'll need a SAS forward breakout cable, too (presuming you're going from the controller directly to the SATA drives).
Tried one of those, didn't work. Also tried an adaptec 6805E, it also did not work.

I'm hoping the Supermicro non-raid card pulls through, or I'm :supaburn:

IOwnCalculus
Apr 2, 2003





What do you mean by "did not work"? That seems like an overly simple description of a potentially very complex issue.

Anime Schoolgirl
Nov 28, 2002

IOwnCalculus posted:

What do you mean by "did not work"? That seems like an overly simple description of a potentially very complex issue.
I tried an LSI 9211-8i and it wasn't recognized by anything I had, nor could I flash it to "IT mode" (ie the only thing useful for one of these, and none of those being sold come flashed like that) It might have been a DOA card, in which case my luck is amazing.

The 6805E just didn't work at all, despite the BIOS seeing it existing. Windows 10 :downsgun: just wasn't having it, neither did Ubuntu.

Krakkles
May 5, 2003

What does it mean if my freenas install stopped sending me the daily report, but still sends the daily security report?

I dug through the settings, couldn't find anything on it. Nothing changed about mail/network config, it's still working fine. N40L with Freenas.

necrobobsledder
Mar 21, 2005
Lay down your soul to the gods rock 'n roll
Nap Ghost

Anime Schoolgirl posted:

I tried an LSI 9211-8i and it wasn't recognized by anything I had, nor could I flash it to "IT mode" (ie the only thing useful for one of these, and none of those being sold come flashed like that) It might have been a DOA card, in which case my luck is amazing.
Something sounds really wrong, you may have gotten a DOA but maybe you had something similar to what I experienced. Of those LSI chipset based cards, I've had an M1015 and a Dell PERC H200 and my BIOS behaved strangely with the M1015 (it detected ok at a point) and it resulted in a damaged BIOS stuck with a partial IBM logo where video cards won't work properly and I had to replace the otherwise perfectly functional motherboard with a different one. I wound up dropping it into another machine that had some temporary BIOS bootup screen corruption that cleared up later after I flashed the card to IT mode, and eventually I managed to fry the stupid card while blowing dust off of it somehow. $200 of hardware bullshit to go back to square one.

mayodreams
Jul 4, 2003


Hello darkness,
my old friend

Anime Schoolgirl posted:

I tried an LSI 9211-8i and it wasn't recognized by anything I had, nor could I flash it to "IT mode" (ie the only thing useful for one of these, and none of those being sold come flashed like that) It might have been a DOA card, in which case my luck is amazing.

The 6805E just didn't work at all, despite the BIOS seeing it existing. Windows 10 :downsgun: just wasn't having it, neither did Ubuntu.

You are trying to use a server part on a consumer OS in Windows 10, so I am not surprised it doesn't work. The server equivalent to Windows 10 is not out yet, so there are no drivers that would probably work. Ubuntu will probably need drivers too.

I have that exact card running on ESXi passed through to my FreeNAS VM for 3 years and have never had a problem with it.

Skandranon
Sep 6, 2008
fucking stupid, dont listen to me

Anime Schoolgirl posted:

Tried one of those, didn't work. Also tried an adaptec 6805E, it also did not work.

I'm hoping the Supermicro non-raid card pulls through, or I'm :supaburn:

To be specific, I am running my AOC-SAS2LP-MV8 in a Windows 2012 box with SnapRaid.

Don Lapre
Mar 28, 2001

If you're having problems you're either holding the phone wrong or you have tiny girl hands.
Synology supports amazon cloud drive now for backup. Good alternative to crashplan i think and no futzing around with hacks.

Anime Schoolgirl
Nov 28, 2002

Skandranon posted:

To be specific, I am running my AOC-SAS2LP-MV8 in a Windows 2012 box with SnapRaid.
A simple driver install and it worked. Thanks for the suggestion on the card~ :hfive:

Anime Schoolgirl fucked around with this message at 00:18 on Aug 18, 2016

Chuu
Sep 11, 2004

Grimey Drawer
Looking to buy four 8TB Western Digital Reds, and want to get them from different vendors to maximize the chances they're from different batches. Any suggestions for online vendors that know how to ship an OEM drive? I've gotten too many OEM drives shipped extreemly poorly from highly rated vendors to trust online reviews.

GokieKS
Dec 15, 2012

Mostly Harmless.
Honestly, I'm not convinced that going through a large amount of hassle/expense to get drives from different vendors in an effort to maximize reliability is worth it, especially when you need a large number of drives. I would just get them from wherever, and then make sure to properly stress-test them before being put into production (e.g. running a badblocks destructive read-write test - just keep in mind that it's 4 passes which will take days for large drives).

Chuu
Sep 11, 2004

Grimey Drawer

GokieKS posted:

I would just get them from wherever, and then make sure to properly stress-test them before being put into production (e.g. running a badblocks destructive read-write test - just keep in mind that it's 4 passes which will take days for large drives).

Do you have a link for the test?

GokieKS
Dec 15, 2012

Mostly Harmless.
Badblocks is built in to most *nix OSes. I'm not familiar with any Windows equivalent tools, but you can easily run a portable Linux off a USB / DVD and run it off that. Usage is very simple, just the -w flag for the destructive test, but you can find documentation for it here. Only thing to note is that with advanced (4K) sector drives, you need to specify that with the -b flag.

Here's the command I use when I'm testing my drives:

code:
# badblocks -wsvb 4096 /dev/disk/by-id/$DISK_ID$
I use /dev/disk/by-id/ because I'm doing this on my file server with existing drives and this allows me to make sure I'm running it on the right ones, but if you're doing from a live disk / USB with no other drives, you can just use /dev/sdX. Also, you probably want to save the SMART information of the drive before and after the test, so you can compare them.

DizzyBum
Apr 16, 2007


I'm dragging my heels on building a home NAS. I just want to get something done already because my desktop is nearly out of room and the newest drive I own is almost 5 years old. Where can I find some recommended build ideas?

I live less than a mile from a Micro Center and I figure I can snag some of the OEM drives they have in stock. I just want something that's simple bulk storage and relatively low-power.

I also have no idea what sort of RAID to plan to use because I feel like everyone has something bad to say about every RAID level. I was hoping for something that allows for easy storage expansion for whenever I'm able to pick up a new drive here and there.

Don Lapre
Mar 28, 2001

If you're having problems you're either holding the phone wrong or you have tiny girl hands.

DizzyBum posted:

I'm dragging my heels on building a home NAS. I just want to get something done already because my desktop is nearly out of room and the newest drive I own is almost 5 years old. Where can I find some recommended build ideas?

I live less than a mile from a Micro Center and I figure I can snag some of the OEM drives they have in stock. I just want something that's simple bulk storage and relatively low-power.

I also have no idea what sort of RAID to plan to use because I feel like everyone has something bad to say about every RAID level. I was hoping for something that allows for easy storage expansion for whenever I'm able to pick up a new drive here and there.

Use xpenology then

fletcher
Jun 27, 2003

ken park is my favorite movie

Cybernetic Crumb

DizzyBum posted:

I'm dragging my heels on building a home NAS. I just want to get something done already because my desktop is nearly out of room and the newest drive I own is almost 5 years old. Where can I find some recommended build ideas?

I live less than a mile from a Micro Center and I figure I can snag some of the OEM drives they have in stock. I just want something that's simple bulk storage and relatively low-power.

I also have no idea what sort of RAID to plan to use because I feel like everyone has something bad to say about every RAID level. I was hoping for something that allows for easy storage expansion for whenever I'm able to pick up a new drive here and there.

What's your budget and what's your use case?

Boxman
Sep 27, 2004

Big fan of :frog:


I just ordered a Synology 216j and a couple WD reds to back up my home mac. I'd like to back this up online as well just in case, I dunno, a fire wipes out my building. Is there a goon consensus "best online drive" for this sort of thing?

I was going to go with Amazon Drive because of the good price:storage ratio and the fact that Synology's software can interface with it no muss no fuss, but I noticed that it won't back up any single file larger than 10GB. I plan on backing up a Photos library and a Time Machine backup. To a mac, they show up as single packages that would exceed that 10 GB limit, but I have no idea if that's how they'd appear to a non OS X machine.

If the Synology software could automatically back it up with a bare minimum of fuss on my end that'd be great. I'm an idiot when it comes to Linux things.

phosdex
Dec 16, 2005

The Time Machine bundles are actually thousands of 800ish MB files. I believe it should work but I've never tried it.

PerrineClostermann
Dec 15, 2012

by FactsAreUseless
I think crash plan had been recommended as a general online
backup

Don Lapre
Mar 28, 2001

If you're having problems you're either holding the phone wrong or you have tiny girl hands.

Boxman posted:

I just ordered a Synology 216j and a couple WD reds to back up my home mac. I'd like to back this up online as well just in case, I dunno, a fire wipes out my building. Is there a goon consensus "best online drive" for this sort of thing?

I was going to go with Amazon Drive because of the good price:storage ratio and the fact that Synology's software can interface with it no muss no fuss, but I noticed that it won't back up any single file larger than 10GB. I plan on backing up a Photos library and a Time Machine backup. To a mac, they show up as single packages that would exceed that 10 GB limit, but I have no idea if that's how they'd appear to a non OS X machine.

If the Synology software could automatically back it up with a bare minimum of fuss on my end that'd be great. I'm an idiot when it comes to Linux things.

Synology just added amazon drive support to hyper backup. It is in beta and apparently fails on large sets so people are adding a folder, waiting for it to finish, adding another, then it can do delta just fine.

Krakkles
May 5, 2003

PerrineClostermann posted:

I think crash plan had been recommended as a general online
backup
I was one to recommend this, but I have to say - lately, I'm having more and more trouble doing so. I've had two failures in the last year where the software thinks the files locally are missing (they're not, I can access them fine through windows explorer), and says "well! No files to back up, 100% done. Oh by the by we'll delete your existing complete backup soon if you don't find a way to reconnect it".

The first time, they blamed memory allocation. I fixed that, waited a month or two to let the software figure out / upload whatever it wanted, got back to 100% - then it happened again. Upped the memory again, verified everything I could, searched for solutions, nothing worked.

I'm currently waiting for a tech support response that isn't "did you change the drive letter? Don't change the drive letter." and stressing a bit because all of my files are potentially not backed up.

Thermopyle
Jul 1, 2003

...the stupid are cocksure while the intelligent are full of doubt. —Bertrand Russell

Can you find others complaining about this? It seems like a serious problem and that if it was happening in circumstances other than your unique circumstances, there'd be lots of talk about it.

Skandranon
Sep 6, 2008
fucking stupid, dont listen to me

Thermopyle posted:

Can you find others complaining about this? It seems like a serious problem and that if it was happening in circumstances other than your unique circumstances, there'd be lots of talk about it.

I've been having issues with Crashplan as well. About at the 1.5tb uploaded part and it struggles to upload anything more. Haven't really looked further into it as I'm frustrated enough with it and would rather be doing other things. I was pretty excited about Amazon Drive until I saw it has a 2gb filesize cap.

Nulldevice
Jun 17, 2006
Toilet Rascal

Skandranon posted:

I've been having issues with Crashplan as well. About at the 1.5tb uploaded part and it struggles to upload anything more. Haven't really looked further into it as I'm frustrated enough with it and would rather be doing other things. I was pretty excited about Amazon Drive until I saw it has a 2gb filesize cap.

I'm using Amazon Cloud Drive and I've got files as large as 14GB stored there. I'm using rclone to get them up to the site tho, so I don't know where this 2GB limitation is coming from.

Thermopyle
Jul 1, 2003

...the stupid are cocksure while the intelligent are full of doubt. —Bertrand Russell

Skandranon posted:

I've been having issues with Crashplan as well. About at the 1.5tb uploaded part and it struggles to upload anything more. Haven't really looked further into it as I'm frustrated enough with it and would rather be doing other things. I was pretty excited about Amazon Drive until I saw it has a 2gb filesize cap.

IIRC, this is the issue you experience when you haven't adjusted the memory usage thingamabob.

Yes, that was super useful information.

(if it helps you any, I've got multiple terabytes of data up on Crashplan)

DizzyBum
Apr 16, 2007


fletcher posted:

What's your budget and what's your use case?

$300.

I have a lot of digital media - mostly music, movies, TV shows, and photos. I use Sonos for music and Plex for everything else. I don't have any video transcoding needs above 1080p.

What I would like to do is offload the persistent server stuff (Plex, Sonos controller, torrents, etc.) from my gaming PC and set it up on something else, with a dedicated NAS available on my home LAN to serve the media files. That way I don't have to leave my power-hungry desktop on 24/7 when, for example, someone at home wants to watch a movie at odd hours.

I'm thinking about picking up another Raspberry Pi 3 and using that as a dedicated Plex server/transcoder (not a fileserver, just Plex), but I don't know if it's powerful enough. Still doing research.

Skandranon
Sep 6, 2008
fucking stupid, dont listen to me

Thermopyle posted:

IIRC, this is the issue you experience when you haven't adjusted the memory usage thingamabob.

Yes, that was super useful information.

(if it helps you any, I've got multiple terabytes of data up on Crashplan)

I did adjust the memory, I bumped it up to 4gb. I have 16gb available. I should try taking another look at it.

Adbot
ADBOT LOVES YOU

Krailor
Nov 2, 2001
I'm only pretending to care
Taco Defender

DizzyBum posted:

$300.

I have a lot of digital media - mostly music, movies, TV shows, and photos. I use Sonos for music and Plex for everything else. I don't have any video transcoding needs above 1080p.

What I would like to do is offload the persistent server stuff (Plex, Sonos controller, torrents, etc.) from my gaming PC and set it up on something else, with a dedicated NAS available on my home LAN to serve the media files. That way I don't have to leave my power-hungry desktop on 24/7 when, for example, someone at home wants to watch a movie at odd hours.

I'm thinking about picking up another Raspberry Pi 3 and using that as a dedicated Plex server/transcoder (not a fileserver, just Plex), but I don't know if it's powerful enough. Still doing research.

If that budget includes new drives it's going to be tough to find something that meets your needs.
Just getting two 3TB drives is going eat up about half of that budget leaving you $150 for an enclosure. At that price you're looking at a Synology DS216se, which I don't think will be able to transcode anything, or kludging something together which is going to be tough for $150. I also doubt that a RPi3 will be beefy enough for transcoding either.

If your budget doesn't include drives then it opens you up to much better options.
If you want something DIY I'd suggest the TS140. This will give you a nice server that you can then install whatever NAS OS you want on.
If you want something smaller or more plug-and-play then you could get the DS216+II. This is a nice 2-bay Synology unit that should be able to handle whatever you throw at it.

  • 1
  • 2
  • 3
  • 4
  • 5
  • Post
  • Reply