|
I'm looking to set up a doubly redundant backup system for my photo archives. I generate 2-3tb of files a year. Currently, I have all of that data stored on drives in my computer and previous years scattered across various external drives. My thought for consolidating things and increasing reliability is to keep my current year on my computer, and then back everything up to a NAS (which I can also use to provide clients with FTP access). I'd then want to back that up. It doesn't necessarily need to be anything constantly accessible or provide all the features of a NAS, but looking at newegg I can't find much in thte way of reliable RAID enclosures. I'm looking at either a Synology DS412+ or a QNAP TS-469-PRO-US, filled with 4tb WD red drives in RAID 5. What should I look at for the second backup, and which of those NAS units is likely to work better?
|
#
?
Mar 7, 2014 03:58
|
|
|
|
| # ? May 26, 2024 20:28 |
|
|
You can certainly spend several hundred dollars on a second NAS, and then several more hundred dollars on HDDs every year to fill it, or you can do what my wife does with all her photos that are more than a year old: At the end of every year, she takes two external HDDs of an appropriate size and simply makes a full copy of the year's files. One she simply unplugs and leaves in a closet. The other she mails to her mother (in case the house burns down or something). Stick a date label on them and keep them all together on a shelf or something. You can also look at cloud storage--it'll hurt pushing that first chunk over, but after that you can sync every few days without too much trouble as long as your internet connection is fast enough to allow it (let it run overnight off the NAS or something if need be). The point is that for cold backup, you shouldn't be looking so much at a second RAID system, as much as getting the data somewhere physically removed from your first NAS and live-data PC. e; With the cloud backup, you can also ask the honesty question of "Do I really need to keep the RAWs?" Think about how often you've ever needed to drag out a RAW from 3 years ago, vice just the finished .jpg or whatever to re-order prints with. Just storing the .jpgs makes cloud backup that much more attractive (plus it's usually easier to provide archived files to clients that way, vice having to go sort through drives and manually pull them back over to your FTP). DrDork fucked around with this message at 04:18 on Mar 7, 2014 |
|
#
?
Mar 7, 2014 04:14
|
|
|
I had a similar debate, I would look at the Synology 411+, it's under 300 on the Synology refurb site. I like their software and you aren't doing anything crazy performance heavy. As for a second backup, why not use smugmug for offsite?
|
|
#
?
Mar 7, 2014 04:16
|
|
|
sellouts posted:As for a second backup, why not use smugmug for offsite?
|
|
#
?
Mar 7, 2014 04:25
|
|
|
sellouts posted:As for a second backup, why not use smugmug for offsite? I'm a product photographer, not a wedding/littleleague/retail photographer, so smugmug doesn't work so well. I'm usually delivering handfuls of large TIF files, rather than having people order prints. I do need to hold onto all my files, raw included, so a web based backup is somewhat suboptimal. As best I can tell, amazon s3 gets very expensive if you're trying to use it to store multiple tb of data, carbonite will allow you to backup your entire computer, but just backs up the whole computer, rather than being a backup of a specific group of files, and I haven't particularly looked into other options. Basically, I need to be able to have reliable local cold-storage of all my data incase the NAS controller dies and I'm unable to rebuild it without data loss. I'd prefer to keep that cold storage on a single solution rather than creating a new drive every year, but if there are no reliable ways to do that then multiple external drives is what I'll use.
|
|
#
?
Mar 7, 2014 04:55
|
|
|
Smugmug does all of that via smugvault, and they aren't zenfolio who was recently hacked, largely due to insecure passwords. But I am not a smugmug shill. You'd want amazon glacier, not S3 for backup anyways. As for crashplan it can be run on the synology. I haven't done it and it doesn't do well with real time backups, but putting final files on there should be fine. I have no idea what they'll say when you dump terabytes of data on there though. sellouts fucked around with this message at 05:07 on Mar 7, 2014 |
|
#
?
Mar 7, 2014 05:02
|
|
|
sellouts posted:I have no idea what they'll say when you dump terabytes of data on there though. They'll say nothing at all. Seriously man, just get yourself some form of off-site backup. Crashplan, Carbonite, or a roll-your-own thing with Amazon Glacier. Just compare them and see what works best.
|
|
#
?
Mar 7, 2014 05:15
|
|
|
MrMoo posted:Conversely BTRFS is almost ready too and by the general articles should be better than ZFS. The huge upside of ZFS is the easy integration with iscsiadm and NFS, plus years of stability, not so much how trendy the filesystem is.
|
|
#
?
Mar 7, 2014 05:32
|
|
|
ConfusedUs posted:They'll say nothing at all. Not loving kidding, either. 
|
|
#
?
Mar 7, 2014 05:33
|
|
|
ConfusedUs posted:They'll say nothing at all. Right, and crashplan does look like a good solution for off-site backup. I'd still like redundancy in my on-site backup, but everything I see on newegg has a ton of bad reviews for power supplies failing. Am I just being unrealistic in wanting a way to back the NAS up on-site without simply using a second NAS?
|
|
#
?
Mar 7, 2014 06:06
|
|
|
Booley posted:Right, and crashplan does look like a good solution for off-site backup. I'd still like redundancy in my on-site backup, but everything I see on newegg has a ton of bad reviews for power supplies failing. Am I just being unrealistic in wanting a way to back the NAS up on-site without simply using a second NAS? You're probably being a bit overly paranoid when it comes your personal backup. The cloud backup is your ultimate fallback. Usually you won't need your backup at all (especially if you have some sort of redundancy in your RAID array). If you do need a backup, your regular backup to disk locally is highly likely to be there. Most restore scenarios are not total disasters. The cloud backup is there if all of the above fails or your house burns down or something.
|
|
#
?
Mar 7, 2014 06:16
|
|
|
Man, I can not understand how crash plan makes money at those prices. If I store 2TB of data, it will take them literally years at $5/mo to just buy the drives my stuff is on.
|
|
#
?
Mar 7, 2014 06:56
|
|
|
AlternateAccount posted:Man, I can not understand how crash plan makes money at those prices. If I store 2TB of data, it will take them literally years at $5/mo to just buy the drives my stuff is on. Most customers don't put up anywhere nearly that much data. The outliers with several TB of data are just that: outliers.
|
|
#
?
Mar 7, 2014 06:58
|
|
|
Most people probably don't have the patience or bandwidth to back up terrabytes of data.
|
|
#
?
Mar 7, 2014 07:28
|
|
|
Crashplan makes money because of block-level deduplication, which applied to a petabyte or possibly exabyte data set likely results in *MASSIVE* savings. For backup, unless you have a large always-needed dataset (i.e. a company's customer database can't be down, they'll lose millions of dollars an hour), a hot backup is probably useless. My current backup system, which is undeniably overkill, is: Desktops/Laptops Cross-Backup to each other (using Crashplan) in a one-to-many setup. This means that each device has all of every other device's data on it. Just user data. Desktops/Laptops backup to Crashplan. Whole-Disk. Desktops back up to a dedicated backup drive inside each computer, only user data though, not full backups. This is my "hot-backup" but it's really more of a "warm" "Just reinstalled, SATA Bus is faster than Gigabit" My main user folder is Synced and Backed Up with Spideroak (general files that I work on everywhere, common settings/tools that i want to be the same across all computers, etc.) "Cold" data (old schoolwork, 1:1 images of CDs/DVDs of music or games, all my family and work pictures) are burned to Blu-Ray discs, one copy kept locally, one copy sent ~800 miles away to a family member's house. Most of this doesn't change much (school stuff never does, discs never do, pictures are only added to, etc) so this usually just entails an additional disc or two being burned each year. There's 10 or 11 of them at this point. Some of it is stored in other places, but things like the discs and schoolwork are just kept in a "outgoing to backup" folder on the file server. There are some other redundancies in there (Computers are clonezilla imaged monthly so that some types of failure condition can be recovered from via wipe-and-reimage or new-harddrive-and-reimage in an hour or two instead of days of re-setting-up everything), and their used to be more (external backup drive for each computer as well as the internal one, things like that) but this is *Way* more than enough backups. I can recover from minor errors in seconds, major errors in minutes, and dedicated and well-planned attacks might put me out for a few hours after the metaphorical bullets start flying. I guess what I'm saying is that in all my paranoia even I think a dedicated hot backup server is a bit much. "Hot" backups are for live-failover things that can't be down for half a second. I should put some of this in the OP, I think.
|
|
#
?
Mar 7, 2014 11:18
|
|
|
teamdest posted:Crashplan makes money because of block-level deduplication, which applied to a petabyte or possibly exabyte data set likely results in *MASSIVE* savings. No, this is wrong. Crashplan's deduplication works only on an individual account's dataset.
|
|
#
?
Mar 7, 2014 12:35
|
|
|
DNova posted:No, this is wrong. Crashplan's deduplication works only on an individual account's dataset. Not that I'm saying you're wrong, but even if they do it on your individual dataset (probably they do, to avoid redundant uploads, make a bunch of backend work easier, whatever), There is likely deduplication happening on the servers, too.
|
|
#
?
Mar 7, 2014 12:58
|
|
|
teamdest posted:Not that I'm saying you're wrong, but even if they do it on your individual dataset (probably they do, to avoid redundant uploads, make a bunch of backend work easier, whatever), There is likely deduplication happening on the servers, too. Why do you feel the need to speculate? You're wrong, full stop. Dropbox does what you are saying, and that is a major indicator that Dropbox is a severe piece of poo poo that nobody should ever use for anything important.
|
|
#
?
Mar 7, 2014 13:27
|
|
|
You can't reliably deduplicate salted encrypted data. What you're saying implies that they're circumventing encryption (which is done client-side, so you'd also have to do client-side deduplication) by doing deduplication on it before encrypting it - effectively rendering the encryption pointless. EDIT: ↓ I'm fairly sure what you did was input your own key, rather than let CrashPlan manage it. BlankSystemDaemon fucked around with this message at 13:51 on Mar 7, 2014 |
|
#
?
Mar 7, 2014 13:43
|
|
|
It has been a couple years since I setup my crashplan backup, but I recall having to turn encryption on manually... So if be surprised if 25 percent of their users actually do so. I find it hard to believe that they wouldn't dedupe non-encrypted data.
|
|
#
?
Mar 7, 2014 13:50
|
|
|
D. Ebdrup posted:You can't reliably deduplicate salted encrypted data. What you're saying implies that they're circumventing encryption (which is done client-side, so you'd also have to do client-side deduplication) by doing deduplication on it before encrypting it - effectively rendering the encryption pointless. Exactly, and it is alarming to me that OP just carelessly assumes this is the case (and he even uses the product!). Fancy_Lad posted:It has been a couple years since I setup my crashplan backup, but I recall having to turn encryption on manually... So if be surprised if 25 percent of their users actually do so. Crashplan encrypts all data for every user at the source. You can choose to manage your own key, but the default is to let the servers handle it. That does not mean that they are decrypting your data and deduplicating it on their end.
|
|
#
?
Mar 7, 2014 13:53
|
|
|
So then even if they buy drives by the container straight from China, a customer storing several TB of data can't possibly be profitable for years. What am I missing here?
|
|
#
?
Mar 7, 2014 15:54
|
|
|
AlternateAccount posted:So then even if they buy drives by the container straight from China, a customer storing several TB of data can't possibly be profitable for years. What am I missing here? Storage socialism.
|
|
#
?
Mar 7, 2014 16:11
|
|
|
Because I pay Backblaze $50 a year and only have 80GB of stuff on there.
|
|
#
?
Mar 7, 2014 16:14
|
|
|
AlternateAccount posted:So then even if they buy drives by the container straight from China, a customer storing several TB of data can't possibly be profitable for years. What am I missing here? Storage will get cheaper, but in the short term advertising as "truly unlimited" attracts customers who overestimate their needs and upload capability?
|
|
#
?
Mar 7, 2014 16:15
|
|
|
AlternateAccount posted:So then even if they buy drives by the container straight from China, a customer storing several TB of data can't possibly be profitable for years. What am I missing here? They get more customers by advertising unlimited storage. Very, very few people will be backing up multiple terabytes.
|
|
#
?
Mar 7, 2014 17:17
|
|
|
It's good for business because they know that heavy users will be pleased with unlimited storage, and so when asked about backup solutions, they'll be more likely to recommend Crashplan to their less-intensive friends and family.
|
|
#
?
Mar 7, 2014 18:09
|
|
|
DNova posted:Crashplan encrypts all data for every user at the source. You can choose to manage your own key, but the default is to let the servers handle it. That does not mean that they are decrypting your data and deduplicating it on their end. What's the default? If the default key is the same for each user it's possible they are deduplicating across all users using the default key.
|
|
#
?
Mar 7, 2014 18:44
|
|
|
Thermopyle posted:They get more customers by advertising unlimited storage. Very, very few people will be backing up multiple terabytes. Amusing thing about that comparison though is the idea of unlimited data from wireless phone providers is dying off because as smartphones get fancier and more common peoples usage on average gets higher. I'm not suggesting we draw any conclusions from this comparison, certainly not about the future of cloud storage services, but it's just something that popped into my head. Col.Kiwi fucked around with this message at 19:49 on Mar 7, 2014 |
|
#
?
Mar 7, 2014 19:46
|
|
|
Ninja Rope posted:What's the default? If the default key is the same for each user it's possible they are deduplicating across all users using the default key. BlankSystemDaemon fucked around with this message at 21:19 on Mar 7, 2014 |
|
#
?
Mar 7, 2014 21:13
|
|
|
A question for the Synology users out there. Seeing as the 1513+ has USB 3 & eSATA ports on the back and my ~3TB of files I want to transfer onto it are currently on a USB3 seagate external, I am anticipating that I would just be able to plug the external directly into the Synology to copy files over. 1) Is this an accurate assumption, that it would work like that? I'm not sure how it shows up in your filesystem if you plug an external drive into the NAS 2) Would I be able to then leave an external plugged into the Synology and use some scheduled backup software (open to recommendations) to perform a folder sync on a weekly or monthly cadence for localized redundancy? I'd be planning on running it in SHR or Raid-5 mode, depending on what you guys recommend. From reading the discussion here it seems like it may be worth looking into using Crashplan. So far I've just been dumping all of my raw photography & personal files on a drive and leaving it at my parents place once a year or so, as they live in the same city.
|
|
#
?
Mar 7, 2014 21:23
|
|
|
Yes to everything above. You can use an attached USB drive as a backup target if you want, and you browse it through a web-based file browser in the NAS UI. For offsite backup I have an AWS account and use the Synology Glacier app to backup to Amazon.
|
|
#
?
Mar 7, 2014 21:34
|
|
|
MMD3 posted:From reading the discussion here it seems like it may be worth looking into using Crashplan. So far I've just been dumping all of my raw photography & personal files on a drive and leaving it at my parents place once a year or so, as they live in the same city. Buddy of mine hid a router with a usb drive attached to it at his parents house and backs up some of his stuff that way.
|
|
#
?
Mar 7, 2014 21:37
|
|
|
I checked and it seems like they actually are handling the encryption correctly which is nice, but certainly not something I assume. I use my own key, so their default handling practices weren't of much concern, but I thought I remembered it doing something silly and insecure by default. I actually might have been thinking about dropbox, which I don't use in favor of Spideroak. Either way, my bad.
|
|
#
?
Mar 7, 2014 21:58
|
|
|
MMD3 posted:A question for the Synology users out there. Yes to everything. I would do SHR2 though if things are really important to you, it will cost you another drive though. Here is a live demo of DSM 5 (the synology software) http://www.synology.com/en-global/products/dsm_livedemo Your device will come with 4.3 but can be upgraded to 5 (5 is still in open beta)
|
|
#
?
Mar 7, 2014 22:05
|
|
|
Don Lapre posted:Yes to everything. I would do SHR2 though if things are really important to you, it will cost you another drive though. My understanding of SHR1 vs. 2 is just that it has 2 drives as a fail-safe. I don't know that I've ever had a hard drive straight up fail on me, knock-on-wood. So the thought that two would fail before I had the chance to notice that one had gone and replace it seems very slim.
|
|
#
?
Mar 7, 2014 22:22
|
|
|
It looks like only the "custom key" option actually keeps them from knowing your password. D. Ebdrup posted:and if you're ever taught about salting in programming you can't avoid also learning about global unique identifiers. Do they actually ask you to provide a salt, or are you saying they always use a salt but it's chosen for you? MMD3 posted:My understanding of SHR1 vs. 2 is just that it has 2 drives as a fail-safe. It's more about what happens after that first drive fails. If the act of reading all of the data present on all of the remaining drives causes one to fail (or more likely, exposes a latent failure on one of the drives), you lose data unless you opted for an extra level of redundancy.
|
|
#
?
Mar 7, 2014 22:43
|
|
|
Ninja Rope posted:It looks like only the "custom key" option actually keeps them from knowing your password. hmmm, I see... so I'll just have to play w/ the calculator on synology's site and figure out what i'm willing to go with. drives are pretty cheap right now thankfully.
|
|
#
?
Mar 7, 2014 22:48
|
|
|
Anyone that has a HP N40L have access to the latest up to date BIOS / drivers (if any are required for FREENAS)? I just picked up a spare and this one is apparently out of warranty and apparently any access to download drivers / BIOS updates from HP now requires a service contract/registration. What the heck were HP thinking?
|
|
#
?
Mar 8, 2014 20:51
|
|
|
|
| # ? May 26, 2024 20:28 |
|
|
MMD3 posted:From reading the discussion here it seems like it may be worth looking into using Crashplan. So far I've just been dumping all of my raw photography & personal files on a drive and leaving it at my parents place once a year or so, as they live in the same city. Crashplan has some memory size limitations if I remember right. I have a 1GB 412+ and am using Symform instead; they have a contribution model which is good if you have unlimited internet and some spare drive space. Contribute 2x what you use and you don't pay a cent to back your stuff up to the cloud.
|
|
#
?
Mar 8, 2014 21:03
|
|
