|
I've been given a somewhat odd backup task that I would love to get some suggestions on. One of my colleagues wants to periodically mirror roughly 30TB of data from one of our institutional network shares onto a pair of Drobos he has in his office. Both Drobos are configured with 8 4TB drives, giving each device ~22TB usable for storage exposed as a pair of NTFS-formatted "16" TB volumes for a total of 4 volumes with ~11TB (usable) each. Since the total size of the source data to be mirrored exceeds the size of any single Drobo volume, my colleague's objective is to have a tool that will automatically distribute the source data across the four target volumes. (Of course, since optimal bin packing is NP-hard, he's not expecting a perfectly even distribution, he just doesn't want to have to manually partition the data.) The core of this objective could be satisfied by a fairly simple script backed by du and rsync. However, the data on the network share changes at a rate of ~500GB/week, so it's unlikely that the "ideal" distribution calculated during the first sync will remain ideal, so the script would also have to be capable of rebalancing, which adds a layer of complexity that, while not intractable, I'd rather avoid. So my question is: does there already exist software that addresses this need? We're open to solutions coming in the form of anything from an already-existing script describing the above to a full-fledged backup system to a method for simply exposing the four Drobo volumes as a single large volume. Free would be ideal, but we'd rather pay for a good solution that "just works" than spend lots of time on even a great solution. A couple of other notes that may be relevant:
I appreciate any suggestions anyone can provide!
|
#
¿
Aug 20, 2015 04:07
|
|
|
|
| # ¿ May 4, 2024 07:54 |
|
|
Thanks for the suggestions, everyone. It sounds like there isn't going to be a clean and easy way for my colleague to get precisely what he wants using only the hardware resources already available to him. I think at this point that the path of least resistance will be to statically assign specific source directories to be mirrored to a specific destination volume, per ConfusedUs' recommendation. thebigcow posted:A mountain of symlinks maintained by a million lines of shell script to make it all seamless. Haha, yeah, I had that same thought for about 5 seconds before snapping back to my senses!
|
|
#
¿
Aug 21, 2015 00:49
|
|