NAS/Storage Megathread: What is this "File Deletion" You Speak of

The Something Awful Forums > Discussion > Serious Hardware/Software Crap > NAS/Storage Megathread: What is this "File Deletion" You Speak of

«‹›2 »

MrMoo: Sep 14, 2000

Open-e provides an interesting low-end alternative, buy your NAS box fill up with disks and use a USB based flash storage device to run the NAS/iSCSI system. Competitors include Wasabi Systems, and you can get FreeNAS to operate similarly.

With software vs. hardware, I think RAID 1 software is almost always faster. RAID 5 usually needs a big memory cache to get good performance due to bus speeds. Software RAID carries the bonus that you can swap the disks immediately into another server, hardware RAID needs the same vendor card. Some hardware RAID cards go obsolete and are incompatible on-disk with future models, it's unsurprisingly not a highlighted feature.

If you lose one disk in a RAID 5 it's highly probable that a remaining disk can fail during the rebuild. This is why hot-spares are preferred, and lower capacity disks. 1 TB disks can take an quite a long time to re-write. Following on, if you have a hot-spare you can be better off using that disk by bumping the level up to RAID 5e or RAID 6 if supported.

Some NAS devices only support code-page encodings for file names, e.g. Buffalo, so you cannot use multiple written languages: e.g. Simplified & Traditional Chinese.

NAS performance varies drastically, from 2-3 MB/s for single disk adapters to ~20MB/s for many current SOHO units. Jumbo frames are a method of bumping the performance to ~40MB/s using larger packets on the wire, but compatibility is limited and the technology slightly dead-end. The performance is directly linked to the processor, various methods are used to avoid the more expensive processors such as hardware RAID 5 calculations and ToE network interfaces. Raw disk speed in comparison is 60MB/s and above. This can mean RAID 5 is faster on SOHO devices than RAID 1, and RAID 10 (if supported).

Vendors are starting to compete more on features as performance is limited by cost. Qnap, Synology, and Thecus trail behind media support of Infrant/NetGear's system and this year started up forums, SSH access and more firmware updates than you usually see for consumer/SOHO hardware. Media compatibility includes iTunes servers, Xbox 360 & PS3 streaming, Slingbox, and other set-top style radio devices.

Synology recently added a IP camera surveillance system similar to ZoneMinder, it is unusually license restricted to 5 cameras. Although this might be tied to performance limitations of the hardware, although it easily performs better than ZoneMinder which is designed for direct attached CCTV cameras.

MrMoo fucked around with this message at 07:17 on Mar 19, 2008

# ¿ Mar 19, 2008 07:13

Adbot: ADBOT LOVES YOU

# ¿ May 2, 2024 05:30

MrMoo: Sep 14, 2000

QNAP with its 409 looks to competing with ReadyNAS NV+ for looks, but Synology on features. It doesn't have a LCD panel or that much media support, but they've bumped up the power to support RAID 6, which they've started to advertise in Asia with 256MB system memory and a 500Mhz random processor speed.

# ¿ Mar 22, 2008 14:17

MrMoo: Sep 14, 2000

Cidrick posted:

Or, this motherfucker holds 24:

http://www.supermicro.com/products/chassis/4U/846/SC846TQ-R900.cfm

That's pretty clever, 24 bays often ships in 5u:

http://usa.chenbro.com/corporatesite/products_cat.php?pos=14.

Haven't seen an equivalent of the Thumper case yet though.

# ¿ May 28, 2008 03:53

MrMoo: Sep 14, 2000

H110Hawk posted:

Xyratex makes one as well, the 48-in-5u is a pretty common form factor now.

I'd hardly call the top-end-only models of a few companies common, side-note: Xyratex website is rear end, the size appears to be 48/4u like the x4500.

(edit) Neither NetApp or EMC appear to have any top loading chassis models.

MrMoo fucked around with this message at 08:45 on May 28, 2008

# ¿ May 28, 2008 08:42

MrMoo: Sep 14, 2000

Col. Mustard posted:

Any other ideas to tweak FreeNAS (FreeBSD) SMB share to improve performance?

As discussed a few times previously, FreeBSD simply sucks at SMB/CIFS. Try going gigabit.

# ¿ Jun 9, 2008 17:20

MrMoo: Sep 14, 2000

DLCinferno posted:

fibre

That's just the interconnect, not the networking technology. The steps up from gigabit ethernet are InfiniBand (4GB+ for modern versions at 4x speed) and 10 gigabit ethernet. The latter is more practical and prices are always dropping. Chelsio make some awesome equipment that can run over copper.

As The Pro stated you might be better off going with a dual or quad Intel E1000 card and using link aggregation (LACP).

MrMoo fucked around with this message at 11:17 on Feb 14, 2009

# ¿ Feb 14, 2009 11:15

MrMoo: Sep 14, 2000

Methylethylaldehyde posted:

Look at rsync, it's designed for poo poo like that.

But stick to robocopy for Windows.

# ¿ Jun 29, 2009 03:51

MrMoo: Sep 14, 2000

Another Chinese review of FreeNAS, this time comparing on Atom hardware to the latest QNAP appliance. QNAP are usually third rate behind Synology and ReadyNAS.

Performance numbers:

QNAP SS-439 Pro, Atom 1.6Ghz, 1GB DDR2, Seagate 7200.4 500GB + WD Scorpio Black 320GB RAID 0.

code:

                    PC -> NAS  NAS -> PC
SAMBA:  big file    68.81MB/s  71.94MB/s
        many files  56.50MB/s  53.93MB/s

FTP:    big file    79.13MB/s  98.92MB/s
        many files  65.92MB/s  65.92MB/s

Power consumption:  idle 28W, active 35W.

FreeNAS 0.7RC1 on Intel D945GCLF2D, Atom Dual-Core 330, 2GB DDR2, Seagate 7200.4 500GB + WD Scorpio Black 320GB RAID 0.

code:

                    PC -> NAS  NAS -> PC
SAMBA:  big file    36.81MB/s  43.97MB/s
        many files  30.42MB/s  42.38MB/s

FTP:    big file     4.15MB/s  24.87MB/s
        many files   0.99MB/s  22.39MB/s

Power consumption:  idle 30W, active 38W.

# ¿ Jul 1, 2009 05:11

MrMoo: Sep 14, 2000

Allistar posted:

I've run into an issue where my throughput is very "bursty".

Likely network problem with packet drops causing Nagle to kick in and limit your transfer speed.

# ¿ Jul 6, 2009 03:23

MrMoo: Sep 14, 2000

Terpfen posted:

Their review is pretty kind:

That lovely site that has consistently proven it has no loving idea how to test performance:

quote:

NOTE: The maximum raw data rate for 100Mbps Ethernet is 12.5 MBytes/sec and 125 MBytes/sec for gigabit Ethernet. Throughput above these values is due to memory caching effects in the client OS and NAS under test.

# ¿ Jul 18, 2009 05:00

MrMoo: Sep 14, 2000

You mean Windows Server 2008 right?

# ¿ Jul 23, 2009 03:54

MrMoo: Sep 14, 2000

IOwnCalculus posted:

*Only one boot drive, and nonredundant power supplies? I hope they have all customers redundant across multiple boxes.

Already covered in the article, much like Google they have replication and de-duplication occurring at a higher level.

IOwnCalculus posted:

*How do you swap drives in that thing readily? The amount of downtime has got to be painful.

Much like Google they don't, it's too expensive in time when your engineers should be racking up new servers instead.

IOwnCalculus posted:

*Why not low-speed green drives? Clearly (see bus concern above) performance is a nonissue. Low-power drives would have made this a lot easier to power and cool.

Presumably they also carry a "green" premium that makes it cost inefficient and too slow on response time. Google's DIY designed PSUs suggest that green devices are good but I presume 5,400 rpm disks would simply be too slow?

# ¿ Sep 4, 2009 08:29

MrMoo: Sep 14, 2000

EnergizerFellow posted:

for the most part.

Finally they updated their utterly retarded NAS performance tests. They used to list several products with faster than wire speed performance :buddy:

# ¿ Sep 10, 2009 07:27

MrMoo: Sep 14, 2000

Synology just upgraded the 2009 2-disk version again for faster performance, claimed speeds are 58MB/s to NAS, 87MB/s from NAS.

http://www.synology.com/enu/products/DS209+II/index.php

Looks like they flattened the front plate too,

# ¿ Sep 28, 2009 04:26

MrMoo: Sep 14, 2000

Like these: http://eshop.macsales.com/shop/hard-drives/RAID/Desktop/

or the more insane 8-way boxes listed here: http://www.directron.com/externalhd.html

(SANS Digital)

Drobo have big rear end 8 and 9 bay boxes too: http://store.apple.com/us/product/TW754LL/A?fnode=MTY1NDA0Nw&mco=MTcwNzc3ODE&s=topSellers

MrMoo fucked around with this message at 10:22 on Mar 10, 2010

# ¿ Mar 10, 2010 10:09

MrMoo: Sep 14, 2000

Look for the JBOD only ones,

4-bay for US$139 http://www.directron.com/tr4u.html
8-bay for US$299 http://www.directron.com/tr8u.html

Also don't forget supply & demand, 2-bays are going to be a lot more popular.

# ¿ Mar 10, 2010 11:33

MrMoo: Sep 14, 2000

The_Frag_Man posted:

Shuttle is available here I think, but I don't know of a shuttle server that can take 5 drives.

I think the biggest can take 4, http://au.shuttle.com/product_detail.jsp?PLLI=14&PI=217

Synology have a 5-bay DX1010 which can pair up with the DX510 for 10-bays,

http://www.synology.com/us/products/DS1010+/index.php

QNAP have a a few variations of 8-bay monsters, SS-839, TS-859, TS-809 Pro

http://www.qnap.com/pro_detail_feature.asp?p_id=124
http://www.qnap.com/pro_detail_feature.asp?p_id=146
http://www.qnap.com/pro_detail_feature.asp?p_id=109

With the SS-839 using 2.5" disks for smaller form factor.

MrMoo fucked around with this message at 06:45 on Mar 13, 2010

# ¿ Mar 13, 2010 06:42

MrMoo: Sep 14, 2000

Wow, newegg has a 20-bay real bare bones DAS shelf, for only US$350

http://www.newegg.com/Product/Product.aspx?Item=N82E16811219033

The mentioned cons are not surprising, no documentation and poor build quality.

# ¿ Jun 1, 2010 13:57

MrMoo: Sep 14, 2000

OatmealRocks posted:

I've been looking at the Synology DS110j. I don't know much about NAS and have been reading up. I have read performance is to be expected (not great).. but does this means in terms for file read and write speeds via wireless?

I plan on putting a 2TB and have it sit in the other room which I will have it hooked up to the modem. My computer will be in the next room. I wanted to know what is the expected performance on file transfer wireless and wired to the wireless router?

Another possible setup would be having my iMac via cat5 to wireless router along with the NAS. WOuld file transfer between the two much faster? if so.. what can I expect? 75MB/sec?

Speeds are here, http://www.synology.com/enu/products/1bay_perf.php

WiFi is always going to be limited, you need tri-band 802.11n for best performance and it's not easy finding that.

# ¿ Aug 2, 2010 04:01

MrMoo: Sep 14, 2000

Synology v3 screenshots leak here, seems going a bit too far on the UI,

http://www.mediaplayerworld.com/index.php?topic=1861.0

# ¿ Aug 9, 2010 16:55

MrMoo: Sep 14, 2000

MJP posted:

Quick, stupid question: a regular gigabit Ethernet connection basically negates any advantage of a SATA 6.0gb/sec HDD? Just making sure before I buy a DNS-323 and SATA 3.0 HDDs.

The faster speeds, along with USB 3, are useful for stacking multiple disks or very fast SSD's on one connector. As such it would be beneficial to a minor degree for file serving whilst insignificant for raw block serving like iSCSI.

# ¿ Aug 16, 2010 15:50

MrMoo: Sep 14, 2000

You generally want the filing system to block IO layer to be faster than the network filing layer, hence why fibre & 10 GigE SANs.

Look at the Synology DS1010 with expansion boxes for a cheap high performance NAS unit. The XServes are still pretty good, the new models even have internal space for a SSD, above HDD capacity, which is pretty neat. I think the nearest rack equivalent is the RS810+.

MrMoo fucked around with this message at 03:13 on Sep 15, 2010

# ¿ Sep 15, 2010 03:11

MrMoo: Sep 14, 2000

Shaocaholica posted:

How do these consumer NAS boxes handle file locking? ie. two or more clients trying to modify the same file?

Works as expected unless you have CIFS Windows Vista+ pre-emptive locking enabled.

(edit) Microsoft calls it opportunistic locking.

# ¿ Sep 24, 2010 02:53

MrMoo: Sep 14, 2000

Buffalo never update anything and Netgear have a terrible UI interface, Synology looks a lot better. From a recent advert though it looks like Netgear is investing more into the lucrative NAS/iSCSI SAN market for virtual machines as they have a lot more expensive rack models, with single and multi-core Xeons, now:

http://www.readynas.com/?cat=27

The third option should be Qnap, the fourth should be Thecus, pretty much avoid anyone else unless you want unresolvable issues.

# ¿ Oct 2, 2010 03:45

MrMoo: Sep 14, 2000

angelfoodcakez posted:

In managing my media, i'm finding it very necessary to move files around a lot and it gets very slow over a network link.

Do I really just want an external RAID5/eSATA enclosure?

Yup, eSATA will be 3 Gbs (or 6 Gbs) and the box will be cheaper. Synology offer both.

# ¿ Oct 4, 2010 02:54

MrMoo: Sep 14, 2000

Latest Thecus NAS supports ZFS, shame their website, much like their products looks like rear end, useless for finding any useful details.

http://www.thecus.com/products_spec.php?cid=10&pid=220&set_language=english

# ¿ Oct 17, 2010 13:54

MrMoo: Sep 14, 2000

With 5400rpms you're looking at 1.7-2.5" form factor, they could probably ram 200 disks into one server for one amazing Super Thumper.

# ¿ Oct 19, 2010 03:42

MrMoo: Sep 14, 2000

With a NAS hooked up with a UPS the UPS is actually louder, although for sleeping the lights on the NAS are the most distracting.

# ¿ Nov 28, 2010 06:07

MrMoo: Sep 14, 2000

lol, the ReadyNAS 4200 & 3200 are the fastest low-end units currently available and certified for VM hosting but the 4200 price tag is US$8,000.

http://www.readynas.com/?cat=96
http://www.cdw.com/shop/products/default.aspx?EDC=2028983

I'd go with the Synology for backup storage, the expansion support system would probably be most useful.

# ¿ Nov 30, 2010 10:06

MrMoo: Sep 14, 2000

frogbs posted:

It never tells me that a drive is bad, it just tells me that the volume has crashed and then it re-verifies the array.

In Linux the RAID device breaks the volume whenever an IO operation fails. This would suggest you should see a lot of relocated sectors in the SMART report if it is a disk issue, otherwise something fruity with the controller. Sometimes this is a hardware failure, sometimes I guess it could be a drive incompatibility with the NAS controller.

# ¿ Dec 14, 2010 04:53

MrMoo: Sep 14, 2000

Synology just announced a 3.1 beta, cleaning up the MSIE-only poo poo from Surveillance Station seems the biggest improvement:

quote:

DSM 3.1 evaluates the efficiency of various operations. Synology DiskStation is the very first to support the sharing of print, fax and scan on a multifunction printer. Multiple administrators can now log on the same Synology DiskStation. In addition, File Browser supports preview of photos, videos, PDF, and Office documents, along with various searching criteria, making searches more precise and quickly. Getting the results has become timelier by performing database index to file names.

Enterprises call for data availability and reliability. The DSM 3.1 provides a workaround by allowing users to schedule synchronization of specific shared folders from one Synology DiskStation to another, providing an immediate recovery solution and allowing documents to be shared seamlessly. In addition, the Synology Hybrid RAID technology now supports two disks fault tolerance. Furthermore, users can choose multiple destinations in network backup for stacked layers of protection while the Windows ACL support allows data to be restored along with its complex access permission settings.

Synology is gearing up to meet the ever increasing demand for mobile device support. The industry�s first AirPrint support allows users wirelessly print documents from iPhone and iPad to any AirPrint-enabled printer connected to DiskStation. Additionally, the iPad-native DS photo+ and DS audio applications are now available. Their tailor-made UIs bring intuitive hands-on experience. By shaking iPad gently, the playing song will immediately jump to the next. Furthermore, the new DS file application allows the users of iPhone download and upload documents from and to Synology NAS server.

The DSM 3.1 extends users� multimedia experience by offering several overhauled applications. The Download Station automates every process, from searching downloadable files on the Internet to aggregating the latest available torrent through the RSS feeds. The Audio Station 3 features an equalizer to facilitate hearing sensation, a mini player to contribute to multitasking, personal playlists and editable ID3 Tags. The Personalized Photo Station allows every account user to host his or her own photo album and blog. Users can easily share the link of a photo album on facebook, twitter, and Plurk�.

Surveillance Station 5 now runs on Internet Explorer�, Firefox�, and Chrome�, and allows users to detect any motions, missing and foreign objects, camera occlusion and focus lost, and adjust image quality for detailed viewing. The Mail Station 2 supports multiple email domain fetching and multiple SMTP servers to offer centralized management of all emails and protect users� privacy. Additionally, selected Synology DiskStations are VMware� vSphere�and Citrix� XenServer�certified, and compatible with Microsoft Hyper-V�, providing a seamless storage solution for virtualization servers.

http://www.synology.com/enu/support/beta/Synology_DSM3.1_2011.php

# ¿ Jan 21, 2011 04:37

MrMoo: Sep 14, 2000

Because that's the largest 2.5" disk capacity drive available?

# ¿ Feb 3, 2011 03:18

MrMoo: Sep 14, 2000

The backup is a basic rsync and works pretty well. For an office NAS backup you will want some rotation, rsnapshot works pretty well:
rsnaphot.conf:

code:

interval        hourly  6
interval        daily   7
interval        weekly  4
interval        monthly 3

Crappy interface in ReadyNAS:

Resultant directories, efficiently de-duped using hard links.

# ¿ Mar 11, 2011 03:15

MrMoo: Sep 14, 2000

Synology DX510?

http://www.synology.com/enu/products/DX510/index.php

# ¿ Mar 12, 2011 10:47

MrMoo: Sep 14, 2000

The DX510 is the expansion unit, commonly recommended for the DS1511+.

# ¿ Mar 12, 2011 11:19

MrMoo: Sep 14, 2000

nimh posted:

Yes and if i understand the website correctly,

You're not, but it looks like I'm not either, this model is custom made for only expansion of their NAS devices not as a generic DAS:

quote:

The Synology DX510 is connected to the Synology DiskStation using an eSATA cable with custom-designed connectors on both ends,

So, looking towards other vendors, Thecus might be more appropriate:

http://www.hexus.net/content/item.php?item=9094

Never heard of Areca before:

http://www.pc-pitstop.com/das/arc-5040.asp

MrMoo fucked around with this message at 11:56 on Mar 12, 2011

# ¿ Mar 12, 2011 11:45

MrMoo: Sep 14, 2000

Thecus' current DAS is USB-3 only and a 2-bay unit whilst Promise have a Thunderbolt device.

http://www.thecus.com/product.php?PROD_ID=20
http://www.promise.com/storage/raid_series.aspx?m=192&region=en-global&rsn1=40&rsn3=47

# ¿ Mar 12, 2011 13:52

MrMoo: Sep 14, 2000

Just to reiterate, never use rsync on Windows, prefer robocopy instead.

I think the issue is down to weak timestamps that can drift meaning that rsync will copy the entire dataset again and again unless you use checksums for file comparisons which cause it to be impracticably slow. Also, robocopy is engineered to restart and resume the transfer when interrupted, it's not usually necessary with rsync.

# ¿ Mar 15, 2011 07:38

MrMoo: Sep 14, 2000

That's why I stopped using ReadyNAS devices but in the end this is why ZFS was invented, hopefully BTRFS can catch up. They all use relatively low end components and it's a complete bitch to get repairs done quickly and within the warranty period.

# ¿ Apr 7, 2011 15:16

Adbot: ADBOT LOVES YOU

# ¿ May 2, 2024 05:30

MrMoo: Sep 14, 2000

Factory Factory posted:

My first instinct was a 32 GB SDHC card for each and every show, but that's a bunch of wasted space per card, and $45 in card costs per gig. It might be worth it, and it would certainly store well and be durable, if you want to spring for it.

RAID 1 a set of compact flash cards, capacity is always increasing, the form factor is reasonably sturdy. The only concern is reliability and hence the RAID option, easy with say a Mac Pro laptop and Disk Utility.

# ¿ Apr 27, 2011 22:17

The Something Awful Forums > Discussion > Serious Hardware/Software Crap > NAS/Storage Megathread: What is this "File Deletion" You Speak of

«‹›2 »