Register a SA Forums Account here!
JOINING THE SA FORUMS WILL REMOVE THIS BIG AD, THE ANNOYING UNDERLINED ADS, AND STUPID INTERSTITIAL ADS!!!

You can: log in, read the tech support FAQ, or request your lost password. This dumb message (and those ads) will appear on every screen until you register! Get rid of this crap by registering your own SA Forums Account and joining roughly 150,000 Goons, for the one-time price of $9.95! We charge money because it costs us money per month for bills, and since we don't believe in showing ads to our users, we try to make the money back through forum registrations.
 
  • Post
  • Reply
Wibla
Feb 16, 2011

I rebuilt my old NAS (9x8TB) with some surplus gear, so now it's in an Antec P182 (used to host a 10x4TB NAS), with a Z170 + i5-6400 and 16GB ram. It's more than sufficient for backups.

Pretty happy with how it turned out considering I put minimal effort into (re)building it. Those 1 molex -> 5 SATA power cables are pretty neat.



Adbot
ADBOT LOVES YOU

necrobobsledder
Mar 21, 2005
Lay down your soul to the gods rock 'n roll
Nap Ghost

Charles Leclerc posted:

Pretty much. You can get more drives than specified into the Node 804 with some of Fractal's bracket mounts and a bit of redneck engineering, but it's a right little toaster unless you put in insanely loud, high CFM, high static pressure fans. It just gets to the point where you're forcing a poor solution to such a niche problem that it doesn't make sense.
Yeah, I've been messing with my Supermicro 3U fan setup and after moving it and trying to figure out what could possibly have caused all my drives to stop being detected now. I tied everything down I thought but there's probably some random stray screw that shorted something randomly.

Combat Pretzel
Jun 23, 2004

No, seriously... what kurds?!

Harik posted:

that's the same thing? or you mean that cyberpower doesn't bother with the double-sided tape on the batteries?
Cyberpower doesn't require you to use their batteries. I put some third party ones in. I got my replacement ones from Battery Direct in Germany.

Tiny Timbs
Sep 6, 2008

THF13 posted:

CA Backup/Restore is supposed to be one of the more reliable ways to get a backup of your appdata because it shuts down your docker containers before making its backup. Did you put the files back in place manually or use the restore function of the plugin? I've had permissions issues manually copying files from its backup to a single borked application, but for a full cache drive replacement you should just need to hit restore and have everything copied back correctly.

Yeah I used the restore function. I was able to fix the *arr DBs on my computer; for some reason the command line utility didn’t work.

IShallRiseAgain
Sep 12, 2008

Well ain't that precious?

If I get a UNC error and a I/O disk error while adding a new drive to the pool, should I return it or not? Should I do an extended SMART scan first?

Paul MaudDib
May 3, 2006

TEAM NVIDIA:
FORUM POLICE
The OWC M.2 shuttles are really cute, they take 4x M.2 and put it on a PCIe switch chip and fit it inside a 3.5" U.2 form factor, and give it some cooling. Consumer drive prices have collapsed, FX900 Pro (PCIe4+DRAM) 4TB are $250 and Ultrastar SN640 SAS SSDs are $390 for 7.68tb (for a semi legit looking listing). Is there a reason to prefer the enterprise SAS over the M.2? Endurance?

Wild EEPROM
Jul 29, 2011


oh, my, god. Becky, look at her bitrate.
This should work, right? I’m not seeing a reason why it shouldn’t

- pcie x16 slot bifercated to 4x4x4x4x
- pcie x16 card with 4x m.2 in said slot
- 4x m.2 to pcie x4 “riser” cables
- pcie x4 card in the risers (eg a nic) provided they receive enough power

BlankSystemDaemon
Mar 13, 2009



IShallRiseAgain posted:

If I get a UNC error and a I/O disk error while adding a new drive to the pool, should I return it or not? Should I do an extended SMART scan first?
RMA it, it's DOA.

Paul MaudDib posted:

The OWC M.2 shuttles are really cute, they take 4x M.2 and put it on a PCIe switch chip and fit it inside a 3.5" U.2 form factor, and give it some cooling. Consumer drive prices have collapsed, FX900 Pro (PCIe4+DRAM) 4TB are $250 and Ultrastar SN640 SAS SSDs are $390 for 7.68tb (for a semi legit looking listing). Is there a reason to prefer the enterprise SAS over the M.2? Endurance?
M.2 can be SATA or PCIe (ie. NVMe) depending on keying and what controller it's connected to - but in general SATA < SAS < NVMe is a good rule of thumb.

The technical details basically just boil down to the number of disk operation queues and their size: SATA has 1 queue that can fit 32 items, SAS has 2 queues that can fit 64 items, and NVMe has 65536 queues that can fit 65536 items.
Endurance is measured in Drive Writes Per Day, and a good enterprise drive will usually be able to do several TB/day.

Wild EEPROM posted:

This should work, right? I’m not seeing a reason why it shouldn’t

- pcie x16 slot bifercated to 4x4x4x4x
- pcie x16 card with 4x m.2 in said slot
- 4x m.2 to pcie x4 “riser” cables
- pcie x4 card in the risers (eg a nic) provided they receive enough power
In theory that should all work, sure.

In practice, the experience of working as a system integrator has taught me that even when a vendor promises something, they might completely fail to deliver on it - and while it was quite some time ago that I did that, everyone I know who's in the industry tells me it's still exactly like that.
So buyers beware, I guess.

IShallRiseAgain
Sep 12, 2008

Well ain't that precious?

BlankSystemDaemon posted:

RMA it, it's DOA.

Sigh, I'm never going to finish setting up my NAS. I already had 2 other disks arrive DOA because UPS delivered the box completely wrecked.

Olympic Mathlete
Feb 25, 2011

:h:


Apologies, I imagine this has been answered a hundred times or more already but I couldn't see in the opening posts about adding drives. I have a Synology DSplay218 which is serving my little home setup quite nicely with the addition of an Nvidia Shield to do all the heavy lifting with transcoding etc.

The Synology currently has a single 4TB drive sat in it about 75% full. I'd like to add more storage but also add redundancy. I know I can use the Synology SHR raid type to mirror the content but is the extra space sat unusable until I put in a matching sized drive at a later date? Thanks.

Kivi
Aug 1, 2006
I care

BlankSystemDaemon posted:

M.2 can be SATA or PCIe (ie. NVMe) depending on keying and what controller it's connected to - but in general SATA < SAS < NVMe is a good rule of thumb.
There's also XP941/SM951 Samsung that have AHCI over PCIe but those are super rare.

Thanks Ants
May 21, 2004

#essereFerrari


If you have one drive then the only place you can go from there is redundancy, see https://www.synology.com/en-us/support/RAID_calculator?hdds=4%20TB%7C8%20TB

Paul MaudDib
May 3, 2006

TEAM NVIDIA:
FORUM POLICE
Has AsRack or Supermicro done an epyc board that is just a big pile of oculink yet? Bonus points for OCP2.0 but not really needed

priznat
Jul 7, 2009

Let's get drunk and kiss each other all night.

Paul MaudDib posted:

Has AsRack or Supermicro done an epyc board that is just a big pile of oculink yet? Bonus points for OCP2.0 but not really needed

Oculink is probably dead, MCIO is the current champ. Nice thing about it is the connectors work to plug in mezzanine cards too so you can have the option of a mezz card or a x8 cable (possibly x16 too)

The new Supermicro H13/X13 boards in their rackmount systems they have a whole mess of MCIOs and there is a mix of cabled and plugged in mezzanine connectors. I don't know if they add any proprietary bullshit in their mcios but hopefully not and if someone like serialcables makes some mcio capable boards that might be pretty nifty. I think I saw on supermicro's site their workstation version with EATX also has mcio. Asrock Rack also definitely does.

Regarding the question earlier about the slot -> M.2 -> Cable -> interposer, that stuff is doable however the clocking will probably be the tricky bit. The NICs probably won't work using separate refclks and the x16 -> 4x4 card will need a clock fanout buffer because you can't just tap off of 100MHz HCSL lines. So it will have to have some components on it and those aren't super cheap - not as expensive as a full on pcie switch but especially with scarcities the clock fanout buffers can add a lot to a BOM cost.

VelociBacon
Dec 8, 2009

My home server/NAS is a windows 10 PC. Is there any way to stop windows updates from pushing updates/restarts? Every time a windows update gets pushed that restarts the server PC, I can't log into it over RDP until I turn the machine off and on again with the power button. I don't keep it plugged into a monitor/kb/etc. I'm not having to hard-reset it or anything but it's annoying and frequently I won't be at home when I realize this has happened.

Olympic Mathlete
Feb 25, 2011

:h:


Thanks Ants posted:

If you have one drive then the only place you can go from there is redundancy, see https://www.synology.com/en-us/support/RAID_calculator?hdds=4%20TB%7C8%20TB

Gotcha, thanks.

BlankSystemDaemon
Mar 13, 2009



Kivi posted:

There's also XP941/SM951 Samsung that have AHCI over PCIe but those are super rare.
:mad:

priznat posted:

Oculink is probably dead, MCIO is the current champ. Nice thing about it is the connectors work to plug in mezzanine cards too so you can have the option of a mezz card or a x8 cable (possibly x16 too)

The new Supermicro H13/X13 boards in their rackmount systems they have a whole mess of MCIOs and there is a mix of cabled and plugged in mezzanine connectors. I don't know if they add any proprietary bullshit in their mcios but hopefully not and if someone like serialcables makes some mcio capable boards that might be pretty nifty. I think I saw on supermicro's site their workstation version with EATX also has mcio. Asrock Rack also definitely does.

Regarding the question earlier about the slot -> M.2 -> Cable -> interposer, that stuff is doable however the clocking will probably be the tricky bit. The NICs probably won't work using separate refclks and the x16 -> 4x4 card will need a clock fanout buffer because you can't just tap off of 100MHz HCSL lines. So it will have to have some components on it and those aren't super cheap - not as expensive as a full on pcie switch but especially with scarcities the clock fanout buffers can add a lot to a BOM cost.
Needs U.3, OP.

priznat
Jul 7, 2009

Let's get drunk and kiss each other all night.

I don't see the point of trimode phy anymore personally especially with SAS pretty much reaching the end of the road, U.3 just seemed like unnecessarily confusing :haw:

EDSFF ftw!

Inept
Jul 8, 2003

VelociBacon posted:

My home server/NAS is a windows 10 PC. Is there any way to stop windows updates from pushing updates/restarts? Every time a windows update gets pushed that restarts the server PC, I can't log into it over RDP until I turn the machine off and on again with the power button. I don't keep it plugged into a monitor/kb/etc. I'm not having to hard-reset it or anything but it's annoying and frequently I won't be at home when I realize this has happened.

IMO figure out what's breaking RDP instead

Wizard of the Deep
Sep 25, 2005

Another productive workday
If you're using RDP that means you're either on Win10 pro or hacked Home to enable RDP. There's a checkbox in Pro that should disable updates, and there used to be a registry entry for Home, too.

Inept posted:

IMO figure out what's breaking RDP instead

But yea, this is the right answer.

VelociBacon
Dec 8, 2009

Hmm, alright. What kind of RDP shenanigans would cause an error like that? FWIW when it's in a no-connect state, it also can't be connected to from my android phone's rdp client.

I've had this issue across two hardware builds of this server so yeah I dunno.

Nitr0
Aug 17, 2005

IT'S FREE REAL ESTATE

VelociBacon posted:

Hmm, alright. What kind of RDP shenanigans would cause an error like that? FWIW when it's in a no-connect state, it also can't be connected to from my android phone's rdp client.

I've had this issue across two hardware builds of this server so yeah I dunno.

Have you checked your windows firewall? You can confirm by temporarily disabling the firewall. If RDP starts working then turn your firewall back on and fix the rules.

You sure your ip isn’t changing?

Other than that, RDP is so mature, it just works once you turn it on.

VelociBacon
Dec 8, 2009

Nitr0 posted:

Have you checked your windows firewall? You can confirm by temporarily disabling the firewall. If RDP starts working then turn your firewall back on and fix the rules.

You sure your ip isn’t changing?

Other than that, RDP is so mature, it just works once you turn it on.

Thanks, I'll try that next time. The IP is static. I've been running this setup for a couple years or more at this point and I just got used to having to power down/up the server every now and then due to what I thought was windows updates.

Kivi
Aug 1, 2006
I care

Paul MaudDib posted:

Has AsRack or Supermicro done an epyc board that is just a big pile of oculink yet? Bonus points for OCP2.0 but not really needed
Would you settle for SlimSAS? https://www.asrockrack.com/general/productdetail.asp?Model=ROMED4ID-2T#Specifications

Windows 98
Nov 13, 2005

HTTP 400: Bad post
Ladies and gentlemen, we got him


I got them to show up with my internal SAS controller. So it was definitely the LSI 9305-16e HBA fighting me. I checked and the firmware is 2 versions behind the internal SAS card. I literally cannot find version 16 of the firmware so I can't attempt to cross flash. I can find version 14, which is what it already has. Sad! I contacted the ebay seller and he is going to send me a card he has already tested the shelves on and knows it works. Good ebay seller. If the card works I will just pay him for the card. I have a feeling it will have SAS2 firmware and not SAS3. Hopefully that is fine with the SAS3 drives. If it is not I will kill myself, then start looking to find someone who knows how to flash these loving cards and pay them money to fix it and send it back to me.

IOwnCalculus
Apr 2, 2003





That is bizarre as hell. Found this that has v16 firmware for the 9305-16i, but I don't know that I'd be comfortable flashing that onto a -16e: http://images.45drives.com/Firmware/LSI9305/

Windows 98
Nov 13, 2005

HTTP 400: Bad post

IOwnCalculus posted:

That is bizarre as hell. Found this that has v16 firmware for the 9305-16i, but I don't know that I'd be comfortable flashing that onto a -16e: http://images.45drives.com/Firmware/LSI9305/

The cross flash guide I read said the firmwares are mostly identical across same “model” boards (9305). It will probably work. And if it doesn’t work I’m still in the same spot I am in now. I’ll take a look and thank you!

BlankSystemDaemon
Mar 13, 2009



IOwnCalculus posted:

That is bizarre as hell. Found this that has v16 firmware for the 9305-16i, but I don't know that I'd be comfortable flashing that onto a -16e: http://images.45drives.com/Firmware/LSI9305/
A v16 driver can usually cope with a v15 or older firmware, but a v15 or older driver can't cope with a v16 firmware without risking dataloss or ZFS issues galore - so please make sure that the driver is at least newer than the firmware.
It's of course always best if they match, since it's not impossible that they could've introduced a breaking change without documenting it - since they don't publicly document their APIs.

Windows 98
Nov 13, 2005

HTTP 400: Bad post
Every comment posted I am more and more convinced that I should just ship my card to someone who is knowledgeable on this stuff and have them fix it for money. I am not interested in data loss or ZFS problems. I simply cannot lose 160tb of data I already have. At the end of the day this is something that needs to be done once. I am not really interested in knowing the complete ins and outs of HBA card firmware. Sometimes the best option is to hire someone who can do the job correctly. What do you think the going rate is on something like that?

kri kri
Jul 18, 2007

Artofserver or whatever can probably do it

Purchased 2 hba cards via their store and they worked perfect

VelociBacon
Dec 8, 2009

Windows 98 posted:

What do you think the going rate is on something like that?

Probably around $800-2000 USD? I thought data recovery was super expensive.

Windows 98
Nov 13, 2005

HTTP 400: Bad post

VelociBacon posted:

Probably around $800-2000 USD? I thought data recovery was super expensive.

It is not data recovery. It is flashing firmware onto an HBA.

Wild EEPROM
Jul 29, 2011


oh, my, god. Becky, look at her bitrate.
Looks like one of these will solve your problem.
https://www.amazon.com/Mini-SAS-Adapter-SFF-8643-SFF-8644-Low-Profile/dp/B01L3H4N10

Just turns a mini sas internal into external, no flashing or anything

BlankSystemDaemon
Mar 13, 2009



I'm still not sure I understand the problem, other than not using a LSI/Awago/Broadcom HBA.

Windows 98
Nov 13, 2005

HTTP 400: Bad post

Wild EEPROM posted:

Looks like one of these will solve your problem.
https://www.amazon.com/Mini-SAS-Adapter-SFF-8643-SFF-8644-Low-Profile/dp/B01L3H4N10

Just turns a mini sas internal into external, no flashing or anything

There are no connections for these shelves internal. The testing I did to get the shelves to work was just that, testing. My internal SAS card has all its connections being used by the internal drives already in the server. I had disconnected them just to use the card to troubleshoot and rule out that the 9305-16e was not functional. Which I proved that theory. I need an external facing HBA that works with appropriate firmware. That's all that I need.

I am not doing data recovery. I am not doing anything special. I am hooking up 2 disk shelves to my server. That's all.

The (likely) problem is that the 9305-16e that I bought has incompatible firmware with the disk shelves. Even though it has IT firmware enabled, which is what is often the suggestion of the fix online, it still has firmware that is two versions behind the card that I know works. The internal card that works is already fully in use. But it is also a 9305, just a 9305-24i. Which means that if my internal card is running version 16 of the firmware and the new HBA is running version 14, the problem is that I more than likely must upgrade the firmware on the HBA for it to work. I am not interested in learning how to cross flash firmware and bios onto an HBA card. I would rather pay someone else to do it who knows how to do it.

All of this is moot. Because the ebay seller who sold me the shelves has sent me a new HBA card that supposedly works and has been tested on the shelves already. I am making a presumption that I will receive a card that is flashed for SAS2, and not SAS3 (because life is unfair). The firmware for handling SAS3 is different than SAS2. I have SAS3 drives, and would not like them gimped. So the plan is to take this supposedly working card from the ebay seller, get everything working and connected. Get my pool and ZFS situated with the new drives. Verify what firmware its running. If its compatible with SAS3 cool thats the end of all of this. If it doesn't and is only supporting SAS2 I will then try and find someone who can take my 9305-16e and cross flash the firmware to be the appropriate version and see if that fixes the card's problem. Otherwise my option is to just buy another card that is already flashed and functional and verified. Not keen on buying an entirely new one when I dropped a bunch of money on the 9305-16e and it is likely salvageable. Also double dislike spending money on yet another HBA card since I also bought and tried to use a NetApp HBA that turned out to be a bust because that too is running odd firmware that is specifically for use in NetApp ONTAP.

These cards are like $300. Spending $1000 on 3 different cards for getting shelves to connect is not an ideal situation and I would love to mitigate the cost as much as I can. I already spent $7500 on shelves and hard drives. I am not made of money lol. So thus the question becomes, who knows how to properly flash the firmware on these cards and how much would something like that cost.

If you are still unclear maybe scroll back a page or two and read my other posts about the situation.

BlankSystemDaemon posted:

I'm still not sure I understand the problem, other than not using a LSI/Awago/Broadcom HBA.

both my 9305-16e and my 9305-24i cards are LSI lol. respectfully, it kind of feels like you are out of the loop and haven't read the things I have typed. I respect it though. I don't read things either.

Wibla
Feb 16, 2011

Flashing LSI firmware is no big deal, I'm not sure what the fuss is about? Here's a guide. 9305-16e P16 IT firmware

Takes about 10 minutes from start to finish.

E: I was a bit hasty :v:

Wibla fucked around with this message at 14:34 on Jul 27, 2023

Windows 98
Nov 13, 2005

HTTP 400: Bad post

Wibla posted:

Flashing LSI firmware is no big deal, I'm not sure what the fuss is about? Here's a guide. 9305-16e P16 IT firmware

Takes about 10 minutes from start to finish.

E: I was a bit hasty :v:

Thank you for the links. I will try and give it a go myself before I start searching for someone who wants to help. I am an idiot though, soooo.......

The new HBA the ebay seller sent me arrives today. Tomorrow when I have more time I am going to try it out! If his card works it will at least put me in a position where I can get my poo poo working and running, then play with firmware on the 9305-16e in my spare time. Or at least reduce the urgency on needing to find someone else and pay them.

EDIT: The download links on this list of firmwares is not working properly. It's some goofy rear end FTP protocol link. I don't know why they would do that. It won't let me just click the link to download. Very annoying. If anyone is able to grab it and toss it on google drive or some other service for me to grab I will suck you off in the bathroom.

Windows 98 fucked around with this message at 14:41 on Jul 27, 2023

Scruff McGruff
Feb 13, 2007

Jesus, kid, you're almost a detective. All you need now is a gun, a gut, and three ex-wives.
Kri Kri mentioned him already but yeah, if you want video walkthroughs of flashing firmware on HBA's, ArtOfServer is the place to go. The dude has videos for almost every model LSI card and is a great resource for anything HBA related. When I bought my first card and cables from his ebay store he was also great at answering all my questions about how to hook that poo poo up.

Wibla
Feb 16, 2011

Windows 98 posted:

EDIT: The download links on this list of firmwares is not working properly. It's some goofy rear end FTP protocol link. I don't know why they would do that. It won't let me just click the link to download. Very annoying. If anyone is able to grab it and toss it on google drive or some other service for me to grab I will suck you off in the bathroom.

<zip>

Scruff McGruff posted:

Kri Kri mentioned him already but yeah, if you want video walkthroughs of flashing firmware on HBA's, ArtOfServer is the place to go. The dude has videos for almost every model LSI card and is a great resource for anything HBA related. When I bought my first card and cables from his ebay store he was also great at answering all my questions about how to hook that poo poo up.

Also very good advice :sun:

Wibla fucked around with this message at 16:30 on Jul 27, 2023

Adbot
ADBOT LOVES YOU

Windows 98
Nov 13, 2005

HTTP 400: Bad post

Thank you so much

  • 1
  • 2
  • 3
  • 4
  • 5
  • Post
  • Reply