Register a SA Forums Account here!
JOINING THE SA FORUMS WILL REMOVE THIS BIG AD, THE ANNOYING UNDERLINED ADS, AND STUPID INTERSTITIAL ADS!!!

You can: log in, read the tech support FAQ, or request your lost password. This dumb message (and those ads) will appear on every screen until you register! Get rid of this crap by registering your own SA Forums Account and joining roughly 150,000 Goons, for the one-time price of $9.95! We charge money because it costs us money per month for bills, and since we don't believe in showing ads to our users, we try to make the money back through forum registrations.
 
  • Locked thread
YOLOsubmarine
Oct 19, 2004

When asked which Pokemon he evolved into, Kamara pauses.

"Motherfucking, what's that big dragon shit? That orange motherfucker. Charizard."

cheese-cube posted:

3Gb/s or 6Gb/s?

At the "handful of 10k SAS drives" level it's not likely to matter.

Adbot
ADBOT LOVES YOU

YOLOsubmarine
Oct 19, 2004

When asked which Pokemon he evolved into, Kamara pauses.

"Motherfucking, what's that big dragon shit? That orange motherfucker. Charizard."

File system block size should generally match the storage block size or be evenly divisible by it to avoid wasted space. What the ideal block size is will be determined by the application workload.

YOLOsubmarine
Oct 19, 2004

When asked which Pokemon he evolved into, Kamara pauses.

"Motherfucking, what's that big dragon shit? That orange motherfucker. Charizard."

Cross posting from the Cert thread: I've got a home lab system that I'm looking to get rid of, if anyone is interested. It's this Shuttle case with an i7-3770s quad core, 16GB of memory and a 240GB mSATA drive. I'm looking to get around $500 for it. I've also got a Cisco SG300-10 layer 3 capable switch that I'll include for another $100. PM me if interested, or just drop a line here and I'll get in touch with you. I can probably also throw in some VMware NFR licenses that are usually good for at least six months.

YOLOsubmarine
Oct 19, 2004

When asked which Pokemon he evolved into, Kamara pauses.

"Motherfucking, what's that big dragon shit? That orange motherfucker. Charizard."

GobiasIndustries posted:

I want to do a total reboot of my home lab on ESXI6; I had started studying last year with an ESXi 5.5 setup but work and university got in the way, I'm also not an expert at this in any way shape or form, I'm planning on using it for learning how to set up a MS domain (AD/DNS/DHCP server, SQL server and whatever else I have a student license for), a few linux servers, and some networking practice. I was browsing around and found this video where the guy sets up ESXi and the web client before installing any VMs, would it be best/easiest to do that? It seems like it'd be faster to get the virt. stuff up and running first but I don't know if it matters at all in the grand scheme of things.

https://www.youtube.com/watch?v=Hm7kQHI4YnM

Yes, you generally want to set up VCenter before your other guests for manageability reasons. You may need to set up a VM serving DNS first though, as The VSphere 6 VCSA install will fail if it cannot resolve its own name via the DNS server you provide it.

YOLOsubmarine
Oct 19, 2004

When asked which Pokemon he evolved into, Kamara pauses.

"Motherfucking, what's that big dragon shit? That orange motherfucker. Charizard."

Dr. Arbitrary posted:

That's 2 cores, so as soon as you exceed two vms, you're going to have contention. This might not be a problem depending on the OS, (I think), especially if you've got good solid state storage.
I would go with i5 at a minimum.

Most people use consolidation ratios of 4 or even 8 vcpus to cores in production environments, a lab certainly isn't going to do 1 to 1.

If you're doing NUCs you're probably doing multiple ones in a cluster so the 2 core limitation isn't that serious.

YOLOsubmarine
Oct 19, 2004

When asked which Pokemon he evolved into, Kamara pauses.

"Motherfucking, what's that big dragon shit? That orange motherfucker. Charizard."

What is the great security threat you're trying to prevent by letting someone set up an isolated test environment? Put them on their own network, firewall everything but ingress management traffic, and let them go nuts.

People are probably already experimenting on your network whether you want it not. You can do a whole lot of experimenting in VMware workstation or Fusion.

YOLOsubmarine
Oct 19, 2004

When asked which Pokemon he evolved into, Kamara pauses.

"Motherfucking, what's that big dragon shit? That orange motherfucker. Charizard."

SEKCobra posted:

You're not gonna run a lab without any internet access. The only internet access in the data center is gonna be the companies.

Of course you can. Especially if all you're doing is setting up GNS3 or packet tracer or whatever. If you really want to lock it down then you just force them to use a jump box to move software into the lab, or even make them put it on removable media.

Of course, even with Internet access a virtual lab running a few server OSes is still a much smaller attack surface than your user workstations.

YOLOsubmarine
Oct 19, 2004

When asked which Pokemon he evolved into, Kamara pauses.

"Motherfucking, what's that big dragon shit? That orange motherfucker. Charizard."

SEKCobra posted:

Meh, maybe I'm paranoid from hospital work, but I wouldn't allow that.

Can you articulate your actual concerns beyond "not on my network!"?

YOLOsubmarine
Oct 19, 2004

When asked which Pokemon he evolved into, Kamara pauses.

"Motherfucking, what's that big dragon shit? That orange motherfucker. Charizard."

Ciaphas posted:

Where do you see that I've tagged VLAN 5 onto the NUC port? Do you mean something's wrong with my vSphere config? Switch said VLAN 5 is untagged across ports 3-8.

You have VSphere configured to accept frames tagged for VLAN 5, but the switch is sending frames destined for VLAN 5 as untagged. Look at the VLAN ID for the LAN port group that contains one of your pfsense interfaces. If you're sending untagged then the port group should not have a VLAN ID. Alternately, you should send traffic destined to VLAN 5 as tagged out of port 3. Mixing tagged and untagged traffic on the same port can be confusing.

YOLOsubmarine
Oct 19, 2004

When asked which Pokemon he evolved into, Kamara pauses.

"Motherfucking, what's that big dragon shit? That orange motherfucker. Charizard."

Ciaphas posted:

For your alternate, what I found last night when I set port 3 vlan 5 to tagged on the Netgear switch was that I could no longer communcate from my desktop PC (port 4) to the ESXi box on port 3, so I dropped that approach out of sheer confusion. (And I think I can't tag all traffic coming out of ports 4-8 because most local devices don't understand tagged frames, right?)

Other than that... You're suggesting I move the LAN virtual NIC on the vSwitch from the vlan 5 port group to the no-vlan port group, correct? Does that change how I would configure pfSense, i.e. whether I set it up to use VLAN interfaces or use the default em0 (WAN) and em1 (LAN) interfaces?

Which network is VLAN 5? 192.168.0.x? Which network is VLAN 4? What network should your VMs be a part of? You've got some mismatches on your port groups between tagged and untagged but it's hard to know where the problem is without knowing what's supposed to be where.

edit: I said it's usually best to tag everything if you're tagging anything on a port. So a single device passing traffic over a single VLAN can sit on an untagged port with the appropriate PVID. That's what most of your edge devices will do because they likely aren't capable of sending 802.1q tagged frames. ESXi can send tagged frames though, and you are using that on the port groups that are tagged as VLAN ID 4 and VLAN ID 5. However you have other port groups on that vSwitch that do not have a VLAN ID associated, so they are sending untagged traffic. This is what's causing confusion, as you've got both tagged and untagged traffic hitting the same switch port and you're trying to handle them in different ways despite them all ultimately having the same goal, which is to get tagged. A better solution is to set the VLANs you want as trunks on the switch (allow tagged frames on those VLANs) and then make sure all of you port groups in ESX have the appropriate VLAN tags. Then there's no confusion about what's lives on what VLAN or where it's tag is being applied. The switch is just accepting tagged frames and forwarding them to other ports that allow those frames.

YOLOsubmarine fucked around with this message at 21:28 on Aug 17, 2016

YOLOsubmarine
Oct 19, 2004

When asked which Pokemon he evolved into, Kamara pauses.

"Motherfucking, what's that big dragon shit? That orange motherfucker. Charizard."

Ciaphas posted:

Still not home from work but I think as soon as both of you said 'subnet' I maybe realized my problem. For the sake of testing I assigned the switch IP address 192.168.0.2, the pfsense router 192.168.0.1, and my working PC 192.168.0.3. Now that I've had more time to think on it, that sounds like a whoopsie? I'm not really sure though, since the router needs to be able to communicate between the WAN VLAN and the LAN VLAN. AN AN AN.

To answer your question, NippleFloss, my reckoning was that my cable modem and ESXi host (specifically, the pfSense VM) should be on the WAN VLAN (4, right now) and all of the VMs (in the same ESXi host machine! [ed: also including the router!]) plus other network devices (like my desktop PC) would be on the LAN VLAN (5). I guess the ESXi host needing to be on both VLANs while plugged into one switch port is what's causing the confusion here.

So, like I said, your issue is that your port groups don't match. Your ESXi host management network is on a port group on you vSwitch that is sending traffic untagged. The Port group that your pfsense router lives on is expecting tagged traffic. These two port groups need to have matching VLAN settings because the interfaces connected to them are on the same network and connected to the same switchport. If one is expecting to send/receive untagged traffic (the management port group) and the other is expecting to send/receive tagged traffic (the pfsense LAN interface) then one of them is going to be unhappy irrespective of the switch config. The quickest fix would be to remove the VLAN ID from the LAN port group. It's less clean than explicitly tagging all port group traffic, but it will get things correct quicker.

The way router on a stick would work in this scenario :

1) Traffic comes into your network through your cable modem
2) The modem forwards this to the PF sense VM's WAN IP address. When the frame arrives at the switch on port 2 it will add the 802.1q VLAN header, tagging it as belonging to VLAN 4, because that port is configured with with a PVID of 4 (untagged frames belong to VLAN 4).
3) The switch will then forward this frame out of port 3 (the arp table will tell it where the packet it going) with the VLAN 4 tag still in place, because port 3 is set to forward traffic destined to VLAN 4 with tags in place (T port not U port in the GUI). It will arrive at the ESX host and get forwarded to the vSwitch that contains the PFsense WAN interface. ESXi network stack will look at the VLAN tag, see that it is tagged for VLAN 4, verify that the interface being targeted belongs to a port group that allows VLAN 4, and so it will strip the header and forward the packet along to the PFsense WAN interface. The important thing here is that the tag gets applied at one end of the communication (ingress to the switch from the modem) and stripped at the other end (leaving the vSwitch destined for the VM).
4) The PFsense VM receives the (no longer tagged) frame and sees that it is destined for your internal network. It routes the VM onto your LAN segment and forwards it out of the PFsense LAN interface.
5) The LAN interface belongs to a port group with no VLAN ID, so it the vSwitch forwards the frame back to your Netgear switch with no VLAN tag applied.
6) The switch accepts the frame on port 3 and tags it for VLAN 5, because the switch is configured with a PVID of 5 on port 3.
7) It then forwards the frame out of port whatever towards your PC, on whatever port it is connected to. That port is set to pass traffic to VLAN 5 as untagged (U port, not T port) so the tag is stripped off and the traffic is delivered to your PC as untagged ethernet traffic, which it can read and understand. When it responds the frame is retagged as VLAN 5 when it enters the netgear switch, then forwarded back toward the PFsense WAN interface out of port 3 with the tags stripped, because port 3 is set to send traffic to VLAN 5 as untagged. The routing happens in reverse and then the frame is forwarded out of the ESXi host now with tags for VLAN 4 because it is leaving the PFsense WAN interface. It arrives at the switch with VLAN tag 4, and is forwarded on to the modem with the tags being stripped as it leaves port 2.

The crux of this is you must be mindful of what the netgear switch and the vswitch are expecting. If one is sending tagged traffic then the other must be configured to expect tagged traffic. If one is sending untagged then the other must be configured to expect untagged. The endpoints (physical or virtual) don't really know or care about any of this, the tags are added and removes as the packets move through the switching layer. It's helpful to remember that the ESXi host isn't an endpoint, it's a switch with endpoints behind it. You've got two switches that are sending VLAN tagged traffic back and forth between them, and they need to agree.

YOLOsubmarine
Oct 19, 2004

When asked which Pokemon he evolved into, Kamara pauses.

"Motherfucking, what's that big dragon shit? That orange motherfucker. Charizard."

VSAN will require at least three hosts. Each host will need a minimum of 1 free ssd and one free hdd. And each host needs to be running full ESXi, obviously, so you'd need to rebuild your media PC as a VM on the cluster. It's also not a requirement, but dual NICs are a good idea.

YOLOsubmarine
Oct 19, 2004

When asked which Pokemon he evolved into, Kamara pauses.

"Motherfucking, what's that big dragon shit? That orange motherfucker. Charizard."

MC Fruit Stripe posted:

I should be able to build it in a nested environment though, no? 3 ESXi VMs on the same PC, each with an additional, empty drive for VSAN to use. Slow or not, without having even Googled it yet, someone has installed VSAN inside a workstation environment, surely.

Sure, you can nest it, but it's going to run like poo poo so you'll never run VMs on it so all you'll really be testing is setting it up, which you could just do just as well with VMware hands on labs. VSAN setup is like a 10 minute task. It's really not worth going through the trouble of doing it at home if you aren't actually going to use it.

YOLOsubmarine
Oct 19, 2004

When asked which Pokemon he evolved into, Kamara pauses.

"Motherfucking, what's that big dragon shit? That orange motherfucker. Charizard."

rafikki posted:

Any suggestion for a NUC or NUC sized device with two NICs that I can put ESXI on? I want to spin up a palo alto VM and use it in my home network, but I'm not going to go the full 1U server or anything. A NUC would be great, but I know things like NICs can be touchy with ESXI and wondered if anyone here had set up something similar.

Do you need dual NICs for any particular reason? You can use a single NIC with ESXi, it only really limits you during certain migration scenarios.

YOLOsubmarine
Oct 19, 2004

When asked which Pokemon he evolved into, Kamara pauses.

"Motherfucking, what's that big dragon shit? That orange motherfucker. Charizard."

rafikki posted:

WAN and LAN ports.

Get a VLAN capable switch and do a router on a stick. I've looked before and there aren't really any offerings like the NUC with two network interfaces. Closest thing I've seen are some bare bones SFF pcs from Shuttle and the like.

YOLOsubmarine
Oct 19, 2004

When asked which Pokemon he evolved into, Kamara pauses.

"Motherfucking, what's that big dragon shit? That orange motherfucker. Charizard."

rafikki posted:

I thought about it, but I was hoping to just do the firewall for now and worry about a managed switch later.

There are definitely dual NIC NUCs out there, like http://a.co/45JBbPv or some Logic Supply ones, just curious if anyone has setup something themselves. If nothing else, I know I could just get a micro-atx case and build it out with a second NIC in there.

That and the logic supply option both have very low limits on memory which makes them fairly useless as virtualization hosts. Best option is going to be to build one out yourself, but it'll end up being more expensive, bigger, and probably louder.

YOLOsubmarine
Oct 19, 2004

When asked which Pokemon he evolved into, Kamara pauses.

"Motherfucking, what's that big dragon shit? That orange motherfucker. Charizard."

The new 6th gen NUCs have the benefit of being supported out of the box by vanilla ESXi without having to build your own custom image to get nics working.

You can do 32GB of memory, and M.2 SSD and a 2.5" drive, so you can get a decent amount of storage and memory in there, and they are dead quiet and put out very minimal heat.

YOLOsubmarine
Oct 19, 2004

When asked which Pokemon he evolved into, Kamara pauses.

"Motherfucking, what's that big dragon shit? That orange motherfucker. Charizard."

The big appeal of a second NIC for VMWare stuff is if you're trying to get onto or off of a DVswitch it can be nice to have. Can always work around it with nested virt though.

YOLOsubmarine
Oct 19, 2004

When asked which Pokemon he evolved into, Kamara pauses.

"Motherfucking, what's that big dragon shit? That orange motherfucker. Charizard."

You're very likely going to run out of memory before CPU on a NUC in a virtual lab scenario. Unless you're doing fairly compute intensive tasks CPU is not likely to be the bottleneck and I'd save the money and do the i5.

YOLOsubmarine
Oct 19, 2004

When asked which Pokemon he evolved into, Kamara pauses.

"Motherfucking, what's that big dragon shit? That orange motherfucker. Charizard."

There are some pretty serious performance issues with the native ACHI driver in ESXi 6.5, so if you're using sata drives in your home lab you may want to disable that driver to revert to the legacy one if you get poor performance. This affects anything that uses the SATA bus, so M.2 SSD as well.

esxcli system module set --enabled=false --module=vmw_ahci

YOLOsubmarine
Oct 19, 2004

When asked which Pokemon he evolved into, Kamara pauses.

"Motherfucking, what's that big dragon shit? That orange motherfucker. Charizard."

Docjowles posted:

Also I just realized that I left "get super drunk, order a set of rails at random, and force them into the rack any way you can" off the list of important rackmount server activities. Because lord knows whoever built the racks you're going to be servicing some day did exactly that. Coming out of the data center looking like you just lost a bare-knuckle boxing match is a critical career milestone.

I'm a big fan of the "rails don't fit, guess we will just set it on top of this other thing that's mounted with working rails" approach where you end up with one set of rails supporting four or five pieces of equipment.

Adbot
ADBOT LOVES YOU

YOLOsubmarine
Oct 19, 2004

When asked which Pokemon he evolved into, Kamara pauses.

"Motherfucking, what's that big dragon shit? That orange motherfucker. Charizard."

H2SO4 posted:

I'm looking for a layer 3 switch with four 10g SFP+ ports in addition to regular copper 1G ports. I see some Juniper boxes that look good but I'm cautious because I assume L3 features are license based so I'd prefer not to drop a grand only to find out the box doesn't have the appropriate licensing. Is this assumption correct, and are there any other options I'm not thinking of? I'm running Ubiquiti gear currently but I'm not married to it by any means.

EX switches do not require a license for L3. JunOS is really good, you should give them a shot.

  • Locked thread