|
I've been having this trouble for a while. I've tried several times to get it to work, and I cannot find the solution online (and I've even followed several video tutorials on Youtube step-by-step) so I figured I would need to ask. I have a box running ESXi, and on it I have a few VMs. They all use the same on-board NIC, but I also have a separate 2-port NIC in the computer as well. I initially got it because I wanted to try playing with pfsense. Of the 2-port NIC, one will go to my cable modem (Cisco DPC3008) and the other will go to an 8-port switch for other computers on the network. The two ports are, according to vSphere: vmnic1 - WAN vmnic2 - LAN and according to pfsense: WAN -> em0 -> (I never get an ip address here) LAN -> em1 -> 192.168.1.254/24 (This is statically set) I cannot ping 192.168.1.254 from any computer on the network, and pfsense cannot ping anything else including other VMs. vSphere shows that the port I'm using for WAN does get the ip address when plugged into the modem, but pfsense doesn't get that IP address on the port. I really don't know what to try next. It kind of seems as though the NIC ports are not being properly associated with the interfaces within pfsense. Does that sound right? How do I fix it? Thank you in advance for any help/advice.
|
#
?
Mar 1, 2015 04:32
|
|
|
|
| # ? May 6, 2024 11:49 |
|
|
Yeah set the vSwitch into promiscuous mode, also you may want to flip the vNics binding if you haven't already.
|
|
#
?
Mar 1, 2015 05:00
|
|
|
By flipping the vnic bindings, you mean set LAN to em0 and WAN to em1? If so, I have flipped back and forth several times. I just enabled promiscuous mode, but the result is the same.
|
|
#
?
Mar 1, 2015 05:51
|
|
|
Ok so basically it needs to work like this: Cable modem -> NIC1 -> vSwitch 1 -> pfSense VM WAN interface Then: pfSense VM LAN interface -> vSwitch 2 -> NIC2 -> 8-port switch What you have here is essentially two separate layer 2 networks, being bridged by the pfSense VM. ESX requires a vSwitch for each layer 2 network you want to use, and treats each vSwitch and the interfaces on it like a VLAN. So you create a vSwitch with your WAN NIC and one of the pfSense VM's NICs, then a second vSwitch with your LAN NIC and the other NIC in the pfSense VM. You should then be able to tell pfSense which is which, and it will bridge the two layer 2 networks. Make sense? edit: You can assign the other VMs' vNICs to the second vSwitch, you don't have to have a separate network for that. It's all on layer 2 with the physical switch and the LAN side of the router, so you're fine. Saves you a switchport.
|
|
#
?
Mar 1, 2015 17:18
|
|
|
That is how I've been trying to set it up. Here is a screenshot from vSphere.
|
|
#
?
Mar 1, 2015 19:39
|
|
|
Your test machine should not be on vSwitch 1, unless you want it to have direct access to the internet (DMZ). If you want it to be behind the firewall and receive an IP, it needs to be on vSwitch 2. edit: Can you get a screenshot of the settings page for the pfSense VM? fatman1683 fucked around with this message at 20:13 on Mar 1, 2015 |
|
#
?
Mar 1, 2015 20:05
|
|
|
The test machine is not going to be permanent. I only created it to check some things. It will probably be deleted within the next day or so. It isn't even powered on right now. Here is the settings page from vSphere:  I don't think this would be too valuable, but just in case: 
|
|
#
?
Mar 1, 2015 21:54
|
|
|
v0v Only thing I can think of then is that you've got the interface misassigned/misconfigured in pfSense. Check the MACs of your interfaces in pfSense against the MACs in the settings page.
|
|
#
?
Mar 1, 2015 22:34
|
|
|
I just verified the MAC addresses, and they are the same in pfsense and vSphere. I'm extremely confused.
|
|
#
?
Mar 1, 2015 22:53
|
|
|
Just for a giggle, try setting your LAN interface in pfSense to DHCP and see what happens.
|
|
#
?
Mar 1, 2015 23:06
|
|
|
The internal side still works, it just changed my ip address to 192.168.1.24. Nothing else is different.
|
|
#
?
Mar 1, 2015 23:23
|
|
|
Sorry, I'm out of ideas. I don't know pfSense's internal configuration all that well. I'd suggest going over the config docs and checking every setting, or maybe just reinstalling the VM with a fresh image.
|
|
#
?
Mar 1, 2015 23:36
|
|
|
I appreciate all of your help. I've reinstalled a handful of times in the past few days from different images. It's always the same issue. I just found something interesting, though. Apparently Realtek NICs are not supported in ESXi 5.5. However, the issue that people seem to have is that the NIC won't even appear, mine does (they've been able to resolve this with a new driver, which I just installed for the hell of it). Also, the onboard NIC in the motherboard has the exact same LAN chipset and that is working perfectly. ~ # esxcli network nic list Name PCI Device Driver Link Speed Duplex MAC Address MTU Description ------ ------------- ------ ---- ----- ------ ----------------- ---- ------------------------------------- vmnic0 0000:005:00.0 r8168 Up 100 Full 44:8a:5b:8b:ad:cd 1500 Realtek Realtek 8168 Gigabit Ethernet vmnic1 0000:008:00.0 r8168 Down 0 Half 00:13:3b:0f:69:50 1500 Realtek Realtek 8168 Gigabit Ethernet vmnic2 0000:00a:00.0 r8168 Up 100 Full 00:13:3b:0f:69:51 1500 Realtek Realtek 8168 Gigabit Ethernet ~ # ethtool -i vmnic0 driver: r8168 version: 8.013.00-NAPI firmware-version: bus-info: 0000:05:00.0 ~ # ethtool -i vmnic1 driver: r8168 version: 8.013.00-NAPI firmware-version: bus-info: 0000:08:00.0 ~ # ethtool -i vmnic2 driver: r8168 version: 8.013.00-NAPI firmware-version: bus-info: 0000:0a:00.0
|
|
#
?
Mar 1, 2015 23:56
|
|
|
Yeah I've never had very good luck with Realtek NICs in ESX/ESXi. A dual-port Intel adapter is cheap and you'd probably have better luck.
|
|
#
?
Mar 2, 2015 00:01
|
|
|
Something just occurred to me, does the (DHCP) on the WAN interface mean that it's a DHCP client, or a DHCP server?
|
|
#
?
Mar 2, 2015 01:00
|
|
|
1) run a tcpdump on the wan interface to see if you are seeing any network traffic. 2) if you have had anything else plugged into that cisco cable modem, power cycle it.
|
|
#
?
Mar 2, 2015 01:08
|
|
|
The DHCP is saying that it is a DHCP client. I don't think I'm able to set the WAN port as a DHCP server. I can set the LAN port as a DHCP server, but right now it is not set that way. When I set the LAN port to be a DHCP client, the "DHCP" appears the same way it is on the WAN interface. When I look at the packet capture for the WAN port, it shows a ton of ARP requests coming from various Comcast IPs. I've never looked at a packet capture from the WAN port on something like this (a big reason for setting up a pfsense machine is as a learning experience), but I'm kind of surprised/confused by what I see. I'll paste a small portion of it below. As for power cycling, I've done it many times throughout my troubleshooting. code:
|
|
#
?
Mar 2, 2015 03:33
|
|
|
yilduz posted:The DHCP is saying that it is a DHCP client. I don't think I'm able to set the WAN port as a DHCP server. I can set the LAN port as a DHCP server, but right now it is not set that way. When I set the LAN port to be a DHCP client, the "DHCP" appears the same way it is on the WAN interface. So when you set the LAN interface to be a DHCP client, what IP address does it receive? Also, I think having two machines on that first vSwitch might've broken something on the ISP end. You're only allowed one IP, but if you had both the pfSense VM and the test VM on at the same time you would've been requesting two. Might explain the ARP flood.
|
|
#
?
Mar 2, 2015 09:28
|
|
|
Just a thought, but I've noticed that with several ISPs that when you switch your WAN interface to a new device, you have to call them to release the DHCP lease to the old device first, otherwise you will not receive a new lease to the new device. Try giving them a call and see about that. It could be something that simple.
|
|
#
?
Mar 2, 2015 15:12
|
|
|
fatman1683 posted:So when you set the LAN interface to be a DHCP client, what IP address does it receive? When I set it to DHCP, it gave me the ip address 192.168.1.24. I have a Windows Server VM acting as a DHCP server and I told it to hand out dynamic addresses starting at 192.168.1.20. The test machine wasn't originally there. I created it yesterday because of all this stuff so I'm confident that it isn't causing the problem. It hasn't even been powered on most of the time. I took it off that vSwitch this morning and I still have the same issue. Nulldevice posted:Just a thought, but I've noticed that with several ISPs that when you switch your WAN interface to a new device, you have to call them to release the DHCP lease to the old device first, otherwise you will not receive a new lease to the new device. Try giving them a call and see about that. It could be something that simple.
|
|
#
?
Mar 2, 2015 16:01
|
|
|
|
| # ? May 6, 2024 11:49 |
|
|
yilduz posted:I wouldn't put it past Comcast to do something stupid like that. But before I spend several hours on the phone with those morons for a simple task, I'll test it by plugging in a couple of laptops directly to the modem after work and see what happens. I'll try some different things to see if I might be doing something wrong, and how I most easily get a new IP address when directly plugged into a different device. Thank you. There's a field where you can spoof the WAN interface MAC to whatever your old router's MAC address is/was
|
|
#
?
Mar 2, 2015 16:08
|
|
