Register a SA Forums Account here!
JOINING THE SA FORUMS WILL REMOVE THIS BIG AD, THE ANNOYING UNDERLINED ADS, AND STUPID INTERSTITIAL ADS!!!

You can: log in, read the tech support FAQ, or request your lost password. This dumb message (and those ads) will appear on every screen until you register! Get rid of this crap by registering your own SA Forums Account and joining roughly 150,000 Goons, for the one-time price of $9.95! We charge money because it costs us money per month for bills, and since we don't believe in showing ads to our users, we try to make the money back through forum registrations.
 
  • Post
  • Reply
MustardFacial
Jun 20, 2011
George Russel's
Official Something Awful Account
Lifelong Tory Voter
Background:
  • Ubuntu Server running KVM
  • 2x NICs, one onboard (Realtek) and one PCIe 1x slot (Intel)
  • Intended to do VFIO passthrough of the Realtek NIC to the pfsense VM for the WAN interface
  • Network Diagram

Turns out that both my onboard and PCI NIC are in the same IOMMU group. Which means I would have to passthrough both devices to get passthrough working at all.

To solve this, I found out about the linux-acs-kernel patch to breakout the devices into separate IOMMU groups. I downloaded a patched kernel and set pcie_acs_override=downstream,multifunction in /etc/default/grub and applied it with sudo update-grub and a reboot.

Doing find /sys/kernel/iommu_groups/ -type l after the reboot shows that both NICs are now in different IOMMU groups. Great.

However now when I try to pass the onboard NIC by following this guide doing a lspci -nnv gives me Kernel driver in use: r8165 (or whatever it is) whereas before applying the ACS patch I could do the exact same process in the guide and that would give me Kernel driver in use: vfio-pci I cannot for the life of me get this to work with the ACS patch applied.

So that brings me to my current question:

If I scrap the VFIO passthrough and just run the WAN interface as macvtap with virtio - Private and isolate the traffic with VLANs (basically following my initial network diagram, but cross out the words "IOMMU Passthrough") are there any major security implications of this? Will that be enough to isolate the hostOS? Or will the host still see all of the traffic and still be exposed to the open internet?

[edit]
All of those code blocks were annoying to read so I replaced them with bold italics

MustardFacial fucked around with this message at 20:57 on Nov 21, 2019

Adbot
ADBOT LOVES YOU

MustardFacial
Jun 20, 2011
George Russel's
Official Something Awful Account
Lifelong Tory Voter
Turns out you only need to do boot time binding to vfio-pci for devices that don't like hot plugging (like GPU's). So all I had to do was remove all of the modprobe.d stuff and add the device as custom hardware in virt-manager.

Two weeks of loving around because I didn't finish reading to docs and missed a checkbox.

  • 1
  • 2
  • 3
  • 4
  • 5
  • Post
  • Reply