VMware VLAN and pfSense setup for your homelab

For those who want to go the extra mile in their homelab builds and wish to further isolate network traffic, the easiest solution is to implement VLANs. Possibly a VMware homelab paired with pfSense virtual router are the most popular configuration. The purpose of this article is to quickly cover the basics when it comes to creating VMware VLAN setup. This article follows the creation of VLANs created on physical TPlink switches used for Production and Fail-over. When implementing VLANs all the pertinent network ports need to support the VLAN tagging. This includes VMware virtual switches, routers and physical switches involved in the process. Making reference to the network diagram used in the previous article the idea in this case of create a Port Group specific for each VLAN and also add the VLAN ID. This allows for any VM part of a group to be configuration free. All the tagging is done somewhere else. VMware supports 3 types of VLAN tagging: External Tagging (EST), Virtual Switch Tagging (VST) and Virtual Guest Tagging (VGT). This article focuses on the VST method.

How does VMware and VLAN work with pfSense?

VMware virtual Switches can tag VLAN traffic before leaving the uplink network card and reach a physical switch and connect to another VMware vSphere Host part of the same vCenter deployment. In order for the VMs across different Hosts to communicate over a VLAN it is important both the virtual switches and the physical switches support the same VLAN configuration. All this traffic is managed at a Layer2. With reference to the ISO-OSI 7 layers model, the second layer is the “Data Link”. This is where switch and bridges devices work to transmit data between location. What if one network on a VLAN needs to communicate with a different one on a separate VLAN ID? For example the SQL VLAN and the Applications VLAN? This is where the routers (or even some advanced Layer3 switches) come into play just routing traffic from one network to another. Routers work at Layer3 “Network” and can switch packet to other networks. And the is where a virtual router like pfSense tops up the VMware VLAN configuration for homelab environments. This article focuses on the VMware side of things. The previous article covers the physical side from a network switch point of view. Next an article series on pfSense and how to support VMware VLAN configuration.

Putting all together

In this example, the VMware homelab supports different environments. Each one is isolated in its own VLAN and VMware Port Group: Production, LAB (for Dev and Testing), Nested (to run additional hypervisors like Hyper-V and Nutanix AHV inside VMware) and separate Network for the VSAN traffic. There are also 3 system Port Groups to further separate the Management Network from the ones used for Hot and Cold Provisioning. As a special Port Group, an additional one where the Port Group VLAN configuration is in “Trunk Mode” and will be used by pfSense to trunk all connections to the firewall. The pfSense firewall will manage the connections between different networks sitting on different VMware VLAN.

In particular the pfSense (or any other VLAN capable router) will have two network cards. One communicating with the external world and the other communicating with all VLANs. All Intel NUC Hosts share the same configuration and use the blue cable for the Management Network, Green for all Provisioning and Orange for all VM traffic.

In this example the second pfSense network card is part of the “0-4094 LAN Trunks” Port Group. This VMware Port Group has the VLAN enabled in trunk mode with all possible VLAN IDs. This Port Group uses as uplinks the Green and Orange, which are exactly the physical connections used for the VLANs. By selecting VLAN trunking option in the VLAN setting just for this Port Group it is possible to specify which VLAN IDs the Port Group will “in/egress” through the Port Groups uplinks. From 0 to 4094 is a sort of catch all and of course it posibble to restrict these only to the desired ones. For example from 0 to 100 and 1000, 2000 and 3000 the value would be something like 0-100,1000,2000,3000. Only this range and specific ones will be allowed to transit.

The pfSense virtual machine will also have a network card part of the Management Port Group. This allows not only the option to access the pfSense for management operations but also the ability for pfSense to reach the internet and act as a web proxy for the VMs in the segregated VLANs. Also it is worth noticing the VMs have access to the internet through the pfSense router only.

When moving to the other Port Groups the concept is similar. It is matter of creating the required Port Group and make from each VMware Host it includes the dedicated VMkernel and network cards. This way for example it possible to direct a special type of traffic (let’s say vMotion or Provisioning) to particular VMnic, routing table and VLAN.

Of course in this case the VLAN type will use the dedicated VLAN ID.

What it is important for each Port Group and especially when creating VMware VLAN, is to associated the intended active uplinks. To make things easier the advice is to use a nice naming convention maybe based on color coding. Uplinks are managed by the VMware virtual Switch configuration. In this example both the vMotion and Provisioning networks share the same green cable using two separate VLANs “21” and “22”.

A similar approach of course can be adopted for the other traffic types. For example separating Prod, Dev, Test and Storage Networks. The sample below shows the VM Production Port Group using the VLAN 30. Across all VMware Hosts the vmnic33 is the uplink dedicated to the VM traffic for different environments.

As per usual it is important to check the VLAN ID each Port Group will use. VMs in Production will use VLAN ID 30. And these will share the same Orange uplink cable together with other VLANs for Dev and Test on 31, nested into other hypervisors on 32 and still separate from VSAN traffic on 33. And the list could go on.

One final bit worth checking when configuring VMware VLAN is to make sure these are using specific VMkernels. Typically when creating VLANs these are sitting on separate domain broadcasts which might contain the same VLAN IDs, making them easier to recognize. By creating additional VMkernels, the right traffic type will be automatically routed through the appropriate routing table, with a specific Gateway IP Address. In addition all packets will be contained within same domain broadcast making networks more efficient.