In response to the poor control I have over the included Modem/router unit provided by our ISP (outlined in this post), I wanted to deploy a firewall software application in a virtual environment to leverage the capabilities of consumer hardware.
The original setup is a Hitron CGN3 Gigabit modem without AC wireless. With 4 gigabit ports, we were unable to connect all network drops in the house. An extra switch was connected to attach more wired devices but that would mean those devices shared a single gigabit up-link.
Since there are more than six connected network drops throughout the house, the original 5 port switch I had installed wouldn’t be enough. I Amazon primed (it’s way too convenient…) a
Trendnet NetGear 8 port Gigabit switch to act as the backbone switch. With the dual gigabit specifically for Sophos and the integrated NIC for management/connection to the FreeNAS shares, we can have dedicated interfaces for internal and external connections.
There would be two gigabit interfaces – one connected to the modem, and another connected to the switch that connects all the network drops. Therefore, in theory, all devices would have to share the gigabit up-link in the future when we get gigabit internet (highly unlikely – everyone would have moved out at this point).
With the Proxmox server up and running, the Sophos UTM VM was ready to take over the role of routing from the combo unit.
However, the configuration of the WAN link was a little more complicated that I expected – but that was only because I didn’t know what I was doing. I’ve only ever tested Sophos in a double pass through configuration – which meant all traffic was still being routed by the combo unit. Bridging it was a whole other environment.
During the initial installation, I didn’t set up the WAN link since it wasn’t physically connected to the network yet. This meant that there was no automatic masquerading rule created. And so when I went to plug it in and apply DHCP to the WAN interface, there was no internet connection to the internal interface. Frustrated, I reinstalled UTM and it worked perfectly.
Basically if you choose to skip setup WAN/internet link during the setup, it won’t create a rule to map internal IPs to the external link.
It worked perfectly… until the performance didn’t seem to be as good as I had expected. The download was great – a full 200mbps down. But the upload was atrocious – 3mbps? I was expecting 15mbps.
Change the MTU!
After a bit of troubleshooting, I noticed that the MTU was a lot lower – 576 – the minimum for IPv4. Ok, let’s adjust it. The link went down, and then back up; but the MTU of 1500 that I set went back to 576. A bit of research revealed that UTM has some sort of bug that doesn’t apply MTU settings on the WAN interface since it is negotiated during DHCP. In order to adjust this, we would need to actually edit the UTM config file (more on that later).
MTU only helped so much. changing the MTU to 1500 bumped the upload speed to about 5mbps. Even bumping the MTU to the IPv4 maximum gave us a max upload speed of 12mbps. A lot better, but now we’re experiencing packet loss. Interestingly enough, a Windows 10 VM running within the Proxmox system is getting the full 15mbps.
Perhaps the NIC I got wasn’t good enough to handle the net speeds… But that doesn’t make sense – gigabit should mean gigabit… right?
But no fear. The PRO/1000 came a couple days ago and so I will be able to swap the cards and see if there is any performance difference. In any case, the Proxmox box should have a more reliable NIC with more than two ports. I may install the dual gigabit NIC into the FreeNAS server and play around with channel bonding to get higher throughput for the VMs that are using the FreeNAS storage through iSCSI. Stay tuned for more troubleshooting and performance tuning!