Multi-site SSL – Server Name indication

I had a lot of issues setting up multiple HTTPS sites on a single webserver. After building out a VM on the Proxmox machine (FreeBSD), I imported the content of my website, the capstone project for George Brown’s T147 program, the development site for church, and a couple already migrated virtual hosts. The only challenge now was to get all sites encrypted with LetsEncrypt.

Migration

The migration consisted of copying all resources and httpd.conf, and also adjusting the DNS entry on the pfSense DNS resolver to point to the new VM.

The Issue

The first issue I encountered was installing python2.7 on the VM. And since it’s been a few weeks, I honestly cannot remember what I did to fix the issue. The main error that occurred was that C+ could not be compiled on the system. I used portsnaps, pkg install, removing, reinstalling but it didn’t work. I do recall changing the permissions on one of the folders. Maybe I’ll recreate another FreeBSD environment to find the fix.

The actual issue I have the biggest struggle fixing was SNI – server name indication. The purpose of SNI is to indicate what to serve when requested by a remote client. Since we only have one public IP, we could use name based virtual hosting. By specifying example.com, we would get redirected to content hosted under the example.com directive. One of the sub issues is when we encounter HTTPS requests. Since server name is encrypted within the HTTPS handshake, we don’t really know what hostname is being requested. This is supposed to be resolved before the HTTPS redirect occurs.

The Setup

So the virtual hosts entries are created. The HTTP is redirected to its HTTPS counterparts, the certificates for each site generated, if I were to go to mincading.ca I should be redirected to https://mincading.ca. It works!

I try the other site… Interesting – a warning from chrome stating that the certificate served is meant for mincading.ca… Perhaps there was a mismatch. I continue onto the site and it serves the content of this site instead. Weird. I try different configurations:

  1. Changing the virtual host order
  2. Explicitly specifying NamedVirtualHost 443/80
  3. Changing the order of 443 virtual hosts…

No luck. Keep in mind I’m configuring this through SSH and using external clients to connect. I kept clearing the cache, changing browsers, restarting the computer… Nope.

The aha! moment

I remote into a local machine on the network (the windows 10 VM to the rescue) and I test the sites. It worked! I check the certificates… It matches the server name / URL.

I decide to park the project and revert to standard HTTP while I give it more thought throughout the week.

Then while I’m configuring the NAT on pfSense for FTP (I can’t seem to figure out the permissions model for that…), I see the NAT entry for HTTP.  I’ve correctly redirected HTTP to the internal IP of the new FreeBSD server, but the HTTPS NAT entry still points to… the old Jail on the FreeNAS server. I completely missed this crucial step when it came to migrating the sites to the new server. Unbelievable.

Explanation

The symptoms reflected the setup as it should. Port 80 requests get forwarded to the new server, and since it redirects to the HTTPS counterparts, it gets forwarded to the old server where only one SSL cert is configured for one site – mincading.ca. That’s why it served the “wrong” certificate and content when outside, meanwhile the internal DNS entry pointed to the right IP.  This meant internal clients would be redirected to the right server thus receiving the correct certs and loading the correct content.

Conclusion

SNI was working as intended. It knew what hostnames were requested, and where to forward it to.  The symptoms were just because I missed out on changing the NAT on the firewall side to the correct IP.