How does a VPN work on Linux?

Categories: Linux, Network, Security

Introduction

This article was motivated by a problem I had: VPN access to my work environment from my new Linux install didn’t work - and some basic searches on the internet provided no obvious answers. It was therefore necessary to look into how VPNs work in some more detail…

The VPN solution I am using is Cisco Anyconnect (aka Openconnect), but much of the info below should apply to other VPN products.

The environment I am setting up the VPN on is Ubuntu 19.10 with Gnome; this implies network-manager for network configuration and systemd-resolved for DNS.

Note that I am a developer, not a networking specialist - any corrections welcome.

Initial Install

Installing Openconnect is pretty easy:

sudo apt install openconnect network-manager-openconnect network-manager-openconnect-gnome

Then go to Gnome’s Settings/Network and add a VPN definition. The VPN can then be enabled from either the settings window or the status-tray.

What a VPN does

So what does “enabling” the VPN actually do?

State With VPN Not Active

Here’s the relevant state of my network without the VPN active:

$ ifconfig

enp0s31f6: flags=4099<UP,BROADCAST,MULTICAST>  mtu 1500
         ether 98:fa:9b:00:7b:d3  txqueuelen 1000  (Ethernet)
         ...

lo: flags=73<UP,LOOPBACK,RUNNING>  mtu 65536
         inet 127.0.0.1  netmask 255.0.0.0
         inet6 ::1  prefixlen 128  scopeid 0x10<host>
         loop  txqueuelen 1000  (Local Loopback)
         ...

wlp3s0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
         inet 192.168.1.147  netmask 255.255.255.0  broadcast 192.168.1.255
         inet6 fd96:ffc0:5eb9:0:d46b:1e4:5624:5f20  prefixlen 64 scopeid 0x0<global>
         inet6 fd96:ffc0:5eb9::43a  prefixlen 128  scopeid 0x0<global>
         ...

$ route -n

Kernel IP routing table
Destination     Gateway         Genmask         Flags Metric Ref    Use 
Iface
0.0.0.0         192.168.1.1     0.0.0.0         UG    600 0        0 wlp3s0
169.254.0.0     0.0.0.0         255.255.0.0     U     1000 0        0 wlp3s0
192.168.1.0     0.0.0.0         255.255.255.0   U     600 0        0 wlp3s0

$ cat /etc/resolv.conf

nameserver 127.0.0.53
options edns0
search lan

$ systemd-resolve --status

...
Link 3 (wlps30)
  Current Scopes: DNS
  DefaultRoute setting: Yes
  Current DNS Server: 192.168.1.1
  DNS Domain: ~.

The ifconfig entries show that this system has one external network interface named “wlp3s0”, which has been allocated a local IP address of 192.168.1.147 (by the DHCP server it talked to on initialisation). This network interface name effectively identifies a driver context within the Linux kernel; in this case the context is associated with a Wifi driver. The interface named “lo” is associated with a “loopback” driver context, and is not really relevant here.

The second routing entry specifies that all packets sent to addresses of form 169.254.* (ie 169.254.0.0/16) are to be processed by the driver context “wlp3s0”. This range is the link-local address range - a special set of addresses that can only be used to access stations on the same local network segment (ie even more restricted than the private address range).

Similarly, the third routing entry specifies that any packets sent to 192.168.1.* are to be processed by that same context. I presume this is set up to match the gateway address (see below).

The first routing entry specifies that any packet whose destination address is not matched by a more-precise rule should be handled by “wlp3s0”. That first routing entry is special in that its flags include a “G” (Gateway) and it has a gateway address. This tells the code associated with “wlp3s0” that before it sends the packet out over the wireless device, it should wrap the packet in an envelope whose destination-address is the gateway address - ie the packet is redirected. That address happens to be the incoming-packet interface within the router that is attached to the wifi device; the driver within the router will then extract the original packet from the envelope and send it on to one of the ports attached to it, depending on the original address. For addresses that are “not local”, the packet is sent through the port that leads to an ISP which will then forward it into the wide internet.

DNS resolution has had a long and somewhat complicated history in Linux. The main Linux distributions now use systemd-resolved for resolving names; calls from applications to the standard C library functions that map name-to-address get forwarded to this daemon which then emits DNS queries to obtain the info (or returns values from a local cache).

The systemd-resolve output indicates that wlp3s0 is a valid interface to send DNS queries over, and gives the address to send them to. It also specifies for which domain-names this interface should be used to find ip addresses - “~.” means all domain-names.

State With VPN Active

After enabling the Cisco Openconnect VPN via the Gnome UI, things now look somewhat different. Clearly the VPN is at least partially working:

enp0s31f6: (unchanged)
lo: (unchanged)
wlp3s0: (unchanged)

vpn0: flags=4305<UP,POINTOPOINT,RUNNING,NOARP,MULTICAST> mtu 1300
         inet 172.17.35.95  netmask 255.255.254.0  destination 172.17.35.95
         inet6 fe80::440c:186a:c381:d61  prefixlen 64  scopeid 0x20<link>
         ...

$ route
(same 3 entries above)
111.11.111.11   192.168.1.1     255.255.255.255 UGH   600 0        0 wlp3s0
188.188.188.18   0.0.0.0         255.255.255.255 UH    50 0        0 vpn0
199.199.99.0    0.0.0.0         255.255.255.0   U     50 0        0 vpn0
.. plus many entries similar to the above two

A new interface “vpn0” has been defined; this corresponds to a context within the kernel that is associated with the Cisco VPN driver code. This code encrypts outgoing packets and wraps them in an envelope that points at the remote VPN endpoint, then resubmits them to the networking layer within the kernel. This context also receives incoming packets, unwraps and decrypts them, then resubmits them to the networking layer for delivery to the relevant application.

The new UGH entry is telling the kernel that all packets with destination 111.11.111.11 (the address on the envelope that the VPN driver adds to all outgoing packets) should be sent via the standard wifi driver and then out via my router’s standard route to my ISP. This address is the remote endpoint of the VPN - obviously all packets need to go over my wifi driver and then to my ISP in order to reach the VPN.

The other entries are ensuring that packets destined for specific address-ranges should be handled by the kernel driver responsible for interface “vpn0”.

Note that this VPN does not simply redirect all outgoing traffic; instead it redirects only specific IP-ranges which the server has been configured to intercept. This is certainly more efficient than catching all traffic; it means my employer does not need to act as a relay when I am interacting with sites that are not of interest. On the other hand, it does mean that my employer’s VPN does not protect me from observation or data-manipulation when accessing unrelated insecure sites.

The DNS Problem

The problem that originally motivated me to look into all this is that although the above traffic-routing was working fine, DNS was not.

$ cat /etc/resolv.conf

nameserver 127.0.0.53
options edns0
search lan

$ systemd-resolve --status

...
Link 5 (vpn0)
  Current Scopes: none
  DefaultRoute setting: no

Note here that vpn0 is not marked as being suitable for DNS requests, no DNS server is defined, and no domain-names are associated with the link.

Therefore, any request to map a name to an ip-address still goes out interface wlp3s0 to my ISP’s DNS server, and will return NXDOMAIN (not found) for any hostname that is defined only in my company’s internal DNS servers.

Solving the DNS Problem

One good step to resolving the problem is to install a few extra packages:

sudo apt install resolvconf resolvconf-admin

When activating the VPN, systemd-resolve --status now reports that link vpn0 is suitable for DNS and has a DNS server set.

Sadly, there are still no domain-names associated with that link. Any lookup for a domain-name will therefore still go to the default location, not over the VPN.

Interestingly, assuming my company’s internal DNS names are of form somehost.example.com:

systemd-resolve somehost.example.com  ==> lookup fails
systemd-resolve --interface vpn0 somehost.example.com ==> lookup succeeds

The following manual step therefore resolves the issue by binding one or more domains to the interface:

sudo systemd-resolve --interface vpn0 --set-domain example.com --set-domain otherexample.com

There is in fact a standard package that does this for OpenVPN:

sudo apt install openvpn-systemd-resolved

which applies the (manually configured) domains associated with the virtual network whenever the VPN is enabled.

There is no such package for Cisco “openconnect” (yet) as far as I know. I believe that an openconnect server provides a list of the domains that it hosts, ie it is possible for suitable client-side software to download and apply the domain redirects automatically (which is what happens on other platforms such as windows and mac). Of course that means that when activating the VPN, you are not in control of which domains will be routed via the VPN - a potential privacy issue.

For me, running a single command to bind the necessary domains to the interface after activating the VPN is sufficient. I have a script which does this, and run it manually after activating the VPN - a minor nuisance but bearable.

All this should of course be automatic, and hopefully will be in the not-to-distant future. On the other hand, some of the bug-reports listed in the references below indicate that for a “split VPN” of this type (where not everything is redirected) some developers think that the current behaviour is “correct”. This appears to be a core philisophical difference between convenience (automatically map domains) vs privacy (only remap domains the user wants to remap).

Running set-domain automatically

If you use the suggested systemd-resolve command above to configure routing via the VPN, then the following will ensure it is run every time the VPN is started:

TARGET=/etc/network/if-up.d/02-vpn-init
sudo tee $TARGET > /dev/null << EOF
#!/bin/sh
if [ "$IFACE" = "vpn0" ]; then
  systemd-resolve --interface vpn0 --set-domain ....
fi
EOF
sudo chmod a+x $TARGET

Current versions of Ubuntu use NetworkManager for VPN management, and file /etc/NetworkManager/dispatcher.d/01-ifupdown ensures the traditional scripts in /etc/network/if-up.d are executed.

References