Configuring systemd-resolved in a NetworkManager Environment
Artwork Credit: "Bolo - We are Already in Hell" By Shimmering-Sword A General Motors Mark XXXIII fully AI self-aware Bolo as described in the technical appendix of the second William Keith novel (the post-Laumer stories).
Summary
This is a WIP. It may contain inaccuracies and may be revised from time-to-time.
For certain configurations it might be desired to resolve DNS names using systemd-resolved on a system that also has NetworkManager-dnsmasq installed. One example of such a configuration is Ubuntu 17.10 Artful Aardvark desktop edition which ships with both NetworkManager-dnsmasq and systemd-resolved available. This blog post purports to show how to switch DNS name resolution back-and-forth between these two different DNS paradigms with NO drastic steps (such as uninstalling packages).
The two DNS resolution modes, NetworkManger-dnsmasq and systemd-resolved can co-operate in the setup detailed below.
On and after starts and restarts of the OS, and;
On and after starts and restarts of the NetworkManager service, and;
On and after starts and restarts of the systemd-resolved service, and;
On and after starts and restarts of the systemd-resolved-helper service.
Background Notes
This configuration is used on an Orabuntu-LXC installation which is part of a multi-host deployment of Orabuntu-LXC. The DNS/DHCP LXC container is at 10.207.39.2 and 10.207.29.2 and is attached to OpenvSwitches on host "athens". The configuration shown below is on host "mikonos" and the DNS/DHCP is obtained over a GRE tunnel attached to OpenvSwitch "sw1" on host mikonos which has endpoints on the physical LAN at 192.168.1.5 (mikonos) and 192.168.1.32 (athens). That can be summarized as shown below. Only the relevant switches for this discussion are shown. Additional switches that were not pertinent are not shown. The "patch ports" on the OpenvSwitch sw1 and sx1 allow all traffic on both zones to go over the single GRE tunnel via switch sw1.
Configuration of OpenvSwitches on "mikonos"
ubuntu@mikonos:~$ sudo ovs-vsctl show
870c8bd7-4c05-41eb-acd5-4968c6fa2771
Bridge "sx1"
Port "a3"
tag: 11
Interface "a3"
Port "a1"
tag: 11
Interface "a1"
type: patch
options: {peer="s1"}
Port "sx1"
tag: 11
Interface "sx1"
type: internal
Port "a2"
tag: 11
Interface "a2"
Bridge "sw1"
Port "gre0"
Interface "gre0"
type: gre
options: {remote_ip="192.168.1.32"}
Port "s3"
tag: 10
Interface "s3"
Port "ora73c24"
tag: 10
Interface "ora73c24"
Port "s4"
tag: 10
Interface "s4"
Port "s2"
tag: 10
Interface "s2"
Port "s6"
tag: 10
Interface "s6"
Port "sw1"
tag: 10
trunks: [10, 11]
Interface "sw1"
type: internal
Port "s5"
tag: 10
Interface "s5"
Port "s1"
tag: 11
Interface "s1"
type: patch
options: {peer="a1"}
Port "ora73c25"
tag: 10
Interface "ora73c25"
ubuntu@mikonos:~$
Configuration of OpenvSwitches on "athens"
The nameserver is "olive" and it has two interfaces "olivew" and "olivex" which provide the DNS/DHCP services from container "olive" which has isc-dhcp-server and bind9 installed. The patch ports allow lookups for both 10.207.29.2 and 10.207.39.2 to use the single GRE tunnel.
ubuntu@athens:~/Downloads$ sudo ovs-vsctl show
Bridge "sx1"
Port "a5"
tag: 11
Interface "a5"
Port "oel73c10"
tag: 11
Interface "oel73c10"
Port "a2"
tag: 11
Interface "a2"
Port "a4"
tag: 11
Interface "a4"
Port "sx1"
tag: 11
Interface "sx1"
type: internal
Port "a1"
tag: 11
Interface "a1"
type: patch
options: {peer="s1"}
Port "a3"
tag: 11
Interface "a3"
Port olivex
tag: 11
Interface olivex
Port "a6"
tag: 11
Interface "a6"
Bridge "sw1"
Port "s2"
tag: 10
Interface "s2"
Port "gre0"
Interface "gre0"
type: gre
options: {remote_ip="192.168.1.5"}
Port "ora73c10"
tag: 10
Interface "ora73c10"
Port "sw1"
tag: 10
trunks: [10, 11]
Interface "sw1"
type: internal
Port "s4"
tag: 10
Interface "s4"
Port "s5"
tag: 10
Interface "s5"
Port "s1"
tag: 11
Interface "s1"
type: patch
options: {peer="a1"}
Port "s6"
tag: 10
Interface "s6"
Port olivew
tag: 10
Interface olivew
Port "s3"
tag: 10
Interface "s3"
Port "ora73c11"
tag: 10
Interface "ora73c11"
ovs_version: "2.8.0"
ubuntu@athens:~/Downloads$
Configuration Steps
File: /etc/systemd/resolved.conf
This file is shown below and should be configured with the relevant nameservers and domains required for the system. In this case I've used my domains and nameservers as shown.
ubuntu@mikonos:~$ cat /etc/systemd/resolved.conf
# This file is part of systemd.
#
# systemd is free software; you can redistribute it and/or modify it
# under the terms of the GNU Lesser General Public License as published by
# the Free Software Foundation; either version 2.1 of the License, or
# (at your option) any later version.
#
# Entries in this file show the compile time defaults.
# You can change settings by editing this file.
# Defaults can be restored by simply deleting this file.
#
# See resolved.conf(5) for details
[Resolve]
DNS=10.207.39.2 10.207.29.2
#FallbackDNS=
Domains=urdomain1.com urdomain2.com gns1.urdomain1.com
#LLMNR=yes
#MulticastDNS=yes
#DNSSEC=no
#Cache=yes
#DNSStubListener=udp
ubuntu@mikonos:~$
File: /etc/systemd/system/systemd-resolved-helper.service
Create the following services as shown below and then run "systemctl daemon-reload". Note that the "systemd-resolved-helper.service" is an entirely new service that is added to the system.
ubuntu@mikonos:~$ cat /etc/systemd/system/systemd-resolved-helper.service
[Unit]
Description=systemd-resolved-helper Service
[Service]
Type=idle
User=root
RemainAfterExit=yes
ExecStartPre=/bin/ln -sf /lib/systemd/resolv.conf /etc/resolv.conf
ExecStart=/usr/sbin/service systemd-resolved restart
ExecStop=/usr/sbin/service systemd-resolved stop
[Install]
WantedBy=multi-user.target
ubuntu@mikonos:~$
Update System
Activate these services as shown below. Note that because in the blockcode shown below systemd-resolved-helper had already been previously enabled, there is no output from the enable command but normally there would be output confirming that the systemd-resolved-helper.service had been enabled and the symlink created.
ubuntu@mikonos:~$ sudo systemctl enable systemd-resolved-helper
ubuntu@mikonos:~$ sudo systemctl daemon-reload
ubuntu@mikonos:~$
Switching Between NetworkManager and Systemd-Resolved
Enable NetworkManager-dnsmasq DNS resolution
Add the "dns=dnsmasq" directive to the /etc/NetworkManager/NetworkManager.conf file as shown below.
ubuntu@mikonos:~$ cat /etc/NetworkManager/NetworkManager.conf
[main]
plugins=ifupdown,keyfile
dns=dnsmasq
[ifupdown]
managed=false
[device]
wifi.scan-rand-mac-address=no
ubuntu@mikonos:~$
Run the script below to switch to NetworkManager-dnsmasq DNS resolution.
ubuntu@mikonos:~$ cat enable-nm-dns.sh
#!/bin/bash
cat /etc/NetworkManager/NetworkManager.conf
sudo service systemd-resolved stop
sudo service systemd-resolved-helper stop
sudo systemctl disable systemd-resolved
sudo systemctl disable systemd-resolved-helper
sudo rm -f /etc/resolv.conf
sudo service NetworkManager restart
sleep 5
cat /etc/resolv.conf
nslookup yum.oracle.com
ping -c 3 yum.oracle.com
Or alternatively run the commands manually as shown below.
ubuntu@mikonos:~$ sudo service systemd-resolved stop
ubuntu@mikonos:~$ sudo service systemd-resolved-helper stop
ubuntu@mikonos:~$ sudo netstat -ulnp | grep 53
udp 0 0 0.0.0.0:5353 0.0.0.0:* 753/avahi-daemon: r
udp 0 0 0.0.0.0:49136 0.0.0.0:* 753/avahi-daemon: r
udp 0 0 127.0.1.1:53 0.0.0.0:* 1629/dnsmasq
udp6 0 0 :::5353 :::* 753/avahi-daemon: r
udp6 0 0 :::56879 :::* 753/avahi-daemon: r
ubuntu@mikonos:~$ sudo systemctl disable systemd-resolved
Removed /etc/systemd/system/multi-user.target.wants/systemd-resolved.service.
Removed /etc/systemd/system/dbus-org.freedesktop.resolve1.service.
ubuntu@mikonos:~$ sudo systemctl disable systemd-resolved-helper
Removed /etc/systemd/system/multi-user.target.wants/systemd-resolved-helper.service.
ubuntu@mikonos:~$ sudo rm -f /etc/resolv.conf
ubuntu@mikonos:~$ sudo service NetworkManager restart (takes a few seconds be patient)
ubuntu@mikonos:~$ cat /etc/resolv.conf
# Generated by NetworkManager
nameserver 127.0.1.1
ubuntu@mikonos:~$ nslookup yum.oracle.com
Server: 127.0.1.1
Address: 127.0.1.1#53
Non-authoritative answer:
yum.oracle.com canonical name = public-yum.oracle.com.edgesuite.net.
public-yum.oracle.com.edgesuite.net canonical name = a1186.g.akamai.net.
Name: a1186.g.akamai.net
Address: 23.48.208.91
Name: a1186.g.akamai.net
Address: 23.48.208.81
ubuntu@mikonos:~$ ping -c 3 yum.oracle.com
PING a1186.g.akamai.net (23.48.208.81) 56(84) bytes of data.
64 bytes from a23-48-208-81.deploy.static.akamaitechnologies.com (23.48.208.81): icmp_seq=1 ttl=55 time=17.1 ms
64 bytes from a23-48-208-81.deploy.static.akamaitechnologies.com (23.48.208.81): icmp_seq=2 ttl=55 time=18.0 ms
64 bytes from a23-48-208-81.deploy.static.akamaitechnologies.com (23.48.208.81): icmp_seq=3 ttl=55 time=19.6 ms
--- a1186.g.akamai.net ping statistics ---
3 packets transmitted, 3 received, 0% packet loss, time 2003ms
rtt min/avg/max/mdev = 17.116/18.253/19.618/1.045 ms
ubuntu@mikonos:~$
Enable Systemd-Resolved DNS resolution
Run the following script to switch to systemd-resolved DNS resolution.
#!/bin/bash
cat /etc/resolv.conf
sudo systemctl enable systemd-resolved
sudo systemctl enable systemd-resolved-helper
sudo service systemd-resolved start
sudo service systemd-resolved-helper start
sleep 5
sudo netstat -ulnp | grep 53
cat /etc/resolv.conf
nslookup yum.oracle.com
ping -c 3 yum.oracle.com
Alternatively run the following commands to switch to systemd-resolved DNS resolution as shown below.
Note that the "dns=dnsmasq" directive in /etc/NetworkManager/NetworkManager.conf does NOT have to be removed and can be left as is when switching to systemd-resolved.
ubuntu@mikonos:~$ cat /etc/resolv.conf
# Generated by NetworkManager
nameserver 127.0.1.1
ubuntu@mikonos:~$ sudo systemctl enable systemd-resolved
Created symlink /etc/systemd/system/dbus-org.freedesktop.resolve1.service → /lib/systemd/system/systemd-resolved.service.
Created symlink /etc/systemd/system/multi-user.target.wants/systemd-resolved.service → /lib/systemd/system/systemd-resolved.service.
ubuntu@mikonos:~$ sudo systemctl enable systemd-resolved-helper
Created symlink /etc/systemd/system/multi-user.target.wants/systemd-resolved-helper.service → /etc/systemd/system/systemd-resolved-helper.service.
ubuntu@mikonos:~$ cat /etc/NetworkManager/NetworkManager.conf
[main]
plugins=ifupdown,keyfile
dns=dnsmasq <-- no need to remove this it can stay set as is
[ifupdown]
managed=false
[device]
wifi.scan-rand-mac-address=no
ubuntu@mikonos:~$ sudo service systemd-resolved start
ubuntu@mikonos:~$ sudo service systemd-resolved-helper start
ubuntu@mikonos:~$ sudo netstat -ulnp | grep 53
udp 0 0 0.0.0.0:5353 0.0.0.0:* 811/avahi-daemon: r
udp 0 0 0.0.0.0:5355 0.0.0.0:* 1861/systemd-resolv
udp 0 0 127.0.0.53:53 0.0.0.0:* 1861/systemd-resolv
udp 0 0 127.0.1.1:53 0.0.0.0:* 1675/dnsmasq
udp6 0 0 :::5353 :::* 811/avahi-daemon: r
udp6 0 0 :::5355 :::* 1861/systemd-resolv
ubuntu@mikonos:~$ cat /etc/resolv.conf
# This is a static resolv.conf file for connecting local clients to
# systemd-resolved via its DNS stub listener on 127.0.0.53.
#
# Third party programs must not access this file directly, but only through the
# symlink at /etc/resolv.conf. To manage resolv.conf(5) in a different way,
# replace this symlink by a static file or a different symlink.
#
# See systemd-resolved.service(8) for details about the supported modes of
# operation for /etc/resolv.conf.
nameserver 127.0.0.53
ubuntu@mikonos:~$ nslookup yum.oracle.com
Server: 127.0.0.53
Address: 127.0.0.53#53
Non-authoritative answer:
yum.oracle.com canonical name = public-yum.oracle.com.edgesuite.net.
public-yum.oracle.com.edgesuite.net canonical name = a1186.g.akamai.net.
Name: a1186.g.akamai.net
Address: 23.48.208.81
Name: a1186.g.akamai.net
Address: 23.48.208.91
ubuntu@mikonos:~$ ping -c 3 yum.oracle.com
PING a1186.g.akamai.net (104.91.166.97) 56(84) bytes of data.
64 bytes from a104-91-166-97.deploy.static.akamaitechnologies.com (104.91.166.97): icmp_seq=1 ttl=55 time=16.7 ms
64 bytes from a104-91-166-97.deploy.static.akamaitechnologies.com (104.91.166.97): icmp_seq=2 ttl=55 time=17.4 ms
64 bytes from a104-91-166-97.deploy.static.akamaitechnologies.com (104.91.166.97): icmp_seq=3 ttl=55 time=17.8 ms
--- a1186.g.akamai.net ping statistics ---
3 packets transmitted, 3 received, 0% packet loss, time 2002ms
rtt min/avg/max/mdev = 16.779/17.373/17.880/0.478 ms
ubuntu@mikonos:~$
Further Considerations
There may be other ways to configure for systemd-resolved using systemd "Condition" directives rather than the "systemd-resolved-helper.service" and this should be explored also as this might be a way to program NetworkManager service to choose a DNS resolution mechanism without requiring the add-on "systemd-resolved-helper.service" but rather simply telling NetworkManager to check for certain installed DNS mechanisms and use those mechanisms according to as yet unspecified "Condition" directives yet to be written into the NetworkManager.service. Use of the systemd "Condition" directive is discussed here and here for example.