QA/Networking/Test environment

Connectivity

The following table lists basic IPv4 connectivity configurations. There are numerous less common configuration including a combination of public and private addresses and multiple public addresses.

IPv4 connectivity	Address				Details
	Pub	Priv	LL	Lo
Global	yes	no	no	yes	Host is connected to the Internet using a public address.
Masqueraded	no	yes	no	yes	Host can use services on the Internet using a masqueraded private address.
Local	no	yes	no	yes	Host can use local services using a private address.
Link-local	no	no	yes	yes	Host can use services in the broadcast domain using a link-local address (IPv4LL).
None	no	no	no	yes	Host has only a loopback address.

The following table lists basic IPv6 connectivity configurations. There are numerous less common configuration including a combination of public and ULA addresses or public or ULA address without a link-local one.

IPv6 connectivity	Address				Details
	Pub	ULA	LL	Lo
Global	yes	no	yes	yes	Host has at least one link-local address and one global address, the latter being used for global connectivity.
Local	no	yes	yes	yes	Host has at least one link-local address and one unique local address (ULA) used for local communication.
Link-local	no	no	yes	yes	Host has only link-local (IPv6LL) and loopback addresses.
None	no	no	no	yes	Host has only a loopback address.
Disabled	no	no	no	no	IPv6 has been disabled in the kernel.

The following table lists common or somehow interesting combinations of IPv4 and IPv6 connectivity.

IPv4 \ IPv6	Global	Local	Link-local	None	Disabled
Global	Dual-stack server		IPv4 server
Masqueraded	Dual-stack workstation		IPv4 workstation	IPv4/PPP workstation	Disabled IPv6 workstation
Local		Private network
Link-local			Dual-stack zeroconf	IPv4 zeroconf
None	IPv6 node		IPv6 zeroconf	Isolated node

The following table provides details for selected combinations from the previous table.

Scenario	Details
Dual-stack server	Server available on public IPv4 and IPv6 addresses.
Dual-stack workstation	Workstation connecting via both IPv4 and IPv6 Internet services.
IPv4 server	Server available on IPv4 with IPv6 limited to link-local and loopback communication.
IPv4 workstation	Workstation connecting via IPv4 with IPv6 limited to link-local and loopback communication.
IPv4/PPP workstation	Mobile workstation connecting via IPv4 with IPv6 limited to loopback communication as IPv4-only PPP won't assign IPv6LL.
Disabled IPv6	Workstation connecting via IPv4 with IPv6 disabled in the kernel.
IPv6 node	Server available on public IPv6 address with IPv4 limited to loopback communication.
Private network	Workstation or server on an isolated private network using IPv4 private addresses and IPv6 ULA for local communication.
Dual-stack zeroconf	Workstation or appliance in a zero configuration network after a failed DHCPv4 attempt.
IPv6 zeroconf	Workstation or appliance in a zero configuration network on a system with IPv4LL disabled.
IPv4 zeroconf	Old-school node in a zero configuration network.
Isolated node	Disconnected node limited to loopback IPv4 and IPv6 communication.

How to get connectivity for everyday use

Most home and office setups offer masqueraded IPv4 connectivity. In addition they often offer global IPv6 connectivity, often firewalled. Some corporate setups, on the other hand, don't offer connectivity to remote hosts except via a proxy server. Global and unmasqueraded IPv4 and IPv6 connectivity for servers is typically available in datacenters.

IPv4 and IPv6 loopback addresses are assigned automatically at boot time. IPv6 link-local addresses are specific in that they are automatically assigned by the kernel when an interface gets connected to a network on the link layer. On the other hand, IPv4 link-local addresses only serve as a fallback for unsuccessful DHCPv4 attempt.

Note: It looks like link-local IPv4 addresses never get automatically configured in default Fedora 23 Workstation with NetworkManager. This is not necessarily a problem as it is much better to rely on ubiquitous IPv6 link-local addresses nowadays.

IPv6 tunnels

Many networks still only have IPv4 connectivity. You can overcome this limitation by configuring an IPv6 tunnel to a network with IPv6 connectivity. You can use various tunnel technologies to achieve that. There are also services like http://www.tunnelbroker.net/ that offer free tunnelled connectivity with IPv6 subnets.

How to emulate connectivity in a virtual test environment

Using two virtual machines, one can emulate any type of IPv4 and IPv6 connectivity by picking up an address block intended for documentation and testing ( 192.0.2.0/24, 198.51.100.0/24, 203.0.113.0/24 and 2001:DB8::/32) and setting up DHCPv4, RA, DHCPv6 and DNS services on one of the virtual machines, while using the other as a client node.

Router with automatic configuration

Server always needs to have a valid static configuration. For simple testing, you can configure the server temporarily using iproute package.

interface=eth1

ip address add 192.0.2.1/24 dev $interface
ip address add 2001:db8::1/64 dev $interface
ip link set $interface up

The easiest way to provide a testing IPv4 and IPv6 configuration server together with a local DNS server is using dnsmasq. The command bellow can be easily transformed into permanent configuration in dnsmasq.conf.

dnsmasq -d \
    --dhcp-range=lan,192.0.2.100,192.0.2.200,10m \
    --enable-ra --dhcp-range=lan,2001:db8::1:0,2001:db8::2:0,10m

Automatically configured node

A proper workstation installation will get configured automatically using NetworkManager. A server can do exactly the same especially if the router is configured with static address assignments via DHCP. Or the server can be configured manually.

On the other hand you can test the server configuration using dhcpcd command from the dhcpcd package. Use the following command. If you leave out the -T option, dhcpcd would configure kernel and /etc/resolv.conf using the information from the server.

dhcpcd -d -T eth0

Note: There are other non-default configuration tools including systemd-networkd and connman but those are out of scope of this document for now.

How to test network scenarios using Linux network namespaces

A pair of veth devices connecting two namespaces

clientns="test-client"
clientif=$clientns
serverns="test-server"
serverif="$serverns


# setup the network namespaces
ip netns add $serverns
ip netns add $clientns

# add veth device
ip link add dev $serverif type veth peer name $clientif

# add the veth device endpoints to the appropriate network namespace
ip link set $clientif netns $clientns
ip link set $serverif netns $serverns

# enable the loopback interfaces
ip netns exec $clientns ip link set lo up
ip netns exec $serverns ip link set lo up

Router with automatic configuration

# configure the static ip addresses on the interface where dnsmasq will be run
ip netns exec $serverns ip address add 192.0.2.1/24 dev $serverif
ip netns exec $serverns ip address add 2001:db8::1/64 dev $serverif

# activate the interface
ip netns exec $serverns ip link set $serverif up

# start dnsmasq as the DHCP server and DNS resolver
ip netns exec $serverns \
    dnsmasq -d \
    --dhcp-range=lan,192.0.2.100,192.0.2.200,10m \
    --enable-ra --dhcp-range=lan,2001:db8::1:0,2001:db8::2:0,10m

Automatically configured node

ip netns exec test-node \
    dhcpcd -d -T eth0

ip netns exec test-node \
    ping router.example.net

Notes on connectivity checks

Network application behavior may change based on various connectivity checks. Some of them work with IP addresses, some with routes and some are even based on attempting connections. All of the mentioned types of checks are currently present in glibc name resolution code, so you need to be careful about subtle differences in configuration.

DNS configuration

A client connects to a service using IPv4 or IPv6 addresses. In many cases the client retrieves a domain name from user configuration or user input and uses that domain name to query DNS for the IP addresses. The most universal translation of a service domain to an IP address is through a combination of SRV and A/AAAA records. First the service is translated to a set of host names using the SRV record, then each host name is translated to a set of IPv4/IPv6 addresses using the A/AAAA records. In many cases indirection using SRV records is not supported and the host name must also be used as the service names. In some cases a different record (MX, NS, ...) is used for the indirection.

The intermediate records (SRV, MX, NS, ...) are protocol neutral and vary by service. At least one A and at least one AAAA service is needed for proper operation of a dual-stack service. IPv4 only and IPv6 only services only have one or more A or AAAA record respectively. Configuration with multiple A and AAAA records (especially in combination with multiple SRV or other intermediate records) are useful for DNS based failover.

Key requirements:

• Domain name of the service is known (from user configuration or input)
• Optional: Authoritative DNS zone provides a list of host names for the service domain name
• Authoritative DNS zone provides correct A/AAAA records for the host domain name
• Service is available on addresses in A/AAAA records
• Recursive DNS server must be able to retrieve he above records
• Local host must be able tu query the DNS server and connect to at least some of the addresses

Possible reactions to DNS requests:

Reply type	DNS rcode	Meaning	Details
DNS	NOERROR	Data	Reply with one or more records of the requested type.
DNS	NXDOMAIN	No such domain	Reply saying that there are no records at all for the domain name.
DNS	NODATA	No data	Reply saying that there are records for the domain name but not of the requested type.
ICMP	–	Destination unreachable	The server isn't available for communication.
–	–	No reply at all	The server failed to reply or the reply was dropped on the firewall.

The test environment often requires a local recursive and authoritative DNS server. In most cases a dnsmasq instance or any other forwarding nameserver with some authoritative capabilities will do the job. For testing, domains can be chosen under example.net and example.com subtrees.

DNS server accessibility

Domain name servers serve different roles. Authoritative servers provide zone data configured by service administrators. Recursive servers retrieve zone data from authoritative servers on behalf of the clients.

Recursive servers are part of the client infrastructure. In a dual-stack network they need to be be available on both IPv4 and IPv6 to also serve single-stack clients using any of the protocols and addresses for both protocols should be announced using automatic configuration protocols. In setups with additinal forwarding DNS servers that do not perform full recursion this applies to those DNS servers that actually serve clients.

The following table contains common combinations of dual-stack, IPv4 only and IPv6 only network network clients and the respective requirements on recursive DNS service availability.

Client IPv4/IPv6 support			RDNSS requirements
Dual-stack	IPv4 only	IPv6 only	IPv4	IPv6
yes/no	yes	yes	yes	yes
yes/no	yes	no	yes	optional
yes/no	no	yes	optional	yes
yes	no	no	at least one of them

Authoritative servers are part of the service infrastructure. For dual-stack services they need to be available on both protocols in order to serve single-stack recursive DNS servers. For single-stack services they need to be available on the respective protocol.

Service connectivity	DNS records		Authoritative DNS
	A	AAAA	IPv4	IPv6
Dual-stack	yes	yes	yes	yes
IPv4 only	yes	no	yes	optional
IPv6 only	no	yes	optional	yes

All packages providing a DNS server should listen on both protocols by default.

Configuration examples

Dual-stack server

Using dnsmasq, /etc/dnsmasq.conf follows:

address=/server.example.net/192.0.2.1
address=/server.example.net/2001:DB8::1

Testing with netresolve:

$ netresolve --node server.example.net --protocol raw

Dual-stack server with SRV record

Using dnsmasq, /etc/dnsmasq.conf follows:

srv-host=_http._tcp.example.net,server.example.net,80,10,10
address=/server.example.net/192.0.2.1
address=/server.example.net/2001:DB8::1

Testing with netresolve:

$ netresolve --srv --node example.net --service http --protocol tcp

Common misconfigurations

No answer to DNS requests

Under normal circumstances each request is followed by a DNS reply or ICMP error packet. Failure to deliver any reply at all usually implies a misconfigured firewall dropping all DNS packets for a given protocol.

This can be easily simulated using the local firewall, see below.

# iptables -A OUTPUT -p tcp --dport 53 -j DROP                           
# iptables -A OUTPUT -p udp --dport 53 -j DROP
# ip6tables -A OUTPUT -p tcp --dport 53 -j DROP                           
# ip6tables -A OUTPUT -p udp --dport 53 -j DROP

No answer to DNS AAAA requests

This issue is unique because of the selective nature of answer absence. The local resolver typically receives answers for all DNS requests except those asking for AAAA records due to a bug in the authoritative server or any DNS server on the way from the local resolver.

This issue is not so easy to simulate as you would have to selectively drop DNS AAAA packets. Possible solutions are through delegating the firewall to userspace or using pcap.

TODO: Provide a mechanism to drop DNS AAAA packets, possibly using pcap or broken-dns-proxy.

Test cases:

• Lost AAAA request or reply

Note: A theoretical issue with dropped DNS A records hasn't been observed in the wild.