DNS Infrastructure SOP

We have two DNS server, the primary is at ns1.fedoraproject.org and is hosted at ServerBeach, from there it replicates to dns1.j2solutions.net.

Contact Information

Owner: Fedora Infrastructure Team

Contact: #fedora-admin, sysadmin-main

Location: ServerBeach and ibiblio

Servers: ns1.fedoraproject.org, ns2.fedorparoject.org

Purpose: Provides DNS to our users

Troubleshooting, Resolution and Maintenance

DNS update

DNS config files are puppet managed on puppet1. The update is standard to the puppet configs at http://fedoraproject.org/wiki/Infrastructure/Puppet/QuickStart

From puppet1:

git clone /git/puppet
cd puppet/modules/bind/files/master
vi fedoraproject.org # Don't forget to increment the serial!
cd ../..
git commit -m "What you did"
git push

It should update within a half hour. You can test the new configs with dig:

dig @ns1.fedoraproject.org fedoraproject.org
dig @ns2.fedorpaorject.org cvs.fedoraproject.org

Adding a new zone

First name the zone and generate new set of keys for it. Run this on ns1. Note it could take SEVERAL minutes to run:

/usr/sbin/dnssec-keygen -a RSASHA1 -b 1024 -n ZONE c.fedoraproject.org ; /usr/sbin/dnssec-keygen -a RSASHA1 -b 2048 -n ZONE -f KSK c.fedoraproject.org

Then copy the created .key and .private files to the private git repo (You need to be sysadmin-main to do this). The directory is private/private/dnssec.

Once that is done you can add the new zone to the puppet/modules/bind/files/master/ directory. Then add the new zone to puppet/modules/bind/files/zones.conf. Note: If you're adding a new geodns zone, you need to do that directly in the named.conf file, not zones.conf.

The last thing you need to do is make sure to put the .key file contents at the bottom of your zone file. See other zone files for an example. The .key file you generated earlier.

If you add a new child zone, such as c.fedoraproject.org or vpn.fedoraproject.org you will also need to add the contents of dsset-childzone.fedoraproject.org (for example), to the main fedoraproject.org zonefile, so that DNSSEC has a valid trust path to that zone.

GeoDNS

As part of our Content Distribution Network we use geodns for certain zones. At the moment just fedoraproject.org and *.fedoraproject.org zones. We've got proxy servers all over the US, in Europe and in the UK. We are now sending users to proxy servers that are near them. The current list of available 'zone areas' are:

• DEFAULT
• EU
• GB
• NA

DEFAULT contains all the zones. So someone who does not seem to be in or near the EU, GB, or NA would get directed to any random set. (South Africa for example doesn't get directed to any particular server).

Don't forget to increase the serial number in the fedoraproject.org zone file. Even if you're making a change to one of the geodns IPs. There is only one serial number for all setups and that serial number is in the fedoraproject.org zone.

Non geodns fedoraproject.org IPs

If you're adding as server that is just in one location, and isn't going to get geodns balanced. Just add that host to the fedoraproject.org zone.

Adding and removing countries

Our setup actually requires us to specify which countries go to which servers. To do this, simply edit the named.conf file in puppet. Below is an example of what counts as "NA" (North America).

view "NA" {
        match-clients { US; CA; MX; };
        recursion no;
        zone "fedoraproject.org" {
                type master;
                file "master/NA/fedoraproject.org.signed";
        };
        include "etc/zones.conf";
};

IP Country Mapping

The IP -> Location mapping is done via a config file that exists on the dns servers themselves (it's not puppet controlled). The file, located at /var/named/chroot/etc/GeoIP.acl is generated by the "GeoIP.sh" script (that script is in puppet).

This is known to be a less efficient means of doing geodns then the patched version from kernel.org. We're using this version at the moment because it's in Fedora and works. The level of DNS traffic we see is generally low enough that the inefficiencies aren't that noticed. For example, average load on the servers before this geodns was .2, now it's around .4

resolv.conf

In order to make the network more transparent to the admins we do a lot of search based relative names. Below is a list of what a resolv.conf should look like.

Any machine that is not on our vpn or has not yet joined the vpn should _NOT_ have the vpn.fedoraproject.org search until after it has been added to the vpn (if it ever does)

Phoenix

search fedora.phx.redhat.com vpn.fedoraproject.org fedoraproject.org

Non-Phoenix

search vpn.fedoraproject.org fedoraproject.org

The idea here is that we can, when need be, setup local domains to contact instead of having to go over the VPN directly but still have sane configs. For example if we tell the proxy server to hit "app1" and that box is in PHX, it will go directly to app1, if its not, it will go over the vpn to app1.