No edit summary |
(update list of master mirrors, add rsync config snippit) |
||
Line 10: | Line 10: | ||
* '''master''': The Fedora servers dl*.fedoraproject.org | * '''master''': The Fedora servers dl*.fedoraproject.org | ||
* '''Tier 0''': The fast mirrors which pull from Red Hat's Internet2-connected master | * '''Tier 0''': The fast mirrors which pull from Red Hat's Internet2-connected master | ||
* '''Tier 1''': The fast mirrors which pull from the Tier 0 servers | * '''Tier 1''': The fast mirrors which pull from the master or Tier 0 servers | ||
* '''Tier 2''': The mirrors that pull from the Tier 1 servers. | * '''Tier 2''': The mirrors that pull from the Tier 1 servers. | ||
Line 21: | Line 21: | ||
* Must have at least 2 Internet2-connected Tier 1 mirrors. | * Must have at least 2 Internet2-connected Tier 1 mirrors. | ||
* Must have at least 1 Tier 1 mirror on each continent for which we have Tier 2 mirrors | * Must have at least 1 Tier 1 mirror on each continent for which we have Tier 2 mirrors | ||
* Must serve private rsync | * Must serve private rsync (see below for configuration) | ||
== Master Mirrors == | |||
* dl0[12345].fedoraproject.org, in Phoenix, AZ, USA. | |||
** dl.fedoraproject.org is a DNS round-robin to dl[12345]. | |||
* download-i2.fedora.redhat.com in Raleigh, NC, USA (Internet2, NLR, and those reachable over NLR only) | |||
== Tier 0 Mirrors == | == Tier 0 Mirrors == | ||
Line 38: | Line 44: | ||
== Tier 1 Mirrors == | == Tier 1 Mirrors == | ||
Tier 1 mirrors pull from one of the Tier 0 mirrors. | Tier 1 mirrors pull from one of the master or Tier 0 mirrors. | ||
{| border="1" | {| border="1" | ||
Line 56: | Line 62: | ||
| sunsite.mff.cuni.cz || | | sunsite.mff.cuni.cz || | ||
|- | |- | ||
| ftp.heanet.ie || IPv6 and Internet2 connectivity. ftp.heanet.ie::fedora-enchilada, ftp.heanet.ie::fedora-epel || mirrors at heanet.ie | | ftp.heanet.ie || IPv6 and Internet2 connectivity. ftp.heanet.ie::fedora-enchilada, ftp.heanet.ie::fedora-epel || mirrors at heanet.ie | ||
|- | |- | ||
| mirror.speedpartner.de || IPv4 and IPv6 || mirror at speedpartner.de | | mirror.speedpartner.de || IPv4 and IPv6 || mirror at speedpartner.de | ||
Line 63: | Line 69: | ||
|} | |} | ||
== | |||
* | == Tier 1 Rsync configuration == | ||
* | Below is an example rsyncd.conf file for a Tier 0 or Tier 1 mirror that provides private rsync access to select downstream Tier 2 mirrors. You may do this via either IP or DNS-based access control, or by a shared username/password which you give to your selected Tier 2 mirrors directly. | ||
* | |||
The key to this is that the Tier 0/1 mirror rsyncs content using a user account (e.g. ''mirror'' used below), and you serve content to Tier 2 mirrors using a private rsync module that runs as that same user account, while providing public non-authenticated rsync using the ''nobody'' account. In this way, Tier 2 mirrors may obtain content before the permissions are made world readable. | |||
<pre> | |||
use chroot = yes | |||
uid = nobody | |||
gid = nobody | |||
dont compress = *.gz *.tgz *.zip *.z *.rpm *.deb *.bz2 *.iso *.ogg *.ogv *.tbz | |||
exclude = .snapshot/ .~tmp~/ /.private/ /.private/** **/.nfs* | |||
ignore nonreadable = yes | |||
list = true | |||
read only = yes | |||
refuse options = checksum | |||
[ fedora-enchilada ] | |||
comment = Fedora - The whole enchilada | |||
path = /srv/pub/fedora | |||
[ fedora-epel ] | |||
comment = Extra Packages for Enterprise Linux | |||
path = /srv/pub/epel | |||
## | |||
## The following are not seen and are limited by IP. | |||
## | |||
[fedora-enchilada0] | |||
comment = Fedora Enchilada for Tier0|1 Mirrors | |||
path = /srv/pub/fedora/ | |||
list = no | |||
uid = mirror | |||
gid = mirror | |||
hosts allow = 152.46.7.122 200.17.202.1/28 zeus1.kernel.org ... | |||
[fedora-epel0] | |||
comment = Fedora EPEL for Tier0|1 Mirrors | |||
path = /srv/pub/epel/ | |||
list = no | |||
uid = mirror | |||
gid = mirror | |||
hosts allow = 152.46.7.122 200.17.202.1/28 zeus1.kernel.org ... | |||
</pre> | |||
[[Category:Infrastructure]] | [[Category:Infrastructure]] |
Revision as of 17:21, 23 May 2011
Tiering
Fedora mirror servers use Tiering, whereby a select few fast mirrors get read access to the master rsync servers, and all the other mirrors pull from those mirrors.
It turns out, 9 of our 10 Tier 0 or Tier 1 mirrors are available over Internet2. And, over half of our total mirrors are reachable over Internet2. So, let's make use of that whereever we can.
For our purposes, define:
- master: The Fedora servers dl*.fedoraproject.org
- Tier 0: The fast mirrors which pull from Red Hat's Internet2-connected master
- Tier 1: The fast mirrors which pull from the master or Tier 0 servers
- Tier 2: The mirrors that pull from the Tier 1 servers.
Properties of Tier 0 and 1 mirrors:
- Limit the number of Tier 1 mirrors to 10, to ensure adequate bandwidth for these. Adjust number up or down depending on capability of the masters.
- Must carry everything under fedora-enchilada and fedora-epel. This allows Tier 2 mirrors to exclude what they wish, but get everything if they so wish. This means at least 1TB of disk space for the Fedora portion of this server.
- Must have a 1 Gigabit connection to the Internet, or faster.
- Must have an active, available, responsive mirror administrator during the days content is staged.
- Must have at least 2 Internet2-connected Tier 1 mirrors.
- Must have at least 1 Tier 1 mirror on each continent for which we have Tier 2 mirrors
- Must serve private rsync (see below for configuration)
Master Mirrors
- dl0[12345].fedoraproject.org, in Phoenix, AZ, USA.
- dl.fedoraproject.org is a DNS round-robin to dl[12345].
- download-i2.fedora.redhat.com in Raleigh, NC, USA (Internet2, NLR, and those reachable over NLR only)
Tier 0 Mirrors
Tier 0 mirrors can pull from Red Hat directly over the Internet2 connection.
Server | Comment | Contact for ACL |
fedora-archives.ibiblio.org | Internet2 / National Lamba Rail (NLR) connected hosts. | <fedora-admin@ibiblio.org> No ACLs - open for syncing. |
archive.linux.duke.edu | Internet2. Uses ACL from MirrorManager database. | Drew Stinnett <drew.stinnett at duke.edu> (spacepope on IRC) |
Tier 1 Mirrors
Tier 1 mirrors pull from one of the master or Tier 0 mirrors.
Server | Comment | Contact for ACL |
kernel.org | mirrors1.kernel.org, mirrors2.kernel.org - USx2, mirrors3.kernel.org - NL, mirrors4.kernel.org - SE Do not sync from mirrors.kernel.org, choose one of the ones above and use that. |
<ftpadmin at kernel.org> |
wpi.edu | IPv6-connected or Internet2-connected mirrors only | Chuck Anderson <cra at wpi.edu> |
rsync.hrz.tu-chemnitz.de | rsync.hrz.tu-chemnitz.de::fedora-enchilada/. Uses ACL from MirrorManager database . | guenther.fischer at hrz.tu-chemnitz.de |
fedora-rsync.ftp.pub.2iij.net | rsync://fedora-rsync.ftp.pub.2iij.net/fedora-enchilada | mirror-contact at iij.ad.jp |
sunsite.mff.cuni.cz | ||
ftp.heanet.ie | IPv6 and Internet2 connectivity. ftp.heanet.ie::fedora-enchilada, ftp.heanet.ie::fedora-epel | mirrors at heanet.ie |
mirror.speedpartner.de | IPv4 and IPv6 | mirror at speedpartner.de |
fedora.c3sl.ufpr.br | South America | Carlos Carvalho carlos at fisica.ufpr.br |
Tier 1 Rsync configuration
Below is an example rsyncd.conf file for a Tier 0 or Tier 1 mirror that provides private rsync access to select downstream Tier 2 mirrors. You may do this via either IP or DNS-based access control, or by a shared username/password which you give to your selected Tier 2 mirrors directly.
The key to this is that the Tier 0/1 mirror rsyncs content using a user account (e.g. mirror used below), and you serve content to Tier 2 mirrors using a private rsync module that runs as that same user account, while providing public non-authenticated rsync using the nobody account. In this way, Tier 2 mirrors may obtain content before the permissions are made world readable.
use chroot = yes uid = nobody gid = nobody dont compress = *.gz *.tgz *.zip *.z *.rpm *.deb *.bz2 *.iso *.ogg *.ogv *.tbz exclude = .snapshot/ .~tmp~/ /.private/ /.private/** **/.nfs* ignore nonreadable = yes list = true read only = yes refuse options = checksum [ fedora-enchilada ] comment = Fedora - The whole enchilada path = /srv/pub/fedora [ fedora-epel ] comment = Extra Packages for Enterprise Linux path = /srv/pub/epel ## ## The following are not seen and are limited by IP. ## [fedora-enchilada0] comment = Fedora Enchilada for Tier0|1 Mirrors path = /srv/pub/fedora/ list = no uid = mirror gid = mirror hosts allow = 152.46.7.122 200.17.202.1/28 zeus1.kernel.org ... [fedora-epel0] comment = Fedora EPEL for Tier0|1 Mirrors path = /srv/pub/epel/ list = no uid = mirror gid = mirror hosts allow = 152.46.7.122 200.17.202.1/28 zeus1.kernel.org ...