From Fedora Project Wiki
(update duke admin info)
No edit summary
 
(48 intermediate revisions by 13 users not shown)
Line 4: Line 4:


Fedora mirror servers use Tiering, whereby a select few fast mirrors get read access to the master rsync servers, and all the other mirrors pull from those mirrors.
Fedora mirror servers use Tiering, whereby a select few fast mirrors get read access to the master rsync servers, and all the other mirrors pull from those mirrors.
It turns out, 9 of our 10 Tier 0 or Tier 1 mirrors are available over Internet2.  And, over half of our total mirrors are reachable over Internet2.  So, let's make use of that whereever we can.


For our purposes, define:
For our purposes, define:
* '''master''': The Red Hat servers download*.fedora.redhat.com.
* '''master''': The Fedora-owned servers dl.fedoraproject.org
* '''Tier 0''': The fast mirrors which pull from Red Hat's Internet2-connected master
* '''Tier 1''': The fast mirrors which pull from a master mirror.
* '''Tier 1''': The fast mirrors which pull from the Tier 0 servers (or one of the other masters).
* '''Tier 2''': The mirrors that pull from the Tier 1 servers.
* '''Tier 2''': The mirrors that pull from the Tier 1 servers.


Properties of Tier 0 and 1 mirrors:
Properties of Tier 1 mirrors:


* Limit the number of Tier 1 mirrors to 10, to ensure adequate bandwidth for these.  Adjust number up or down depending on capability of the masters.
* Limit the number of Tier 1 mirrors, to ensure adequate bandwidth for these.  Adjust number up or down depending on capability of the masters.
* Must carry everything under fedora-enchilada and fedora-epel.  This allows Tier 2 mirrors to exclude what they wish, but get everything if they so wish.  This means at least 1TB of disk space for the Fedora portion of this server.
* Must carry everything under fedora-enchilada and fedora-epel.  This allows Tier 2 mirrors to exclude what they wish, but get everything if they so wish.  This means at least 1TB of disk space for the Fedora portion of this server.
* Must have a 1 Gigabit connection to the Internet, or faster.
* Must have a 1 Gigabit connection to the Internet, or faster.
* Must have an active, available, responsive mirror administrator during the days content is staged.
* Must have an active, available, responsive mirror administrator during the days content is staged.
* Must have at least 2 Internet2-connected Tier 1 mirrors.
* Must have at least 2 Internet2-connected Tier 1 mirrors.
* Must have at least 1 Tier 1 mirror on each continent for which we have Tier 2 mirrors
* Must have at least 1 Tier 1 mirror on each continent for which we have Tier 2 mirrors.
* Must serve private rsync
* Must serve private rsync (see below for configuration).
 
== Master mirrors ==
 
* dl0[123].fedoraproject.org, in Ashburn, VA, USA.
* dl0[45].fedoraproject.org, in Ashburn, VA, USA - tier1 mirrors only.
** dl.fedoraproject.org is a DNS round-robin to dl0[123].
** dl-tier1.fedoraproject.org is a DNS round-robin for dl0[45].


== Tier 0 Mirrors ==
== Master mirror rsync modules ==


Tier 0 mirrors can pull from Red Hat directly over the Internet2 connection.
The master mirrors provide two additional rsync modules which provide pre-bitflip content.  Fedora tiered mirrors should use these modules to be able to get pre-bitflip content.


{| border="1"
{| border="1"
|-
|-
| Server || Comment || Contact for ACL
! Module name || Content
|-
| fedora-buffet0 || Everything under /pub/, including pre-bitflip content
|-
|-
| fedora-archives.ibiblio.org || Internet2 / National Lamba Rail (NLR) connected hosts. || Don Sizemore <dls at metalab.unc.edu>
| fedora-enchilada0 || Everything under /pub/fedora/, including pre-bitflip content
|-
|-
| archive.linux.duke.edu ||Internet2.  Uses ACL from MirrorManager database. || Drew Stinnett <drew.stinnett at duke.edu> (spacepope on IRC)  
| fedora-epel0 || Everything under /pub/epel/, including pre-bitflip content (even though EPEL doesn't do bitflips)
|}
|}


== Tier 1 Mirrors ==
== Tier 1 mirrors ==


Tier 1 mirrors pull from one of the Tier 0 mirrors.
Tier 1 mirrors pull from one of the master mirrors.


{| border="1"
{| border="1"
|-
|-
| Server || Comment || Contact for ACL
! Server || Organization || Location || Network || Modules || Comment || Contact for ACL
|-
|-
| mirrors.kernel.org || USx2, SE, NL || <ftpadmin at kernel.org>
| archive.linux.duke.edu || Duke University || US East Coast || IPv4, Internet2 || fedora-enchilada and fedora-epel || uses ACL from [https://admin.fedoraproject.org/mirrormanager MirrorManager database] || Drew Stinnett <drew.stinnett at duke.edu> (spacepope on IRC)
|-
|-
| wpi.edu || IPv6-connected or Internet2-connected mirrors only || Chuck Anderson <cra at wpi.edu>
| mirrors.kernel.org || Linux Kernel Organization || US West Coast || IPv4 and IPv6 || fedora-buffet, fedora-enchilada, fedora-epel, fedora-secondary, and fedora-alt || || ftpadmin at kernel.org
|-
|-
| rsync.hrz.tu-chemnitz.de || rsync.hrz.tu-chemnitz.de::fedora-enchilada/.  Uses ACL from [https://admin.fedoraproject.org/mirrormanager MirrorManager database] . || guenther.fischer at hrz.tu-chemnitz.de
| rsync.hrz.tu-chemnitz.de || Technische Universität Chemnitz || Chemnitz, Germany || IPv4 || fedora-enchilada and fedora-epel || uses ACL from [https://admin.fedoraproject.org/mirrormanager MirrorManager database] || support at hrz.tu-chemnitz.de
|-
|-
| fedora-rsync.ftp.pub.2iij.net || rsync://fedora-rsync.ftp.pub.2iij.net/fedora-enchilada || mirror-contact at iij.ad.jp
| ftp-stud.hs-esslingen.de || Hochschule Esslingen || Esslingen, Germany || IPv4 and IPv6 || fedora-buffet, fedora-enchilada, and fedora-epel || || Adrian Reber <adrian at hs-esslingen.de>
|-
|-
| sunsite.mff.cuni.cz ||
| fedora-rsync.ftp.pub.2iij.net || Internet Initiative Japan || Tokyo, Japan || IPv4 || fedora-enchilada and fedora-epel || || mirror-contact at iij.ad.jp
|-
|-
| ftp.heanet.ie || IPv6 and Internet2 connectivity. ftp.heanet.ie::fedora-enchilada, ftp.heanet.ie::fedora-epel || mirrors at heanet.ie ||
| mirror.twds.com.tw || Taiwan Digital Streaming Co. || Taipei, Taiwan || IPv4 and IPv6 || fedora-buffet0 || || mirror at twds.tw
|-
|-
| mirror.speedpartner.de || IPv4 and IPv6 || mirror at speedpartner.de
| fedora.c3sl.ufpr.br || Universidade Federal do Paraná || Curitiba, Brasil (South America) || IPv4 and IPv6 || fedora and fedora-alt || || Carlos Carvalho <carlos at fisica.ufpr.br>
|-
|-
| fedora.c3sl.ufpr.br || South America || Carlos Carvalho carlos at fisica.ufpr.br
| ftp.linux.cz || CZLUG || Brno, Czech Republic || IPv4 and IPv6 || || || ftp-admin at fi.muni.cz
|-
| mirror.gtlib.gatech.edu || Georgia Tech || US East Coast || IPv4 and IPv6 || fedora-enchilada and fedora-epel || || Neil Bright <neil.bright at oit.gatech.edu>
|-
| mirrors.rit.edu || Rochester Institute of Technology || US East Coast || IPv4 and IPv6 || fedora-buffet, fedora-enchilada, and fedora-epel || || mirrors at rit.edu
|-
| mirror.liquidtelecom.com || Liquid Telecom || East Africa Datacenter, Nairobi, Kenya || IPv4 and IPv6 || fedora-buffet, fedora-enchilada, and fedora-epel || || anthony.somerset at liquidtelecom.com
|-
| fr2.rpmfind.net || RpmFind || Lyon, France || IPv4 || fedora-enchilada, fedora-secondary and fedora-epel || || fabrice at bellet.info
|-
| download-ib01.fedoraproject.org || Fedora || North Carolina, USA || IPv4 and IPv6 || fedora-buffet0 || Uses acls from master mirrors || admin at fedoraproject.org
|-
| download-cc-rdu01.fedoraproject.org || Fedora || North Carolina, USA || IPv4 and IPv6 || fedora-buffet0 || Uses acls from master mirrors || admin at fedoraproject.org
|}
|}


== Master Mirrors ==
== Tier 1 rsync configuration ==
* download-i2.fedora.redhat.com in Raleigh, NC, USA (Internet2 users only, but requires static routes to use.  Contact < mirror-admin at fedoraproject dot org> before using this mirror to ensure proper routes are in place).
 
* download1.fedora.redhat.com is offline, no ETA on its return. DNS for it points at download3.
Below is an example rsyncd.conf file for a Tier 1 mirror that provides private rsync access to select downstream Tier 2 mirrors.  You may do this via either IP or DNS-based access control, or by a shared username/password which you give to your selected Tier 2 mirrors directly.
* download2.fedora.redhat.com is offline, no ETA on its return. No DNS.
 
* download3.fedora.redhat.com in Phoenix, AZ, USA
The key to this is that the Tier 1 mirror rsyncs content using a user account (e.g. ''mirror'' used below), and you serve content to Tier 2 mirrors using a private rsync module that runs as that same user account, while providing public non-authenticated rsync using the ''nobody'' account.  In this way, Tier 2 mirrors may obtain content before the permissions are made world readable.
* download4.fedora.redhat.com in Phoenix, AZ, USA
 
* download5.fedora.redhat.com in Phoenix, AZ, USA
<pre>
uid = nobody
gid = nobody
use chroot = yes
dont compress = *.gz *.tgz *.zip *.z *.rpm *.deb *.bz2 *.iso *.ogg *.ogv *.tbz
exclude = .snapshot/ .~tmp~/ /.private/ /.private/** **/.nfs*
ignore nonreadable = yes
list = true
read only = yes
refuse options = checksum
 
[ fedora-buffet ]
        comment = Fedora -- the whole buffet (all you can eat)
        path = /srv/pub
 
[ fedora-enchilada ]
        comment = Fedora -- the whole enchilada
        path = /srv/pub/fedora
 
[ fedora-epel ]
        comment = Extra Packages for Enterprise Linux
        path = /srv/pub/epel
 
##
## The following are not seen and are limited by IP.
##
 
[fedora-buffet0]
      comment = Fedora Buffet for Tier0|1 Mirrors
      path = /srv/pub/
      list = no
      uid = mirror
      gid = mirror
      hosts allow = (IP or DNS address) ...
 
[fedora-enchilada0]
      comment = Fedora Enchilada for Tier0|1 Mirrors
      path = /srv/pub/fedora/
      list = no
      uid = mirror
      gid = mirror
      hosts allow = (IP or DNS address) ...
 
[fedora-epel0]
      comment = Fedora EPEL for Tier0|1 Mirrors
      path = /srv/pub/epel/
      list = no
      uid = mirror
      gid = mirror
      hosts allow = (IP or DNS address) ...
</pre>
 
== Tier 2 mirrors ==
 
The number of mirrors is too large to list them here; you can find them in the [https://mirrors.fedoraproject.org/ MirrorManager].


[[Category:Infrastructure]]
[[Category:Infrastructure]]

Latest revision as of 03:22, 20 August 2024

Tiering

Fedora mirror servers use Tiering, whereby a select few fast mirrors get read access to the master rsync servers, and all the other mirrors pull from those mirrors.

For our purposes, define:

  • master: The Fedora-owned servers dl.fedoraproject.org
  • Tier 1: The fast mirrors which pull from a master mirror.
  • Tier 2: The mirrors that pull from the Tier 1 servers.

Properties of Tier 1 mirrors:

  • Limit the number of Tier 1 mirrors, to ensure adequate bandwidth for these. Adjust number up or down depending on capability of the masters.
  • Must carry everything under fedora-enchilada and fedora-epel. This allows Tier 2 mirrors to exclude what they wish, but get everything if they so wish. This means at least 1TB of disk space for the Fedora portion of this server.
  • Must have a 1 Gigabit connection to the Internet, or faster.
  • Must have an active, available, responsive mirror administrator during the days content is staged.
  • Must have at least 2 Internet2-connected Tier 1 mirrors.
  • Must have at least 1 Tier 1 mirror on each continent for which we have Tier 2 mirrors.
  • Must serve private rsync (see below for configuration).

Master mirrors

  • dl0[123].fedoraproject.org, in Ashburn, VA, USA.
  • dl0[45].fedoraproject.org, in Ashburn, VA, USA - tier1 mirrors only.
    • dl.fedoraproject.org is a DNS round-robin to dl0[123].
    • dl-tier1.fedoraproject.org is a DNS round-robin for dl0[45].

Master mirror rsync modules

The master mirrors provide two additional rsync modules which provide pre-bitflip content. Fedora tiered mirrors should use these modules to be able to get pre-bitflip content.

Module name Content
fedora-buffet0 Everything under /pub/, including pre-bitflip content
fedora-enchilada0 Everything under /pub/fedora/, including pre-bitflip content
fedora-epel0 Everything under /pub/epel/, including pre-bitflip content (even though EPEL doesn't do bitflips)

Tier 1 mirrors

Tier 1 mirrors pull from one of the master mirrors.

Server Organization Location Network Modules Comment Contact for ACL
archive.linux.duke.edu Duke University US East Coast IPv4, Internet2 fedora-enchilada and fedora-epel uses ACL from MirrorManager database Drew Stinnett <drew.stinnett at duke.edu> (spacepope on IRC)
mirrors.kernel.org Linux Kernel Organization US West Coast IPv4 and IPv6 fedora-buffet, fedora-enchilada, fedora-epel, fedora-secondary, and fedora-alt ftpadmin at kernel.org
rsync.hrz.tu-chemnitz.de Technische Universität Chemnitz Chemnitz, Germany IPv4 fedora-enchilada and fedora-epel uses ACL from MirrorManager database support at hrz.tu-chemnitz.de
ftp-stud.hs-esslingen.de Hochschule Esslingen Esslingen, Germany IPv4 and IPv6 fedora-buffet, fedora-enchilada, and fedora-epel Adrian Reber <adrian at hs-esslingen.de>
fedora-rsync.ftp.pub.2iij.net Internet Initiative Japan Tokyo, Japan IPv4 fedora-enchilada and fedora-epel mirror-contact at iij.ad.jp
mirror.twds.com.tw Taiwan Digital Streaming Co. Taipei, Taiwan IPv4 and IPv6 fedora-buffet0 mirror at twds.tw
fedora.c3sl.ufpr.br Universidade Federal do Paraná Curitiba, Brasil (South America) IPv4 and IPv6 fedora and fedora-alt Carlos Carvalho <carlos at fisica.ufpr.br>
ftp.linux.cz CZLUG Brno, Czech Republic IPv4 and IPv6 ftp-admin at fi.muni.cz
mirror.gtlib.gatech.edu Georgia Tech US East Coast IPv4 and IPv6 fedora-enchilada and fedora-epel Neil Bright <neil.bright at oit.gatech.edu>
mirrors.rit.edu Rochester Institute of Technology US East Coast IPv4 and IPv6 fedora-buffet, fedora-enchilada, and fedora-epel mirrors at rit.edu
mirror.liquidtelecom.com Liquid Telecom East Africa Datacenter, Nairobi, Kenya IPv4 and IPv6 fedora-buffet, fedora-enchilada, and fedora-epel anthony.somerset at liquidtelecom.com
fr2.rpmfind.net RpmFind Lyon, France IPv4 fedora-enchilada, fedora-secondary and fedora-epel fabrice at bellet.info
download-ib01.fedoraproject.org Fedora North Carolina, USA IPv4 and IPv6 fedora-buffet0 Uses acls from master mirrors admin at fedoraproject.org
download-cc-rdu01.fedoraproject.org Fedora North Carolina, USA IPv4 and IPv6 fedora-buffet0 Uses acls from master mirrors admin at fedoraproject.org

Tier 1 rsync configuration

Below is an example rsyncd.conf file for a Tier 1 mirror that provides private rsync access to select downstream Tier 2 mirrors. You may do this via either IP or DNS-based access control, or by a shared username/password which you give to your selected Tier 2 mirrors directly.

The key to this is that the Tier 1 mirror rsyncs content using a user account (e.g. mirror used below), and you serve content to Tier 2 mirrors using a private rsync module that runs as that same user account, while providing public non-authenticated rsync using the nobody account. In this way, Tier 2 mirrors may obtain content before the permissions are made world readable. 

uid = nobody
gid = nobody
use chroot = yes
dont compress = *.gz *.tgz *.zip *.z *.rpm *.deb *.bz2 *.iso *.ogg *.ogv *.tbz
exclude = .snapshot/ .~tmp~/ /.private/ /.private/** **/.nfs*
ignore nonreadable = yes
list = true
read only = yes
refuse options = checksum

[ fedora-buffet ]
        comment = Fedora -- the whole buffet (all you can eat)
        path = /srv/pub

[ fedora-enchilada ]
        comment = Fedora -- the whole enchilada
        path = /srv/pub/fedora

[ fedora-epel ]
        comment = Extra Packages for Enterprise Linux
        path = /srv/pub/epel

##
## The following are not seen and are limited by IP.
##

[fedora-buffet0]
       comment = Fedora Buffet for Tier0|1 Mirrors
       path = /srv/pub/
       list = no
       uid = mirror
       gid = mirror
       hosts allow = (IP or DNS address) ...

[fedora-enchilada0]
       comment = Fedora Enchilada for Tier0|1 Mirrors
       path = /srv/pub/fedora/
       list = no
       uid = mirror
       gid = mirror
       hosts allow = (IP or DNS address) ...

[fedora-epel0]
       comment = Fedora EPEL for Tier0|1 Mirrors
       path = /srv/pub/epel/
       list = no
       uid = mirror
       gid = mirror
       hosts allow = (IP or DNS address) ...

Tier 2 mirrors

The number of mirrors is too large to list them here; you can find them in the MirrorManager.

Retrieved from "https://fedoraproject.org/w/index.php?title=Infrastructure/Mirroring/Tiering&oldid=717699"