From Fedora Project Wiki
m (Cleanup.)
No edit summary
 
(55 intermediate revisions by 14 users not shown)
Line 4: Line 4:


Fedora mirror servers use Tiering, whereby a select few fast mirrors get read access to the master rsync servers, and all the other mirrors pull from those mirrors.
Fedora mirror servers use Tiering, whereby a select few fast mirrors get read access to the master rsync servers, and all the other mirrors pull from those mirrors.
It turns out, 9 of our 10 Tier 0 or Tier 1 mirrors are available over Internet2.  And, over half of our total mirrors are reachable over Internet2.  So, let's make use of that whereever we can.


For our purposes, define:
For our purposes, define:
* '''master''': The Red Hat servers download*.fedora.redhat.com.
* '''master''': The Fedora-owned servers dl.fedoraproject.org
* '''Tier 0''': The fast mirrors which pull from Red Hat's Internet2-connected master
* '''Tier 1''': The fast mirrors which pull from a master mirror.
* '''Tier 1''': The fast mirrors which pull from the Tier 0 servers (or one of the other masters).
* '''Tier 2''': The mirrors that pull from the Tier 1 servers.
* '''Tier 2''': The mirrors that pull from the Tier 1 servers.


Properties of Tier 0 and 1 mirrors:
Properties of Tier 1 mirrors:


* Limit the number of Tier 1 mirrors to 10, to ensure adequate bandwidth for these.  Adjust number up or down depending on capability of the masters.
* Limit the number of Tier 1 mirrors, to ensure adequate bandwidth for these.  Adjust number up or down depending on capability of the masters.
* Must carry everything under fedora-enchilada and fedora-epel.  This allows Tier 2 mirrors to exclude what they wish, but get everything if they so wish.  This means at least 250GB of disk space for the Fedora portion of this server.  If they can carry everything under fedora-enchilada, even better.
* Must carry everything under fedora-enchilada and fedora-epel.  This allows Tier 2 mirrors to exclude what they wish, but get everything if they so wish.  This means at least 1TB of disk space for the Fedora portion of this server.
* Must have a 1 Gigabit connection to the Internet, or faster.
* Must have a 1 Gigabit connection to the Internet, or faster.
* Must have an active, available, responsive mirror administrator during the days content is staged.
* Must have an active, available, responsive mirror administrator during the days content is staged.
* Must have at least 2 Internet2-connected Tier 1 mirrors.
* Must have at least 2 Internet2-connected Tier 1 mirrors.
* Must have at least 1 Tier 1 mirror on each continent for which we have Tier 2 mirrors
* Must have at least 1 Tier 1 mirror on each continent for which we have Tier 2 mirrors.
* Must serve private rsync
* Must serve private rsync (see below for configuration).
 
== Master mirrors ==
 
* dl0[123].fedoraproject.org, in Ashburn, VA, USA.
* dl0[45].fedoraproject.org, in Ashburn, VA, USA - tier1 mirrors only.
** dl.fedoraproject.org is a DNS round-robin to dl0[123].
** dl-tier1.fedoraproject.org is a DNS round-robin for dl0[45].


== Tier 0 Mirrors ==
== Master mirror rsync modules ==


Tier 0 mirrors can pull from Red Hat directly over the Internet2 connection.
The master mirrors provide two additional rsync modules which provide pre-bitflip content.  Fedora tiered mirrors should use these modules to be able to get pre-bitflip content.


{| border="1"
{| border="1"
|-
|-
| Server || Comment || Contact for ACL
! Module name || Content
|-
| fedora-buffet0 || Everything under /pub/, including pre-bitflip content
|-
|-
| fedora-archives.ibiblio.org || Internet2 / National Lamba Rail (NLR) connected hosts. || Don Sizemore <dls at metalab.unc.edu>
| fedora-enchilada0 || Everything under /pub/fedora/, including pre-bitflip content
|-
|-
| archive.linux.duke.edu ||Internet2.  Uses ACL from MirrorManager database. || Kambiz Aghaiepour <kambiz.aghaiepour at duke.edu>
| fedora-epel0 || Everything under /pub/epel/, including pre-bitflip content (even though EPEL doesn't do bitflips)
|}
|}


== Tier 1 Mirrors ==
== Tier 1 mirrors ==


Tier 1 mirrors pull from one of the Tier 0 mirrors.
Tier 1 mirrors pull from one of the master mirrors.


{| border="1"
{| border="1"
|-
|-
| Server || Comment || Contact for ACL
! Server || Organization || Location || Network || Modules || Comment || Contact for ACL
|-
|-
| mirrors.kernel.org || USx2, SE, NL || <ftpadmin at kernel.org>
| archive.linux.duke.edu || Duke University || US East Coast || IPv4, Internet2 || fedora-enchilada and fedora-epel || uses ACL from [https://admin.fedoraproject.org/mirrormanager MirrorManager database] || Drew Stinnett <drew.stinnett at duke.edu> (spacepope on IRC)
|-
|-
| wpi.edu || IPv6-connected or Internet2-connected mirrors only || Chuck Anderson <cra at wpi.edu>
| mirrors.kernel.org || Linux Kernel Organization || US West Coast || IPv4 and IPv6 || fedora-buffet, fedora-enchilada, fedora-epel, fedora-secondary, and fedora-alt || || ftpadmin at kernel.org
|-
|-
| rsync.hrz.tu-chemnitz.de || rsync.hrz.tu-chemnitz.de::fedora-enchilada/.  Uses ACL from [https://admin.fedoraproject.org/mirrormanager MirrorManager database] . || guenther.fischer at hrz.tu-chemnitz.de
| rsync.hrz.tu-chemnitz.de || Technische Universität Chemnitz || Chemnitz, Germany || IPv4 || fedora-enchilada and fedora-epel || uses ACL from [https://admin.fedoraproject.org/mirrormanager MirrorManager database] || support at hrz.tu-chemnitz.de
|-
|-
| fedora-rsync.ftp.pub.2iij.net || rsync://fedora-rsync.ftp.pub.2iij.net/fedora-enchilada || mirror-contact at iij.ad.jp
| ftp-stud.hs-esslingen.de || Hochschule Esslingen || Esslingen, Germany || IPv4 and IPv6 || fedora-buffet, fedora-enchilada, and fedora-epel || || Adrian Reber <adrian at hs-esslingen.de>
|-
|-
| sunsite.mff.cuni.cz ||
| fedora-rsync.ftp.pub.2iij.net || Internet Initiative Japan || Tokyo, Japan || IPv4 || fedora-enchilada and fedora-epel || || mirror-contact at iij.ad.jp
|-
|-
| ftp.heanet.ie ||
| mirror.twds.com.tw || Taiwan Digital Streaming Co. || Taipei, Taiwan || IPv4 and IPv6 || fedora-buffet0 || || mirror at twds.tw
|-
|-
| mirror.speedpartner.de || IPv4 and IPv6 || mirror at speedpartner.de
| fedora.c3sl.ufpr.br || Universidade Federal do Paraná || Curitiba, Brasil (South America) || IPv4 and IPv6 || fedora and fedora-alt || || Carlos Carvalho <carlos at fisica.ufpr.br>
|-
|-
| fedora.c3sl.ufpr.br || South America || Carlos Carvalho carlos at fisica.ufpr.br
| ftp.linux.cz || CZLUG || Brno, Czech Republic || IPv4 and IPv6 || || || ftp-admin at fi.muni.cz
|-
| mirror.gtlib.gatech.edu || Georgia Tech || US East Coast || IPv4 and IPv6 || fedora-enchilada and fedora-epel || || Neil Bright <neil.bright at oit.gatech.edu>
|-
| mirrors.rit.edu || Rochester Institute of Technology || US East Coast || IPv4 and IPv6 || fedora-buffet, fedora-enchilada, and fedora-epel || || mirrors at rit.edu
|-
| mirror.liquidtelecom.com || Liquid Telecom || East Africa Datacenter, Nairobi, Kenya || IPv4 and IPv6 || fedora-buffet, fedora-enchilada, and fedora-epel || || anthony.somerset at liquidtelecom.com
|-
| fr2.rpmfind.net || RpmFind || Lyon, France || IPv4 || fedora-enchilada, fedora-secondary and fedora-epel || || fabrice at bellet.info
|-
| download-ib01.fedoraproject.org || Fedora || North Carolina, USA || IPv4 and IPv6 || fedora-buffet0 || Uses acls from master mirrors || admin at fedoraproject.org
|-
| download-cc-rdu01.fedoraproject.org || Fedora || North Carolina, USA || IPv4 and IPv6 || fedora-buffet0 || Uses acls from master mirrors || admin at fedoraproject.org
|}
|}


== Master Mirrors ==
== Tier 1 rsync configuration ==
* download1.fedora.redhat.com in Raleigh, NC, USA (on Internet2, but requires static routes to use, so for Tier 0 mirrors only.)
 
* download2.fedora.redhat.com in Tampa Bay, FL, USA
Below is an example rsyncd.conf file for a Tier 1 mirror that provides private rsync access to select downstream Tier 2 mirrors. You may do this via either IP or DNS-based access control, or by a shared username/password which you give to your selected Tier 2 mirrors directly.
* download3.fedora.redhat.com in Phoenix, AZ, USA
 
The key to this is that the Tier 1 mirror rsyncs content using a user account (e.g. ''mirror'' used below), and you serve content to Tier 2 mirrors using a private rsync module that runs as that same user account, while providing public non-authenticated rsync using the ''nobody'' account.  In this way, Tier 2 mirrors may obtain content before the permissions are made world readable.
 
<pre>
uid = nobody
gid = nobody
use chroot = yes
dont compress = *.gz *.tgz *.zip *.z *.rpm *.deb *.bz2 *.iso *.ogg *.ogv *.tbz
exclude = .snapshot/ .~tmp~/ /.private/ /.private/** **/.nfs*
ignore nonreadable = yes
list = true
read only = yes
refuse options = checksum
 
[ fedora-buffet ]
        comment = Fedora -- the whole buffet (all you can eat)
        path = /srv/pub
 
[ fedora-enchilada ]
        comment = Fedora -- the whole enchilada
        path = /srv/pub/fedora
 
[ fedora-epel ]
        comment = Extra Packages for Enterprise Linux
        path = /srv/pub/epel
 
##
## The following are not seen and are limited by IP.
##
 
[fedora-buffet0]
      comment = Fedora Buffet for Tier0|1 Mirrors
      path = /srv/pub/
      list = no
      uid = mirror
      gid = mirror
      hosts allow = (IP or DNS address) ...
 
[fedora-enchilada0]
      comment = Fedora Enchilada for Tier0|1 Mirrors
      path = /srv/pub/fedora/
      list = no
      uid = mirror
      gid = mirror
      hosts allow = (IP or DNS address) ...
 
[fedora-epel0]
      comment = Fedora EPEL for Tier0|1 Mirrors
      path = /srv/pub/epel/
      list = no
      uid = mirror
      gid = mirror
      hosts allow = (IP or DNS address) ...
</pre>
 
== Tier 2 mirrors ==
 
The number of mirrors is too large to list them here; you can find them in the [https://mirrors.fedoraproject.org/ MirrorManager].


[[Category:Infrastructure]]
[[Category:Infrastructure]]

Latest revision as of 03:22, 20 August 2024

Tiering

Fedora mirror servers use Tiering, whereby a select few fast mirrors get read access to the master rsync servers, and all the other mirrors pull from those mirrors.

For our purposes, define:

  • master: The Fedora-owned servers dl.fedoraproject.org
  • Tier 1: The fast mirrors which pull from a master mirror.
  • Tier 2: The mirrors that pull from the Tier 1 servers.

Properties of Tier 1 mirrors:

  • Limit the number of Tier 1 mirrors, to ensure adequate bandwidth for these. Adjust number up or down depending on capability of the masters.
  • Must carry everything under fedora-enchilada and fedora-epel. This allows Tier 2 mirrors to exclude what they wish, but get everything if they so wish. This means at least 1TB of disk space for the Fedora portion of this server.
  • Must have a 1 Gigabit connection to the Internet, or faster.
  • Must have an active, available, responsive mirror administrator during the days content is staged.
  • Must have at least 2 Internet2-connected Tier 1 mirrors.
  • Must have at least 1 Tier 1 mirror on each continent for which we have Tier 2 mirrors.
  • Must serve private rsync (see below for configuration).

Master mirrors

  • dl0[123].fedoraproject.org, in Ashburn, VA, USA.
  • dl0[45].fedoraproject.org, in Ashburn, VA, USA - tier1 mirrors only.
    • dl.fedoraproject.org is a DNS round-robin to dl0[123].
    • dl-tier1.fedoraproject.org is a DNS round-robin for dl0[45].

Master mirror rsync modules

The master mirrors provide two additional rsync modules which provide pre-bitflip content. Fedora tiered mirrors should use these modules to be able to get pre-bitflip content.

Module name Content
fedora-buffet0 Everything under /pub/, including pre-bitflip content
fedora-enchilada0 Everything under /pub/fedora/, including pre-bitflip content
fedora-epel0 Everything under /pub/epel/, including pre-bitflip content (even though EPEL doesn't do bitflips)

Tier 1 mirrors

Tier 1 mirrors pull from one of the master mirrors.

Server Organization Location Network Modules Comment Contact for ACL
archive.linux.duke.edu Duke University US East Coast IPv4, Internet2 fedora-enchilada and fedora-epel uses ACL from MirrorManager database Drew Stinnett <drew.stinnett at duke.edu> (spacepope on IRC)
mirrors.kernel.org Linux Kernel Organization US West Coast IPv4 and IPv6 fedora-buffet, fedora-enchilada, fedora-epel, fedora-secondary, and fedora-alt ftpadmin at kernel.org
rsync.hrz.tu-chemnitz.de Technische Universität Chemnitz Chemnitz, Germany IPv4 fedora-enchilada and fedora-epel uses ACL from MirrorManager database support at hrz.tu-chemnitz.de
ftp-stud.hs-esslingen.de Hochschule Esslingen Esslingen, Germany IPv4 and IPv6 fedora-buffet, fedora-enchilada, and fedora-epel Adrian Reber <adrian at hs-esslingen.de>
fedora-rsync.ftp.pub.2iij.net Internet Initiative Japan Tokyo, Japan IPv4 fedora-enchilada and fedora-epel mirror-contact at iij.ad.jp
mirror.twds.com.tw Taiwan Digital Streaming Co. Taipei, Taiwan IPv4 and IPv6 fedora-buffet0 mirror at twds.tw
fedora.c3sl.ufpr.br Universidade Federal do Paraná Curitiba, Brasil (South America) IPv4 and IPv6 fedora and fedora-alt Carlos Carvalho <carlos at fisica.ufpr.br>
ftp.linux.cz CZLUG Brno, Czech Republic IPv4 and IPv6 ftp-admin at fi.muni.cz
mirror.gtlib.gatech.edu Georgia Tech US East Coast IPv4 and IPv6 fedora-enchilada and fedora-epel Neil Bright <neil.bright at oit.gatech.edu>
mirrors.rit.edu Rochester Institute of Technology US East Coast IPv4 and IPv6 fedora-buffet, fedora-enchilada, and fedora-epel mirrors at rit.edu
mirror.liquidtelecom.com Liquid Telecom East Africa Datacenter, Nairobi, Kenya IPv4 and IPv6 fedora-buffet, fedora-enchilada, and fedora-epel anthony.somerset at liquidtelecom.com
fr2.rpmfind.net RpmFind Lyon, France IPv4 fedora-enchilada, fedora-secondary and fedora-epel fabrice at bellet.info
download-ib01.fedoraproject.org Fedora North Carolina, USA IPv4 and IPv6 fedora-buffet0 Uses acls from master mirrors admin at fedoraproject.org
download-cc-rdu01.fedoraproject.org Fedora North Carolina, USA IPv4 and IPv6 fedora-buffet0 Uses acls from master mirrors admin at fedoraproject.org

Tier 1 rsync configuration

Below is an example rsyncd.conf file for a Tier 1 mirror that provides private rsync access to select downstream Tier 2 mirrors. You may do this via either IP or DNS-based access control, or by a shared username/password which you give to your selected Tier 2 mirrors directly.

The key to this is that the Tier 1 mirror rsyncs content using a user account (e.g. mirror used below), and you serve content to Tier 2 mirrors using a private rsync module that runs as that same user account, while providing public non-authenticated rsync using the nobody account. In this way, Tier 2 mirrors may obtain content before the permissions are made world readable. 

uid = nobody
gid = nobody
use chroot = yes
dont compress = *.gz *.tgz *.zip *.z *.rpm *.deb *.bz2 *.iso *.ogg *.ogv *.tbz
exclude = .snapshot/ .~tmp~/ /.private/ /.private/** **/.nfs*
ignore nonreadable = yes
list = true
read only = yes
refuse options = checksum

[ fedora-buffet ]
        comment = Fedora -- the whole buffet (all you can eat)
        path = /srv/pub

[ fedora-enchilada ]
        comment = Fedora -- the whole enchilada
        path = /srv/pub/fedora

[ fedora-epel ]
        comment = Extra Packages for Enterprise Linux
        path = /srv/pub/epel

##
## The following are not seen and are limited by IP.
##

[fedora-buffet0]
       comment = Fedora Buffet for Tier0|1 Mirrors
       path = /srv/pub/
       list = no
       uid = mirror
       gid = mirror
       hosts allow = (IP or DNS address) ...

[fedora-enchilada0]
       comment = Fedora Enchilada for Tier0|1 Mirrors
       path = /srv/pub/fedora/
       list = no
       uid = mirror
       gid = mirror
       hosts allow = (IP or DNS address) ...

[fedora-epel0]
       comment = Fedora EPEL for Tier0|1 Mirrors
       path = /srv/pub/epel/
       list = no
       uid = mirror
       gid = mirror
       hosts allow = (IP or DNS address) ...

Tier 2 mirrors

The number of mirrors is too large to list them here; you can find them in the MirrorManager.

Retrieved from "https://fedoraproject.org/w/index.php?title=Infrastructure/Mirroring/Tiering&oldid=717699"