Mirror

Requirement

rsync

Usualy i use rsync for create mirror, it s easy and fast. you can exlude and delete old entries easily too.
If you are behind a proyx tou may also use this env RSYNC_PROXY='<host>:<port>' and rsync will use it.

apache

You have to serve your mirror with a web server, i use apache, but you may use nginx or lighttpd, in fact nginx and lighttpd is better for serving static file, so it s better to use it, but this server is not in front of internet so it s for internal use only, performances are not a problem. I shoudn't have to change my web setup just for 1 server.

System

For Redhat like system (Centos, Scientific linux,...) you mays use any distrib, but if you want a debian mirror, it's a pain in the ass with a distrib other than debian like (because of apt-mirror), so I use a debian system for all my mirror infrastructure.

File system

Recently i use a btrfs file system, because i can extend it easily, defrag, make a snapshot before backuping, and it works well.

Before i use btrfs i use xfs, because ext[34] may have trouble with many file on it (inode resctrictions) and reizerfs is,....sorry,

For Redhat Like

Scientific Linux

LIST_VERSION="6 6.0 6.1 6.2"

for version in $LIST_VERSION; do

  rsync -avkSH --delete --exclude=archive/debuginfo --exclude=archive/obsolete --exclude=SRPMS --exclude=i386 --exclude=x86_64/iso rsync://rsync.scientificlinux.org/scientific/$version/ $version/

done

adjust LIST_VERSION to suit you, you can ot not exclude i386 system (i don t include it, who use 32 bits system ?)

set or not RSYNC_PROXY

Centos

Shit, mirror for Centos is only for the last version.

LIST_VERSION="5 6"

for version in $LIST_VERSION; do

  rsync  -avSHP --delete --exclude="local*" --exclude="*/i386" --exclude="iso" rsync://ftp.belnet.be/centos/$version/ $version/

done

Debian/Ubuntu

I use apt-mirror for this

create a config file and use it.

apt-mirror.conf:

############# config ##################

set base_path           /data/mirror/ubuntu

set mirror_path         $base_path/mirror

set skel_path           $base_path/skel

set var_path            $base_path/var

set cleanscript         $var_path/clean.sh

set defaultarch         amd64

set run_postmirror      0

set nthreads            10

set _tilde              0

### distrib - precise - ###

deb http://cdn.archive.ubuntu.com/ubuntu precise main/debian-installer

# Repository

deb http://cdn.archive.ubuntu.com/ubuntu precise main restricted universe multiverse

deb http://cdn.archive.ubuntu.com/ubuntu precise-security main restricted universe multiverse

deb http://cdn.archive.ubuntu.com/ubuntu precise-updates main restricted universe multiverse

deb-i386 http://cdn.archive.ubuntu.com/ubuntu precise main restricted universe multiverse

deb-i386 http://cdn.archive.ubuntu.com/ubuntu precise-security main restricted universe multiverse

deb-i386 http://cdn.archive.ubuntu.com/ubuntu precise-updates main restricted universe multiverse

### distrib - squeeze - ###

deb http://ftp.fr.debian.org/debian squeeze main/debian-installer

# Repository

deb http://ftp.fr.debian.org/debian squeeze main contrib non-free

deb http://ftp.fr.debian.org/debian-security squeeze/updates main contrib non-free

### cleaning

clean http://cdn.archive.ubuntu.com/ubuntu

clean http://ftp.fr.debian.org/debian

And after seeting it you can start to mirror it.

apt-miror apt-mirror.conf

Same as before if you are behin a proxy you may use this env http_proxy='<host>:<port>' or set it in apt.conf

echo 'Acquire::http::Proxy "http://hal.paris.exalead.com:8118";' >/etc/apt/apt.conf.d/99proxy