16.9. Building a Complete Debian Mirror with apt-mirror

Problem

You want to provision your Debian hosts over the network from a local server, and you want a complete local mirror for this. How do you do this?

Solution

You need a Debian-based HTTP server to do this. There are two Debian applications for creating a local mirror: apt-mirror, which creates a complete mirror, and apt-proxy, which creates a partial mirror. In this recipe, we'll make a complete mirror with apt-mirror.

You will need anywhere from 40–120 GB of storage, according to which releases you want and how many CPU architectures. First, install apt-mirror in the usual Debian manner, plus Lighttpd:

	# aptitude install apt-mirror lighttpd

Then, edit /etc/apt/mirror.list to include your chosen repositories. You want to use Debian mirrors that are close to you, which you will find at the mirror list page (http://www.debian.org/mirror/list). They may not be geographically close, but you should run some ping and tcptraceroute tests to get an idea of which ones perform well. This example uses the mirror at http://linux.csua.berkeley.edu/debian:

	## /etc/apt/mirror.list
	############# config ##################
	# 
	# set base_path    /var/spool/apt-mirror
	# set mirror_path  $base_path/mirror
	# set skel_path    $base_path/skel
	# set var_path     $base_path/var
	# set cleanscript $var_path/clean.sh
	# set defaultarch
	# set nthreads 20
	set tilde 0
	#
	############# end config ##############

	# debian Etch (stable)
	deb http://linux.csua.berkeley.edu/debian etch main contrib non-free
	deb-src http://linux.csua.berkeley.edu/debian etch main contrib non-free
	deb http://linux.csua.berkeley.edu/debian etch main/debian-installer

	#debian Lenny (testing)
	deb http://linux.csua.berkeley.edu/debian lenny main contrib non-free
	deb-src http://linux.csua.berkeley.edu/debian lenny main contrib non-free
	deb http://linux.csua.berkeley.edu/debian lenny main/debian-installer

	#debian Sid (unstable)
	deb http://linux.csua.berkeley.edu/debian sid main contrib non-free
	deb-src http://linux.csua.berkeley.edu/debian sid main contrib non-free
	deb http://linux.csua.berkeley.edu/debian sid main/debian-installer

Now, run this command to start downloading files:

	# apt-mirror /etc/apt/mirror.list 
	Downloading 66 index files using 20 threads...
	Begin time: Sun Jul 22 22:43:46 2007
	[20]... [19]... [18]... [17]... [16]... [15]... 14]... [13]... [12]... [11]... [10]..
	. [9]... [8]... [7]... [6]... [5]... [4]... [3]... [2]... [1]... [0]...
	End time: Sun Jul 15 22:57:52 2007

	Proceed indexes: [SSSSPPPPPPP]

	52.7 GiB will be downloaded into archive.
	Downloading 81257 archive files using 20 threads...
	Begin time: Sun Jul 15 22:58:37 2007

Packages download into /var/spool/apt-mirror/mirror/, and this is obviously going to take some time, so you might as well find something else to do. Like configure Lighttpd. First, create a directory in your HTTP root:

	# mkdir /var/www/debian

Then, link your package mirror to this directory:

	# ln -s /var/spool/apt-mirror/mirror/linux.csua.berkeley.edu/debian \
	/var/www/debian

You could also set up a cron job to update the server every night. apt-mirror installs /etc/cron.d/apt-mirror, so all you have to do is uncomment the command line:

	0 1   * * *   apt-mirror   /usr/bin/apt-mirror > /var/spool/apt-mirror/var/cron.
	log

This runs it every day at 1 a.m.

Discussion

The nice thing about having your own Debian mirror is that it's always current. Once the initial download is completed, subsequent downloads will be small.

It doesn't hurt to run apt-mirror /etc/apt/mirror.list a few times after the initial download is completed, just to be thorough.

Security updates are not cached on the server in this recipe, though you could do this if you wanted to. Some admins prefer to configure each client to download them directly to ensure they get fresh security updates.

You could also run apt-mirror on a different Linux distribution, such as Fedora or Slackware, or whatever you like. Download and install it from the source tarball, and then you'll have to create the directory structure and configure cron yourself.

See Also