CrunchBang Linux

brokenpike

#! Member

Registered: 2010-03-21

Posts: 55

jigdo and updating Statler using your own offline repositories

This weekend I was charged with salvaging an old ppc imac and some old PC hardware while off the grid.
Step one was collect 60 G of ppc , i386  and amd64 squeeze repos for use offline or where bandwidth is expensive. 

Getting iso images of the squeeze repositories:
1. apt-get install jigdo-file
2. download .jigdo and .template file for desire architecture at
http://www.debian.org/devel/debian-installer/
3. find a fast mirror
sudo netselect-apt --nonfree squeeze -o $HOME/documents/squeeze.list
4. wait for 30G per architecture to download

Jigsaw download takes a large image/iso  of a repo and breaks it down into chunks of the list of packages and their md5sume.  These instruction are used to download the the packages from any  debian repo and reconstruct the checksummed iso locally.  The iso may then be mounted locally and used by apt.   

Important observations:
1. each .jigdo file early in the process create an full size iso   called iso.temp.  Your iso´s will not appear to grow through the process
2. the iso.temp file may be mounted and scanned to avoid duplicate  downloads and to recover from stopped downloads.
3. jidgo-lite can scan old mounted iso images and avoid unneeded downloads.
4. different instance of jigdo-lite can share a database of mounted images.
5.  it is best to mount all of the old iso´s each in a sub directory of /mnt/  and have only on instance of jigdo-lite search /mnt/ and create the .db file in the directory with the .jigdo file
6.   after the the .db file of the mounted  iso´s is created other instances of jigdo-lite  may quickly it.
7.  a jigdo-lite dvd.jigdo may exploit a mounted cd of blueray image
8. jigdo-lite recovers well from download interuptions

Installing and updating:

Stater and the debian live installer are a breeze to use form either a bootable cd or usbdrive.
The process only take about five minutes even on a slow old computer.  The wireless also tends to work well out of the box.

The updating and installing of specialized software is more onerous.   
The initial dist-upgrade pulls 450M of archives and my chosen apps pull another 1.2G (2.2 if I do not avoid KDE apps). 

Using the repo images mounted from a local external usb drive the initial dist-upgrade (from statler A2)took 22 minutes and the installation 1.2G of archives took only 10 minutes on a i5-540m notebook.

non-free, statler, security  and multimedia are not local repos.

Updating  from the local repos required the following steps:

1 modify sources.list
## CrunchBang Linux 10.xx aka Statler
## Compatible with Debian Squeeze, but use at your own risk.
deb http://packages.crunchbanglinux.org/statler statler main
## Official Debian Repositories:
deb http://ftp.de.debian.org/debian/ squeeze non-free
deb file:/mnt/i1/debian/ squeeze main contrib
deb file:/mnt/i2/debian/ squeeze main contrib
#deb-src http://ftp.de.debian.org/debian/ squeeze main contrib non-free
deb http://security.debian.org/ squeeze/updates main
deb http://www.debian-multimedia.org squeeze main non-free
# deb-src http://security.debian.org/ squeeze/updates main

2. mount images:
cd /mnt
sudo mkdir i1 i2 i3 i4 i5 i6 i7 i8
mount -t iso9660 -o loop pathtoiso/debian-testing-amd64-BD-1.iso /mnt/i1
mount -t iso9660 -o loop pathtoiso/debian-testing-amd64-BD-2.iso /mnt/i2

3. update
apt-get update
apt-get dist-upgrade #wait 20 minutes, no worse than a debian netinstall
apt-get install  geany geany-plugins texlive-base-bin texlive-latex-base pybliographer texlive-bibtex-extra latex-beamer  ipython python-scipy python-numpy python-matplotlib  python-sympy spe openoffice.org openoffice.org-gtk thoggen texmaker apt-build vim  jigdo-file shotwell network-manager-gnome netselect-apt #wait 10 minutes

4. Unmounting iso drives:

Your external usb drive may have a phantom processes the complicate unmounting it.  the following steps worked for me.
sudo su
umount /media/isodrive
umpunt /mnt/i*
if that fails find  close  offending apps
lsof -a | grep isodrive
if that fails force umount
umount  -l -f /media/isodrive

5.  updating weekly image of debian repositories:

sudo apt-get install jigdo-file
down load Blue Ray .jigdo and .template file to working directory.
make mount points for old squeeze iso´s. and mount
cd /mnt
sudo mkdir i1 i2 i3 i4 i5 i6 i7 i8
mount -t iso9660 -o loop OLDISODIR/debian-testing-amd64-BD-1.iso /mnt/i1
mount -t iso9660 -o loop OLDISODIR/debian-testing-amd64-BD-2.iso /mnt/i2
etc...

open non-root terminal in working directory.
jigdo-lite debian-testing-amd64-BD-1.jigdo

at prompt tell script to search /mnt/
allow to chug for about 30 minute while it builds a database

pick a fast server using netselect-apt or similar, will default to statler repo and fail otherwise

after database is built for first jigdo download (*.db ) in working directory you may start other  jigdo-lite downloads agains the same mounted images.

About 2.5% of the packages changed from November 22 to November 29 and the update took about 1 hour of download time with a 400kps repo .

This was a great solution in search of a problem.

V/R

Brokenpike

Last edited by brokenpike (2010-11-29 18:39:15)