Thanks to pghjake's post I did some thinking about different achitectures. The "selections" method of installing seems to refer to the architecture-specific packages, so in the "barebones" package above I changed it back to 'apt-get install ...' In this case you seem to be able to use a generic name and apt will figure out which version you need.
That is, except for the kernel itself and a couple of other things like libc6-i686. However, those have already been dealt with by the Debian Netinstall, so I thought maybe by going down the list of packages the script will install and removing all those already put in by Debian, the remaining ones will all be architecture-agnostic? At least, after doing that, no packages in the "barebones" list had names including 386 or 686.
I wonder if someone with an AMD system and, maybe, a virtualbox, could confirm that the "barebones" installer works for them too? It should take less than an hour to run, depending on your system power.
Last edited by johnraff (2012-09-04 17:59:25)
Wow, just stumbled onto this ... incredible, fantastic, wow. I'm screwing with some droid hacks right now but the moment I get finished with that I will be all over this. Inspired stuff.
Find something useful here? Help fund CrunchBang, donate to the project!
Your last post inspired me to try the latest version of your script for an AMD64 install. I downloaded today's Debian DAILY testing netinst AMD64 iso (2012-09-05). Followed your instructions to wget the script from dropbox, unpacked it, edited the pkgs-manual file, ran install script, grabbed a coffee, posted this from Midori after a reboot. I'm sure it was a good bit shorter than an hour from start to finish, but I forgot to note exact start time. No hangs, no errors, just minimalist goodness. I added a few of my essential apps to the pkgs-manual file, which is easy to edit before running the install script since nano is included in the base install. My only suggestion would be to include cups, cups-pdf, and optionally, system-config-printer. Thanks again for your work on this. I'm sure it will make other's life easier, as it did mine.
Hi pghjake, thanks for checking it out! It was the "barebones" version I posted a couple of days ago which you ran? Like the name, that's meant to install only a very minimal set of apps, for people who want that, but mainly to test out new versions of the script because it will take a lot less time to run than the full version, and also just as an example of how it's possible to customize the install.
For that reason, I'm reluctant to add other packages like cups etc. to cb-barebones. If you want anything else you can add it to pkgs-manual as you did, install it yourself after the script has finished, or if you want a complete CrunchBang setup, wait a few days and...
Now that system seems to be working I've adjusted the "full crunchbang" install setup and as soon as I've tested it out (fingers crossed) I'll upload it and amend the first post.
Meanwhile, thank you again for confirming that AMD (that's 64 bit?) worked OK
64 bit yes, on AMD 4 core processor. I appreciate your design plan, just thought it might be nice to be able to print to pdf web pages from Midori for off line reference if needed for further mods to basic install. Looking forward to full version. Could this be the easy solution to the recent sid based CB variant discussion?
64 bit yes, on AMD 4 core processor.
Ah that's good to know. So the script works as-is for 64 bit too.
Looking forward to full version.
Coming right up!
Could this be the easy solution to the recent sid based CB variant discussion?
To be honest, I don't think so really. There are no #! packages built for Sid, so a whole new repo would have to be re-compiled from the source. There are already scripts for making a usable system out of a Debian Sid Netinstall, for example VastOne's. Still, I tried to make this installer as modular as possible, so someone might be able to use it as a base for building something on a sid netinstall.
@zacharias I think the current script ought to work with any architecture as it is.
The "full" script referred to in post 1 has now been updated to the same method as the "barebones" script. The url is the same, but I've discovered that it's not necessary to use https. This url is OK: http://dl.dropbox.com/u/10808732/cb-netinstall.tar.gz
*) I've removed references to any packages that will have been installed by the Debian Netinstaller. That includes the kernel, and a couple of other architecture-related packages, so now the script should run OK on any (Debian Wheezy) system, with a PAE or non-PAE kernel. You set up all that stuff at the Debian Netinstall phase, before running the cb-netinstall.
*) Just installing the manual-marked packages leaves out some dependencies - my guess is they were recommends at some point - that are still needed, so there's also a list of auto-marked packages to check, and any that haven't been brought in while installing the packages on the "manual" list are installed separately and marked as "auto-installed".
*) You can set up a mirror for apt downloads during the Debian Netinstall phase, and instead of overwriting the sources.list file that was created then, the CrunchBang repositories are now added to that file, along with the Debian contrib and non-free repos. Using that local mirror should make downloading the 1,000 or so packages needed for #! a bit quicker.
*) Other aspects of the installation have been taken out of the "install" script and put in separate files, so that the same script can be used in any version (eg barebones), and all customization can be done by editing the external files. ( This makes maintaining the script easier too. )
install - the script
greeting - message to user at start of script
extra_repos - repositories to add to /etc/apt/sources.list
sysfiles1 - directory tree of system files to install before installing packages (apt stuff right now)
apt-keys - commands to run to add any necessary GPG APT keys.
pkgs-manual - packages to install
pkgs-auto - checklist of auto-installed packages to install if missing, and mark as auto.
sysfiles2 - directory tree of system files to install after packages (/etc/skel and a couple of other things)
postinstall_commands - commands to run after installing packages and files (couple of small tweaks)
README - some general info
cb-netinstall-logs - a folder that will hold incredibly verbose records of what was done
I'll post a Customization Guide soon for people who might like to adapt this for their own CrunchBang respin, or any Debian setup for that matter.
I ran your script on this image-
Debian GNU/Linux testing _Wheezy_ - Official Snapshot amd64 NETINST Binary-1 20120905-09:20]
after I changed the sources.list to sid and ran apt-get dist-upgrade.
Your cb-netinstall.tar.gz created this sources.list (I commented out the wheezy entries in your extra_repos file as you can see in the 2 stanzas your script added below.)
## Assuming "main" repositories have already been added
## by the Debian netinstall, and best mirror chosen.
#deb http://ftp.debian.org/debian/ wheezy contrib non-free
##deb-src http://ftp.debian.org/debian/ wheezy contrib non-free
edit: forgot to mention as a precaution I changed your sysfiles1/etc/apt/preferences file to be:
Pin: release a=waldorf
Pin: release a=sid
Everything was perfect except for one small hiccup re: missing geode xorg driver when it checked the default files, but the script handled it gracefully. Video, sound, network, thunar drives, all works so far. Posting from iceweasel.
So now we know that an AMD64 "sid" netinst works with your script and sid repos as of 9/5 build.
Last edited by pghjake (2012-09-08 00:15:08)
^pghjake I'm glad it worked, but this is pretty dangerous territory. The packages in that Crunchbang repo were made for wheezy, not sid, so your system could hit the rocks at any moment. You haven't got anything important there I hope, or at least well backed up...
Still, I appreciate you stress-testing the script - thanks!
My main working desktop system is the latest Waldorf. I just had some time to play and a second empty HD, but it did work out better than I expected. I'm sure things will get less compatible very soon. Timing is everything.
Incredible work John!
I have just now been able to look at these scripts and have learned an incredible amount from what you have created.
Added to Quick References.
If you build it, they will come...
Words That Build Or Destroy
If anyone really wanted to use sid with CrunchBang packages, without ever having tried it myself I would suggest: set the pinning for Waldorf in sysfiles1/etc/apt/preferences to 100 instead of 1001, and install only the theme/config type packages from crunchbang, leaving compiled biraries like thunar to the sid default packages. You'd lose the compiled-in CrunchBang customization, but get compatibility with the sid libraries. It might help.
@V1 thanks for the encouragement!
Apart from providing a workaround for people who can't boot from a USB stick or DVD, the second motivation for writing this installer was to give people a platform for sharing their own CrunchBang, or Debian Wheezy, respins, or a quick way to restore your personal setup of CrunchBang. (ie the system, not data of course)
Anyway, some hints on making your personal installer out of this.
Of course, first download and unpack the .tar.gz file so you have a cb-netinstall folder.
If you just want to change a few of the default apps, eg use your favourite text editor, terminal emulator or image viewer, try this:
1) Open pkgs-manual, find the apps you don't want and comment them out with a #. Now add your extra packages. Dependencies will be installed automatically, but not recommends, so if you want some recommended packages you'll have to add them too. The pkgs-auto list is less important, but if you see some dependencies of the apps you commented out of pkgs-manual, it's safe enough to comment the dependecies out of pkgs-auto too.
2) Now you have to edit the menu entries and keyboard shortcuts to match. Edit sysfiles2/etc/skel/.config/openbox/menu.xml and rc.xml .
3) If you want to edit or add any config files, everything in sysfiles2/etc/skel will end up in the user's home folder.
More extensive customization in the next post.
Of course, you presumably have your own nicely set up system - maybe Crunchbang or maybe Debian Wheezy without the Crunchbang repositories. (called "pilot system" below) You want to make an installer so you can put it on other machines or share it with your friends. There are other tools around which can duplicate a system, but this script is an alternative method.
A couple of handy utilities:
apt-mark in Wheezy is more useful than before. You can now run 'apt-mark showmanual' to get a list of manually installed packages, among other things.
comm is perhaps a simplified diff, but handy for comparing two lists, and its output can be used in scripts. For example 'comm -23 file1 file2' will show all the entries unique to file1. It needs the files to be sorted though, so often you need to use process substitution: 'comm -23 <( sort file1 ) <( sort file2 )'
Meld is somewhat useful for comparing and editing two files, or two directory trees. (If you are comparing system directories you need to open it with gksu, even if you only want to look, not edit.)
These files come in the tar.gz bundle, along with the install script itself. It should be possible to do most, or all, of the customization there, without having to edit the script itself:
README greeting extra_repos apt-keys pkgs-manual pkgs-auto postinstall_commands
and these folders contain files that will be installed into the target system:
You can put anything you like in the README file.
Anything in the greeting file will be echoed out to the terminal, but not to the log-file.
sysfiles1 contains system files to be copied into the netinstalled base system before installing packages. This means mainly apt configuration. For example, put the sources.list you want to install in sysfiles1/etc/apt/sources.list. On the other hand, if you have an extra_repos file in your base folder the repositories in extra_repos will be added to whatever the Debian netinstall put in /etc/apt/sources.list. This lets you keep the mirror that Debian chose for the main Debian repositories, if you prefer. (sysfiles1/etc/apt/sources.list will be ignored if it exists.) You can add apt-pinning in sysfiles1/etc/apt/preferences.
You can put any other files you want to install before the package installation process in sysfiles1.
Any commands in apt-keys will be run before upgrading the system. This can be used to install the keys for whtever repositories you have added. NB the script itself should be run as a normal user, so sudo is needed for system calls. (see the existing apt-keys file)
Maybe the main point. Put a list of all the packages you want to install in pkgs-manual. # style comments are allowed. You can extract a list of all the packages that were installed manually on your pilot system by running:
One point: recommends. This script sets apt not to install recommends, so if your own system did the same then just a list of the manually installed packages should pull in all the dependencies needed. If you installed some recommends as dependencies at some point, then the manual apps alone will not be enough. This was the case with the Crunchbang Waldorf beta I used; for example python-xdg was auto, but needed. If you think this might be the case with your system you can get round it by running
and putting the result in pkgs-auto. Packages in pkgs-auto will be installed after the main install (if they haven't been already) and then marked as auto-installed.
comm can be useful. Example from the script:
missing_auto=$( comm -23 <( sed 's/\#.*$//' pkgs-auto | sort ) <( apt-mark showauto | sort ) )
missing_auto is a list of packages in pkgs-auto that haven't been auto-installed on the system.
Config files you want to put in /etc (or anywhere else) after the package installation can be put in sysfiles2. Any existing system files will be backed up to /backup before being overwritten. User config files should go in sysfiles2/etc/skel. Here's where you tweak themes, wallpaper, openbox menus, tint2... or config for whatever apps you install. The contents of skel will also be copied into the existing user's home directory, making backup copies of any existing files, marked with whatever is set in backup_suffix on line 37 or so.
You could also install extra theme files in syfiles2/usr/share/themes for example. In principle you could bundle fonts and icon sets too, but this would likely make the whole tar.gz file much bigger! Easier to stick with what's in the repos.
Meld can be a handy tool for comparing what's in your pilot system's /etc with the netinstalled system, or with sysfiles2/etc, or to compare the two users' home directories.
Put any commands you want to run at the end in the postinstall_commands file. Check out the existing file for hints. Here we can fix things like group membership or general system settings. Don't forget to use sudo if necessary.
The install script is arranged so that error messages go to ~/.cb-netinstall-logs/install.log and get echoed out to the terminal too. This is done with stuff like:
command > >( tee -a "$logfile" ) 2>&1 || error_message
The redirection is so that the || at the end can still get the return value of the original command, not that of tee. There are some logging and message functions at the top of the script, eg warnlog to put a warning message in the logfile. You might want to use some of this stuff in any commands you add to apt-keys or postinstall_commands. Have a look at what's there now.
If you run into any problems putting together your custom installer, feel free to post questions here. Also, please post your resulting customized Debian installer somewhere!
Johnraff, I cannot thank you enough for making this script! I was completely unable to install #! in any way, shape, or form until I found this script. It worked like a charm and I am now happily working within the nearly #! environment. Thanks again!
Do you need a basic Debian Wheezy install, prior to running this script and does the script recognize seperate "/home" partitions?
I did a basic debian wheezy netinstall, but at the last part of the process chose nothing but "Standard System Utilities" and then ran the script after that from the terminal. It threw some error but whatever it was it must not have been a big deal 'cause the system is working beautifully now.
I did the same basic install using the wheezy-beta4 installer. I used the x86_64 installer, the only thing I installed form the netinstall was the ssh packages.
I just had two issues.
1.) The script had an issue finding some 32-bit xfree package. After I removed the package from pkgs-manual and reran the script everything was "working". (with the exception of SNA)
2.) The main reason I went the netinstall route was the need for EFI booting. I wasn't exactly thrilled the script downgraded the grub-efi-amd64 package, but that's life.
@everyone I have been thoroughly slacking off on keeping the installer up to date, spending time on other things, so the system files modified by the script have not been changed to reflect the October 2012 release. Meanwhile, in practice it should not matter too much. The CrunchBang packages in the repositories are of course being updated by Philip, and will be brought into your system by an 'apt-get upgrade'. The latest user config files in /etc/skel and hence in your ~/.config directory will have to be modified by hand if you want them. You might also want to look at /etc/apt/sources.list and possibly modify it to get the best mirror. Apologies, and I intend to make an updated version soon.
@HelloLion Thanks, and always happy to hear if it's helped someone.
@DapperMe17 You only need to run the basic netinstall. You don't even need to check the "standard system utilities" box as most of them will be installed by the script anyway, but no harm if you do. This will give you just a command-line-only system from which you can run the script. More instructions are in README in the script package.
@mitchejj Yes, there might be obsolete packages in the pkgs-manual list as Debian Wheezy is still changing. Till I make a new version, commenting the problem packages out as you did should get you through OK. The grub downgrade was triggered by adding the #! repositories and apt-pinning them to 1001. This reproduces the standard #! setup, but you could consider editing /etc/apt/preferences and changing the waldorf pin-priority to 500. This would allow grub upgrades, but might have implications for other packages like Thunar.
@all If you get any error messages I would appreciate it if you could post them here, in case they show a bug that needs fixing. Have a look at ~/.cb-netinstall-logs/install.log. It's long and boring, but any error messages should be there.
@johnraff: your script works 1000xs better than my two previous attempts to go from a Debian netinstall to Waldorf... I took the EFI grub download in stride and an few outdated packages easy to deal with thanks for the script.
trying the script now , and i got errors with 3 packages in auto-install : libisc83, libdns81 and xserver-sorg-video-geode, these packages were not available.
I am unsure about what they do , and am trying the script anyways a second time, after commenting them out. Hope this works. Any light on those packages ?
Tried to run the script a second time after errors the first time around, I get the error, first two arguments of install_sysdir() must be existing directories.
and then it said now exiting and it quit
Tried again afresh from a freshdebian netinstall and commented out the 3 packages in auto-install list. Still the same error. I would really appreciate some help with this as currently this is turning out as my only method to install crunchbang easily as I have a UEFI system and its causing me a lotta trouble.
So wit these errors , I am still greeted by a login manager and once I enter I get a black screen ( regular openbox screen ) and a cursor and thats about it. So since I guess the error caused was at a stage before any config files were copied to my user, there is no openbox folder in .config ( hence no autostart.sh and stuff) so no applications starting by default. Also maybe other files didnt get copied which I might not be aware of.
The script runs pretty decent but help to rectify these errors will help me out a lot !
Many Thanks !
Last edited by ayushjsh (2013-01-22 03:09:15)
i got errors with 3 packages in auto-install : libisc83, libdns81 and xserver-sorg-video-geode, these packages were not available.
I am unsure about what they do , and am trying the script anyways a second time, after commenting them out. Hope this works. Any light on those packages ?
Hi ayushjsh - thank you for your feedback, it is appreciated. The contents of the Debian Testing repositories are still changing, and those packages have probably been removed, replaced by something else. This netinstall script was based on an earlier #! Waldorf release, and I have been working on releasing a new one in a week or two (now to be based on the lastest 20130119 release) which will probably fix that problem. Meanwhile, commenting the files out of pkgs-auto should work OK.
I get the error, first two arguments of install_sysdir() must be existing directories.
and then it said now exiting and it quit hmm
This is more worrying. That error has never ocurred for me. Could you do me a favour and look in the .cb-netinstall-logs folder in your home directory, where there should be a file "install.log". Can you post here the last few (say 20 or so) lines in that file, before the script exited? There might be a hint there.
Anyway I'll be testing out an updated version of the script soon - I hope this issue can be fixed too.
Thanks for your super cool script ! I was determined to install #! any way possible and did a fresh install over that one so I am afraid that I have lost those files. But alas the new method I was trying did not work out either so I am back to no working system atm. Will give your script a shot again tomorrow as Im a bit busy today. Will try to duplicate the error again and post the file here.