SEARCH

Enter your search query in the box above ^, or use the forum search tool.

You are not logged in.

#1 2012-08-04 05:34:02

johnraff
#!Drunkard
From: Nagoya, Japan
Registered: 2009-01-07
Posts: 2,843
Website

A Waldorf netinstall script

Several good people have posted scripts in the past for installing a system from the Debian netinstall core system, though none specifically for #! Waldorf so far. This one, however is somewhat different:
*) It's monstrously clunky and bloated.
*) It requires a download of a tar.gz file.
also
*) It might take hours to install the system.
*) You don't get a live session.

However, I'm sort of hopeful it might eventually turn out to be useful.
*) It might take some of the heat out of the 700MB limit question.
*) It's intended to be usable by a relatively inexperienced Linux user.
*) It should make an identical #! system, making forum support easier.
*) It should be quite easy to use it as a platform for a CrunchBang Lite or any other CrunchBang respin you want to make and share with the rest of us.
*) The bloatedness is partly because I've tried to make the installer as fail-safe and stable as possible. A full record of system changes is made, and it's possible to run the script more than once (maybe after tweaking a config file) without breaking anything. It ought to be possible to write an "uninstall" script if there is a demand for one. I figured some more lines of code and extra milliseconds to execute don't matter much in this context. edit 2014/4/5 There is now an experimental "undo" script provided, which can take your system back to something like what it was before. This would mainly be useful for developers who don't want to go through the Debian netinstall again before retrying the script.

Basic structure: install script, some lists of apps to install, two directory trees of system files to install (before and after app install) and a couple of other config files.

To customize, you could edit the apps lists and sysfiles2/etc/skel/.config/openbox/{autostart,rc.xml,menu.xml}
You could go on to put new themes in sysfiles2/etc/skel/.themes ...

If anyone would like to play with it and report back with problems and ideas for improvment I'd be more than grateful.

The whole rhing is now on Github:
https://github.com/johnraff/cb-netinstall
Pull requests for improvements or fixes are most welcome, but maybe it would be good to discuss them here first. smile

The dropbox link has now been removed.

So to use it, in a new netinstall Debian Wheezy core system you'd do:

wget https://github.com/johnraff/cb-netinstall/archive/master.tar.gz
tar -xpf master.tar.gz
cd cb-netinstall-master
./install

There's a bit more info, and hints about the Debian netinstall process, in the README file in that package.

There is now a "How To" thread for users here:
http://crunchbang.org/forums/viewtopic.php?id=25098
I would suggest posting questions about, and problems with, usage on that thread, and keep this thread for discussion of the script itself - how it works, customization, suggestions and bugs.

edit 120808 A couple of tweaks to slightly increase the chance of the script completing successfully and make the backups a bit more robust. (The dropbox url will always point to the latest version.)
edit 120819 Now installs the latest version of Waldorf, and the script has had some more tweaking to make it a bit more solid.
edit 120823 Download and install the CrunchBang GPG key to avoid those error messages.
edit 120908 More modular, a bit faster, and hopefully architecture-independent. See this post.
edit 130220 Finally updated to match the #! 130119 release. Yet more cofiguration has been pulled out of the script into the external files. Thanks to corenominal's new packaging system this installer might stay usable even after CrunchBang version upgrades. Customized versions might also stay usable even after the script itself has been upgraded. Some bugs have been fixed. smile A bit more here. (Also edited the explanation above a bit to match the current situation better.)
edit 130301 Added link to how-to thread.
edit 130420 The whole package is now on Github.
edit 131106 Replaced the script here with a link to github.
edit 140405 Removed dropbox link, replaced with github, amended install instructions to match.

Last edited by johnraff (2014-04-05 04:54:19)


John
--------------------
( a boring Japan blog , idle twitterings  and GitStuff )
#! forum moderator

Offline

Be excellent to each other!

#2 2012-08-04 09:12:32

sunfizz98
Carbonated Orange Juice
From: su terminal
Registered: 2011-05-12
Posts: 1,876
Website

Re: A Waldorf netinstall script

Nifty script. I will have to dig through it and try to understand the functions for myself smile

Offline

#3 2012-08-07 16:08:29

johnraff
#!Drunkard
From: Nagoya, Japan
Registered: 2009-01-07
Posts: 2,843
Website

Re: A Waldorf netinstall script

Hi sunfizz - you were one of the people I was thinking of who'd already worked on this kind of stuff. Any advice you might have will be appreciated. smile

Those functions - most of them are about sending messages to a logfile and the terminal, with extra bits like ***WARNING*** tacked on or whatever. Just to save me from typing out the same code multi times. That big install_dir function just does the same as rsync or something, but also makes a record of all the files changed, in the hope of being able to undo it all later if necessary.


John
--------------------
( a boring Japan blog , idle twitterings  and GitStuff )
#! forum moderator

Offline

#4 2012-08-07 16:20:42

machinebacon
#! unstable
From: China
Registered: 2009-07-02
Posts: 6,619
Website

Re: A Waldorf netinstall script

I love the idea. Thank you, John. This will open new opportunities for users to share their spins - simply by providing a link to pastebin or forum code (applist/sysfiles). Big +1! Calls for a future sub-section big_smile

Last edited by machinebacon (2012-08-07 16:21:43)

Offline

#5 2012-08-08 04:50:41

johnraff
#!Drunkard
From: Nagoya, Japan
Registered: 2009-01-07
Posts: 2,843
Website

Re: A Waldorf netinstall script

Hi mb, thanks for the moral support! smile

It's still kind of messy at the moment, and extra bits of checking and logging keep getting added, but a lot of that is for bugfixing and the final version ought to be a bit simpler, at least from the end user's point of view.


John
--------------------
( a boring Japan blog , idle twitterings  and GitStuff )
#! forum moderator

Offline

#6 2012-08-08 10:28:56

corenominal
WRONG
From: Lincoln, UK
Registered: 2008-11-20
Posts: 5,057
Website

Re: A Waldorf netinstall script

I have not had chance to actually test it yet, but I really like what I see. Nice work, John, and thank you for sharing. smile

Offline

#7 2012-08-08 11:35:21

rhowaldt
#!*$%:)
Registered: 2011-03-09
Posts: 4,396

Re: A Waldorf netinstall script

@john: looks very good! i skimmed through the script and i noticed all the logging functions, indeed a script as you describe, 'rough draft' kinda thing. can't try it out right now but i might be getting a spare PC in the near future, which will then be open for experimentation, and i'll be sure to check out the script by then!

Offline

#8 2012-08-08 11:57:44

el_koraco
#!/loony/bun
From: inside Ed
Registered: 2011-07-25
Posts: 4,749

Re: A Waldorf netinstall script

This is very high quality stuff.

Offline

#9 2012-08-09 01:30:08

pghjake
#! Member
From: Pittsburgh, PA, USA
Registered: 2009-02-07
Posts: 79

Re: A Waldorf netinstall script

Just did a new AMD64 wheezy install using the dual-arch netinstall image dated 8/4 and your script. I commented out the linux-image and linux-headers in the apps-to-install file since I had a shiny new 3.2.0-3-amd64 kernel. I had to change the libc6 to i386 rather than i686 as the i686 was not available. There are 3 versions of libboost-iostreams in the apps-to-install, but only 1.49.0 was available. With those changes the script ran through with no issues. Thanks so much!

Last edited by pghjake (2012-08-09 01:31:40)

Offline

#10 2012-08-09 06:02:23

johnraff
#!Drunkard
From: Nagoya, Japan
Registered: 2009-01-07
Posts: 2,843
Website

Re: A Waldorf netinstall script

@pghjake thanks for catching that!
My apps-to-install file was made by running 'apt-mark showmanual' on a freshly installed Waldorf, so of course it would have been an i686. I should mention somewhere that there's no support for AMD etc at the moment. In fact I'm a bit surprised it worked for you with only a couple of tweaks.

Going over the apps in that list is one little project ahead, but I want to try to make the resulting install as close as possible to the standard Waldorf.

Otherwise, atm the script seems to basically work, but it's still sort of fragile, easily upset by changes in the Debian repos etc, and things can get messed up if you run it twice in a row. Making it a bit more robust is the major aspect I'm thinking about right now.


John
--------------------
( a boring Japan blog , idle twitterings  and GitStuff )
#! forum moderator

Offline

#11 2012-08-09 12:48:32

pghjake
#! Member
From: Pittsburgh, PA, USA
Registered: 2009-02-07
Posts: 79

Re: A Waldorf netinstall script

You may want to consider adding your "recent files" and "places" scripts to the menu.xml file, since corenominal added them to the latest Waldorf spins. I put them in their own section above "Lock Screen" but you would probably like them near the top of the menu. :-)

Offline

#12 2012-08-09 18:17:43

johnraff
#!Drunkard
From: Nagoya, Japan
Registered: 2009-01-07
Posts: 2,843
Website

Re: A Waldorf netinstall script

Now both corenominal and Debian have just upgraded I'll have to go back, reinstall Waldorf, update the sysfiles and apps-to-install in the script to match, then test it out on the new Wheezy beta 1 netinstaller.

This could take a few days, especially as I'm just about to go off on a little summer holiday...

Meanwhile the script itself is due to get a couple of tweaks, to make it a bit safer if things get interrupted half-way through eg if the user presses Ctrl+C.

Maybe next week some time... smile


John
--------------------
( a boring Japan blog , idle twitterings  and GitStuff )
#! forum moderator

Offline

#13 2012-08-09 19:46:53

omns
#! wanderer
From: ~/
Registered: 2008-11-25
Posts: 5,131

Re: A Waldorf netinstall script

This is a really nice project johnraff. Keep up the great work big_smile

Offline

#14 2012-08-17 00:20:32

zacharias
#! Member
From: Tokyo 3
Registered: 2009-10-19
Posts: 76
Website

Re: A Waldorf netinstall script

gr8 work mate. thoug, is this valid also for amd64 versions? i ask this just out for curiosity, since ill be using a 386 version.
kudos

Offline

#15 2012-08-17 04:33:02

johnraff
#!Drunkard
From: Nagoya, Japan
Registered: 2009-01-07
Posts: 2,843
Website

Re: A Waldorf netinstall script

@zacharius The sad truth is I have no experience of systems beyond i386/32bit, so the script was built without a thought for anything else. pghjake above was able to tweak the apps-to-install file to get an AMD install, but apps-to-install no longer exists...

@everyone sorry, but the thing is in a very fluid state atm, and as a web download is the easiest way to get the script into my test system the version up on dropbox might be broken at the moment you download it. roll

The install method has switched from apt-get of manual-marked packages to setting the selections pulled from waldorf to get all the auto-marked packages too. I'm hoping the big bugs will be ironed out in a day or two and it will be there for folks to help find all the little bugs. smile At that point I'll have a think about if anything can be done about other architechtures and 64bit systems. (The Debian netinstaller might already be taking care of most of that.)

I'll post again when it's sort of ready to use, but meanwhile be very careful and have a look at the script before running it!

Thanks for all your encouragement!


John
--------------------
( a boring Japan blog , idle twitterings  and GitStuff )
#! forum moderator

Offline

#16 2012-08-18 17:39:39

johnraff
#!Drunkard
From: Nagoya, Japan
Registered: 2009-01-07
Posts: 2,843
Website

Re: A Waldorf netinstall script

I think it's now reasonably safe to try out, if anyone feels like giving it a go. If you hit an issue you can exit the script, tweak something and run it a second (or third...) time with no harm done.

I've just used it to install Waldorf in a virtualbox machine and it seems OK, except that apt-xapian-index eats all the memory (removed it). Hoping that's a Wheezy issue that wil get fixed.

Anyway, eager to hear of any bugs you might find!


John
--------------------
( a boring Japan blog , idle twitterings  and GitStuff )
#! forum moderator

Offline

#17 2012-08-20 22:45:29

zacharias
#! Member
From: Tokyo 3
Registered: 2009-10-19
Posts: 76
Website

Re: A Waldorf netinstall script

johnraff wrote:

I think it's now reasonably safe to try out, if anyone feels like giving it a go. If you hit an issue you can exit the script, tweak something and run it a second (or third...) time with no harm done.

I've just used it to install Waldorf in a virtualbox machine and it seems OK, except that apt-xapian-index eats all the memory (removed it). Hoping that's a Wheezy issue that wil get fixed.

Anyway, eager to hear of any bugs you might find!

i'll be giving it a go on a virtualbox. i'll feed you back propperly once i run it.
kudos

Offline

#18 2012-08-21 04:34:54

johnraff
#!Drunkard
From: Nagoya, Japan
Registered: 2009-01-07
Posts: 2,843
Website

Re: A Waldorf netinstall script

Thanks zacharias - any feedback will be appreciated! smile


John
--------------------
( a boring Japan blog , idle twitterings  and GitStuff )
#! forum moderator

Offline

#19 2012-08-21 07:46:00

rich
#! Junkie
From: barcelona
Registered: 2009-01-26
Posts: 414
Website

Re: A Waldorf netinstall script

Just saw this, nice work johnraff smile

Have a whole bunch of Wheezy VMs at the moment for various things, am looking forward to testing this on a fresh netinstall.

Incidentally, i've created similar build scripts for custom Debian's, and have also used the same system where the build files were in a downloadable tar.gz.

A nice option is to add a switch where the person has the option to cp from a USB Pendrive as well as wget from a server. Useful for systems with unusual Nic (Non-Free) Firmware and therefore no internet access after the netinstall.

May try and find some free time to adapt this to 64bit as well.

Thanks for sharing.

Last edited by rich (2012-08-21 07:51:30)

Offline

#20 2012-08-21 18:53:40

zacharias
#! Member
From: Tokyo 3
Registered: 2009-10-19
Posts: 76
Website

Re: A Waldorf netinstall script

johnraff wrote:

Thanks zacharias - any feedback will be appreciated! smile

well, mate youre welcome.
i dont like just to give feedback. i also like to tell dev's what in my opinion as a end-user should/could be improved, so here goes.

Suggestions
1) welcoming screen is just ok, filled with warning and the goal behind the script. after this first screen, another one should pop and letting the user choose the machines architecture in order to asjust the update to it's propper version and or HW. this means that the update process/script needs to be slightly changed instead of beeing so straight forward (it's pretty damn good wink ) this would allow a multi-architecture support of this script.

2) i understand the goal behind the script is to get to closest #! experiece as possible to the original, but i think the user coul'd have the choose option in what in fact to install, seperated by topics. meaning, the ideia is to sperate the updates, kinda like: 1st one installs basic system  #! feautures like DM WM, thunar, conky scripts, etc. after this, tools like gimp, abiword etc, (that i dont need right away) could be an option. (yes yes, i also think #! waldorf iso is waaaaay 2 big ; ) by deviding it in categories like: Office, Graphic, Multimedia, etc. etc.
this would also allow to decrese significantly the download to the user's and installation time. yeah, i know, it would be kinda like a custom install, but i think i've made my point.

The script itself
ran like a charm. it's not pretty (yet! wink ) but it does what it says. to be noticided that a keyring warning showed up after updating mirror list. i just found a nag, to had to configure again! the keyboard. everything else went just smoothly.
to be noted: i installed with with a mini.iso. prior to all during the installation of wheezy, on must be suggested no to allow root login. why? in my experience, after first reboot, my username didn't show up on the sudoers list, and this might be a nag for someone, specially as "green" as me.

i'll be more than happy to help you with further testing of this script.
kudos

Offline

#21 2012-08-22 03:44:57

johnraff
#!Drunkard
From: Nagoya, Japan
Registered: 2009-01-07
Posts: 2,843
Website

Re: A Waldorf netinstall script

rich, thanks for your feedback smile

rich wrote:

A nice option is to add a switch where the person has the option to cp from a USB Pendrive as well as wget from a server. Useful for systems with unusual Nic (Non-Free) Firmware and therefore no internet access after the netinstall.

I'm not quite sure what you're suggesting here. wget is used to get the script itself into the netinstall core system. Of course the user is free to download it on another box and import it via a usb stick, but that is before the script itself is even run... When it's running, of course internet access will be needed for apt to install packages. Are you talking about an option to download packages elsewhere and for apt to get them from a usb stick? Another way might be to import the necessary non-free packages before doing the install?

May try and find some free time to adapt this to 64bit as well.

Please! I know nothing about 64bit or non-386 architectures. roll Any help there would be most welcome.

Last edited by johnraff (2012-08-23 04:57:43)


John
--------------------
( a boring Japan blog , idle twitterings  and GitStuff )
#! forum moderator

Offline

#22 2012-08-22 03:55:11

omns
#! wanderer
From: ~/
Registered: 2008-11-25
Posts: 5,131

Re: A Waldorf netinstall script

Hmm, it wouldn't be too hard to come up with a 64bit version. If I get time I'll have a look smile

Offline

#23 2012-08-22 04:17:00

johnraff
#!Drunkard
From: Nagoya, Japan
Registered: 2009-01-07
Posts: 2,843
Website

Re: A Waldorf netinstall script

zacharias, this kind of feedback is extremely valuable - thank you!

zacharias wrote:

Suggestions
1) welcoming screen is just ok, filled with warning and the goal behind the script.

Maybe a bit less warning and a bit more welcome? Well when that was written it was very much an alpha sort of thing for developers and the brave to experiment with, so I was trying to avoid any blame... Later if/when it all goes stable the interface might be made a bit less off-putting.

after this first screen, another one should pop and letting the user choose the machines architecture in order to asjust the update to it's propper version and or HW.

This would be great! It's definitely a long-term goal. Right now I don't have the knowlege, or any AMD etc machines to test on. I need to study up or get help. If it's just a question of adjusting the list of packages to install it might not be too difficult.

2) i understand the goal behind the script is to get to closest #! experiece as possible to the original, but i think the user coul'd have the choose option in what in fact to install, seperated by topics. meaning, the ideia is to sperate the updates, kinda like: 1st one installs basic system  #! feautures like DM WM, thunar, conky scripts, etc. after this, tools like gimp, abiword etc, (that i dont need right away) could be an option. (yes yes, i also think #! waldorf iso is waaaaay 2 big ; ) by deviding it in categories like: Office, Graphic, Multimedia, etc. etc.
this would also allow to decrese significantly the download to the user's and installation time. yeah, i know, it would be kinda like a custom install, but i think i've made my point.

I was thinking about a CrunchBang-Lite installer from the beginning, and tried to make things so it would be easy to customize. The choice between one script with multi-options and a number of customized installer scripts is a tricky one. Right now I'm just trying to make a clone of the regular CrunchBang system, so that if a user comes to the forum for help they won't have to explain how they installed it. When the installer system itself has settled down I'll post a guide to making custom installers, so anyone can share their favourite CrunchBang respin. It should be quite easy, and not require any editing of the script itself. cool

to be noticided that a keyring warning showed up after updating mirror list.

This should be fixed very soon.

i just found a nag, to had to configure again! the keyboard.

Yes, that's because one of the installed packages calls for a reconfigure. It can be got over with a dpkg option to always use the default setting, but there's a possibility something more important might be missed, so maybe it's better just to accept these little prompts. In any case, any non-free software is likely to require you to agree to its conditions. (I had that for my Intel wireless stuff.)

to be noted: i installed with with a mini.iso. prior to all during the installation of wheezy, on must be suggested no to allow root login. why? in my experience, after first reboot, my username didn't show up on the sudoers list, and this might be a nag for someone, specially as "green" as me.

In the netinstall hints (in the README file) I made a point of not entering a root password. The Debian installer says that if you leave it blank you will be given sudo powers instead, which is what happened in my case (on several installs). You can still enable root login later if you want. Anyway, the script tests if the user can sudo, and exits with a message if there's a problem.

Thanks again for your comments. smile


John
--------------------
( a boring Japan blog , idle twitterings  and GitStuff )
#! forum moderator

Offline

#24 2012-08-22 17:21:55

johnraff
#!Drunkard
From: Nagoya, Japan
Registered: 2009-01-07
Posts: 2,843
Website

Re: A Waldorf netinstall script

installing packages in scripts
For anyone who's irritated by all this "selections" stuff, and to anyone who might know a bit more about using apt in scripts than I do...

OK we've got a Waldorf machine with #! Waldorf newly installed the regular way, and a Target machine waiting to be Waldorfed...

To duplicate Waldorf's packages on Target the obvious thing to do would be something like  'apt-get install $list_of_apps' on Target, if you can get the list from Waldorf.
The apt-mark utility (options have changed between squeeze and wheezy) is quite useful.
'apt-mark showmanual' on Waldorf will now give you a list of all packages that were specifically installed on the system so

# on Waldorf
apt-mark showmanual > apps_to_install
# on Target
apt-get $( < apps_to_install )

is the first method I tried.

However, even after installing everything in apps_to_install, along with all the dependencies that apt brought in, there were still some auto-installed packages in Waldorf that weren't installed in Target. Running 'apt-get autoremove' on Waldorf took nothing out, so Waldorf obviously considered those extra packages to be dependencies of the manually-installed ones. Still, apt on Target hadn't seen any need to install them with the same "manual" list.

The only thing I could think of was that the CrunchBang APT settings include NOT regarding recommends as dependencies. Could it be that those extra auto packages had been brought in at some point because they were recommends, and even after setting no-install-recommends they don't get marked as automatically removable?

Some of those extra auto packages are important, so a way of getting them on Target was needed. neutral

To clone a system's packages, around the web and in 'man dpkg' the method you see referred to is to do

# on source box
dpkg --get-selections > selections
# on target box
dpkg --set-selections < selections
apt-get dselect-upgrade

Hardly anyone mentions that --set-selections won't work unless you also have dselect installed, but that's what I found on Wheezy. (Now I've just tried those same commands on Statler with no need for dselect so it looks as if there might have been some changes around here.)
Anyway, another difference I've just discovered with 'dpkg --get-selections' compared with 'apt-mark showmanual' is that dpkg outputs the architecture-specific package name, ie packagename:i386 instead of plain packagename. This makes that list less portable, obviously.

All this feels too messy, so now I'm thinking of going back to plain apt-get with the output of apt-mark. Something like:
'apt-mark showmanual > mark-manual' on Waldorf.
'apt-get install $( < mark-manual ) on the Target.
Comparing the output of 'apt-mark showauto' on the two systems (using eg comm) ought to give a list of packages missing from Target which can be installed with apt-get and marked as auto with apt-mark. Does that seem to make sense?

If it still feels right in a day or two maybe I'll amend the script (again).

edit Cleaned it up a bit. Should avoid latenight/drunk posting. roll

Last edited by johnraff (2012-08-23 04:55:12)


John
--------------------
( a boring Japan blog , idle twitterings  and GitStuff )
#! forum moderator

Offline

Be excellent to each other!

#25 2012-09-03 16:10:07

johnraff
#!Drunkard
From: Nagoya, Japan
Registered: 2009-01-07
Posts: 2,843
Website

Re: A Waldorf netinstall script

barebones Crunchbang imitation

This is a very basic install on Debian Wheezy netinstall, just putting in a text editor (leafpad), terminal emulator (urxvt), file manager (thunar), image viewer (viewnior) and web browser (midori), and a bit of background stuff. No display manager, just cli login and auto-startx, 'sudo halt' to shutdown. No pipe-menus, except for places & recent files. ( tongue ) However, it looks just like CrunchBang, even the compositing. On a virtualbox allocated 380MB, it boots up with ~40MB of RAM.

Made it to play with the new package install method and develop the script to be more generic without having to wait 2 hours for all the packages to install each time. Also, it demonstrates how the same script can fairly easily be customized to do almost anything with a Wheexy base, editing the files that come in the package, but leaving the script itself alone. I'll post a Customization Guide in due course...

Have fun!

wget http://dl.dropbox.com/u/10808732/cb-barebones-netinstall.tar.gz
tar -xpf cb-barebones-netinstall.tar.gz
cd cb-barebones-netinstall
./install

...and  please report any problems - thank you! smile

Last edited by johnraff (2012-09-23 15:48:37)


John
--------------------
( a boring Japan blog , idle twitterings  and GitStuff )
#! forum moderator

Offline

Board footer

Powered by FluxBB

Copyright © 2012 CrunchBang Linux.
Proudly powered by Debian. Hosted by Linode.
Debian is a registered trademark of Software in the Public Interest, Inc.

Debian Logo