Live Build crib sheet

pvsage · 2015-02-24 08:33:42

Crib Sheet for everyone on the team interested in the Live Build process

We'll assume that you've already used Debian and derivatives from Live media enough to have an idea about initrd, vmlinuz, SquashFS, etc., so we'll dispense with what they do (for now) and why you'd want make a live build in the first place. If you're reading this, you're probably already waaay over that bar. (If these terms are new to you, you might want to at least scan the first few chapters of the Live Build manual, and especially read Chapter 4.1.)

We'll assume also that you're only interested in building live images within a working Debian installation, and your target distro will always match build system.

sudo apt-get install live-build

Create a directory in which to build "default live system without X":

mkdir live-default && cd live-default

Execute live-build config (default options)

lb config

Build the farking system:

lb build

This may take a while, so have a coffee, or a beer...

...and 20 minutes or so later, live-image-i386.hybrid.iso

dd to USB drive (It's only about 200MB, so it should fit on just about anything other than commercial promotional malware sticks), reboot, and enjoy the emptiness of your bare-bones Live system.

N.B.: The Live Build manual says "you should automatically be logged in on the console to the user account". At the time of this writing, this is not true; you'll see a login prompt.
account: user
password: live

I'll be posting about how to build something other than the default system as I dig it out of the manual...

Last edited by pvsage (2015-02-24 08:38:18)

AnInkedSoul · 2015-02-24 17:46:04

live-build is very VERY complex...can be finicky...and a tough nut to crack when something isnt working...

it use to change like every day, hopefully developement changes have slowed nowadays

the complexity, changes, and my trouble with it is probably the biggest reason I created refracta actually

just sayin

brontosaurusrex · 2015-02-24 19:07:04

subscribed.

pvsage · 2015-02-24 20:20:53

@AnInkedSoul: Thanks for sharing that; it's always good to hear from someone speaking from experience.

tknomanzr · 2015-02-25 02:44:04

@pvsage I posted some long thing then somehow lost it. Sent you a pm also. I have done some work but not 100% there yet. It has been my intention to share my success AND my pain but had temporarily put that project on hold, as it was about to kill that older Toshiba. I'll play with it some more and see if I can get over the last couple of hurdles I have with it. At any rate, Ill also post the configs back to github. I had them up there, then took them down. Been juggling so many different thigs lately, I get myself lost

pvsage · 2015-02-25 04:12:50

^ I know the feeling. I've had to put a few personal projects (most not related to Linux or computing, but nothing terribly pressing) on hold until we get BunsenLabs off the ground.

zazen23 · 2015-02-26 21:22:53

AnInkedSoul is spot on with my experience with live-build also(recently as six months ago). Troubleshooting breakage is painful. Refracta is pretty straight forward. Thank you AnInkedSoul for creation of Refracta.

zazen23 · 2015-02-27 04:05:28

pvsage you might find Handylinux how to inspirational Building your own HandyLinux

Head_on_a_Stick · 2015-02-27 08:07:17

https://wiki.archlinux.org/index.php/Archiso

SCNR...

tknomanzr · 2015-03-02 00:25:02

Intro

I promised a walkthrough of my last Live Build, which I consider successful, though I am aware of many details we have discussed which would change my approach to certain things. Be aware that this may end up a TL:DR post unless this stuff really interests you as I have a lot to cover.

For the record, I have built up my own minimal type debian respin for the express purpose of attempting to understand this tool. This respin is not intended in any way to reflect the Bunsen Labs effort. It is my personal test bed and as such is very different from Bunsen Labs.

I am going to attempt to document my process and then point out where Bunsen Labs will deviate as we really want to build it up in the proper Debian way.

Real quick, let's cover the territory that @pvsage already covered in his initial post. The Live Build environment consists of three commands: lb clean, lb config and lb build. Use lb clean any time you make changes to your chroot environment to ensure that those changes get reflected in your builds.

lb clean is used to clean your environment up. Issued with no options, it clears your chroot up. It could also be used to clean up your binary environment up using lb clean --binary. The command lb clean --purge cleans up your chroot along with your cache.

Already there are a few things to note:

The chroot environment is where most of the build work happens. I have done most of my work with this section.
Binary sections are used to modify run-time behaviors in Live Boot (Live Boot is what you are running when you boot up a live iso.) For instance language, keyboard and time-zone settings would be modified here.
Finally, when doing a build apt-get(or apt if set in your configs), along with dpkg, are very busy. Your cache is used to hold all downloaded packages so you don't have to re-download them every build. Live Build will check for updates, however and update packages as necessary.

The chroot environment gets built up in stages. The first stage is bootstrap and in this stage, Live Build will download and install everything necessary to build up a base Debian installation. The next stage, I don't know what it would be called, is where most of the work of installing our packages and copying desktop configurations in will happen. Finally, Live Build will build a squashfs and build up a usable Live ISO. Note, it is also possible to build hdd installs, net installs and the like but I'm just covering how to build up a hybrid Live environment.

Next up is how to set up the Iso Config environment

Last edited by tknomanzr (2015-03-02 00:27:39)

tknomanzr · 2015-03-02 01:04:26

Section 2: The environmental configs

I'm calling these environmental configs because they are used to modify the configuration of the Live Build tool itself.
lb config issued by itself will build up a default config and fill out most of the relevant sections for us. I usually start from this basic framework and make my modifications on it. A look at man lb_config will show the entire list of lb config options. I refer to this often.

I am not going to cover everything in that man page, partly because I don't yet understand every option and partly because we aren't going to need them all at this point.

lb config --apt <--- Specify whether to use apt or apt-get. I set it for apt-get as I am more familiar with the tool.
lb config --apt-ftp-proxy <----- If we need to setup proxies
lb config --apt-http-proxy
lb config --apt-indices <----- Whether to allow apt or apt-get to use indices. Not using them
allows for smaller installs at the expense of having to rebuild them
post install.
lb config --apt-options <----- Modify apt or apt-get options in some custom way.
lb config --apt-recommends <----- I set this to false for my installs. I have three options here, true, false, or use custom
apt-pinning on a per package basis to turn it off for specific packages. I chose to set it to
false and turn off recommends for now
lb config --apt-secure <----- I left at default but it will need to be set to false if using unsigned packages.
lb config --architecture <----- I left it at amd64. I could do a 32 bit also. However, an arm, ppc or whatever build would have to
be done with a machine running Debian on that architecture.
lb config --binary-images <----- Should be set to iso-hybrid. That's what we want for a Live iso.
lb config --config <----- This option will allow you to pull a config environment in from git.
lb config --debian-installer <---- Set this to Live to ensure that Debian installer copies your Live system over on install.
lb config --debian-installer-gui <----- A matter of preference.
lb config --distribution <----- Can be any valid Debian enviroment, IE, testing, stable, unstable, wheezy, jessie, sid.
lb config --firmware-binary
lb config --firmware-chroot <----- Set these both to true to ensure non-free firmwares get loaded.
lb config --grub-splash <---- This option is supposed to allow for a custom grub splash. I couldn't get it to work but found a work
around.
lb config --init-system <----- If you are a systemd hater.
lb config --interactive <---- lets you pause after building chroot to do stuff from the command line. Use exit to unpause the build.
lb config --iso* <---- All of the options let you modify the metafile info for your particular brand
lb config --memtest <---- memtest86+ is a good idea to include in the build.
lb config --archive-areas <--- Set these to non-free and contrib if including non-free packages, including firmware. If not set and you
include non-free packages, the build will abort.
So basically, from my base configuration, I issued these commands to build up my environment. lb config also gives you an auto folder inside your build root. IE if you chose live_build to build your live build then auto would be in -->live_build-->auto.
You can then build up a file as follows that will build your configs up for you

 Edit auto/config, adding any options as you see fit. For instance:
342

#!/bin/sh
lb config noauto \
     --architectures i386 \
     --linux-flavours 686-pae \
     --binary-images hdd \
     --mirror-bootstrap http://ftp.ch.debian.org/debian/ \
     --mirror-binary http://ftp.ch.debian.org/debian/ \
     "${@}"

I was having some difficulty getting this to work initially and resorted to issuing my lb config commands one-by-one. Note: you only need to build this up once, especially if you push your configs to git.

tknomanzr · 2015-03-02 01:45:32

Section 3: The Desktop Environment

The final step is to build our desktop environment. The first, and possibly most straight-forward portion is to list the packages we need to install. From your Live Build folder

cd configs
mkdir package-lists
lb config

lb config should notice that you created the package-lists folder and supply you with a file called live.list.chroot. To list all of the packages for my install, I opened this folder and listed out my packages one per line. Below is a sample based off of my install.

live-boot
live-config
live-config-systemd
dbus
eject
sudo
policykit-1
udisks2
xorg 
wireless-tools
firmware-linux-free
firmware-linux-nonfree
firmware-iwlwifi
firmware-ralink
firmware-ipw2x00 
firmware-realtek
firmware-atheros
firmware-brcm80211
intel-microcode
amd64-microcode
user-setup
sakura
network-manager-gnome
network-manager-openvpn-gnome 
network-manager-pptp-gnome
network-manager-vpnc-gnome
e2fsprogs
xfsprogs
reiserfsprogs
reiser4progs
jfsutils
ntfs-3g
fuse
gvfs-fuse
fusesmb
openbox
lightdm 
obmenu 
pcmanfm
python-xdg 
tint2 
suckless-tools
nitrogen
hsetroot
conky-all 
compton 
xfce4-power-manager
geany 
geany-plugins 
lxappearance
lxappearance-obconf
xfce4-notifyd
libnotify-bin 
gksu
zenity
arandr 
xinput
ntp
curl
xsel
xdotool
htop
fbxkb
scrot
fonts-dejavu
fonts-droid
ttf-freefont
ttf-liberation
ttf-mscorefonts-installer
xarchiver
dmz-cursor-theme 
gtk2-engines-murrine
gtk2-engines-pixbuf
gtk2-engines
alsa-base
alsa-utils
volumeicon-alsa
iceweasel
flashplugin-nonfree
gnome-keyring
bash-completion
lintian
avahi-utils
avahi-daemon
libnss-mdns
gvfs-bin
rsync
anacron
usbutils
wmctrl 
menu
bc
screen
whois
ftp
rpl
openssh-client
sshfs
cpufrequtils
debconf-utils 
apt-xapian-index
build-essential
unrar
unace
unalz
unzip
lzop
rzip
zip
xz-utils
arj
bzip2
p7zip

Remember, this is not Bunsen Labs and my list may or may not be 100% correct. I have attempted to create as complete a list as possible for Bunsen Labs and have that file stored somewhere.
To install custom BunsenLabs packages, we will need to consider whether they are coming out of a repository or not. If they are, we can include something similar to this in our environment config:

You may add more repositories, broadening your package choices beyond what is available in your target distribution. These may be, for example, for backports, experimental or custom packages. To configure additional repositories, create config/archives/your-repository.list.chroot, and/or config/archives/your-repository.list.binary files. As with the --mirror-* options, these govern the repositories used in the chroot stage when building the image, and in the binary stage, i.e. for use when running the live system.
391

For example, config/archives/live.list.chroot allows you to install packages from the debian-live snapshot repository at live system build time.
392

deb http://live-systems.org/ sid-snapshots main contrib non-free

Finally, if we need to pull them off of git and install them locally, that can be accomplished like so:

you may place package files in your config/ tree, which is well suited to testing of new or experimental packages before they are available from a repository.

Next comes the theming. In order to do this, I chose to populate /etc/skel and in some cases, /usr/share. There is some discussion going on currently about moving /etc/skel stuff to /usr/lib, possibly. If so, it can still be done this way.
so let's cd into includes.chroot, then create /etc/skel and /usr/share:

Notes about /etc: you will want a lightdm config here. IE lightdm --> keys.conf; lightdm.conf; lightdm-gtk-greeter.conf; users.conf.
The gtk-greeter should specify background, gtk theme, icon theme and font settings.
Notes about /etc/skel/.config: be sure compton, gtk-3.0, nitrogen, openbox, and tint2 are here. This is the basic desktop.
Notes about /etc/skel This is a mirror of the user's home directory. Place conky configs, gtkrc-2.0, gtkrc.mine, and possibly .bashrc here. Also place the following empty folders here: backup, bin, Documents, Downloads, Music, Pictures, possibly templates with the templates placed in there, and Videos.

Notes about /usr/bin: any binaries or executable shell script you place here will show up in your install. For instance, since I have not yet tackled Debian package building, I placed tint2restart, conkywonky, and my version of cb-exit here and they are available in your Live environment as well.

Notes about /usr/share: place a backgrounds folder with any backgrounds you want, a fonts folder with any custom fonts, an icons folder with the icons you the build to come with, and themes with your gtk themes.

Notes about Grub Splash As I noted above, I was having some difficulty theming grub. I did however find a workaround. One the machine you are doing the build with, look in /usr/share/live/build/bootloaders. Back this stuff up and theme it however you want. Your changes will show up in your Live Build.

Once all this is set up, issue lb build and wait awhile. If all goes well, sometime later you will end up with a live iso you can test out in a VM or dd to a usb stick and test out.

I hope this helps to shed some light on the Live Build process. Aside from the manual and the man pages, I have been unable to find much good information on the net regarding this tool. It is powerful, seems finicky, but is highly configurable. I am going to try and clean up my git repository some and post the configs alone into it somewhere. While this is not specifically how things will be done while building BunsenLabs, I think it provides a goo starting point.

zazen23 · 2015-03-03 02:04:04

tknomanzr thank you for taking the time to post to-do.

Ozitraveller · 2015-03-03 02:09:35

Could I ask what version of live-build you are using? I can't tell from the information here.

tknomanzr · 2015-03-03 02:12:49

I started out using the version from Sid. My sid box sadly needs some work on the hardware end and it cannot handle the work-load that live build puts on it atm. So I ended up moving it to my Jessie Netinstall. Truthfully, I hadn't noticed any difference between the two. I could pull from .git I guess and attempt to build. I xas sorta waiting til they get EFI boots working in it to test it out.

Last edited by tknomanzr (2015-03-03 02:13:20)

damo · 2015-03-03 02:22:58

tknomanzr wrote:

...sorta waiting til they get EFI boots working in it to test it out.

@spacex posted this earlier (if you want to test it now )

Ozitraveller · 2015-03-03 02:23:54

I'm doing live-builds on sid using 4.0.5-1 without issue.

I have a build script in the same folder as the auto and config folders. Notice the "lb init --distribution" line. This was added in live-build 4.x.
Q: How to live-build sid?

Jessie is version 4.0.3-1.

build.sh

#!/bin/sh

# sudo lb clean --purge
# sudo lb clean noauto --all

sudo lb clean
sudo lb init --distribution jessie
sudo lb config
sudo lb build

I'm also waiting for the UEFI hook/scripts to be finished and included.

I also understand that the devs are working to move to python instead of bash scripts.

Ozitraveller · 2015-03-03 02:32:12

lightdm is also here: /usr/share/lightdm/lightdm.conf.d/01_debian.conf

# Debian specific defaults
#
# - use lightdm-greeter session greeter, points to the etc-alternatives managed
# greeter
# - hide users list by default, we don't want to expose them
# - use Debian specific session wrapper, to gain support for
# /etc/X11/Xsession.d scripts

[SeatDefaults]
greeter-session=lightdm-greeter
greeter-hide-users=false
# user-session=xfce
session-wrapper=/etc/X11/Xsession

The developer has split lightdm.conf into 2.

I would suggest that you install the debian rc1 somewhere, if you can.

Last edited by Ozitraveller (2015-03-03 02:32:41)

tknomanzr · 2015-03-03 02:41:30

Ok. I will see what I can move around. My sid laptop needs some new heatsink paste and possibly ripped out of it's case and given some Lifehacker treatment to solve it's overheating issues. mksquashfs overheats it 4 out of 5 builds. It has been relegated to light duty until I am able to do some work on it.

I inherited an HP g7 that has barely been touched. It is running Jessie off of USB atm. I could probably put sid on it's internal drive once I get my dad's password cracked and see what is on the Win 8.1 install, if anything.

tknomanzr · 2015-03-03 04:33:44

damo wrote:

tknomanzr wrote:
...sorta waiting til they get EFI boots working in it to test it out.
@spacex posted this earlier (if you want to test it now )

Thanks. I'm gonna try this out soonish. I guess I need to figure out a way to keep tabs on Live Build's progress with EFI as well. It may take a bit before we would be fully able to support a Live EFI build. Depends how fast they can release stuff, I guess. I just have a feeling BunsenLabs Hydrogen may hit release stage before all of that stuff ends up finalized. We will see though as Jessie still has a lot to be ironed out. I ran into some grave bugs in Network Manager the other day.

damo · 2015-03-03 04:53:21

tknomanzr wrote:

...BunsenLabs Hydrogen may hit release stage before all of that stuff ends up finalized. ....

That's OK - we can release an update when it is all sorted: an isotope of Hydrogen.... "BunsenLabs Deuterium"

tknomanzr · 2015-03-03 05:03:08

You know I would expect a wallpaper similar to Hydrogen with the color bars for that isotope

Ozitraveller · 2015-03-03 05:06:02

tknomanzr wrote:

damo wrote:
tknomanzr wrote:
...sorta waiting til they get EFI boots working in it to test it out.
@spacex posted this earlier (if you want to test it now )
Thanks. I'm gonna try this out soonish. I guess I need to figure out a way to keep tabs on Live Build's progress with EFI as well. It may take a bit before we would be fully able to support a Live EFI build. Depends how fast they can release stuff, I guess. I just have a feeling BunsenLabs Hydrogen may hit release stage before all of that stuff ends up finalized. We will see though as Jessie still has a lot to be ironed out. I ran into some grave bugs in Network Manager the other day.

Re: Assistance enabling EFI in live-build

damo · 2015-03-03 05:14:54

tknomanzr wrote:

You know I would expect a wallpaper similar to Hydrogen with the color bars for that isotope

Already in hand I am making an image where I can add layers as masks so the relevant frequencies show through - one for each element and isotope (the first few anyway). I'm thinking ahead O:)

tknomanzr · 2015-03-25 17:59:08

I wanted to report success with Live Build using a Live Install method. There is currently a bug in the installer as mentioned in another thread. This suggestion from Debian Live list-group is what worked for me

What I ended up haing to do was:

On 03/17/2015 11:32 AM, debian@tutanota.de wrote:
> I built several Jessie images with the live tools, but when I choose the
> "Install" option from the boot screen I found no support for the ext
> file systems in the partitioning screens. I tried many configurations
> but with the same results.
> 
> Is this a known issue?

I heard a report from a user on #debian-live @ irc.oftc.net that
building with --debian-installer-distribution daily works around the
problem for now. You should revert to using
--debian-installer-distribution jessie after the issue is resolved in
the next release candidate, as dailies are not stable and therefore are
prone to other kinds of unexpected breakage.

Ben

As mentioned above, using a daily build is not precisely stable but hopefully, RC2 will bring us lots of goodness.

I still have some configs to work out and .xsession-errors and /var/log/syslog made pretty apparent some stuff that is needed if no-install-recommends is set to false, to wit:

Locales is not setup correctly, I need to look into this.
Accessibility not installed, though I know how to set this up.
Same for polkit related automounting and ejecting.
sendmail did not get installed. Cron wants it.
A few more minor tweaks to various settings need to be updated.

Overall, I consider the install a success though. It is very light at 742 mb, with just the basics: terminal, file-manager, editor and web-browser, along with all the necessary packages to configure the system. Overall the install took me roughly 15 minutes.

Next up, I found an article on Kali Linux forums that suggests how to get an efi install working through Live Build, so that's my next goal. Secure boot will be the last thing I tackle, as the way it stands now, it's not something I would consider an average Joe wanting to take on.

Anyway, I hope this info helps.

CrunchBang Linux

SEARCH

#1 2015-02-24 08:33:42

Live Build crib sheet

#2 2015-02-24 17:46:04

Re: Live Build crib sheet

#3 2015-02-24 19:07:04

Re: Live Build crib sheet

#4 2015-02-24 20:20:53

Re: Live Build crib sheet

#5 2015-02-25 02:44:04

Re: Live Build crib sheet

#6 2015-02-25 04:12:50

Re: Live Build crib sheet

#7 2015-02-26 21:22:53

Re: Live Build crib sheet

#8 2015-02-27 04:05:28

Re: Live Build crib sheet

#9 2015-02-27 08:07:17

Re: Live Build crib sheet

#10 2015-03-02 00:25:02

Re: Live Build crib sheet

#11 2015-03-02 01:04:26

Re: Live Build crib sheet

#12 2015-03-02 01:45:32

Re: Live Build crib sheet

#13 2015-03-03 02:04:04

Re: Live Build crib sheet

#14 2015-03-03 02:09:35

Re: Live Build crib sheet

#15 2015-03-03 02:12:49

Re: Live Build crib sheet

#16 2015-03-03 02:22:58

Re: Live Build crib sheet

#17 2015-03-03 02:23:54

Re: Live Build crib sheet

#18 2015-03-03 02:32:12

Re: Live Build crib sheet

#19 2015-03-03 02:41:30

Re: Live Build crib sheet

#20 2015-03-03 04:33:44

Re: Live Build crib sheet

#21 2015-03-03 04:53:21

Re: Live Build crib sheet

#22 2015-03-03 05:03:08

Re: Live Build crib sheet

#23 2015-03-03 05:06:02

Re: Live Build crib sheet

#24 2015-03-03 05:14:54

Re: Live Build crib sheet

#25 2015-03-25 17:59:08

Re: Live Build crib sheet

Board footer