CrunchBang - Code, Design & GNU/Linux

Interesting Stuff Elsewhere

I have created a new page on my website. For lack of a better name, I have called the page "Interesting Stuff Elsewhere". The page features a list of links to "stuff" which I have deemed interesting enough to share via my Google Reader account.

The "stuff" is a hodgepodge collection of articles, blog posts, podcasts and videos; having said that, the "stuff" could be anything and the only real way to find out what the "stuff" is, is to click on the links. One thing is for sure, all the links lead to dead good "stuff" :)

See: http://crunchbang.org/elsewhere/

A(nother) Regular Expression Test Tool

I came across another regular expression test tool today. This one is an Ajax enabled regex tool which lets you evaluate regex expressions in several languages, including PHP PCRE and PHP POSIX, with instant results. You can choose which functions to use, such as match, match all, replace, split etc. I much prefer it to the similar regex tool I mentioned a couple of months ago. Everything considered, it's a very handy resource for when you are struggling with those pesky expressions.

URL: http://regex.larsolavtorvik.com/
Blog: Lars Olav Torvik - Programming and computer stuff.

Wiki Rewrite

Over the last couple of nights I have completely rewritten my personal wiki. The wiki previously used the PHP WikkaWiki wiki engine, it now uses a bunch of custom PHP scripts. The scripts are similar to those used by my blog software, Whird. I decided to perform the rewrite for numerous reasons, some of which I have listed below:

1. I was unhappy with how WikkaWiki was formatting the underlying HTML, specifically the way in which it would never use the paragraph tag, opting instead to insert break tags. While this probably sounds like a minor issue, it was really beginning to bug me.

2. I started this site [crunchbang.org] with the intention of coding all of the software/scripts myself. Therefore, and somewhat obviously, my use of WikkaWiki was always going to provide reason for my conscience to niggle me.

3. As mentioned before, WikkaWiki is very hackable; however, it was never going to be as hackable as something I had produced myself.

4. I wanted both my blog and my wiki to use the Markdown markup language. While this was not a problem for my blog [it has always used Markdown] I could not find any suitable plugins/hacks for enabling Markdown within WikkaWiki.

The rewrite is pretty much complete and is now live. I have tried to make sure any URLs used by WikkaWiki are either reused or redirected. Please feel free to drop me a comment if you notice anything funky occurring.

Create MySQL User Accounts from the Command Line

I have my LAMP install process down to fine art, which is a good thing considering the amount of Linux installs I have been performing lately. Anyhow, just recently I have been experiencing some issues with creating MySQL user accounts. Normally, for ease of use, I would simply login to phpMyAdmin and create a new user account, but, for some unknown reason, this method has not been working; maybe there is a compatibility issue between the latest versions of MySQL Server and phpMyAdmin available from the Ubuntu repositories?

Anyway, to get around this little problem, I have been creating new MySQL user accounts via the command line. It is a fairly straightforward procedure, but like a lot of administration tasks, it is not something I do on a daily basis. With this in mind, I thought it would be a good idea to record the process for future reference.

Follow the steps below to create a new user MySQL account. Note, these instructions assume that you have already set-up Apache, MySQL and PHP.

1. Open a terminal and login to MySQL as root:

mysql --user="root" --password="your_root_password"

2. Create a new user and password with the following command:

CREATE USER 'new_username'@'localhost' IDENTIFIED BY 'password_for_new_username';

3. Assign privileges to the new user, enter the command:

GRANT ALL ON *.* TO 'new_username'@'localhost';

4. Exit MySQL interface, enter:

exit

Notes: The above instructions will create a new user account on "localhost" and grant the user all privileges. While this is safe for my development environment, you would obviously want to grant only the necessary permissions in a production environment.

For more developer information, including the MySQL Reference Manual, visit: http://dev.mysql.com/

Perplexed by Web Frameworks

The latest LugRadio episode features a discussion about Django and other web frameworks. I found the feature interesting, but I have to admit that I find the subject of web frameworks somewhat perplexing. I have yet to fully embrace any such framework, although I have played with the Zend Framework. I think there are a number of reasons I have not fully adopted any frameworks:

1. I am put off by having to learn all the new classes, structures and methods employed by said frameworks. Surely my time would be better spent actually learning more about the core language?

2. I fear that using a framework would somehow stifle innovation. I am under no illusions of being the most innovative player, however, I find it hard to shake. I guess I am questioning where the innovation comes from if everyone is using the same framework?

3. It is fun to write your own code, functions, classes and routines etc. I like to experiment with code, I like to make mistakes, before fixing them — it is this that keeps me interested. I would be concerned that using a framework would takeaway much of that.

Having said all that, I'm not totally opposed to frameworks and I think they have their place; three PHP frameworks of interest to me are:

1. Zend Framework: http://framework.zend.com/
2. CakePHP: http://www.cakephp.org/
3. Symfony: http://www.symfony-project.org/

I am going to look into the above to see what they have to offer; however, I think I will probably continue to hack together my own code for a while to come.

TwitterZoid PHP Script

It's a rather silly name, I know; however, TwitterZoid is the chosen name of my PHP script for parsing Twitter RSS feeds. I've been using Twitter quite steadily for a couple of weeks now and I thought it might be nice to include my latest tweets on my blog, so I wrote TwitterZoid to do just that.

TwitterZoid differs to other PHP based Twitter RSS parsers, at least the ones I tried before I wrote it, in that it will automatically link both lexicons and URLs found within individual tweets. It also tries to mimic Twitter's timestamping, although this could be improved.

TwitterZoid usage

I wanted to make TwitterZoid as simple to use as possible. Therefore I decided to write the script to be used as a simple include file which can be used on any PHP page. Basically, to use TwitterZoid all you need to do is set a couple of variables, include twitterzoid.php and then echo the main $TwitterZoid variable where you would like your list of tweets to appear.

Example set-up:

$twitter_username = "corenominal";
$twitter_feed = "http://twitter.com/statuses/user_timeline/99713.rss";
require_once('twitterzoid.php');

Call on the main TwitterZoid variable to produce the list of tweets:

echo $TwitterZoid;

Don't worry if this reads like gibberish, I've included an example page within the download.

TwitterZoid examples

There are currently two demonstrations of TwitterZoid in action, see:

My official "What am I doing?" Twitter page:
http://crunchbang.org/what-am-i-doing/

A more stylised version of "What am I doing?", included within the download:
http://crunchbang.org/projects/twitterzoid/demo/

Download TwitterZoid

Location: http://crunchbang.org/projects/twitterzoid/twitterzoid-0.1.tar.gz
MD5: 99dace9f9872cf1ebf93588bb2d36458

TwitterZoid license

This program is free software; you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation; either version 2 of the License, or (at your option) any later version.

This program is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details.

http://www.gnu.org/licenses/

Regular Expression Test Tool

I've used this web based regular expression test tool a couple of times over the last day or so. While I don't normally struggle with regular expressions, this tool has still come in handy; it has saved me from the "code it and cross your fingers" approach I normally take.

This is a great tool for anyone who works with PHP, especially as the service uses PHP regular expression functions as a base for its operations.

URL: http://www.solmetra.lt/scripts/regex/

Wikka Wakka Wiki

Over the last couple of days I have mainly been playing with WikkaWiki, a flexible, standards-compliant and lightweight wiki engine written in PHP. I've toyed with the idea of creating a personal wiki for sometime, but, I've been reluctant to start one for fear of it becoming just another thing to update and maintain. Anyhow, I now seem to have conquered this fear and I've created my first wiki.

Now, being a new wiki it naturally doesn't contain much of any content, but, at this moment that's not too important, the important thing is that I've started it, the content will come later. I'm planning on using the wiki to document my Ubuntu/CrunchBang Linux projects. I'll also be using it to privide supporting supporting material for my blog. For example, stuff like this crontab howto, and these GIMP brushes would be better suited to the wiki.

Regarding the WikkaWiki software, it's quite impressive and fairly easy to work with. I looked at several other PHP based wiki projects, but IMHO WikkaWiki was the best — apart from maybe MediaWiki, but then I considered that to be overkill for my needs. If you're starting a new wiki then I'd certainly recommend at least looking into the possibility of using WikkaWiki. It has many great features, some that I noticed include:

• a super easy installation process
• a nice default design and easy to customise CSS
• code base is very hackable, including actions/plugins
• RSS feeds for recently updated content, comments etc
• syntax highlighting of published code, courtesy of the GeSHi plugin
• it's super scary fast

All-in-all WikkaWiki has impressed me a lot; besides, if it's good enough for Puppy Linux, it's good enough for me ;)

PHP Function: Word Filter

So, last night I was mainly experimenting with Whird, more specifically I was trying to build an internal search feature. This is something I've been putting off for long enough, and I'd really like to get it coded up. I didn't finish it last night, but at least I've made a start.

Anyhow, during my experimentation I wrote this little PHP function for filtering words out of a string. Basically the function takes 2 strings as arguments before filtering words out of the first string based on words found in the second. I've posted it below for future reference:

function word_filter($string1,$string2){
    $string1 = trim($string1);
    $string1 = preg_replace('/s+/', ' ', $string1);
    $string1 = explode(" ",$string1);
    $wordcount = count($string1);
    $i = 0;
    while ($i < $wordcount) {
        $string = $string1[$i];
        if (strstr(strtolower($string2),strtolower($string))) {
            $string1[$i] = "";
        }
    $i++;
    }
$string1 = implode(" ",$string1);        
return($string1);
}

Example usage

This is probably not the best example, but this:

$poem = <<<EOD
<p><em>GIVE me women, wine, and snuff <br />
Untill I cry out "hold, enough!" <br />
You may do so sans objection <br />
Till the day of resurrection: <br />
For, bless my beard, they aye shall be <br />
My beloved Trinity.</em></p>
EOD;

$common_words = 'and be but do for is it may me my not of the they there so was you';

echo word_filter($poem,$common_words);

Would output something like this:

GIVE women, wine, snuff
Untill cry out "hold, enough!"
sans objection
Till day resurrection:
For, bless beard, aye shall
beloved Trinity.

John Keats would be proud, not! Please feel free to optimise, or let me know if a one line equivalent already exists :)

Parsing RSS feeds with PHP & SimplePie

Tonight I have mainly been playing around with SimplePie, a PHP class for parsing RSS and Atom feeds. To quote the SimplePie site:

SimplePie is a very fast and easy-to-use class, written in PHP, that puts the "simple" back into "really simple syndication". Flexible enough to suit beginners and veterans alike, SimplePie is focused on speed, ease of use, compatibility and standards compliance.

I'm totally impressed with SimplePie and I'll certainly consider using it within future web projects. I performed a quick test with it tonight by parsing a couple of feeds and displaying the results on my philipnewborough.com domain, it worked a treat. Good stuff.

Cruciforum - Crucially Simple

Stuart Langridge has updated his really simple forum software, Cruciforum.

Cruciforum is really, really simple. There's no database required to run it — you don't need MySQL or anything — it doesn't have user accounts, it doesn't need an administrator. It's really low-impact on your server because all the pages are plain HTML — if someone's browsing the forum then they don't even use PHP! Just fire it up and go; all you need on the server is PHP support, which most things have.

Cruciforum is built with Joel Spolsky's thoughts on forum design in mind, and a few other tweaks that might be interesting. It's designed to be crucially simple.

I really like simple web software [the simpler the better if you ask me] and Cruciforum is about as simple as it gets. I really like the fact that there is no need to create an account before you can post a message.

Cruciforum also features built in support for Akismet spam protection [which I guess is a must when there's no account system.] Textile is used for converting user input and Gravatar support is also a nice touch.

The only thing missing at the moment is support for RSS — this has been recognised and there's a bug report here.

I can see many uses for Cruciforum, it has a lot of potential. Check it out here: http://www.kryogenix.org/code/cruciforum/

Ternary Operator in PHP

Over on Planet PHP there's a running debate [see here, here & here] over the use of the ternary operator. I love a good debate so I thought I'd chip in with my 2 pennies.

The ternary operator looks attractive and can reduce the amount of code that you write. I know this but I still don't use it. I think PHP is an awesome language and a major contributing factor to its awesomeness is its simplicity. It's a relatively easy language to learn and I think that the wide spread use of the ternary operator would only increase the barrier to entry [not a good thing!]

Also, I find it interesting that the ternary operator is hardly ever referenced in the comments and code examples at PHP.net. Indeed chapter 16 "Control Structures" of the manual hardly mentions it at all.

Ternary operator code example

In case you've no idea what this post is about.

Before: no ternary operator, easy to understand

if ($treat == 'cream') {
    $cat = 'Happy!';
} else {
    $cat = 'Not so happy.';
}

After: ternary operator in use, less code but not so easy to understand

$cat = ($treat == 'cream') ? 'Happy!' : 'Not so happy.';

PHP 5 & Feed Updates

It's been a long time coming but my website [CrunchBang.org] is now finally running on PHP 5. My hosting provider performed the upgrade earlier this month and made the switch optional on a domain basis. Any domain on their servers can run either PHP version 4 [default] or upgrade to version 5 by adding a script handler in .htaccess:

Action php5-script /interpreters/php5-script
AddHandler php5-script .php

My development systems all run PHP 5 so the switch was a breeze. Hopefully from now on I'll be able to avoid stuff like this.

Also, I've been working on my Whird project over the weekend. I've now added feeds for individual tags and updated various features to improve usability. I've had to place some URL rewrites for the new feeds and I'm hoping that the various planets [Planet Ubuntu Users, Planet Ubuntu UK] don't get flooded — I apologise if they do :)

Howto Set-up a Crontab File

Yesterday I mainly worked on some PHP backup scripts [not very exciting, but necessary.] The scripts are run at regular intervals and perform various data backups across numerous domains. To accomplish this I set-up some cron jobs that will automatically execute the scripts at set intervals.

This is not the first time that I've had to set-up cron jobs [I've got several that date back a few years.] I think that having the ability to set-up scheduled tasks is really quite important. Therefore I thought I'd blog about it for future reference.

What is cron?

Cron is a daemon/service that executes shell commands periodically on a given schedule. Cron is driven by a crontab, a configuration file that holds details of what commands are to be run along with a timetable of when to run them.

Creating a crontab file

You can create a crontab file by entering the following terminal command:

crontab -e

Entering the above command will open a terminal editor [Ubuntu uses Nano by default] with a new blank crontab file [or it will open an existing crontab if you already have one.] You can now enter the commands to be executed [see syntax below] before saving the file and exiting the editor. As long as your entries were entered correctly your commands should now be executed at the times/dates you specified. You can see a list of active crontab entries by entering the following terminal command:

crontab -l

Crontab syntax

A crontab file has six fields for specifying minute, hour, day of month, month, day of week and the command to be run at that interval. See below:

*     *     *     *     *  command to be executed
-     -     -     -     -
|     |     |     |     |
|     |     |     |     +----- day of week (0 - 6) (Sunday=0)
|     |     |     +------- month (1 - 12)
|     |     +--------- day of month (1 - 31)
|     +----------- hour (0 - 23)
+------------- min (0 - 59)

Crontab examples

Writing a crontab file can be a somewhat confusing for first time users [and the above table probably doesn't help much!] Therefore I've listed below some crontab examples:

* * * * * <command> #Runs every minute
30 * * * * <command> #Runs at 30 minutes past the hour
45 6 * * * <command> #Runs at 6:45 am every day
45 18 * * * <command> #Runs at 6:45 pm every day
00 1 * * 0 <command> #Runs at 1:00 am every Sunday
00 1 * * 7 <command> #Runs at 1:00 am every Sunday
00 1 * * Sun <command> #Runs at 1:00 am every Sunday
30 8 1 * * <command> #Runs at 8:30 am on the first day of every month
00 0-23/2 02 07 * <command> #Runs every other hour on the 2nd of July

As well as the above there are also special strings that can be used:

@reboot <command> #Runs at boot
@yearly <command> #Runs once a year [0 0 1 1 *]
@annually <command> #Runs once a year [0 0 1 1 *]
@monthly <command> #Runs once a month [0 0 1 * *]
@weekly <command> #Runs once a week [0 0 * * 0]
@daily <command> #Runs once a day [0 0 * * *]
@midnight <command> #Runs once a day [0 0 * * *]
@hourly <command> #Runs once an hour [0 * * * *]

Multiple commands

A double-ampersand "&&" can be used to run multiple commands consecutively. The following example would run command_01 and then command_02 once a day:

@daily <command_01> && <command_02>

Disabling email notifications

By default a cron job will send an email to the user account executing the cronjob. If this is not needed put the following command at the end of the cron job line:

>/dev/null 2>&1

Specifying a crontab file to use

As mentioned at the top of this post, you can create a new crontab file with the "crontab -e" command. However, you may already have a crontab file, if you do you can set it to be used with the following command:

crontab -u <username> <crontab file>

Therefore the following command…

crontab -u tux ~/crontab

…would set Tux's crontab file to that of the file named "crontab" residing in Tux's home directory.

Removing a crontab file

To remove your crontab file simply enter the following terminal command:

crontab -r

Further information

Refer to the man page for further information about crontab. Enter the terminal command:

man crontab

External links

Some external links for your browsing pleasure:

• Cron article on Wikipedia
• Ubuntu Documentation - CronHowto
• Debian Help - Scheduling Backup Jobs using at and crontab in debian
• Matt Cutts - How to fetch a url with curl or wget silently
• Sitepoint - Getting PHP and Cron to Work Together

I think that pretty much covers the subject of cron jobs and crontab. Please feel free to comment if I've missed anything/made any boobs.

User Agent Sniffer

I've currently got several web projects at various stages of development. One thing that all of these projects have in common is that they all capture and manipulate user-agent stings.

What are user-agent strings?

User-agents strings are used by client applications such as web browsers, feed readers, bots and other software to identify themselves to the servers they are connecting to. The strings contain important information such as application type, version, language and operating system. A typical user-agent string might look like this:

Mozilla/5.0 (compatible; Konqueror/3.5; Linux) KHTML/3.5.7 (like Gecko) (Kubuntu)

The above example is used to identify that the client is a Mozilla compatible Konqueror web browser running on Kubuntu Linux.

Collecting user-agent strings

To run tests on my projects I need some sample data to play around with [a list or database table of user-agent strings.] I figured that the best way to get this sample data would be to collect some user-agent strings from the wild. So, I wrote a quick PHP script to do just that.

I ran the script on this site [crunchbang.org] for ten days, starting on October 7th 2007. During that ten day period the script captured a total of 2272 unique user-agent strings. The captured list included both standard and non-standard strings.

A few facts about user-agent strings

After capturing the list I then edited the PHP script so that it would report a few facts. Here is a breakdown of what was returned:

1. At just 6 characters in length the shortest user-agent string captured was:

NG/2.0

2. The longest measured in at 205 characters, it was:

Mozilla/5.0 (compatible; MSIE 7.0; Windows; HTMLAB; .NET CLR 1.1.4322; 
MEGAUPLOAD 1.0; Seekmo; ZangoToolbar4.8.2; Alexa Toolbar; Hotbar 4.2.8.0) (compatible; 
Googlebot/2.1; +http://www.google.com/bot.html)

3. The average computed length of the user-agent strings was 91.2750880282 characters.

4. Most strings contained some non-alphanumeric characters, these were:

/ . ( ; - : ) + _ ! = , @ &  ' [ ] * ~ ? { }

5. The strangest user-agent string was:

Mmm.... Brains....

View the whole report

You can view the whole report here: http://crunchbang.org/misc/user-agent-report-2007-10-17.txt

Get the sample data

I thought it would be good to share the sample data. There's no private or confidential information in the data and I figure it may come in handy for other developers working on similar projects.

You can get the data as an ASCII file [one user-agent string per line] here: http://crunchbang.org/misc/sample-user-agents-ascii.txt

Or, as an SQL statement here: http://crunchbang.org/misc/sample-user-agents-mysql.txt

Get the PHP script

If you fancy having a go at collecting your own samples you can grab my PHP script here: http://crunchbang.org/misc/ua-sniffer.txt

The script requires the use of MySQL. Other than that it's a fairly straightforward affair. Just edit the four settings to define your database name, address, username and password.

I ran the script by calling it with a require_once statement. Note that the script also sets and reads a cookie so you'll need to call on it before outputting any data to the client.

require_once("ua-sniffer.php");

Once the script has collected some user-agent strings it is possible to query it and have it produce a basic report. You can do this by accessing the script through your browser like so:

http://www.example.com/ua-sniffer.php?report=true

Links to external references

• User Agent article on Wikipedia
• List of User-Agents at user-agents.org
• Another list of browser ID strings
• User agent string analysis

301 Redirect with PHP

I've just had to permanently redirect an entire site. The site consisted of several hundred pages and on the face of it, it seemed like a daunting task. However two lines of PHP came to the rescue :-)

 Header( "HTTP/1.1 301 Moved Permanently" );
 Header( "Location: http://www.new-location.com" );

Luckily, each page on the site referred to a single configuration file and this made the process painless.

PHP: strrpos() vs strpos()

I normally wouldn't post about such trivial matters, however this particular trivial matter bugged me for several hours this afternoon, so I thought I mention it.

It seems that one of the problems of coding on a development server that runs PHP 5 for a production server that runs PHP 4 is the subtle differences in the language. These subtle differences can really throw a spanner in the works [like one did to me this afternoon.]

Basically my development code used the strrpos() function to attempt to find the first occurrence of a string within a string. While this worked just fine with PHP 5, it bombed big time with PHP 4 [which will only find the first occurrence of a single character.] Apparently I should have been using the strpos() function instead!

Oh well, live and learn.

7 Reasons He Switched Back to PHP

I didn’t abandon the rewrite IDEA, though. I just asked myself one important question:
“Is there anything Rails can do, that PHP CAN’T do?”
The answer is no.

Derek Silvers writes a good article with 7 good reasons as to why he switched back to PHP after 2 years of coding Ruby on Rails.

I've always liked the idea of learning Rails, [is it just me or has Rails been portrayed as a sexy/trendy language?] but I've never started because PHP has always been good enough for my own projects. I'm glad I didn't waste 2 years learning another language just to find that out!

My favourite reason from the article:

7 - PROGRAMMING LANGUAGES ARE LIKE GIRLFRIENDS: THE NEW ONE IS BETTER BECAUSE YOU ARE BETTER