That’s why this page has always been subject to relatively long pauses without updates, even if I have felt like I’d like to share something. Thinking about alternative solutions, the social media websites, I’ve had a Facebook and Twitter account for quite a while now. I have always felt that Facebook suits best for personal friends and not for public stuff. Twitter is ok for public stuff, but the 140 char limit means that you’ll share only some tiny things, like a little anecdote or just a link to somewhere.

When Google+ came around, I started using it as well. By nature it is like a combination of good things from both Facebook and Twitter, and seems to fit my usage patterns quite well. It allows really easily to select per-post who I’m sharing it with. It doesn’t limit me to short posts when I want to say a little more. Each post is automatically a discussion thread, and since it isn’t hosted by me, I have less to worry about approving posts / preventing spam entries.

So I realized I tend to write there a bit more often – the smaller things, findings and even some more light-hearted observations go there. So, if you want to follow smaller updates from me, join Google+ if you aren’t yet there and add me to your circles. Today, I was originally going to write about my latest findings regarding prototyping with Processing (again), but decided I probably do that in my Google+ stream later and write this endorsement post here instead! ;-D

A common technique for me has been to just make zip files of whatever I want to backup. Simple snapshots of a project, so I end up with bunch of “projectname-YYYYMMDD.zip” files. Or altenatively a common zip of certain folder which gets updated over time. For example, before I moved to using cvs or svn for source code version control, I used a batch file like this to backup all my little coding projects:

backup.bat
@zip -u -r d:\backup\prog_c * -x *.ilk *.pdb *.idb *.ncb *.opt *.plg *.bsc *.sbr *.pch *.obj *.zip *.dll


As you can see, this just updates an existing zip and excludes bunch of unnecessary files which I didn’t want to keep bloating up the backup zip. A worthy note is that I have usually made sure I have two separate hard disks in a computer and put the backups on the other one from the original data. At some point I also tried to switch to using dar instead of the zip files, since it would have had some advantages over the zip files. I didn’t end up using that for long though.

Like I hinted above, nowadays I’m using version control for source code, so I don’t make such zips that frequently anymore. Instead I have made sure that the version control repository gets backed up somehow.

My first solution was to make nightly snapshots of the repository using a script on the server, and then occasionally copy a snapshot file back to my main computer. I used this solution for many years. Below is copy of the backup script I used, which creates a .tar.gz backup file with given prefix from a specified file mask, adding YYYY-MM-DD timestamp to the backup file name, and also keeps only given number of snapshots around cleaning up any older ones.

dobackup.sh
#!/bin/sh

if [[ ! $1 || ! $2 || ! $3 || ! $4 || ! $5 ]]
then
echo $0 destdir name execdir mask backups
echo $'\t'destdir = directory to put backups in
echo $'\t'name = file prefix name of this backup
echo $'\t'execdir = directory to run the backup tar command in
echo $'\t'mask = filemask to give for tar
echo $'\t'backups = number of backups of name to keep in destdir
exit
fi

DESTDIR=$1
NAME=$2
EXECDIR=$3
MASK=$4
BACKUPS=$5

DESTFILE=$DESTDIR/$NAME-`date +%F`.tar.gz

cd $EXECDIR
tar zcf $DESTFILE $MASK

EXTRAFILES=$((`ls $DESTDIR/$NAME-* | wc -l` - $BACKUPS))

if [[ $EXTRAFILES > 0 ]]
then
rm -f `ls -t $DESTDIR/$NAME-* | tail -$EXTRAFILES`
fi


I had then another script which called the above one to create backup files from a few places in the system, and was put in a cron job to be run once per night. The backup script looked something like this:

allbackups.sh
#!/bin/sh
# All automatic nightly backups
cd /home/user/bin
BACKUPDIR=/var/backups/user
umask 007
# /home/share (includes version control repository)
./dobackup.sh $BACKUPDIR share /home share 7
# misc stuff
./dobackup.sh $BACKUPDIR misc /home/user "bin bookmarks.html code" 7


More recently I have also moved to mirroring stuff over to another computer using rsync. This was something I needed to do to create an off-site remote backup for my workplace’s version control repository. To do this, I have simply put ssh-keys in place to allow automatic remote ssh login from a certain machine to run rsync over it, and then I have a cron job line like this to update the mirror every night:

nightly rsync mirror for cron job file:
30 5 * * * /usr/bin/rsync -avz -e ssh user@server:/path/* /backup/mirror/ > /backup/log/last_cron_mirror.txt 2>&1


Additionally I’m also backing up some paths of this web server every now and then using simple tar commands in a script, and the following command to back up a dump of the mysql database:

compressed mysql dump, must be run as root:
/usr/bin/mysqldump --defaults-file=/etc/mysql/debian.cnf -A -Q --opt | /bin/bzip2 > ~/db-mysql-`date +%Y%m%d`.sql.bz2


Finally, since there are multiple computers in our household, we have also been simply copying manually some files over to a share in another computer to have at least a single duplicate. These days I have also set Windows 7 backup software to maintain a backup of certain folders and system state to a share in my home Linux server. In the past I also sometimes made manually a backup of my Windows XP computer’s system state using the OS backup tool. Also recently when a brand new hard disk failed for me, I bought another one and after getting the warranty replacement I put the two into a mirrored RAID disk.

I have experienced so many hard disk crashes over time so I have learned to pay at least some attention to backing up the most important stuff, even if I really still tend to be a little bit lazy about it.

Well, let’s think about what it means to build bug-free software. There actually exists some software, which is built with goal of “zero-defects”. By that, I’m not referring to the constant hype from latest agile method cult who may have some extravagant claims. I mean software where it’s of utmost importance to really have no bugs, no matter the costs. A case where a single bug may mean loss of lives or failure of the whole project. This kind of software is built for space missions, nuclear facilities, and so on. And such a policy on quality is inherently very very expensive.

Imagine if Windows, for example, would have been built with such policy. It’s a really complex and huge piece of software compared to ones normally built with zero-defect policy. But, given the option, would you pay $19,900 of your operating system, if it would never crash? It’s quite a no-brainer that “everybody” would rather pick the same software for $199 and just put up with the few problems they encounter. (The price difference was picked by Stetson-Harrison method. It’s a guess to illustrate the point. The difference could be actually lot worse for niche market software which isn’t aimed for mass market.)

I was away for about 2.5 weeks to visit Japan, more exactly Kyoto and Tokyo. The trip was excellent, and in retrospect it was clearly a good idea to visit both cities. While Kyoto is a modern city, it still has a bit more traditional feel to things. For example, the local laws forbid building tall skyscrapers, so there aren’t much taller buildings than we have in Helsinki.

Tokyo in other hand is very modern city with lots of skyscrapers. When going to see the city view from top of Mori building in Roppongi hills, you can see the city expands in every direction to the horizon as far as you can see. There is also extremely well developed public transportation, just check the Tokyo subway map (PDF). The public transportation in Helsinki is also quite good, but Helsinki is so tiny compared to a big metropolitan city so that our subway is tiny as well – just compare the above one to the current Helsinki metro map. ^_^

Interestingly when going to USA it feels that everything is bigger. When going to Japan, it feels the opposite. Maybe that’s partly because there is a whole different class of small cars which are narrower than we’re used to. I think the average width of streets is a bit less as well. There are also typical big western cars, so there’s just more variety of different sized cars. There’s even mini garbage trucks as you can see. However, not everything is small; I think in Japan they just tend to make things small by default, but aren’t afraid of making things big when there’s a reason to do so.

Also some things seemed to be done in an opposite way of what we’ve used to. Not just the left-hand driving and walking, but cars are also almost always parked backing to parking spot so that it’s slower to park car but easy to leave. And at least in Kyoto the buses worked so that you go in from a single back door and pay when you exit through the front door. Curiously in Japanese language the verb is also put last in sentences (think of Yoda-speak).

When writing you sometimes want to put a sidenote in parentheses, and then end it with a smiley.

This raises a question: Should you write one or two ending parentheses?

This seems to be an awkward problem for many. XKCD even made a comic about it (link at end of this post).

My solution is to use just a single parenthesis, so the ending parenthesis becomes part of the smiley. It just looks better that way! However, this way really sucks if you are writing in e.g. some chat system which automatically replaces smileys with small graphic icons. In that case the ending parenthesis just disappears and it feels like the sidenote keeps going on and on and the whole paragraph starts feeling unbalanced. So, if I know that the system will replace my smileys with icons, I will use double parenthesis.

At this point you’re probably wondering why I don’t show an example? I often overuse such sidenotes for some remarks, which is of course the reason I have even realized this earth-shattering problem. However, I have also realized that the best way to fix it is actually just to not use such sidenotes. Often the sidenote may actually be just a natural thing to say in the next sentence. Or perhaps you can consider using a footnote. I feel footnotes don’t work that well in web pages but I prefer them in print publications.

Anyway, I should save you the trouble (of constructing examples yourselves :).
Or maybe you already tried yourself (I know you did! :-) ).

XKCD comic

In other “news”, I didn’t end up making an entry for the last weekend’s Ludum Dare 48h game dev compo. The theme was “Exploration” – I wasn’t really fond of the theme, but I still had a solid idea very quickly I went forward with. However, I ended up honing gameplay nuances for too long in my head and without actually doing much of implementation. I ended up writing some level generator thing and the supporting draw code etc. Might come back to those later if I want to work more on the idea; or perhaps even integrate with my summer efforts for making a little game for Assembly which also turned out not to be.

On the other hand, at least there was nice chance to have sort of “role reversal” for the weekend, since I was the one who went to buy groceries and made some food, and my wife made a nice little game to entry in the same LD48 compo.

In the past years my typical use case was as follows. I used to make “64 KB intros” (such as this one), and for those I needed to embed a few data files to the executable as object data, so that I could exploit compression of the executable packer without writing a proper compression tool myself (such as UPX).

First I ended up making a simple tool which just reads in a file, byte-by-byte, and outputs something like this:
static unsigned char data[]={13,37,74,47, ... };
That worked just fine and easy – sizeof(data) can be used to get the length of the data. Writing such a tool took probably something like 5-10 minutes. I think I have actually written that same little utility multiple times in the last decade, sometimes just forgetting where the previously created version were and then just re-creating it.

However, at some point I ended up with a solution where music of the production was an XM module with generated samples. The actual playable module file was huge in size – several megabytes – but the trick was that all the samples were generated at initialization time of the application. My quick hack for this was to make a tool which read in the XM module, replaced the sample data with blank data and stored the tiny bits of metadata for sample generation. And then just embed the huge binary file, so that executable compressor would crunch it down a lot (because it’s mostly empty blank data anyway).

Everything was fine, except for the fact that the compiler&linker I used didn’t like that idea at all, declaring a compile-time handled char array of several megabytes. To workaround this, I took the previously mentioned little utility and added simple RLE compression, which was just enough of a workaround and was trivial to decode as well. Problem solved. (Note that this way of adding music to an intro is now considered ancient, and more modern way is something like this).

Recently I found a version of these two little utilities and decided maybe I can save a few minutes of somebody’s time — or at least my own time if I ever consider writing the same thing over again and happen to remember I have added those to my web page.

I called the first one “binso” for reason I can’t remember anymore. The RLE-version is simply “binsorle”. There’s also testrle.c code for testing and decompressing the data made by binsorle. Note that the code has been written no later than 2001 (possibly even earlier than that), and hasn’t been really “stress tested” to be bug-free. Any bug reports are still welcome, of course.

Download source-code and pre-built win32 executable files:
binso.zip

Supported operations include dilation, where color of alpha>0 pixels are dilated to nearby pixels with alpha=0. This is the default operation. An alternative operation is pre-multiplying colors by alpha, i.e. value of any color component won’t be larger than the alpha value after this operation.

Using either of these operations will “magically” fix many graphical glitches related to drawing images with linear interpolation. Generally using pre-multiplied alpha would be better choice, but using it requires modifications to the source code (different blending mode). That is, it should be known in the source code if image to be drawn has pre-multiplied alpha or not. Just doing the dilation operation won’t need any changes to the code, so it is helpful in many situations.

I’m using stb_image to load the images, so some other input formats should be supported as well in addition to png. However, note that only certain kind of input files are supported, e.g. trying to load 64 bpp (16 bits per color component) png files will lead to a crash. Output files are always normalized to 32 bpp (8 bits per color component) and saved in png format.

Download version 1.0, including source code and a pre-built executable for Windows:
pngprep.zip

Disclaimer: Use it for anything, but do so at your own risk.

If you’re wondering what is psdslayer, it is simply a small command-line tool which extracts each layer from a .psd file to separate png (or tga) files. It also outputs some meta information about each layer to the console (stdout), or optionally also as part of the file names of exported images. The meta info contains position and size of each layer region, which is hopefully useful for automating some part of a tool pipeline.

Grab the version 1.1 here, including source code and a pre-built executable for Windows:
psdslayer.zip

Disclaimer: Use it for anything, but do so at your own risk.

Update: The PSD reading code is limited and really handles only a small subset of possible things. Here’s hints how to make “compatible” PSD for processing:

• Use “Maximize compatibility” setting when saving PSD files
• Rasterize all special layers to ordinary layers (text, smart objects, layer effects, …)
• Collapse all groups (layer hierarchy should be completely flat)
• Use 8 bit depth per color component (16 bit per component is not supported)
• Remove all layers which contain a “very small” image, e.g. size being just a few pixels (the loader seems to have a bug with handling of these, or perhaps related to reading layers consisting of raw data rather than rle-compressed?)

Update Feb 2010: There’s also a tool called psdparse which does the same task, apparently even more robustly.