tl;dr

APRSdroid is an Amateur Radio geo-location (APRS) app for Android licensed under the GPL. It started as a Scala learning experience two years ago, and has become a nice auxiliary income since, despite being Open Source and offering free downloads from the homepage. However, using Scala was not always the easiest path to go.

Project history

In mid-December of 2009 a HAM radio friend asked me: "it can't be too hard to make an Android APRS app, can it?" and because there was none yet, I started pondering. On December 31rd, instead of having fun and alcohol, I made the first commit. January 1st, at 03:37 local time, the first placeholder release 0.2 was created.

Over the course of the next weeks I discovered step-by-step what I had encumbered myself with. APRS is a protocol with a long history of organic growth, firmware limitation workarounds, many different ways to say the same thing (at least four just for position reports), countless protocol amendments and, to add insult to injury, base-91 ASCII encoding.

There was no Java code available to abstract away the protocol and to allow me to keep my sanity. So I read the spec, implemented position encoding, re-read the spec, implemented HTTP and UDP sending code, read the amendments, re-re-re-read the spec, etc.

The first usable release became 0.4 from the end of January. Because APRSdroid always was a leisure time project, phases of activity alternated with idle phases, and the app slowly grew features through 2010.

In early 2011, one year into it, I decided it was high time to make the project pay for itself. Real APRS gear (radio transceivers with GPS and packet radio support) was expensive (on the order of US$500), and the app was not only easier to use but also grew more and more features (except for direct access to the amateur radio spectrum, which does not work well on cell phones).

For some time, I went underground (by omitting git push to github and only providing nightly builds to some friends) and worked on the code behind closed doors (there were no other people contributing source anyway).

On April 1st, I decided to fool the community a little, but was not taken too seriously. In the meantime I was polishing a 1.0 release for Android Market.

Income Report

On April 18th, 2011, APRSdroid 1.0 was "commercially" launched to Android Market. It was important for me to keep up the OSS spirit, so I kept providing source code and APK files from the home page. By buying the app instead of just downloading it, the users got automatic updates and a good feeling of supporting what they liked. Also, I did not make it too obvious in the Market description that the app can be downloaded for free as well ;-)

So far, this scheme has paid off very well. Since the beginning, more than 60% of all app users actually bought it (it is possible to monitor the global user activity on the APRS network), with an average of 350 sales per month, at 2.99€ / 4.49US$ (minus the Google "tax" and subject to local income taxes).

Most users I had contact with were ready to pay for the app even though they knew they could download it for free. Only one person so far demanded the free version to be made available on Android Market (using CAPS and three consecutive Twitter messages, though, so I did not feel too pressed).

So far, I invested the income into real APRS hardware, a Desire Z (or G2 or HTC Vision) and am eagerly awaiting the availability of ICS tablets, aiming at finally adding Fragments support to the app.

Scala + Android = Pitfalls

I decided to use Scala because I do my coding in vim and Java is so crammed up with boilerplate code that you can not sensibly use it with anything but a bloated refactoring IDE. Another reason was that I do not like to repeat myself, and Java provides even less usable abstractions than the good old C language with its #define.

Scala was the language of the day, and I liked what I had read about it so far. It sounded good enough for an experiment anyways. Fortunately, people had already figured out how to make it work on Android without carrying the bloat of the full Scala runtime, so all I had to do were some refinements of build.xml.

The first warning sign was that I had to override def $tag() to work around an issue in the 2.7 beta compiler (IIRC). I complied by cargo-cult-copying the code from some place and moved on.

Another major issue was Android's AsyncTask. The API requires the developer to override protected SomeType doInBackground(OtherType... params). Unfortunately, Scala has trouble with overriding abstract varargs methods from Java, and thus your app crashes with the opaque java.lang.AbstractMethodError: abstract method not implemented exception. After triangulating the source of the problem (who would have suspected a compiler bug?), a wrapper class in Java was written. Another bunch of days well spent.

One of my biggest hopes in Scala was to be able to reduce the boilerplate for Android's numerous single-abstract-method function parameter workarounds. Unfortunately, this problem is not yet solved in Scala, requiring to write explicit implicit conversion functions for each SAM type.

However, not everything was bad in Scala-land. Scala's traits allowed to reuse the same code in descendants of Android's Activity, ListActivity and MapActivity. Working string comparisons, type based match and a huge amount of syntactic sugar, added on top of a proper ctags config, actually made life good.

Further, the base-91 decoding was elegantly implemented as a map/reduce operation on the ASCII string. Other interesting solutions were: an UrlOpener for buttons and regex based packet matching (Warning: please do not try to understand the regexes!).

What remains in the end is build time (compilation + proguard), which is subjectively higher for the Scala app than for a Java-only project of comparable size. However, that might be due to a bug in my build.xml and so far I was not impatient enough to investigate.

Conclusion

After two years, I am really glad to have gone this way. Learning Scala was a very pleasant experience, and it improved my ability to see problems from different points of view. However, it also significantly restricted the number of people able to contribute. Of over 500 commits to APRSdroid, only three were by another developer. The APRS parsing code has been replaced by javAPRSlib, a Java library with major contributions from several other people.

APRSdroid remains my only Scala project. My other Android projects are written in Java, either because I did not want to restrict contributors, or because I did not expect the Java code to become complex enough.

Would I start a new Scala project on Android? Probably no, as it is already hard enough to find people who would like to contribute to your pet project if it is written in Java. Scala makes that almost impossible.

Would I contribute to an existing Scala project? Yes!

P.S: Starting around March 2012, I will be looking for Android/IT-Sec related freelance jobs. Check github and Android Market for my other projects.

My love hate aversion to SyncML

Some years ago, I accidentally managed to synchronize my Nokia E65 phone to Evolution using Bluetooth, OpenSync packages from a custom repository, a huge amount of patience and a blood sacrifice to the gods of bloated binary XML protocols.

Unfortunately, soon after that my file system crashed, I reinstalled Debian and the magic setup was forever gone. Whatever I tried, all I got were opaque error messages. After many months of moot efforts, I finally gave up the transfer of events onto my phone and of phone numbers onto my PC. Sigh.

It was only last autumn that I dared challenging my luck again. After setting up a new colo box (it is serving this blog article right now) and having upgraded my Android toy-phone to an Android 2.x firmware, it was time to get my data from the good old Nokia phone to the Android device. Somehow.

The Quest of SyncML, part 1: eGroupWare

I began my quest by simply installing the current version of eGroupWare from the Debian Backports repository. Unfortunately, this version (1.6.002) is flawed with regard to SyncML. It worked partially with my cell phone, and failed miserably with Evolution.

After several days of fruitless efforts, I found a set of SyncML patches for eGroupWare written by Jörg Lehrke, which are already integrated into 1.6.003. Fortunately, eGroupWare.org is offering Debian 5.0 packages as well. I just added the following line to my /etc/apt/sources.list and installed the new version:

deb http://download.opensuse.org/repositories/server:/eGroupWare/Debian_5.0/ ./

Do not forget to import the repository key as well:

wget -O - http://download.opensuse.org/repositories/server:/eGroupWare/Debian_5.0/Release.key | apt-key add -

With the shiny new eGroupWare, I only needed to wipe my previous synchronization efforts and to enable the SyncML application for the Default user group. Et voila, I could access my new RPC server at https://<servername>/egroupware/rpc.php

Part 2: Evolution

This step does work more or less properly, an official HOWTO is available. The only thing I have not automated yet is the fact of synchronization. It still requires manually running

syncevolution <servername>

Update, 2011-05-15: If you are running debian, do not use it's default packages. After my last dist-upgrade (sid), syncevolution thought it was a good idea to parse its plaintext config files, generate an XML-based config and throw it up on me due to strange parser errors.

Uninstalling syncevolution* and using the syncevolution-evolution package from

deb http://downloads.syncevolution.org/apt unstable main

solved my troubles however.

Part 3: Nokia E65

Fortunately, Nokia already includes a SyncML client with their smartphones. It is almost trivial to set up following the official howto. However, with eGroupWare 1.6.003, I could set the SyncML version to 1.2 to obtain the full contacts information.

Fortunately, it was also very easy to add the CAcert root certificate to the Nokia device, allowing end-to-end encryption of my sensitive personal data.

Part 4: Android

Now, the real fun began. Android comes preinstalled with a well-working synchronization service which is pushing all your data to Google servers. Not that I would mind Google having the data, I just wanted to be able to snapshot my contacts and calendar whenever I need to.

There are as well clients for other synchronization protocols. ActiveSync is supported out-of-the-box (and there is the GPL'ed Z-Push ActiveSync server); Funambol and Synthesis implement SyncML on Android.

Because I already had SyncML running and Funambol is Open-Source and looked generally promising, I started my work with it. However, the Android client is "optimized" for interacting with the Funambol server (read: it interoperates with other implementations only by chance).

Besides the hell imposed on the unlucky ones trying to compile android-client for themselves instead of using the Market version, there were various compatibility issues. In addition to that, SSL verification is only possible using the certificates already stored in the system. Neither self-signed nor community-signed SSL connections are possible.

If you have root permissions, there is a workaround to add CAcert:

# adb pull /system/etc/security/cacerts.bks .
# keytool -keystore cacerts.bks -storetype BKS -provider org.bouncycastle.jce.provider.BouncyCastleProvider -storepass changeit -import -v -trustcacerts -alias cacert_org_class_3 -file cacert_class3.crt 
Certificate was added to keystore
[Storing cacerts.bks]
# adb remount
remount succeeded
# adb push cacerts.bks /system/etc/security/cacerts.bks

Nevertheless, the experience was so frustrating that I started my own project to improve SSL certificate management on Android.

After many fruitless attempts at getting reproducible synchronization with Funambol's Android client, I decided to test Synthesis. It installed, allowed me to bypass SSL certificate checking (which is not quite perfect, but at least better than no SSL at all) and synced all my contacts at first attempt. Wow! Considering the time I have put into Funambol, paying 18€ for a Synthesis license really looks inexpensive in hindsight.

However, not everything is as shiny as it looks at first. It seems, Synthesis is not providing its own calendar backend. Instead, it is using whatever is available on the device. My device however seems to be lacking any calendar providers, unless I install the Funambol client. So all in all, I am using Synthesis to synchronize events to the Funambol calendar because Funambol fails at it. Funny, isn't it?

Update: After upgrading eGroupWare to 1.8.001, I can actually synchronize my events to my Android using Funambol. Because they change much more often than my contacts, I might actually stick to this software for some more time without buying Synthesis...

Update, 2011-05-15: I finally found the "bug" responsible for my lack of contacts synchronization. I happened to have a contact with an "&" sign, which was transmitted verbatim by eGroupWare, freaking out the Funambol parser. After renaming the contact, life suddenly became great passable!

Conclusion

SyncML is a friggin' huge pile of shi bloat. Just sync your devices to Google and your experience will be great.

As a long-time Mutt user I always looked with envy at you Thunderbird and Kmail and what-not fans, as you could spawn new windows for reading and writing e-mails with a mere click (or sometimes a double-click).

It was just too bothersome to have $EDITOR block my inbox until I finish writing or give up and postpone the mail, losing track. As I am using Screen to run Mutt anyway, it seemed like a logical step to make it spawn new screen windows for writing mails and opening other mailboxes. The problem of multiplexing Mutt seems to have appeared to other people before as well, however the solution is not quite easy to spot.

The Idea Behind Mutt Multiplexing

The suggestion for people like me is: replace editor with a command line that spawns a new terminal and opens up mutt -H on a copy of the temp file. The original mutt will then see that the message was not changed and drop it silently, while the new instance allows editing. Fortunately, you can call screen from inside screen, thus creating a new window, so project mutt-screen was born! Unfortunately, not everything was as easy as expected.

The Gory Details, version 1

So, to make the whole thing better manageable, lets split that editor setting line into two parts: a script that is called in the new screen window and what needs to be done in the original mutt instance.

Lets call the script ~/bin/mutt-compose.sh:

#!/bin/sh
mutt -H "$1"
rm "$1"

Now, lets make a copy of the mail draft (%s), and run the composer in a new screen window:

# edit messages in a new screen window
set editor="cp %s %s.2; screen -t composer ~/bin/mutt-compose.sh %s.2"
unset wait_key

Great! We did it! Oh wait, no! It's a fork bomb! The new composer of course evaluates editor and launches... another copy of itself.

Edit the editor, version 2

Consequently, we need to override editor for the editing instance. Because we need to prevent the new Mutt from inserting a second .sig and more headers, lets create a second config file. Then we can call mutt -P ~/.mutt_compose -H "$1" from the script, with ~/.mutt_compose containing:

# read main config
source ~/.muttrc

# remove hooks, headers and sig, they are already in the draft
unhook send-hook
unset signature
unmy_hdr *

# call the right editor immediately
set autoedit=yes
set editor="vim +'set tw=72' +/^$/+1"

But now its going to work, isn't it? Not quite - something happened to postponed messages! Why are there now two edit windows? It seems, Mutt has a different code-path for messages which have been edited already. Here, the first instance ignores that the message was not edited and falls through to the compose window, while the second instance runs the editor (and permanently deletes the message if you do not explicitly save it, oops!).

Forking the editor - or not? Version 3

Our forking-out of the editor causes problems in three cases, as it seems: <recall-message>, <edit> and <resend-message>/<forward-message>. The former can be worked around by using a custom macro:

# override the <recall-message> hotkey
macro index,pager R "<shell-escape>screen -t postponed mutt -F ~/.mutt/compose -p<enter>"
# prevent recall on <mail>
set recall=no

This comes at the additional discomfort of not being asked if you want to resume writing your last mail when pressing m. That might be possible to achieve with some workaround - feel free to leave a comment if you find one.

For <edit>, the only viable workaround seems to be resetting editor to its former state, editing the message in-place and setting editor back to the screen call. Inconvenient, blocking your main Mutt, but possible.

# bonus points for outsourcing the two different editor settings into their own
# source'able one-liners...
macro index,pager e '<enter-command>set editor="vim +set\ tw=72 +/^$/+1"<enter><edit><enter-command>set editor="cp %s %s.2; screen -t composer ~/bin/mutt-compose.sh %s.2"<enter>'

With <resend-message>/<forward-message>, there is no workaround. I tried pushing keys into the buffer to quit the compose window, changing relevant settings, making blood sacrifices to various gods, all without success. So far I have to live with manually quitting the first composer and editing the message in the second one. Yikes.

The resulting config, version 4 (final)

When we put everything together, the following settings files should be in place.

~/.muttrc

# do not forget your own settings...

# edit messages in a new screen window
set editor="cp %s %s.2; screen -t composer ~/bin/mutt-compose.sh %s.2"
unset wait_key

# override the <recall-message> hotkey
macro index,pager R "<shell-escape>screen -t postponed mutt -F ~/.mutt/compose -p<enter>"
# prevent recall on <mail>
set recall=no

# set the editor for for editing messages in-place
macro index,pager e '<enter-command>set editor="vim +set\ tw=72 +/^$/+1"<enter><edit><enter-command>set editor="cp %s %s.2; screen -t composer ~/bin/mutt-compose.sh %s.2"<enter>'

# open mailbox listing in a new window
macro index,pager y "<shell-escape>screen -t mboxes mutt -y<enter>"

~/bin/mutt-compose.sh

#!/bin/sh
# set the screen window title to the message receiver
awk -F 'To: ' '/^To:/ { print "\033k" $2 "\033\\" }' "$1"
mutt -F ~/.mutt_compose -H "$1"
rm "$1"

~/.mutt_compose

# read main config
source ~/.muttrc

# remove hooks, headers and sig, they are already in the draft
unhook send-hook
unset signature
unmy_hdr *

# call the right editor immediately
set autoedit=yes
set editor="vim +'set tw=72' +/^$/+1"

The following .screenrc.mutt file can be used to spawn a screen with your inbox in it and a status bar showing the outsourced instances:

~/.screenrc.mutt

hardstatus alwayslastline "%-Lw%{=b BW} %50>%n%f* %t %{-}%+Lw%< "
sorendition = bG
vbell off

# start mutt directly in a window
screen -t INBOX mutt

Conclusion

This solution is not perfect. In certain corner cases (<resend-message>) it just does not behave. In other situations (caching the PGP passphrase does not work) it has less comfort. Nevertheless, it has increased my Mutt productivity and decreased my frustration a lot. Now all is missing is clean support for multiple profiles (read: without having many scripts bound to clumsy keyboard shortcuts) and proper in-mutt mechanism for message tags.

Sometimes it is plain frustrating to see how people try to be smarter than you and hard-code functionality which is (almost) impossible to override.

The rant (skip this section or don't complain!)

The one single most-frustrating feature of my HTC Dream/G1 phone looks like an attempt to be smart and save batteries: Whenever the phone connects to a WLAN, the 3G/mobile data connection is terminated.

"What's the problem?" you might retort. Now, most Internet applications are using TCP as their protocol of choice, and TCP maintains a connection bound to an IP address. Whenever you change your Internet access, you switch IP addresses and all existing TCP connections vanish. Your downloads are aborted, your SSH connections are closed, your IM session is terminated (or, even worse, it looks like online but is not).

The smart G* engineers of course have provided a way to detect the change of connectivity, using a NetworkConnectivityListener. They also probably implemented some really smart synchronization protocol into their binary-only applications, to improve the user experience.

However, they did not provide a way to prevent the deactivation of 3G data service. They added in some complicated code to keep a 3G data connection open to the MMS service, but the "normal" data session is just terminated whenever a WLAN is found. This would not be as bad as it sounds if such a state change would only happen twice a day (WLAN → 3G when you leave home; 3G → WLAN when you come back). However, WLAN is eating your batteries really really fast. Thus, the smart G* engineers made the phone automatically switch WLAN off one minute after the display backlight is disabled. That means: you look at the phone clock, it finds a WLAN, terminates all your connections, goes to sleep, turns off WLAN, terminates all your connections, ... GOTO 10

To add insult to injury, they added ConnectivityManager.requestRouteToHost(int networkType, int hostAddress), which looks like it would set up a route to your destination using the specified network interface. Ha-ha! Fail! That function only works if the requested interface is already up!

For application developers, this basically means that they have to catch the CONNECTIVITY_ACTION events, terminate the stale connection and open a new connection, synchronizing all of the state between the client and server. This of course implies that the application protocol must support re-synchronization. HTTP for example provides the Range: x- header to continue a partial download. For Jabber, there is XEP-0198 (which is still missing in most implementations). Other protocols, like SSH, are basically screwed.

For developers working at a mobile carrier, this is also bad news - there is no way to access the 3G data network when the user is surfing via WLAN.

Compared to this, the Symbian way of presenting the user a list of available networks when an application opens a socket is just a heavenly dream. Sorry, smart G* developers, you f'ed up this one.

The hack

After following the PdpConnectionConnectivityManagerConnectivityService twine of bloat, I saw some really fascinating code in ConnectivityService:

if (!mTestMode && deadnet != null) {
    if (DBG) Log.v(TAG, "Policy requires " +
      deadnet.getNetworkInfo().getTypeName() + " teardown");
    toredown = teardown(deadnet);
    if (DBG && !toredown) {
    Log.d(TAG, "Network declined teardown request");
    }
}

Now, what it means is basically that if mTestMode is enabled, the old connection is not terminated when a new one is established. mTestMode is set as:

mTestMode = SystemProperties.get("cm.test.mode").equals("true")
    && SystemProperties.get("ro.build.type").equals("eng");

On a rooted phone, all we need to get it is to change /system/build.props, reboot, and call requestRouteToHost().

Fortunately, the smart G* engineers fixed this evil exploit for the 2.0 release! GOTO 10 again!

One-and-a-half Xbox360 hackers

Yesterday, I was sitting around and pondering whether to implement some kind of NAND filesystem support in XeLL. After all, there are people out there who would like to boot a Linux kernel without having to attach a USB storage device or insert a CD-R.

On the other hand, XeLL should actually be ported to libxenon, a hardware abstraction library for the Xbox360 hardware, which is desperately lacking developer attention as well. Now, I am only one person, but there are so many missing features. And by missing features I actually mean "things one could rip port from other OSS projects".

The idea of libhomebrew

And that is where the idea enlightened me. Every homebrew-on-$HARDWARE project so far was redoing the same things:

1. hack the hardware (optional)

2. write drivers for hardware components

3. create a library containing/wrapping all the drivers and a libc

4. port libraries with additional functionality (like MP3 playback and JPEG decoding)

5. port libSDL ;-)

6. write apps!

Now, steps 1. and 2. are of course specific to $HARDWARE. Step 3. is often based on how step 2. was performed, but it does not have to. The following steps however could be easily abstracted away from the actual hardware, even though currently, they are redone countless times.

But do we really have to redo them for every new platform?

One lib to rule them all

Instead, we could just create the one homebrew library, libhomebrew. It would contain a basic set of functionality, ports of commonly-used libraries and of course HAL backends for all supported platforms.

Everybody in the homebrew scene would profit from this:

• homebrew authors could do write-once deploy-everywhere development.

• platform hackers would profit too: instead of porting SDHC drivers or libmad to yet another console, they could just add a libhomebrew HAL backend for their hardware, automagically gaining all the libs (and many apps).

• library maintainers could integrate libhomebrew support into their libs, without the fear of creating a forest of hardware adaptations.

*yawn* This is all old news!

Of course, this is nothing I could file a patent for (I would not blog it if it was ;-)). The basic idea already exists for decades in many different implementations, however not in such a homebrew-centric way so far.

The three projects most similar to the presented idea are:

1. The Linux kernel is exactly that, plus a huge pile of bloat completely superfluous for console homebrew.

2. ScummVM is an adventure game "emulator" with a large set of platform backends.

3. devkitPro provides homebrew toolkits for several different platforms, but no common hardware abstraction as far as I could see.

Progress of libhomebrew

So far, all there is is a libhomebrew wiki page on the free60 wiki. After all, I am only one person with a full-time job not related to homebrew in any way.

However, I hope to find some interested developers who are tired of re-writing drivers and porting yet another lib to their favourite platform.

Contribute or at least spread the word!

cgit, a gitweb replacement?

Recently I was made aware of the existence of cgit, a git web frontend written in C. It looked promising, so I tried it out.

It seems to perform better than gitweb, which I kind of expected from the fact that it is a native binary, as opposed to a perl script forking git for the actual work. Also, there is a comparison performed by the author of cgit.

However, I was convinced by the clean, short, nice looking URLs provided by cgit, which allow to checkout files from your default HEAD without nasty hash tags:

/cgit/libmpq.git/tree/Makefile.am

versus the rather ugly gitweb URL:

/git?p=libmpq.git;a=blob;f=Makefile.am;
    h=1f70fdb06cc13ae0338c380139c352383aea7700;hb=HEAD

I think in the good old new days of Web 0.2, this is called RESTful behaviour, which also coincidentally contains the substring "STFU".

Syntax highlighting with cgit

Anyway, once I was at playing with cgit, I wanted to add some syntax highlighting, too. To enable it, cgit already comes with a filter script (you see the elegant URL syntax at work again?). This script uses the highlight command, but it does not support all the file formats of highlight, and might be missing some elegance.

I made an attempt at supporting all the files I have in the different git repositories, which you can find here:

#!/bin/sh
# store filename and extension in local vars
BASENAME="$1"
EXTENSION="${BASENAME##*.}"

# map Makefile and Makefile.* to .mk
[ "${BASENAME%%.*}" == "Makefile" ] && EXTENSION=mk

exec highlight --force -f -I -X -S "$EXTENSION" 2>/dev/null

This variant extracts the file extension of the file to be displayed, contains detection of Makefile.* files and spares one fork by running exec highlight. If the syntax is not known, --force makes the command just pass through the input.

Do not forget to add the CSS required by highlight, as documented in the original syntax-highlight.sh.

What is still missing?

Cgit comes with cache support to further reduce the CPU load. However, I yet have to figure out how to make the caching time long enough to make a difference on this low-traffic server without taking forever to update the cache content whenever I push changes.

A cosmetic issue remains with directories in the tree view. They should get a more decent mark-up for people to notice them. Update: this is easy with a small css tweak (example).

And, last but not least, I am missing a feature to limit the blob-size for generating HTML. This is especially important for binary objects which are served as a hex-dump, generating gazillions of bytes for something you are not going to read anyway. Update: If you want something done, send a patch upstream! (example) :-)

Update 3: All three patches have found their way into the cgit master repository, thanks very much, Lars!

XEP-0198: Stream Management is an XMPP extension adding stanza acknowledgements and stream resumption.

What does that mean in English? If you are using Jabber on your cell phone, you are not going to lose messages anymore whenever you get out of service coverage.

So, why do we need it?

Even though Jabber is using TCP, which is called a reliable protocol, messages can be lost when a user loses connectivity: the server still has an open TCP connection to your client, sends the message, and the message vanishes because you are gone. In the good case, the connection is immediately closed by your network provider, causing further messages to be stored on the server. In the bad case however, there is no reply (because your provider employs paranoid admins), and the server takes the usual TCP timeout (several minutes to hours) before taking you offline.

How can XEP-0198 save our lives?

The new proposal can not magically keep the connection while you are offline. However, it introduces two important elements to provide you with the messages you missed when you come back:

• stanza acknowledgement allows both the server and the client to request a reply (acknowledgement) on every sent message or a group of messages. Once this reply is received, the messages are considered as delivered.

• stream resumption is used when a connection is re-established. By including sequence numbers in the stanza acknowledgements, both client and server can tell each other which messages they received before the interruption, requesting to re-send later messages.

By combining these two enhancements, it is possible to provide a delivery guarantee for every message between you and your server. The same mechanism can also be used on S2S (server to server) links, however these are generally far more reliable.

XEP-0198 also allows to implement *throttling* of connections on busy servers, however this does not have such a big effect on the subjective reliability of Jabber.

Update, 2012-03-31: Protocol Versions

Over the years, the XEP-0198 specification developed, changing the XML stream format on the way. Fortunately, they introduced a protocol version to check for compatibility. Unfortunately, the authors who bothered enough to implement the proposal, only added one single version of the spec, providing no backward compatibility:

Version	Namespace	Annotations	Implementations
0.5 (2008-09-29)	urn:xmpp:sm:0	First version with an official namespace and protocol versioning.	ejabberd
0.6 (2009-03-19)	urn:xmpp:sm:1	Renamed attributes, added (later deprecated) stanzas attribute.
0.8 (2009-04-09)	urn:xmpp:sm:2	Simplified protocol. Added (later out-specced) throttling element <t/>.	Gajim, Prosody, Psi, Swift(en)/Stroke
1.2 (2011-03-02)	urn:xmpp:sm:3	Further simplified protocol. Removed throttling, per-number acks.

Where can we get it?

Unfortunately, after XEP-0198 has been proposed to Draft Standard on 2009-06-17, not much has happened. Update: The following applications already support the extension:

• Gajim (Gnome-oriented XMPP client): version 0.8
• jabberd2: pre-0.5?
• Prosody (a lightweight Jabber server written in Lua): version 0.8
• Psi (multi-platform XMPP client): forks of Psi and Iris: version 0.8
• Swift ("your friendly chat client", multi-platform), by means of Swiften, its embedded XMPP library: version 0.8
• Stroke (a Java port of Swiften): version 0.8

If your favorite server/client is not in that list, there is something you can do:

• provide patches for your favourite Jabber application.
• kindly request the authors to implement it.
• support existing requests.

Here you can find related requests for different Jabber software:

• SMACK (Updated May 2011, original Forum thread) (a Java client library usable in mobile phones)
• BEEM (Android XMPP client based on SMACK)
• ejabberd (Updated Jan 2012: fork with XEP-0198) (scalable Jabber server written in Erlang)
• Pidgin (multi-platform multi-protocol IM client)

Notes

There are several related extensions, like XEP-0184: Message Receipts or XEP-0199: XMPP Ping. However, these do not provide the reliability and efficiency of XEP-0198.

If you know of a client or server that supports XEP-0198, please feel free to leave a comment or drop an email to <georg@op-co.de>.

Am I actually making a blog? I don't even want to know!

Setting up the CSS for iki is not quite a nice & easy job, but at least it is not the usual Web 0.2 bloat.

Update: ikiwiki has the nasty misfeature of overwriting local.css whenever you regenerate the wiki without --refresh. Because I do not like it and I did not want to dig too deep, I added the following to my .htaccess:

Redirect /blog/local.css /ikiwiki.css

Up-Update: The obvious solution is to put local.css into the source directory instead of the destination directory. Thanks to Spida for this hint.