Upgrading iPhone firmware using VMWare on Linux

Well that was a bit scary. I've just updated the firmware on my iPhone to 2.2 (the horrors!). Actually, this normally shouldn't be too much of a trial. The difference here is that I did it from Windows running inside VMWare on a Linux host.

I initially tried the naive approach of applying the update normally using iTunes (but running inside VMWware). Don't do this! The host (Linux) USB subsystem gets in the way leaving you with a somewhat useless iPhone in "recovery mode". It seems that the iPhone firmware upgrade procedure does something tricky with USB that doesn't play nicely with VMWare running on Linux.

To workaround the issue, some USB modules have to not be loaded on the Linux host during the upgrade. Extending the tips on Stephen Laniel's blog I created a blacklist file for modprobe that prevents certain USB modules from loading. Just unloading the modules manually isn't enough. The iPhone reconnects to the USB bus at least once during the upgrade process causing udev to reload the modules you've manually unloaded.

So before attempting the upgrade, create a file named something like /etc/modprobe.d/blacklist-usb with the following contents:

blacklist snd_usb_audio
blacklist usbhid
blacklist ehci_hcd

I'm not sure if ehci_hcd needs to be disabled, but I did this in my case.

Reload udev so that it knows about the new modprobe config file:

sudo /etc/init.d/udev reload

Now make sure these modules aren't already loaded:

sudo /sbin/modprobe -r snd_usb_audio
sudo /sbin/modprobe -r usbhid
sudo /sbin/modprobe -r usbhid

Now go back to VMWare, cross your fingers and attempt the firmware upgrade using iTunes. Things should progress along slowly.

You might find that at one point during the upgrade that iTunes sits there with a dialog saying "Preparing iPhone for upgrade". If this goes on for several minutes it could be that the iPhone isn't connected to VMWare anymore. iTunes is just waiting for the device to appear. If this happens, reattach the iPhone to VMWare using the "Removable Devices" option on VMWare's VM menu. It's a good idea to occasionally check that the iPhone is connected to VMWare during the upgrade.

Once the upgrade is complete you can remove the modprobe config file and reload VMWare:

sudo rm /etc/modprobe.d/blacklist-usb
sudo /etc/init.d/udev reload

For the record, this was done using iTunes 8 running on Vista running inside VMware Workstation 6.5.0 build-118166 on Ubuntu Intrepid (8.10).

All contacts in sync (+1 for the iPhone)

So new employer has given me an iPhone and despite my dislike of vendor lock-in I really like it.

One thing I've now managed to achieve is a single, central set of contact data. Like most geeks I had people's email addresses, phone numbers and postal addresses scattered across various devices and email profiles. In my case this included a Nokia 6130, my laptop's Thunderbird profile, various work Thunderbird profiles, my Gmail account and my trusty old Palm Tungsten E2. None of these stores of contact data were being synced. Contacts were added haphazardly as required.

No longer. The mess has been solved through the use of a variety of tools.

Getting an iPhone was the tipping point. Suddenly it was possible to sync contact (and calendar) data over the air, without needing to connect to a PC. This is very attractive to me. I like being able to sync even when away from the computer where iTunes is installed. I'm using the excellent (and free) NeuvaSync service for this. It hijacks the iPhone's Microsoft Exchange support to push changes to contact data to and from Google Contacts.

I've chosen Google Contacts as the central storage point for this data because it's reliable, I already have a populated gmail account and (mainly) because of the large number of tools that can interface with it.

Syncing to the various Thunderbird instances I use is done using the Zindus extension. This syncs Thunderbird's address book with Google Contacts. It works fairly well as long as you remember to stick to Google's rules about contact entries (eg. email addresses must be unique across all contacts). It will warn you if there's a problem.

So, that covers syncing contact data to everywhere I need it: iPhone, Gmail and Thunderbird.

The final challenge was to pull together all the contact entries I had around the place and merge them into Google Contacts. This was a long and messy process that was helped by some adhoc Python hacking. To get the data out of my Palm I used the pilot-address tool that ships with the pilot-link package. The excellent Gnokii tool pulled the contacts out of my Nokia (over Bluetooth, no less!). I wrote some Python scripts to help merge the highly imperfect sets of data, trying to make intelligent guesses where possible. Finally, OpenOffice Calc was used to manually fix the CSV before importing into Google Contacts.

It took a long time and many editing iterations but I now have a clean, centralised set of contacts. Say what you like about me needing to get out more, but I think this is awesome (and it was a worthwhile learning process).

freshfoo.com now using rest2web

I've now converted 2 popular articles [1] on freshfoo.com so that they are generated via Michael Foord's excellent rest2web. Previously one of these articles was in plain HTML and the other was on the site wiki. I've got a few other articles in the works so I decided it was time to migrate the articles to a clean, consistent format. reStructured Text fits the bill perfectly.

I recently also started using the rst entryparser for PyBlosxom so now most of the site is generated using reStructured Text.

[1] according to the HTTP server logs


Glenn and I discovered markup.py today. It's a lightweight, simple Python module for generating HTML and XML. Tags are generated using Python objects, with calls to methods on those objects dynamically creating child tags (using __getattr__ trickery). A useful feature is that repeated tags of the same type can be blatted out by passing sequences of values.

The module provides a natural way to create correct HTML/XML from Python and suited our needs perfectly. My only complaint is that the output isn't indented so it can be harder to read than it needs to be. It would probably be straightforward to add such an option. Also, it would be nice if the output could be streamed to a file-like object. Currently it builds the output in memory which could be a potential problem for large pages.

markup.py is worth adding to your toolbox if you ever have to generate bits of XML from Python. For simple jobs, it beats using a complex XML library and is less error prone than plain old string interpolation.

PyCon UK 2008

PyCon UK 2008 is approaching fast. If you're a Python programmer in the UK (or are just Py-curious) then you really should be going. The talk schedule looks hugely exciting and the in-the-hallways and at-the-pub action will undoubtably be fun and engaging. I guarantee you'll learn something about Python and come away feeling inspired.

Disclaimer: I'm presenting :) I'll be doing a talk on Python on the Openmoko Freerunner as well as helping out Michael and Christian with a 4 hour tutorial: Developing with IronPython

Neo Freerunner: first impressions

I finally got my hands on my Neo Freerunner two weeks ago and have been playing with it when time allows (so much so that I haven't given myself time to blog about it).

Overall, the hardware is great. The first thing you notice is that the unit feels very solid and the quality of the display is excellent; bright and high resolution. I've had success with wifi, GPS, the SD card slot and basic GSM usage. I haven't had a change to try out the accelerometers yet, mainly due to a (surprising) lack of software that uses them.

Read more…

OpenMoko release

The OpenMoko Freerunner has been released! This is big news for people who'd like an open and free phone (running Linux) with some interesting hardware: GSM, GPRS data, WiFi, GPS, accelerometers, USB host support, accelerated graphics, SD card slot and much more. The software is still a work in progress so the phone is primarily for developers at this stage

The UK distributor has been swamped by the amount of people interested in buying one. This is certainly an encouraging for the potential success of the project. I'm on the list to get one in the next batch, fingers crossed.