Uber Geeky!

This morning I finally decided I couldn’t stand Evolution any longer. I started using it as my mail client at work because we have an Exchange mail server and Microsoft doesn’t make a client for Linux (surprise, surprise). I’ve never worked at a place that uses Exchange, so I’ve always just used Thunderbird for mail in the past. When I got here, this was my first serious experience with Outlook, and I started to get a bit hooked on some of the calendaring niceness. When I got my new Workstation and ditched Windows for Kubuntu I was looking for something that could integrate best with Exchange.

Evolution seemed the most attractive option because it has a plugin for exchange integration called evolution-exchange. You can install directly from the Ubuntu repositories. Basically it’s a scraper for the OWA (Outlook Web Access) web interface, and generally it works really well. I’ve been using it for about 5 months now, but there are just a few things I can’t stand about it that made me decide today to bin it.

1. Password manager is broken and has been for about 12 months it seems. No one cares to fix it. You have to type your password in each session.
2. A bug with the Evolution/Nvidia/Compiz combination of packages causes the cursor to leave garbage on the screen when you use the cursor keys in a new mail message.
3. HTML support sucks, the development community seem hell bent against it.
4. It can’t handle contacts with “>” in the name, which is a convention we use to keep mailing lists at the top of the list. This character will cause it to generate invalid mail headers, destroy the html layout of the email and makes me look like an ass, especially when sending announcements to groups like “>ALL STAFF”.
5. Occasionally the evolution-exchange plugin cache gets corrupted and you loose random mails, but you wouldn’t know unless you check occasionally in OWA or Outlook. You have to delete the whole cache and download them all again.

I was prepared to switch back to using Outlook in a virtual machine, but before doing so I decided to check first to see if there was any possible way to get Thunderbird to read Exchange calendars. Thunderbird has an extension called Lightning that adds Outlook style calendaring integration. The latest release is 0.9. Last time I used Thunderbird I think I had to get the development version of 0.8 in order to get enough working features to call it functional (appointment invites used to be very buggy in the 0.7 version). Version 0.9 has come along way, everything seems to work great… except still no support for Exchange calendars unfortunately. Your calendar is maintained locally or in some 3rd party web calendar that uses a non-proprietry protocol. Not quite good enough.

Then I found DavMail. It’s a gateway that basically does the same as the evolution-exchange plugin. This thing acts as a webservice making data available from Exchange via standard protocols by scraping the Exchange OWA interface. It maintains separate user sessions, so you really can set it up as a server side service, possibly even installed on the mail server itself.

A ray of hope! But would it work?

I had enough problems getting Lightning installed to tell the truth, and that’s just a plugin for Thunderbird! Turns out the link on the Mozilla site was no good for 64bit Linux OS and I had to jigg about with the URL to find the actual release for me. DavMail is a totally unsupported package though, the one thing going in it’s favour is that it’s Java based, so it should just work right? Well lets find out.

They have a deb package ready for me to download. I attempt to install it but I’m missing the dependency “libswt-gtk-3.4-java”. Oh boy, looks like it’s using ugly Swing for the UI (Update: thanks for the education Jurrie, libswt is certainly not Swing). Well, looks like my Kubuntu Hardy distro is a bit behind the times now because the best I could find was a package called “libswt3.2-gtk-java” (yeah, I love the way they switched naming conventions too ^^). I installed that anyway and then just used dpkg to force it to install. It created an icon in my start menu under “Internet” and then seemed to work perfectly, connect to OWA no worries, all was good. Except now my package manager is going boonta because I’ve got a “broken package”. I can’t find the chill button so I uninstalled it for now.

How do you downgrade a dependency? Well this method worked for me…

dpkg -e davmail_3.2.0-1_all.deb
cd DEBIAN/
nano control
tar -czvf control.tar.gz *
mv control.tar.gz ..
cd ..
ar r davmail_3.2.0-1_all.deb control.tar.gz
mv davmail_3.2.0-1_all.deb davmail_3.2.0-01_all.deb

When editing the control file I changed the dependency from “libswt-gtk-3.4-java” to “libswt3.2-gtk-java” and also changed the version number to 3.2.0-01 so as not to conflict with a real version. The deb installer GUI detected something was wrong and wouldn’t install it, but “dpkg -i” worked like a charm. This is probably totally the wrong approach to take to my problem, but I don’t care. While I can sort of understand why Gnome developers might think Swing is cool, I can’t believe the libswt-gtk project can possibly have achieved much in two minor versions that would break compatibility. The latest 32bit Ubuntu has the correct version so most people wont have to care about this at all.

Now I’m rock’n with Thunderbird and Exchange! Without too much tomhackery even. So I settle back in with my old friend, so many features I’ve missed. The fantastic rich text editor (well actually it’s really basic, but light years a head of Evolution). The simplicity of theming it. I toyed with installing an Outlook theme and trying to get it to look and act exactly like Outlook. You can choose which IMAP folders to subscribe, so I picked Inbox and all it’s sub-folders, Drafts, Sent and Deleted Items. Now it’s easy to configure Thunderbird to put your sent mail into the IMAP Sent folder, but you still have the default Thunderbird Trash folder which you cannot easily convert to the IMAP Deleted Items folder. I found out you just have to edit user preferences to change the name. Searching on the net, people are always talking about editing some “prefs.js” file. I always just change the welcome page in Thunderbird to about:config, it was one of the first things I did when it was installed. This allows me to edit my preferences in the exact same way as you would in Firefox. In this case I just added the property as specified, restarted, and it worked like a charm.

I also hooked up the address book to the Exchange Global Address LDAP and then just tweaked the LDAP settings so it finds contacts a little quicker. One thing I notice is that the compose window only completes local addresses and not LDAP addresses. There had to be a fix for this.

With the power of about:config I searched for ldap and quickly found the settings that looked most useful…

• ldap_2.autoComplete.directoryServer
• ldap_2.autoComplete.useDirectory

The first one needed a quick search to figure out the syntax but the second one was just a boolean. Once that was done I had the same, if not better, contact auto-completion as I had previously with Evolution and Outlook. Game over, Thunderbird/Lightning/DavMail wins!

Update May 20, 2009 at 12:45 pm: Set mail.check_all_imap_folders_for_new to true, otherwise Thunderbird doesn’t check for new mail in your subfolders. This can be a problem if you have any Exchange mail filters running.

Update May 26, 2009 at 12:58pm: http://www.trustedbird.org/tb/Multi-LDAP here’s an addon that’s showing the easy way to configure LDAP addressbook lookups. Install that addon if you would like to search multiple LDAPs.

It’s not made very clear, but apparently Picasa don’t allow hotlinking to full sized images, although they seem to make an exception for blogspot blogs. I’ve suspected something like this was the case for a while since the Android splash screen image in my first post tends not to render most of the time (until I’ve visited Picasa, and it gets cached). I just noticed that my dysfunctionally hotlinked image of the android splash screen is on page 2 of a Google image search and decided it was time to get to the bottom of this.

Visiting the image URL directly also always works, so I suspect they check the referrer URL, and if it doesn’t belong to Google, they return zilch. I got a bit of confirmation about this from the Picasa forum, and also found a solution (hopefully). If you click “Link to this photo” and look at the “Embed image ” html, you’ll get a scaled down thumbnail URL in there. You can also specify a larger size thumbnail up to 800 pixels in width. I’m now hoping the full size thumbnail URL I’m using will stay active.

Personally I’d like to keep using Picasa to manage all my images, but all this jumping through hoops is a bit out of control. If this method doesn’t work then I’m shifting to Flickr.

Was just talking today about the future of Flash, considering the G1 and iPhone don’t support it. Hell, I spent a good hour or so getting my tag cloud to look good both with and without Flash support so this blog maintained good compatibility with those handsets.

Looks like they could be making up some ground at the moment with Bsquare porting Flash Lite over to Android for the upcoming release of Dell netbooks. I predict now it’s just a matter of time before it’s ported to these handsets.

I think you’ll agree it’s been bigger-better thinking for such a long time for most of the big players in this industry, particularly Adobe. How massive is their PDF reader theses days? It’s ridiculous. I uninstalled it after version 7 was released and switched to an alternative reader because I couldn’t take it any more. The industry is still catching up with the concept of people wanting more of the desktop experience on less powerful devices. I still suspect Microsoft’s approach is to get a strangle hold on the distributors to ram their outdated wares down our throat rather than actually improve their product, but I have no proof yet so stay tuned. As soon as I find out why there are only Windows netbooks for sale in this country I’ll post it.

Actually, while I’m back on the netbook OS topic again, I stumbled upon this little gem which appears to have no OS whatsoever! The Astone UMPC netbook. The specs aren’t too bad for the price, but mostly I admire their balls for releasing a bare metal netbook to the public. Viva!

Thanks to Gizmodo for the rather flashy heads up!

I originally found it only because I stole his awesome Flash tag cloud. But the content is right up my ally and top quality. This is what my blog wants to be when it grows up!

http://www.roytanck.com/

Tonight I spent some time looking around for places that sell Linux Netbooks again. Amazon had quite a few good deals, that sadly wouldn’t ship to Australia.

In the country I only found a few outlets. Penta and YouPrice are selling the 901 model with 20GB SSD for I think one or two hundred dollars more than what they are worth these days. I seen some people selling the Acer Aspire One with Linux. No one is selling the top end Netbooks with Linux here.

I remember seeing the prototype for the Touch Book a while back. Looks like they are just about ready for the market. I think I’ll hold out for these to hit the shores. Fingers crossed they don’t have Windows support by the time they reach Australia, the distributors wont order the Linux model if the current state of the market is anything to go by. More than likely we wont even see these before someone else releases a knockoff with Windows on it. Then I’ll be out of luck for sure.

Update: Actually it wasn’t the Touch Book prototype I’d seen, it was the Crunch Pad. However, I think the Touch Book looks a little superior with it’s detachable keyboard. Both are rather sexy!

The next step for me was to get root. People are generally recommending to downgrade firmware to take advantage of a glitch. I checked mine and I was still on RC19, which has another simple method. Basically you can simply start the telnet service and connect as root remotely from another computer. I just had to install a terminal app from the Android market and then I was in. From there you can create an su command for use on the handset and then kill the telnet service. I used instructions from Hello Android in case you are interested in the specifics.

It was good that I had the old firmware, as I was able to experience some of the original mods that generally no longer work. I went over the original G1 hacking forum posts and tried a few tricks. My firmware was nice and insecure, that’s for sure. Storm clouds were brewing and I could hear thunder in the distance. It added some mad scientist atmosphere.

While I was exploring the joys of root I received an email from one of the G1 modding sites informing me that the latest firmware was modded and available, including multitouch enabled features, such as multitouch zoom in the browser. This is something you really need, just in case an iPhone fanboy notices you have the G1. Time to get serious and put some custom firmware on this thing. I downloaded it and started reading through the tutorial.

By now it was getting really stormy. The storm knocked out my ADSL, so that was the clincher. No internets, nothing else to do but start the firmware upgrade. I followed the tutorial, which was very mickey mouse. No explanation of what you were doing, just follow the bouncing ball. One step was to wipe all your settings. This meant I had lost my APN settings in the firmware upgrade (the details I’d gotten off the Whirlpool site to get GPRS working). No internet = no Whirlpool = no G1 reactivation… Doh! I had to wait a while to reactivate the handset. Next time I’d better jot those details down somewhere.

So, I took advantage of the opportunity to get some chores done, change and feed the baby, etc. All the stuff I probably wasn’t going to be doing as soon as I got my toy back.

…

Eventually I was back on line and could activate the handset again. I even found the right place to get G1 APN settings for any carrier.

Upgrading to the modded firmware actually turned out to be more of a pain in the butt than I had anticipated. This was only because the tutorial was flawed. It assumed you’d already installed a modded firmware previously and so they missed a crucial step. After pestering the Android devs on IRC and even tracking down JesusFreke (the main guy responsible for the firmware mods) eventually someone put me on to the full instructions.

These instructions rock. Included was a very useful hotkey that actually gives you some text output during the firmware upgrade process. The crucial step missing from the previous tutorial required me to replace the default recovery image with a modified one. Once I had an alternative recovery image that could circumvent some digital signature check, I was able to install the modified firmware with no dramas.

The system is fairly robust in preventing you from bricking it. As long as you don’t nuke the recovery image you can boot with it in the event that the main system wont start up. Once in there you can refresh the system with an image file stored on the sdcard. I’m not sure what would happen if your recovery image was also broken or missing. The place to find out would be over at xda-developers forum.

One downside of the firmware upgrade was I lost all my apps. I spent the rest of the day putting apps back on there with a bit more discrimination this time. Found some useful apps, like sftp clients and an ssh client. The sftp apps I tried kept crashing for some reason that wouldn’t become apparent to me until the next day.

There is also a screenshot app that supposedly worked when granted root access. It was rather unuser friendly, and I couldn’t figure out where the images were supposed to be saving to. I’ve since used it to collect a few screenies that I’ll scatter through the blog at some stage (don’t hold your breath, I’m pretty slack). Stuff was starting to seem buggy, but it was late so I decided to call it a night. That was the end of day 2.

Day 3 I had the pleasure of showing my iPhone wielding boss the multitouch browser. Never mind the fact that multitouch support is limited by the hardware such that you can’t effectively use it if your fingers get closer than about an inch. Apparently a snapping effect then happens where the device only detects a single touch somewhere between the two fingers. You can checkout the specifics on Luke Hutchison’s blog, he’s the mastermind behind the multitouch hacks. The hacked browser doesn’t actually use multitouch gestures properly anyway. It just emulates a general zoom in/out command, it doesn’t centre the action on the point between your fingers like an iPhone does.  Doesn’t matter, you can successfully fool the Apple fans for long enough to get past them and get back on with your life.

Actually I found that wasn’t quite true. My workplace has WPA2 enterprise wireless security. The wireless applet/widget/whatever you kids call programs these days wouldn’t even list the SSID. Funily enough I found my Tricorder application did detect them. I had a moment of bemusement that this purposeless toy could actually be useful for something.

<insert Tricorder screeny here>

So now iPhone had me on a feature that wasn’t just aesthetic. Rrrrrr… GOOGLE SKILLS ACTIVATE! Good on you Goolge, you really brought the true Linux experience to the handset market. Editing text files as root with vi to get stuff working on my phone, that’s pretty special, in some sense of the term anyway. So I followed instructions listed on the xda-developers forums. I didn’t actually need to install any certificates, it just connected as soon as I put details about the SSID, WPA2, and my username and password in a text file. Not ideal for security reasons, but as long as it works I’m a happy man. The SSID now shows up in the wlan applet too.

Something else was amiss that day though, the instability that I noticed last night was still going on and I finally figured out what the pattern was. When anything tried to write to the sdcard it would fail after a couple of files. Some apps would crash, other’s would fail silently, and just one actually handled the exception and displayed a meaningful error message. Go you cowboy Android app developers!

Experimenting with remounting the sdcard confirmed that after a short window of write activity the sdcard would become readonly, once again keeping true to the spirit of Linux. A quick search confirmed my suspicion the solution was to reformat it. Bare in mind, this is the sdcard that came with the handset. I backed up the files, formatted it and restored the data and the problem went away.

I spent the rest of day 3 pondering the next frontier.

In case you’re not familiar with Unix terminology, I’m not talking about some kind of sick techno-fetish. I’m talking about gaining full and unrestricted access to the software that powers your phone. Just like administrator on your Windows PC. You are free to do anything, including making it not go so good no more.

When you purchase a PC, you don’t expect the manufacturer to tell you what operating system or other software you can or can’t run on it. You’re free to do what ever the hardware supports. So what’s so different about phones? The main difference is that mobile phone technology has always been entirely proprietary. Even if you had the access, there is probably not a lot you could legally do that would be of any use to you. But that’s finally starting to change now that Google have provided us with an alternative via the G1. This phone is the first generation of handsets powered by the entirely open source Android OS.

I wasn’t looking for a phone when I decided to get this bad boy. If it was just a phone, there’s no way I’d have gotten anything this big. This is not a phone, this is the beginning of the revolution. A miniature PC that you can even make phone calls with if you like. The Android OS is basically a Linux distribution with it’s own custom Java virtual machine. Most Android developers use Java, but it is possible to compile other Linux packages.

Unfortunately, root access isn’t handed to you out of the box. Understandably the phone manufacturer doesn’t want to give you such power because it can cost them more money in support and returns. However, chatting with the Android development community is a very fast way to find out everything you need to know to unlock it. There are also packages in the Android Market which either rely on root access or have greater functionality once it’s available. I’ll go more into it in the subsequent parts. This part I’ll just cover my first impressions of the handset.

I’d been eyeing off the Eee PC and Eee Boxes for a while now, but it looks like I’ve missed the opportunity to get one from the shops with Linux on them in Australia, seems they just don’t stock them any more. The G1 had been out for about a month in Australia, only a few phone companies are selling them on plans. I decided to just buy the thing outright from an on line store based in Australia called MobiCity. I wanted fast delivery and a warranty, so I paid a little more than I probably should have.

When I got the email from the tracking company, it soon became apparent that the phone wasn’t coming from a warehouse in Australia, it was actually coming from Hong Kong. I thought perhaps it’s a good thing, it might get it faster. But no, it had to go through Sydney first so took the full 5 days to get delivered. I couldn’t really complain about this because they stated 1 to 5 days delivery time. Anyway, it was probably for the best since I was on a training session that week, otherwise I probably wouldn’t have taken much in.

Halfway through the training session I got news that the package had arrived. I collected it around lunch time. I even missed out on the free hot dogs being given out that day because I was tearing off packaging and trying to figure out how to get the thing going. Holding the thing, it was a little ugly, but actually not as bad as some people made out in the reviews I’d read. Also the weight was pretty good considering how powerful it is. I put the battery in and fired it up. The little android animation I’d seen in the emulator popped up. Bonus points for no boot music, something that always bugs me about Nokia.

The phone prompted me for my Google account, and then failed to connect using GPRS. The system refused to budge until I registered with a Google account, after some trial and error I found that the menu button brought up an option to configure APN settings. A quick Google and I found some settings posted on Whirlpool forums that worked like a charm. I was in.

Looking at the accessories, it was apparent that I’d bought a phone that originated in the US, was repackaged in Asia for the Asian market and then repackaged again for Australia. I had several power cords and adapters of different shapes and sizes, some inside the original box, and some outside. The box seal had been opened, though nothing had been touched in side the box, they had obviously just put an Asian power adapter in there. I was a bit bummed about it, considering part of the reason I didn’t mind paying a little extra was because I thought I was buying from a solely Australian distributor. At least I have the 12 month warranty, which is something that wasn’t offered by the Ebay dealers. I’m really a bit frustrated with the consumer electronics market here in Australia. If a product isn’t in the mainstream, you pretty much have to buy from overseas. My disappointment was very short lived once I started actually playing with it.

I was actually pleasantly surprised by the handset. The screen snaps up and down quite solidly. I wouldn’t exactly call it durable, but I don’t think you’d be buying one for the farm or a construction site. I didn’t buy it for looks, but I was concerned that it could be a complete monstrosity from some of the reviews I’d read. Many of the pictures I’d seen looked all right, but they were obviously showing off the handset’s best angles. The keyboard is quite comfortable to use with your thumbs. Anyone who expects to be able to touch type on such a tiny thing is tripping. The back cover is just a thin slither of black plastic hiding the battery, sim card, rear speaker and the camera lens. Like I said previously, the weight was very reasonable for what you get. It weighs about the same as my wallet when it has a little bit of shrapnel in it. I’m predicting this thing isn’t going to age gracefully, I’ve probably got about 24 months of being able to use it in public before it’s just too embarrassing.

The operating system is smooth, stable and very importantly it has a very light footprint. With just 192MB ram and a 528 MHz processor I guess you could try and compare it to an early pentium, except the size, OS and fancy hardware makes that a bit of a joke really, I don’t know why I’m mentioning it. Like most Linux distributions, it has a sort of package management interface where you can download bucket loads of software for free. This particular system is called Android Market, and it currently contains hundreds, if not thousands of free applications to download. I pretty much spent the rest of the first night trying out the apps and setting up this blog (Android browser support being most important).