Categories: Top ::

Palm: Things That Annoy the Developers (Like Me)
Tue, 10 Jan 2006

Well, its that time of year again where things must be purged, the soul laid bare in preparation for a new year of technology silliness. (Okay, I had pleasant time at a Christmas party last night and a nice little lunch out today but I'm still in a mood to let fly the flames of war ;) While I really enjoy developing for mobile devices (and development in general) and have very much enjoyed being on the front edge of mobile technology the last half decade or so, I've become a little weary of dealing with PalmOne's (now Palm again) and PalmSource's sillyness to tell you the truth - one can only have so much time wasted through their sheer stupidity before you start to wonder - when will it end? When will they stop being idiots? Their "historical issues" (mistakes, guffaws, unfortunate changes in the timeline) are forever in the developers faces. After writing this huge rant, I realize - it is a wonder people develop for this platform at all. I've been assisting other developers in forums for years, and I know how tough it is for them to get past all this stuff. This is why after a major Shadow release theres usually a few weeks of quiet .. time to decompress and come to terms with the sheer amount of crap that had to be shovelled. This is why theres so much time between releases sometimes -- its getting harder to get the nerve up to "dive into the mountain of crap" :P

A comprehensive list of things that annoy me would be enormous as I routinely work around dozens of different Palm OS system bugs every day. But off the top of my head right now, heres what comes to mind .. call it the abridged and shortened summary of the greater litany :)

Ben, and my many friends at Palm and PalmSource - this isn't personal or directed, instead just the frustration as I work arond more and more bugs that make me pull my hair out. Gotta vent and get it off my chest :) I enjoy the platform, and I admire the work and dedication you guys have put in, but we still suffer :)

Edit, Jan 2006: Remember, I love this platform and work on it every day. I wrote this after working around some really aggravating OS bugs and answering hundreds of support emails about it. Don't be going and thinking I'm anti-Palm OS or anything ;)

Note: I could equally flame Microsft Pocket PC, WinCE, etc. I will, but after Christmas :)

Backwards and Forwards Compatibility

Microsoft has been doing operating systems for awhile now so I don't give them much slack when they've blown the game of increasing device resolution (requiring new developer APIs to support higher resolution units, instead of one neatly designed API - we all knew that devices would get better over time, right?) but at least they followed Palm's example of making low res apps just work on higher res devices. Well and good. Now, with all Microsoft OS PDAs working the same, compatibility is pretty high - but not so with Palm OS. It literally drives me and most other developers absolutely batty dealing with the volume of OS bugs that infect every Palm OS device - and do so differently with each device incarnation.

I always championed PalmSource by suggesting this is due to the difference in mindset and perhaps a lack of developers - that Palm pushed for diversity of devices and would happily try and warp their OS to fit a new device, while Microsoft clamped down specifications very rigidly. Hence to this day there are quite a variety of Palm OS units from watches to game machines to phones all on "one OS" (riddled with bugs), while in the Microsoft PDA world you'll find a fair variety of devices all working more or less alike (not as many bugs, but surely some duzzies!) Good and bad - Pocket PC is fairly easy to develop for, since all devices have pretty much the same feature set, but Palm OS is fragmented to all heck. Still, I like the variety of devices.... well, I did, until I had to support them.

In the end, to summarize - it is a chore to develop for Palm OS these days; most disregard all the older devices, but I try to maintain compatibility with and not forgot those trusty old friends... but the time is coming when I just start ripping out all that code -- it is difficult enough keeping up with all the new bugs in the OS, let alone old ones. Developers wish to be productive as time is expensive and tight, yet 90% of time I find is being consumed finding OS bugs and investigating work arounds. How much more intuitive would an interface be if we could focus on improving it, rather than dealing with bugs? How many more features would there be? How much lower would prices be? We have no time for 'business logic' when we're wasting it on fluff. This my friends is one of the biggest challenged in Palm OS development.. they're taking the fun out of it, and making it hell.

Palm Competing with Developers

So great, we were all happy PalmOne decided to start growing their PIMs. Typically, they just altered the database format without actually improving it, but worse still - they didn't document their changes for years - I can only assume they resisted this because they feared competition from the very developers they depended upon. What other explanation is there? Certainly there is no patenting or licensing issue at large.. though it could just have been lack of manpower to make it happen. Regardless, it made our lives _miserable_ and to this day we're all still dealing with the fallout.

Remember, when a tire won't fit on your car, you don't blame GM.. you blame the maker of the tire. When users found crashes, slowdowns or weird data loss, they blame the developers. Not PalmOne for breaking their own specs and then not informing anyone how it worked..

What they did was introduce new databases, undocumented; when developers applications opened the old databases, the OS would kick in and ferret data between the new and old. Sounds good on paper, but evidently PalmOne didn't debug or test this process much, since it was flawed in the design and buggy in its implementation - frequent device crashes would occur due to bugs in their 'PIM sync' (my name) code. Things would slow down enormously (especially with the later NVRAM devices) as data was forever being copied back and forth, something that never occurred before. Worse yet is data loss as the PIM sync layer tried to guess the intent of things going on and often got it wrong. So the developers got flack again, for PalmOne's mistakes.

They finally did release some documentation, years late. I don't get it. Months of suffering for no good reasons.

The real tragedy is the lack of learning - this wasn't the first nor last time the documentation didn't come until devices were on the market (sometimes for years) and our apps late to be stable. I'm not even going to talk about the dozens of serious NVRAM flaws - NVRAM being a great little system that users need, but when a large %age of apps become unstable it makes you wonder how much testing went on..

Development Tools - or Lack There-Of

Microsoft, aside from its immoral behaviour, knows how to make some decent tools for developers to work with; their OSes tend to be backwards compatible and they do a pretty good (if pricey) job of ensuring developers can get things done. Their Visual Studio suite is quite popular and its available for C/C++ and Visual Basic development for Pocket PC - a big comfort to many. With their new .net platform you also get C# tossed in, and halfway useful drag and drop GUI building tools.

With Palm OS 5 and earlier, the official development suite was always Metrowerks Codewarrior - a pretty good product. For OS5 "ARM" code development it was useless for awhile (seriously buggy), but they worked hard to move CW 9.0 to become CW 9.3 - flaky, but overall pretty darned decent for doing OS 5 68k+ARM development. A lot of us have been using CW for years, since it was the best overall system - and it was discontinued a year or so back or more. New developers have a hard time locating a copy of the package to use, so they resort to the PRC-Tools (freeware gcc, a thing close to my heart) which is much harder to get into.. discouraging away new developers. CW went away likely because it is hard to make money with development tools, but also because PalmSource was building their own development environment - PODS - for their OS 6 product. So PODS is out and about, but as you all know OS 6 is not. So PalmSource spent enormous resources and time working on OS 6 upgrades and development tools for us, that we couldnt' really use.. as PODS isn't so good for OS 5 and earlier. Great - Palm depending on out of print tools, and PalmSource working away for years on things no one is using. (gcc is great stuff that I use every day, but due to the 68k+ARM system in OS 5 devices its never been all that convenient.)

So in the Palm OS world, developers have been cobbling things together .. a sorry state of affairs .. something Palm should be ashamed of.

OKay, theres some other tools so lets flame them too - Java is a popular thing these days, correct? Well, a shame that Palm only licensed Websphere Micro for a few of their devices, and isn't really committed to it. Certainly it is unusable since how can you afford to base an application (months or years of work) on something which is only for a subset of devices and may be dropped at any time? Certainly, Java isn't available on Pocket PC at all since Microsoft isn't all that nice...

The reverse is also true; MS wants everyone to move to C#, but of course they will never provide a C# environment for Palm OS, and Palm just can't afford to kick off yet another development tool project.

If you want some cross platform code to work, your best bet is C/C++, or maybe somethign like NSBasic or SuperWaba. Still, a painful situation.

One other sub-annoyance; people all want java and C# and python these days as they're sick of building from the metal; sure, productivity is up with these languages as they've all borrowed some of the best inventions of the past and mixed in recent patterns and experience; further, they've got enormous manpower poured ito making new libraries to save time. (C++ and such never quite got so organized as to have these giant libraries bult for them in standard ways.) Anyway, people are forever asking why all our Palm OS apps aren't in java -- they seem to forget the fact that java isn't really available or fast for mobiles (hell, most phones don't even do JITting or garbage collecting these days) - that C and C++ are the really truly only portable codebases out there, to this day. But try explaining this to the newbies fresh from school, without sounding like an old codger. They do likes their java ;) Sadly, me too -- but its not an option so stop bugging me :)

The OS 6 Fiasco

This is much publicized so I needn't go into it much, but in summary - PalmSource (the OS developer) spent years building and pushing Palm OS 6 as the follow-up to Palm OS 5. Numerous developers (including myself) started working on OS 6 versions of our applications - either changes to remain compatibile or re-developed to actually use some of OS 6 features. Alas, OS 6 was not to come .. despite PalmSource pushing it and announcing devices and partners, we all quickly realized it was doomed from the get go, so all that time was pretty much shot. To this day, PalmOne has been knocking out OS 5 devices with their modified (read, buggy) version of OS 5 .. a great overall product, but with so many extensions and PalmOne seemingly out of control with regards to quality and standards, its started to be referred to as "Frankengarnet" by the developers and users.

Mac OS

Mobile developers aren't big companies with enormous budgets to burn, so we like to be cross platform (using one codebase for multiple operating systems) but its nutty over in OSX-land. We have a stable and dependable conduit for Windows, so that Shadow Plan can sync with Shadow Desktop and work very well. It would be nice if that same codebase could work under OSX, but we quickly found out that not only would that not happen, but that Hotsync Manager for Mac OS X is basicly a butchered up version of the Mac OS 9 system - that modern OS X style code cannot work with it. Wonderful how our Mac OS X application is OS X based and due to various dependancies can't tie into this OS 9 Hotsync Manager system. What silliness! Its possible to rebuild the needed dependancies, but I am rather sick of returning to the dark ages, of building up from scratch for every application somewhere along the line. At least Mark/Space is on top of the ball and has built their own Hotsync Manager replacement that offers modern OS X facilities.. but I do not feel it right to saddle our customers with an extra product (and none to cheap!) just to cover up for Palm's mistakes. So we .. as usual, work around the system limits and bugs. Another shameful situation.

Globals, 68k/ARM, and Other 1970s Problems

Again, being spoiled by the wonderful development environments of the 1980s, am sick of painful development situations requiring great devious minds to actually produce great things. Moving from 68k to ARM was a difficult yet amazing thing for Palm - its incredible they pulled it off as smoothly as they have; almost all Palm OS applications to this day are 68k processor applications, running inside of PACE - Palm OS 5's compatibility system for its ARM processors. Amazing stuff, so that our old apps can run on the new architecture without changes. Great!

The problem is when you wish to actually write ARM code, to get some speed. PACE exacts a large performance hit - something you wouldn't notice for GUI applications but when putting together a high performance multimedia application such as a game or video player or the like, you need the real hardware - the ARM system. Ben and the boys worked hard to make Codewarrior 9.3 stable and its a great product overall.. but due to the nastyness of the OS and interfaces, we're all forced to jump through hoops. You see, the 68k suport is quite good of course, but for ARM code you have almost no access to the rest of the application APIs unless you 'stub them' over. So to begin with, we all have to make stubs like mad, so we can make ARM code halfway as easily as 68k code. Ben really helped out here and the community worked together to produce a lot of useful code, but a shame PalmSource didn't provide it - they assumed we'd all be on OS 6 by now, and it just didn't pan out. The whole ARM-68k division is really quite painful - a lot of us have it pretty well mastered, but we're sick of the pain; newbies - forget it. Theres just such a learning curve and hump to get over that few ARM-based applications have come out. Certainly, to this day its still difficult to port applications from standard operating systems that use certain features, and you'd better have CW 9.3 (costware and discontinued) if you want your life to be sane.. using the freeware tools should be enjoyable and powerful, but its taken years for that community to get to this point. Terrible.

At least I did my part.. I got XCade out pretty fast (one of the first all ARM applications).. but its not pretty :)

Another thing that annoys me is the whole launch code and globals situation - its bizarre to be in the year 2005 here and have to deal with things like an app being launched without globals being available, due to how Palm OS works. It was innovative and a must-do-to-survive situation back in OS 1 and 2 and whatnot, but for OS 5 .. we're just sick of having to warp our code in these terrible fashions to make things run. Lets not even talk about 64k segmenting and whatnot .. I hate nothing more than copying functions around, into small little files, so I can re-order them so that functions can reach each other. Good god, that was the 286 days :)

OS Behind the Times

We all like compatibility (see all that ranting above?), but we're also cool with well controlled changes. If your OS is broken - fix it. We'll be there if you manage that process, publishing specs and documentation so that we can be ready when the newer versions roll out. Bring it on.

It seems alien again to be in 2005 with simple bleep-bleep alarms in Palm OS; the user can't easily assign mp3s or wav files to alarms or ringtones, without third party hacks. I've even written and given away a simple hack for years, to let small wav files be used as alarm tones. We need an OS change here, so that all applications can have a modern alarm managing system. Theres some cool stuff in the OS now, so that alarms can be made to vibrate or be silent from a user configuruation level withotu changin the apps.. but wheres the modern stuff?

Missing APIs

This ties back to the question of whethor Palm treats developers as competitors. Where are the APIs that the built in applications can use? How can we, on a Treo, control the radio to the degree the built in applications can? Perhaps its documented in some buried Handspring reference, but when I take a quick look through the usual places (dozens of out dated documents :) I find it hard to find some of these things.. if they're to be found at all. Lets cut to the chase - we want a good solid knowledge base and search, like Microsoft provides. (Well, better than they provide would be nice ;)

Licensee Madness

This is mostly old news, but it drove us all mental over the years. I still support many of these older devices so its still relevent to me but not for much longer I tell you. Especially, Sony as they were King of Bastards.

Sony pioneered high-resolution because PalmSource couldn't get something together fast enough I suppose, but they failed to think ahead and made a pretty dimwitted API. At least it worked more or less... but it was prone to crash. If the user or developer did certain unknown things, the device would just hang up - again, the developers took flack and it took ages to work out the myriad causes. To this day I keep an old device around for testing, since the problem randomly manifests. Sony never fixed this bug.. years would pass and the issue only went away when OS 5 came along and Sony pseudo-adopted the OS 5 high res system.

Course, Sony was evil for refusing to document how their joystick add-on operated. Their documentation to the developer consisted of saying "it just works", but alas it did not - it depended on some undocumented series of events occuring in order to work, so for all my gaming applications it worked ramdomly. (That was fascinating.. it should work, or not work. But working sometimes and not others.. bizarre.) It'd have been nice of they'd taken 20 mins to document the requirements.

They also refused to document the sound system; as usual, developers took flack for not supporting special audio on Sony devices, but such is life. Years passed before Cliepet, a whiz in the community, reverse engineered it all. Bastards.

I'll try not to go on, but the problem was PalmSource's lack in taking control of the reigns - why did Sony and Palm and Dana Alphasmart and Handera all work out different ways of doing things? Shadow Plan supports some 4 or 5 different incompatible techniques for managing high resolution. Again, think how much higher application quality, and how much lower costs would be, if we didn't have to both develop dozens of duplicate ways of doing things, and then support them over years!

Really, its amazing anyone developed for the platform at all when you look at this mess :)

At least Tapwave got most of it right.. a shame they tanked :/

Installations

A standard installer process would have been nice; heck, thinking ahead would've been nice, too, so that Sony and Palm installs could sit nicely side by side without blowing each other up. We rely on Beik's Pilot Catapult installer builder, since its always been simple and inexpensive and fairly stable, but even they can't quite keep it solid due to all the varieties of weird in the registry that the various licensees put in. (Microsoft doesn't help with their eternal OS monkeying on the desktop, making Hotsync Manager sometimes run in Administratior mode while other apps do not, etc.) A shame when PalmSource laid off the fine fellow working on a promising new standardized installer.

Quality Assurance

Last but not least is just the lack of QA here; how does it slip through the testing cracks that sometimes a file can be copied out to SD from RAM and then not back into RAM? Or that devices crash/reset more frequently now than ever? Since the new PIM databases weren't documented we were required ("supposed to") use the old PIM database interface (thus being denied the new features, plus gaining all the slowdowns and bugs) and yet the magic PIM sync would frequently crash. Crazy stuff, and all the blame comes back to developers.. with nothing we can do.

Whew!

When I began this opus I had not intended to write all this out .. I merely wanted to rant a bit, to get it off my chest how miserable the situation has been for years.. but out it poured.

Now that I'm all healed, you're suffering. Merry Christmas! :)

[ Category: / technology / mobile / palm ] [link] [Comments]