A few months ago NASA caused a stir by releasing a new global height dataset called ASTER. I use an earlier dataset (SRTM) for OpenCycleMap, which has a few problems that ASTER, at least initially, promised to solve. The three of primary interest to me are:

• Voids – SRTM has “no data” gaps in certain places of the world, where the radar reflections went haywire. These happen in marshes (not of interest) and mountains (of great interest!), especially over the Alps. ASTER is void-filled already, so the clever-but-inaccurate void-filling I use wouldn’t be necessary
• Resolution – It’s great that SRTM covers the whole world, but I’d love to see it at a higher resolution. ASTER’s nominal resolution is three times greater than SRTM, so it’s very attractive.
• Arctic coverage – SRTM only goes as far north as 60°N, which is a bit of a problem in Scandanavia. Although there’s GTOPO30 data for these areas, that’s got a horizontal resolution measured in kilometres, so not exactly great for me. ASTER covers those areas too, up to 83°N.

So far so good. But when I started work with ASTER in December, things spiralled rapidly downhill. First is the pointlessly irritating “order a dataset” website, that sucked up hours of going round in circles. It’s like a shopping website from 1999. You need to use a stupid interface to order which 1°x1° tiles you want, and “All” isn’t an option, despite there being 22,600 of them. It seems geared up for people who want a couple of dozen at a time, and the whole thing has a feel of being run by men with beards and sandals who’d rather you didn’t use their website in anything newer than Netscape 4 on HP-UX.

When I read the README alarm bells started ringing. There’s a section on “mole runs” and “pit artefacts” that sounded a bit worrying, but I wasn’t sure how much of an issue they’d be – if they were small and few and far between then that’s not much of a problem. But the biggest thing that caught my eye, buried on page six after pages of confirmation of how good the accuracy was, right at the end of a section as a throwaway comment, was the following statement:

Also, while the elevation postings in the ASTER GDEM are at 1 arc-second, or approximately 30 m, the detail of topographic expression resolvable in the ASTER GDEM appears to be between 100 m and 120 m.

That’s a bit of a bomb-shell – it’s saying that although it’s got a much higher nominal resolution than SRTM, it’s effective resolution is about the same – there’s not any more actual detail, just more pixels. That was almost enough to make me give up there and then, but it’s still void filled and covering more area of the planet, which would be good improvements. So I grabbed the DEM (thankfully, they’re straightforward GeoTIFFs) and got to work over Snowdon. I did some colouring and contours, and they both looked excellent and much better than what I made from SRTM. But then I tried hill-shading, and disaster!

Here’s the area around Snowdon (click through for original size):

and a detail of Snowdon itself:

The mole runs are everywhere – all across that image, even on the flat bits. And the pit artefacts are huge – the size of quarries, and really, really obvious. I honestly can’t use that – maybe for a “what the world would look like if it looked like the moon” project, but nothing more serious than that. And considering that SRTM has only a handful of single-pixel voids in that area, the guys making ASTER have made something that’s substantially worse than an oversampled SRTM. And considering they were even using SRTM to fill the gaps in the ASTER data, that’s a pretty poor show. I started reading around and found a few people saying similar things. And when I though about it, the “improvements” to the contours I saw could be recreated with SRTM by using gdalwarp to artificially increase the resolution (with some nice smoothing) before generating the contour lines. So I gain nothing from ASTER in the 95% of the planet that doesn’t have significant voids, and in that same 95% it’s not really usuable.

So for now, I’ve given up with ASTER. I might revisit it for the band between 60°N and 83°N, but it also says in the readme they have voids over eurasia for that area (so much for void-filled, eh?). And it would be interesting to see if someone can fill the large SRTM voids with ASTER (which sounds back to front, hey-ho), but I don’t have time for that. However, as they say in the docs, all these artefacts are happening in the boundaries where they have different numbers of original samples, so maybe a future version will have these automatically smoothed out, and it they can figure out how to stop their 15m sampling getting turned into 120m effective resolution, that would be awesome. But for now I would say it’s not worth using it.

The war against the spam commenters is, it seems, endless. Last week I installed a well-know plugin (Askimet) to help protect this blog – previously all comments were being moderated by me, which led to big delays for honest commenters. After a few days of successful operation, I switched off the moderation on Friday, only to be faced with a handful of spam comments making it through last night. Grrr, indeed.

So the moderation is back on again, but only for first time posters (first time since I upgraded a few weeks ago, I think that is). I’m especially disappointed that Askimet didn’t pick up on comments which were exactly the same format as the previous 200 that I had got.

Still, it’s not really the plugin’s fault – one day, there will be much vengenance wreaked by everyone who has ever run their own blog…

Stumbled across the ToryScum website this morning (I think I’ve seen it before though). The thing that lead me there were the defaced Tory posters – check out the “Subverts” category. Priceless.

Now if only I knew of an organisation that had both ladders and pots of black + white paint to hand. Hmmmm….

(For those of you trying to avoid being roped into caring about the election, feel free to ponder the joys of being able to link directly to a blog category instead of a collection of individual posts. Nice.)

(And for those of you who are really into the whole election thing: “Compulsory voting : bringing an end to low turnout caused by intimidatio and negative campaigning. Discuss.”.)

I’ve had around 700 spam comments and now trackbacks since the start of the holidays, and I’m fed up with it. For the moment all comments are being moderated until I get round to putting in place one of these anti-spam things like Mike has. The trackback spam is a new one today, but since everyone running WordPress is being hammered, I expect the new versions to be able to deal with it better.

And I feel ill, which isn’t helping.

Another of those “I couldn’t find the answer on Google so I’ll write it here” postings.

By default, Thunderbird only checks your Inbox for new messages (which stems from a POP3 legacy). If you have server-side rules (or use a web interface to do your rules), then you’ll want Thunderbird to check subfolders as well. This can be done by right-clicking the folder in question, selecting properties, and ticking the appropriate box. There’s no(t yet an) option to do this globally.

Applies to Thunderbird 0.9.

Update: April 2008

Years later, even with Thunderbird 2.0, this is still a problem. But there’s now a global option, thankfully. Although it’s completely hidden, it saves a lot of time if you’ve got lots of subfolders.

Open up the preferences dialog (either Tools -> Preferences or Edit -> Preferences), and click Advanced -> General -> Config Editor. In the filter box start typing ‘mail.check’ and then double click on mail.check_all_imap_folders_for_new. It should then change the value to true (and it’ll appear in bold since you’ve changed it) and all should hopefully be hunky-dory.

I’ve now added all the posts from my old Chasing Rainbows weblog to this one. Although I’m not exactly proud of some of the posts (to put it mildly), there’s just about enough interesting things in there to make it worth while keeping them. So with a trusty SSH session and some PHP (and a fair bit of luck) I managed to convert my old weblog database into the new Wordpress format, and managed not to screw anything up.

I’m not going to turn on comments for them though, and I’m not going to go through them all and try classifying them, since that would be a bit boring.

Today’s exercise in coding reminded me again why I’ve made the switch from a home-grown weblog to using a properly developed one (not quite off-the-shelf or shrink-wrapped, but you know what I mean). Next time I want to change backend, I know that someone else will have written an import script, and I won’t have to bodge one together myself. And now that I’ve spent about two working-days on the switch, I’ve gone from entering data into the database by hand, to having a proper admin interface, categories, sub-categories, comments, pingback and trackback, drafting, private entries, searching, linkrolls, plugin interfaces… which would have taken quite some time if I’d done it myself. And every time I’m in the pub, or sleeping, or whatever else, I know someone else is improving the software that runs my weblog. Which is nice. And I was getting fed up of the whole wheel-reimplementation thing anyway.