The Jackal Rants

I’ve added some basic EXIF metadata extraction support to the MediaFunctions extension, following a thread on wikitech-l, which uses Tim’s excellent work on media handlers, so we can do things like:

{{#mediaexif:File.jpg|Make}}

I’ve also introduced a {{#mediadimensions}} parser function, which fetches the natural “dimensions string” from the file – for images, this returns a width and height, e.g. “800×600″; for audio, a duration, e.g. “1m30s”, etc.

The MediaFunctions extension is in Subversion, but not live on Wikimedia sites at this time.

Well, these images are getting the better of me, eh? Seems I made an (idiotic, now I look on it) assumption about the “timestamp-like” part of file archive names when I wrote the FileRevertForm and the FileDeleteForm, and thus extracted the wrong timestamp for presentation in the UI. Good job nothing in the backend relied upon this.

In similar news, it seems MediaWiki’s inconsistent image markup tripped me up, too; RandomImage’s “size” parameter was being ignored for goodness knows how long.

Anyway, all fixed now. There’s some talk of parser functions for extracting metadata (EXIF, etc.) from files, which I might poke if Magnus doesn’t implement this as he seems to want to do. I also plan to poke the existing MediaFunctions extension a bit.

Some updates to the RandomImage extension:

• Captions will now be pulled from the image description page, if possible. Text within <randomimage></randomimage> tags is preferred, if present, otherwise, the first paragraph of the description is used. This idea came from Dave Cook.
• The extension can now perform a more complex SELECT to ensure that non-images are excluded from selection.

Update: Dave caught a bug, which I’ve fixed – when a size was specified, the caption wasn’t showing. This is due to the fact that I’m a touch lazy and the extension just generates wiki markup, passing it to the parser to do the actual work. Of course, this results in MediaWiki’s slightly inconsistent sizing/framing behaviour.

Just to follow up from the teaser screenshot I posted a short time ago, the forthcoming Screenshot extension is now demoing at http://wiki.anubite.co.uk/view/Slideshow.

It requires JavaScript, but will now show a decent fallback message if the client doesn’t support this, and messages (including button labels) can now be localised as normal.

I breezed over the “next generation MediaWiki” issues in the last post, but at least one person saw fit to comment on a particular part of it, so I feel obliged to post a bit more about it, with a clarification.

For some months, there have been discussions on the wikitech-l and mediawiki-l mailing lists regarding the next version of MediaWiki after 1.9. Brion Vibber, our lead developer and release manager, has stated that this would be 1.10, due for release this month or early next, and this sparked off a whole discussion about “MediaWiki 2.0″.

The problem (I feel, and it seems, so do some others) with labels such as “MediaWiki 2.0″ is that it encourages people to start planning for a big rewrite with a whole raft of new features, and we had some mailing list posts discussing this. Now, I’m not going to defend the entire code base and state that parts of it don’t suck, and I’m not going to get into a language war, either, but I don’t feel (and again, others actually working on the software agree) that a full-blown rewrite from scratch is at all the approach we need to take; our existing model of continuous development and integration works for us, and it works well.

There’s been a lot of time and effort invested by many people in developing techniques and experience for all aspects of running the Wikimedia cluster, including actual application development, optimisation (including SQL optimisation, etc. and a lot of trial and error) and the entire server operations side of things. A major rewrite of MediaWiki would discard much of this hard work, and would discard something that a lot of people have worked on since 2003.

Therefore, I was skeptical, as were a lot of developers, over the claims of some people that we needed to rewrite MediaWiki for version 2.0. This seems to have been the general consensus at a recent conference in Vancouver, and while I didn’t attend it, I’d like to think some of the comments I made on the general issue prior to that did something to help convince people. (Satisfied, Erik?)

So, as I mentioned before, the outcome of that was that the mediawiki-ng-l mailing list was set up. It’s on moderated subscriptions at the moment, but as explained in the introduction post, membership is not a difficult hurdle, although I am concerned that it seems to be adopting a secretive attitude.
With a bit of luck, we have a think tank established that will begin churning out useful ideas soon. With even more luck, that think tank will remain open to all of us…

We’ve got a lot of big features planned or being implemented which will add to the MediaWiki user experience, although some of them seem to have stalled, which is a shame.

Flagged Revisions

Also known as stable versions, this is being worked on under a contractor, with some volunteer aid, and is coming on quite well, from what I can gather. With a bit of luck, it can undergo the review process and perhaps be tested on a Wikipedia soon; I imagine the Germans are going to want the first pilot.

Single User Login

Not a lot mentioned on this front in a while; last I heard, Brion had completed some initial tests and estimated the number of conflicts, etc. that might occur when migrating user accounts across to the central authentication database. I’ve had a few users asking me about things like cross-wiki blocking, and cross-wiki permissions, and while these aren’t in the plans for square one, central authentication will make them easier, as well as opening up other possibilities.

LiquidThreads

Some sterling work from Dave McCabe during last year’s Summer of Code, which unfortunately appears to have been suspended, although he has been hired by Wikia, and there was talk of it being picked up again, which would be great. If that’s not the case, then I would suggest it would be well worth Wikimedia thinking about contracting someone to finish off development, as improve discussion features in MediaWiki would be damn useful.

Update: I’ve heard that there are talks in place between Wikia staff to have this work continued, and with luck, Dave’ll be the one finishing it.

Media Improvements

Tim Starling has done a huge chunk of work on the new image backend, in the form of a nice piece of storage middleware which handles requests to view, thumbnail, publish and remove media, etc. There’s also a lot of discussion over changes to the various media namespaces in preparation for a big overhaul to MediaWiki’s media handling, which, coupled with the work being done by this year’s Summer of Code student, ought to mean we have some nifty inline players and so forth within a year’s time.

Search Improvements

I’ve spotted some interesting-looking activities in the repository which look suspiciously like somebody’s about to make a whole bunch of improvements to MediaWiki’s Lucene Search extension and the backend daemon. Apache Lucene is a Java-based search engine, and we use it on the live sites, but searches could still do with some improvement, so perhaps we’re about to see some drastic new changes in that direction.

Next Generation

(Section removed in favour of later post)

Wikimania

Who knows what will be requested at Wikimania, where the development team can’t escape the hordes of users demanding autographs and bug fixes in equal measure? Who knows what next big thing might emerge from a five minute catch-up session?

While quite flexible, our existing parser hook mechanism is a bit limited for developers wanting to affect the OutputPage object representing the actual rendered page; things like adding links to stylesheets or attaching JavaScript aren’t easy to do, mainly due to caching.

I’ve been messing about in a branch, preparing a mergeable version of a not-too-ancient patch Brion put together which adds a neat mechanism for working around this; extension output handlers.

An extension output handler is basically an object which is attached to the ParserOutput, and so is serialised and stored in the parser cache along with everything else (page text, metadata, etc.). The idea I had for implementation, prior to inspecting Brion’s code and deciding it was a better solution, was to provide a couple of simple accessors and mutators in ParserOutput and use these, combined with hooks executed when the page is put together, to make the changes.

However, the output handler mechanism is a lot nicer, especially from the developer’s perspective, since it takes a lot of the hook hassle away. By avoiding the hook mechanism, it’s probably more efficient, although I don’t pretend to have any benchmarks to back up that guess at all.

The great news is I barely had to write any code at all…Brion’s patch was easily adaptable to the branch, and all I really did was write my own little test extension to make sure it actually worked…and it does, well. I’ve written up (hopefully) thorough documentation and put a prettified copy of it on the MediaWiki.org web site.

Now I’m waiting for the aforementioned to review and approve it for merging. Once this, or something offering the same capabilities, goes live, then we have much greater flexibility in the quality of parser hook extensions we can write, and for my mind, I’ve got my eye on the idea of a nice little slideshow extension…

The BookInformation extension is more or less done; at least, the first incarnation is. Checked it into Subversion, although it’s without documentation right now and doesn’t support anything other than Amazon Web Services.

I’ve put it live on my test wiki.

Oh, and thanks to Dirk, Raymond, and Niklas for prompt translations for Dutch, German and Finnish.

It’s extension writing time, again – a new one, BookInformation is en route soonish. It will allow site owners to configure web services such as the Amazon Web Services set that can be used to perform basic informational queries when users are accessing Special:Booksources.

The whole thing’s nice and object-oriented, and is based around the concept of a “book information driver”, which is a class implementing a straightforward interface spec, so writing drivers to handle providers other than Amazon will be relatively simple.

This probably won’t make it live on Wikipedia or any Wikimedia wiki, simply because we’d have to find a provider who would allow us to access their web service at a relatively high rate. There’s built-in caching of responses per-ISBN, but it wouldn’t choke the Wikipedian influx enough for quite some time.

I’ve just released version 1.1 of the RandomImage extension; the quick and dirty parser hook extension which tries to bring a bit of spontaneous dynamic content generation to pages.

The new version introduces a new configuration setting; $wgRandomImageNoCache, which defaults to being off. When set to true, however, it disables the parser cache for pages using the extension, which means true randomness. This could be a performance bitch for large sites with huge numbers of images and on large pages where the formatting alone should have been cached. Double-edged sword.

• RandomImage Extension on MediaWiki.org
• Extension code in Wikimedia Subversion
• README file