townx

I got so sick of deleting spam off my website, I decided to install various Captcha modules to stop it happening. I've done a few quick tests, and they work far better than they did about 6 months ago. I've actually got a range of different, random captchas installed, ranging from ascii art, to phrase completion, maths questions, and missing letters. It does mean leaving a comment on my site is a bit like the Krypton Factor (for those of you old enough to remember that), but hopefully it will cut down on my spam (which was up to about 50 comments a day NOT being caught by the filters; upwards of 100-200 per day which were getting caught). I may even turn the spam filters off, as they always seemed to rate comments from my friends as spam, while letting through endless crap about viagra and porn.

Update: Paul's code is MIT licenced, so I'm putting mine out under the same licence.

A while back, I wrote some simple scripts for parsing the BBC iPlayer search pages, so I could get emails about programmes I was keeping an eye on. I also wrote a little Rack server interface so I could browse my search results easily. The downer was that I could only watch them online as downloads aren't supported on Linux, so I ended up having to go to the BBC website to watch them in a tiny window, or boot my Windows VM to download a DRM-crippled version.

However, Rob Styles noted today that there's a Ruby script out there (http://po-ru.com/diary/keeping-up-with-iplayer-dl/ by Paul Battley) which mimics an Apple iPhone, enabling you to download programmes as MP4 files without DRM. I don't have an iPhone, but I can't see why it should only be design know-it-alls who get this privilege. So I integrated that script with my Ruby/Rack web front end, spiced it up a bit, and away I go. I can now search for programmes and download the mp4s to my hard disk (if they exist - not every BBC programme has an mp4 version). Nice. Turns out Rack can also multi-thread in a decent fashion (unlike Rails), so I can even have several downloads running at once.

The code is attached. There's no licence on the download script I'm using, so I hesitate to release it under any kind of open source licence (yet). I'll contact Paul and find out what licence he's releasing under. I made a few modifications to his script to wrap it in a class for use in my app., but not many. See my previous entry (linked at the top of this post) for proper instructions/installation etc..

Also, please don't expect very clean, well-tested code here: it's something I've thrown together for my own use, and I'm putting it out there for others to use if they like. Also, I can't guarantee it will work on Windows at all. Enjoy.

This year's gardening project, since we've recently moved house, is to have a vegetable patch. I'm not particularly fussed about where our fruit and veg. come from: we buy local organic mostly, but I'm not strict about it. So it's not through any sense of wanting to be self-sufficient or mistrusting suppliers. It's more a case of wanting a challenge; plus there's a sort of vegetable patch in the garden already; the aspect is good (south facing with some shady parts); soil is heavy clay but we had loads of compost we could add from pots, growbags and compost bins left by the last inhabitants; and I think it's something Madeleine will enjoy.

Anyway, I decided to start small with half a dozen easy to grow crops (germinate well, grow fast and are reasonably tough) which we enjoy eating or which taste better really fresh. I've gone for:

• Spinach 'Tardy'
• Lettuce 'Catalogna'
• Sweetcorn 'Sweet Nugget' (short plants; I love corn on the cob)
• Wild rocket
• Pumpkins (that one was Madeleine's choice - they take up a lot of room, but I like pumpkin soup etc., you can make Halloween lanterns from them, and they're impressive plants)

My choices of variety were based on reading Organic Gardening by Charles Dowding, and browsing The Organic Gardening Catalogue. I ordered the seeds from the latter, which seems reasonably priced. Delivery was pretty fast and accurate. The book I mentioned is interesting and gives some good tips on when to sow and which varieties to go for. (I've been doing organic gardening for about 8 years and it's taught me to grow the things which want to grow, grow plants which resist the local pests, and to encourage beneficial insects and animals. So I fully expect lots of failures all the time when gardening, and use those failures as object lessons.)

We planted some in the "greenhouse" (more like a green polytunnel) in newspaper pots Madeleine and I made together using this Paper Potter implement, which is part of the fun. Madeleine likes making the pots and filling them with compost, pushing the seeds in, watering them. We also put a few wonky rows of seeds into one of the beds, as per instructions, and labelling them; though the wind and Madeleine's hands ensured that we'll probably just end up with a big mixed patch of all sorts of stuff. All part of the fun.

We're also doing sunflowers and nasturtiums, both of which are easy and produce good results. You can eat the nasturtiums too, of course.

This weekend I also harvested a load of spring onions which were already growing in the garden, planted by the previous owners, and have been making honey and mustard salad dressing to go with them. I'll probably be back onto making pickles before too long.

Here's hoping we get at least a few of our own plants to eat. I keep checking the pumpkin seeds we planted last week, hoping that some of them come up, as Madeleine seems quite keen on those growing. Wish us luck.

First time in my life I've seen one of these birds for real, and I'm 38 years old. I find that incredible. It was eating berries in the neighbours' garden.

Here's what the little fellow looked like:

(Photo released under a Creative Commons by-nc-sa licence by Marko_K; I took a picture, but it wasn't particularly clear, so I used this one instead.)

That'll be your man, Rahsaan Roland Kirk. Later in the video, he invites the audience to take whistles and join in with his band, in the key of W; then plays jazz to a tiger while a child rides on his back; then plays a flute and annoys a fox. Interspersed with the very droll John Cage talking about music while preparing for a performance with a musical bicycle. I just realised John Cage sounds very much like Vincent Price, which may partly be why I like him so much.

On a related note, I'm doing my monthly hunt around YouTube for interesting videos. Here's what I found:

• This little gem is Can in the studio. What I particularly like is the intense concentration, interplay between the band members, and the frenetic activity which initially produces light ripples of sound but builds into a flutter of noise. Also I like their long hair and the way Damo Suzuki is standing in a cubby hole at the back to do his vocals.

• Here's another one, a performance of Bring Me Coffee or Tea by Can.

• Class: Can playing I Want More on Top of the Pops in the 70s.

• I just discovered the largely forgotten and underrated at the time Finitribe did a cover version of I Want More. Not as good as Can's, but interesting.

• Also discovered Finitribe did a cover version of The Electrician, one of Scott Walker's rather extraordinary mid-career experiments (when he was transitioning from his crooner phase to the all-out avant gardism of his current work).

• And this is one my favourite Can tracks, Oh Yeah, bizarrely shown on MTV. As someone points out in the comments on this one, you can definitely hear what Radiohead have been listening to. I liked the man juggling umbrellas in the middle of their live act. This is the song which The Fall pastiched in their Can tribute, I Am Damo Suzuki, which is also great. Here's a version of that.

• Speaking of The Fall pastiches, Spinal Tap's Tonight I'm Gonna Rock You (the first track at the start of the film) has the same bass line as Athlete Cured off The Fall's The Frenz Experiment album.

• Here are Portishead's performances on Later this week, preceding the release of their new album (the first in 11 years):
  • Machine Gun. Pretty brutal that one.
  • The Rip.
  • We Carry On.
    Somehow I think the album's going to lack the dinner party appeal of their previous efforts.

I just read Paul Graham's list of heroes, which was an interesting read. It got me thinking about my heroes. There probably aren't many, and I'm not in the mood to expand on the list, other than to throw out a few names like J.G. Ballard, Scott Walker, Max Ernst, Marcel Duchamp. The thing they have in common is that they all followed their own path, pushed at boundaries (still are in some cases), changed art. Those are my run-of-the-mill heroes.

But when I think about my mum, who died in 2003, I realise how much of a hero she was and is to me. I don't talk about her much, or how painful it was when she died; but I still think about her often, and the older I get, the more admiration for her I have. Here's why she's one of my heroes:

• She always encouraged me to think for myself and be creative. She was very creative herself, and I remember her painting when I was little, then going to art lessons, knitting, crocheting, making cushions. There was a room full of patchwork material left behind when she died.
• She was always proud of me. One time I remember her being critical of one of my stories, and it broke my heart, I cared so much for her opinion.
• She gave me enormous freedom when I got older, basically letting me do what I liked. I used to be nocturnal when I came home from university, and she never complained. She also let us cook our own food and eat what we wanted as soon as we were old enough (though she still guided me into reasonable eating habits). But because she brought me up to be respectful of other people, I knew well enough not to go wild. In fact, she was always encouraging me to go out and socialise, as I preferred to stay home a lot (very shy).
• She was very understanding of my dislike for games lessons, and basically let me skive off them if I felt like it.
• She gave up a lot to give our family stability. I think she was unhappy about living in Spalding (it is a lovely town, but was crap if you wanted any kind of access to the outside world, like music or films or art). She was always talking about moving away. But we stayed because Dad is happy there, and it's a good place to bring up young children (I think). I often think that she yearned for a bigger life, and that she sacrificed that for the happiness of those around her.
• She had no truck with religion. I'm not anti-Christian or anything, and neither was she as far as I know, but I remember asking her "What happens after you die?" as part of my homework when I was about 11, and her replying: "You go into the ground and get eaten by worms. That's why I want to be cremated." (She was.) She was basically an atheist, and had a humanist funeral. But what I admire was that she was straightforward with us and had firm beliefs.
• She sparked my love of science fiction. Her shelves were absolutely full of Michael Moorcock books (dozens of them), along with loads of other fantasy and science fiction. I started reading them when I was about 9 or 10, and she let me, despite them being full of sex, violence and wierdness. As she got older, she moved onto horror, and used to read Stephen King novels, sometimes while knitting with the television on. None of my friends' mums had the faintest interest in what I was interested in. I think she may also have watched every single detective series ever shown on terrestrial television.
• She always made a real effort whenever I went home to visit, getting in all my favourite foods.

She wasn't perfect: she had a foul temper, and I think she struggled a lot when I and my siblings were young. She regretted her treatment of us as we got into our teens. But she was always there for us, always caring, always forgiving.

I've won another £50 on the premium bonds. Result! That's 100 quid since November. Definitely a good investment, and a 10% return on my investment since 9th November 2006. So probably slightly more than I would have earned from a savings account over the same period.

My Rails book has ended up on a free download site. No comment.

In a RESTful web application design, you typically first identify the resources in your application, the nouns. For example, imagine you're writing a library app., and you're working on adding items to a catalogue. So you've got catalogues and items as your nouns.

You then decide to implement the operations on items, which live in the catalogue. In REST, HTTP verbs map onto the operations you want to perform: POST = create a new resource when you don't know what its identifier should be; PUT = update; DELETE = delete; GET = query resources. So you might end up with:

Fairly typical REST.

Then you realise you want to upload a whole pile of items at once, embedded in a single request for efficiency; but you don't want the client to have to wait while the items are inserted into the catalogue and properly indexed etc.. Maybe it will take 5 minutes or something, and you don't want to leave a web client hanging. Or perhaps you want to upload only a single item, but once items are uploaded they are put into a queue for processing by another system, so there's a wait.

What are your choices? Here are some ideas, partly gleaned from the RESTful Web Services book:

• Don't allow bulk uploads. You can only upload one item at a time, and you just have to wait until the operation completes and you get your proper status code back. Not really a solution, though.
• Allow bulk uploads, process the items, and return a multi-status 207 code with the response. You still have to wait for all the processing to finish, but the response body contains a list of response codes and status reports, one for each uploaded item. Again, you have to wait for the upload to finish before you can return anything.
• Allow a bulk upload, but return a 202 Accepted status, spawning asynchronous jobs to do the processing in the background. The response body can contain the URIs for each uploaded item. Each item then has a status which can be queried by asking for the resource again. For example, when an item is first uploaded, you get its URI back as a Location; when you GET that, the object has its status set to Inactive or Under processing or something. When processing is complete, you get a status like Complete on the item instead. The down-side is that your resource representations are polluted with status information, which could be good or bad.
• As above, but your request to /catalogue/items returns a 202 along with a handle to a "transaction" or "job" resource which wraps the resources you uploaded. Effectively, you treat the upload itself as a resource with a status you can query; the resources attached to that resource don't have to be polluted with status information, but you perhaps lose the granularity of individual status codes on resources. Or maybe you produce one transaction resource per uploaded resource?

However, what I'm not so keen on is the idea of a job or service being a resource. Why? Well, if I want to create items in my catalog, I don't want to wrap them in a job and post them to /jobs; if I want to query my items, I don't want to have to go to a query service at /services/query or similar.

What these paths hint at to me is that an operation is being represented by that path, rather than a resource: effectively, calling them is like doing RPC: you pass the resources you want to act on as arguments to the procedure you're calling. Often, there's also some implicit resource hidden away behind the job or service. Compare:

• GET /catalog/items?term=potter: the catalogue is visible, and we know we're querying items within it
• GET /services/query?term=potter: here there's an implicit catalogue and its items behind the service; effectively, these objects are passed invisibly to the procedure we're calling; also what we're querying is not explicit

Or:

• POST /catalog/items: we're appending a new item to the catalogue; we can infer that our new item will then be available at /catalog/items/<some identifier>
• POST /jobs: job is an amorphous category, and we could post pretty much any type of resource into it; and there aren't any hints from the API about how to get at those resources once we've posted them

It's kind of like the difference between object-oriented design (REST) and procedural design (RPC). While a job might look like a resource, my opinion is that it's really an amorphous wrapper around the real resource you should be representing. Typically, jobs get introduced to cope with asynchronous updates; I'd prefer to see asynchronous operations occurring on proper resources, but exposed using the batch processing approaches outlined above. Otherwise I fear you might lose your resources inside some vague blob of a "job" or "service".

I did my Ph.D. in artificial intelligence, so was interested to read a few Wikipedia articles about it. One distinction I'd never heard of was neats vs. scruffies in the field.

I put myself in the scruffies camp, probably, though I always had a yen for predicate logic and formal grammars. To my mind, some of the AI scruffies weren't scruffy enough, and tried to model human intelligence without any reference to psychological data. I tried to redress the balance a bit, and compared my program's output with psychological data on human inference during story comprehension. You can read all about it here.

At the time I did my Ph.D., I was pretty unfashionable, as I was researching symbolic AI approaches, while everyone around me seemed to be doing neural networks. However, I thought that while sub-symbolic approaches might produce intelligent output, I struggled to see how that would lead to a description of the solution, or anything that might be built on or added to by humans. If you're trying to program a reasoning system, for example, is it enough to train a neural network to create associations, or do you need to write something which can reflect on the process by which it reached its solutions? Neural nets are great for recognition tasks, but I was never convinced they were suitable for reflecting on how they completed the task. I'm sure there are plenty of counter-arguments to my limited opinion, so feel free to enlighten me.