Curiouser and Curiouser! on open-data-formats

Word escape velocity attained at last!

Mon, 26 Apr 2004 09:22:41 +0100

Using Open Office to convert MS Word documents. Rickard Öberg recently posted a request for suggestions about using Java to convert MS word docs into HTML. I have been doing some work on this lately using the freely availiable, open-source OpenOffice.org to do the hard parts, making calls... [sockdrawer.org]

Paul has done some very craft work here -- many people want to solve this problem.

In fact it's a problem that he and I worked on last year. At that time we were looking for out of the box tools to do the job and not getting very far with it. Since then he's cooked up a clever solution by implementing RPC with an OpenOffice server. Neat!!

Combining OPML and RSS to create an export format for a blog

Mon, 10 Apr 2006 09:30:46 +0100

Marc Canter links to Joe Brockmeier's post about weblogs having a shared format. Timely.

I've been thinking about this myself because I want such a format too. Although I have written a tool to serve my own needs I won't be using it forever and I (probably) want to take my blog with me. I've also been thinking about how to do backup and restore. The two problems appear to be the same to me.

I think we already have the answer: RSS. It's already a natural format for holding the essential data of a weblog and namespacing is an easy way to store the tool-specific data. A tool that understands another tools metadata (e.g. ENT topics) can import it, a tool that cannot can safely ignore it. Actually why are we even discussing this?

The real question seems to me to be: how best to use RSS for this purpose? Do we have one gigantic RSS feed for a weblog? In my case with about 2100 posts it would be pretty big and unwieldy. Back in 2004 Paolo and I were talking about how to do weblog archives.

I was messing with an approach that combined RSS and OPML to create a weblog archive. For each post/day/month (pick your granularity) create a corresponding RSS feed of weblog entries. These feeds are then referenced from an OPML file that defines the overall structure of the archived weblog. In this way you can quickly narrow down to find an individual post, or suck up the whole thing (useful for tools like Sigmund).

For convenience the whole lot could be wrapped up in a .tar.gz. It might be helpful to include some kind of (optional?) metadata file at the top-level that describes the contents (ala JAR archives).

I'm not sure why I stopped working on that, maybe it just got shoved aside by other things. I might have a go at adding this feature to Squib since we need a backup format anyway.

An experimental OPML+RSS archive for C&C

Wed, 12 Apr 2006 19:48:29 +0100

Over the last couple of days I've hacked together experimental support for OPML+RSS archives in Squib as I described a couple of days ago. You can grab my entire archive from here either directly or as a .tar.gz archive.

The structure of the archive looks like this:

The weblog.opml file is an outline that contains the date-based structure of Curiouser and Curiouser. There is a branch for each year, and each month of each year. At the leaves are pointers to daily RSS files and the ID & title of entries.

It occurred to me that I could just put the entire entry data directly into the OPML file and cut out the RSS. However, with over 2,100 entries, I felt that would lead to a very big and unwieldy file. Being just a file of pointers means it can still be sensibly opened in an OPML editor.

Another reason for using RSS is to ensure that users of the archive can take advantage of all the software out there to parse RSS. Once you've figured out which days entries you want, you can hand the corresponding RSS file to a standard parser and get back the entries.

Update: I've added <link rel="archive" type="application/opml+xml" href=".../archive/data/weblog.opml" /> to my home page to allow archive auto-discovery. I did a minimum amount of research before doing this so please correct me if that's a gross misuse of a link tag or there is some established way of doing this already.