Curiouser and Curiouser! on rss

How many applications could be generating dynamic RSS feeds?

Thu, 13 Jun 2002 11:18:23 +0100

I am familiar with the Livelink knowledge management system from OpenText having implemented it a few years back. They already generate feeds of both news (in the conventional sense of bulletins) and also "what's changed" reports. Both of these could be easily exported from the system in RSS format making it easy for project users with an aggregator to keep up with current events in their project.

As someone on the K-Logs mailing list pointed out it would be quite easy to make blogging part of your "project journal" posting items aggregated from project applications and sources and appending your own ideas, notes and comments (to be aggregated and read by others).

Additionally if a tool such as Livelink was capable of reading and indexing an RSS feed it could then close the loop and bring these "project journals" back into the system for archiving and searching by the wider audience.

Just a bit of speculation....

Weblogs, communities and tools

Thu, 13 Jun 2002 12:51:50 +0100

Weblogs and communities.

There are a lot of great ideas going around and I look forward to the tools and discussion this will generate. However, I think of this issue in a slightly different way, while linking neighborhoods, syndications, etc. all highlight the mechanical connections between related weblogs what interests me most are the conceptual threads of conversation that cross through many weblogs. This means extracting the relevant portions of many conversations, organizing the responses, editorializing the content, etc.

Everyone talks about wanting to create communities, but it seems most of the proposals really address how to create collections. To me, a community is about discourse and participation, not just relationships. I don't only want to know who's like me, I want to interact with them to create great ideas and products drawing from our shared experience. What's more, I want to filter or focus on real analysis, not just the link parroting that Blogdex and Daypop tend to highlight.

[rjsjr :: Robert J. Seymour, Jr.]

Robert is a new voice to me but I'm glad to have met him as he has a great perspective.

So far I've been concentrating on the tools to collect together the members of a disparate community in a dynamic fashion (it's the problem that drove me to think about this in the first place) but Robert highlights that the business of community is really about discourse and exchange.

Part of the BlogPlex manifesto (in progress) is that a BlogPlexa should actually be able to offer you useful services. The first service being of course, that it introduces you to other people in a wider community. Hopefully we can

I think we need a discussion about the services/facilities that we can provide to people.

If chat, threaded discussion and file sharing were not important to people I think that newer mediums such as Groove and organic mediums such as SlashDot wouldn't have them at their heart.

However I think we also need new tools that fit the medium of blogging. In this message on k-log Phil Wainewright discusses the possibility of shared aggregators:

"The key point I'm making is one that I feel would be of immense value in
networks of k-logs: being able to read an RSS feed of someone else's RSS
aggregation:"

I agree and I think it would be interesting to consider how a BlogPlex might offer a shared (& digested) aggregation of the RSS feeds of each the current membership (remember it's dynamic).

DayPop does RSS

Sat, 15 Jun 2002 11:27:33 +0100

Daniel Chan very kindly pointed out to me that DayPop can provide results formatted as an RSS stream. This should be pretty cool for integrating searches. I'd still be keen to see a web services API appear though.

Right on the money

Wed, 19 Jun 2002 20:55:46 +0100

John Robb. How to boost employee productivity by using a news aggregator. [klogs]

A small change in the way we work could shave 45 minutes off of the average workday. That small change is to use a news aggregator to get news instead of gathering it by hand. Applied across a company, that 45 minutes of savings could be worth $1,650,000 a year. The wild part is that the cost to implement this is only $8,000 and requires little if any support from the IT department.

Accurate K-Logging of current activities: status, thinking, plans, projects, etc.
Online presentations, to-do lists, project plans via outlines.
K-Log personal portals that integrate all connection info (e-mail, IM, phone, address, bio, resume, picture).

Very simple stuff can yield big results. [John Robb's Radio Weblog]

» John Robb's right on the money again. I'm really starting to love reading John's blogging (via my aggregator of course)

I'm looking at everything I do now in terms of whether it can be output as OPML for instant outlining, or as RSS for aggregation, or both.

Radio vs. Email

Sat, 22 Jun 2002 01:32:39 +0100

Steve Yost on ubiquitous collaboration tools. Steve Yost, inventor and proprietor of QuickTopic, disagrees with David Weinberger's assertion that collaborative software fails to thrive because companies are afraid to "hyperlink the hierarchy." The real problem is more mundane, Steve says: ... [Jon's Radio]

An intriguing hypothesis on the challenges of getting new technology ideas to take root in organizations

[McGee's Musings]

» Interesting. From a quick scan I'm not sure how QuickTopic differs from, say, using a Yahoo group where participants can either use it as a list (with single & digest options) or a web forum. I have switched from reading radio-dev as a list to using it as a web forum. But that's because I use Outlook for most of my email and it just sucks, sucks, sucks!

But beyond that I would be looking for radio-dev to arrive as an RSS feed. In fact much of the stuff I currently receive as email would be better arriving in my Radio news aggregator. Of course, at that point my aggregator is going to have to become a lot smarter and work much harder for me.

The other technology that could do for email is Instant Outlining (IO). Radio is getting interesting in this regard. It's going to take a lot of work to make it a killer app, but it's certainly on that track.

As someone else has said, Radio: the best $40 I ever spent on software!

Clever RSS

Sat, 22 Jun 2002 17:47:26 +0100

Now that postings can be annotated with topics to describe what they are about it would be interesting to be able to put those topics into the RSS feed somehow. This would allow clever aggregators to offer filtering tools to their users.

Bringing topics to RSS

Sun, 30 Jun 2002 10:10:55 +0100

Bringing metadata back into RSS with subject taxonomies. I hope to be able to spend time experimenting with XML again after a few projects I'm working on settle down. First on my plate will be to read more about how to bring subject-headings/topics into RSS. Specifically, I think XTM for Topic Maps, the RSS taxonomy and Dublin Core modules, and Peter's XFML are where I am going to be spending my time at first. [ia/ - news for information architects]

» Serendipity!

I'm also interested in how to add topic-based metadata to RSS feeds. I'd come across XTM a while back but not had an immediate use for it. Time to start reading I guess.

This post is part of the liveTopics demo.

Tue, 02 Jul 2002 19:30:38 +0100

Radio is a really cool application. You can do lots of interesting things with it. Like publishing information in RSS format.

Email vs. k-logging

Mon, 15 Jul 2002 22:23:33 +0100

Email Email Everywhere.

E-Mail Storage Issues Facing North American Companies

According to a recently-released whitepaper from Osterman Research, 31% of North American companies say the average size of an e-mail mailbox in their message system is between 26 and 50 megabytes (Mb). Additionally, 46% of these companies say that e-mail users in their system send up to 50 messages per day....

There has to be a way for k-logging to help with this for at least a percentage of these people. Luckily, we don't have quotas in place at SLS or else my external email would be a real problem. Here I am with my own blog, I'm trying to move into k-logging, and I really haven't integrated email into that equation yet. How on earth am I going to get my staff to do this?

Are there any guidelines out there yet for how to integrate various information sources (web, email, chat, etc.) into a k-log, or is the format still too young?

[The Shifted Librarian]

» Too many good questions here I'm afraid.

My experience of KM leads me to expect that k-logging will not provide a turn-key answer to managing email. What it will do is, in all practical terms, to kill email. That's the solution.

Many of the business contexts for e-mail could be replaced by publish & subscribe RSS feeds and Wiki leaving e-mail purely for private correspondance. If we could solve this spam thing too then you might see mailboxs drop back to pre-1996 levels again.

I'd be interesting to hear what other people think on this topic.

Using RSS to track software updates

Tue, 16 Jul 2002 21:38:57 +0100

New activeRenderer.root Updates RSS Feed. Following UserLand's lead, I've created a new RSS feed where I log all modifications to activeRenderer. This way, aR users who subscribe to this feed can get an idea of which new aR parts are downloaded . [read more] [s l a m]

» This is a very good idea.

Integrating klogs with Big-KM

Thu, 08 Aug 2002 23:08:31 +0100

In order for klogging to be successfully I think it is going to have to come to an understanding with Big-KM.

Example: BigCo has invested half a million dollars in a big knowledge management system for their world-wide operations. This kind of investment can become a lode-stone around any other systems neck. For klogging to thrive here it is going to have to integrate.

Here's one idea I have for how this could work.

Extend Big-KM System-X so that it can aggregate RSS feeds like Radio, MT and others do now.
Extend your klogging software to allow per-post meta data. (liveTopics does this for Radio)
For each project in System-X define a set of topics that will act as trigger phrases for that project
Get the kloggers to use those topics when they want to involve a post in a particular project
Now subscribe System-X to every klog in the organization and watch as it indexes and archives all that information. Each project grabbing only those postings that are appropriate (by use of the trigger phrases)
This means that the klogs add value to the big-KM system. Suddenly it doesn't just have the dry dusty project documention, but all the live vibrant stuff that people are really doing!
Now extend System-X to generate a per-project RSS feed.
If I am on the project I can subscribe to this feed. Now instead of receiving email from System-X or having to go to an arbitrary web page, I get all the "official" project stuff (new documents, forms etc...) delivered in my RSS stream.

Closing the loop between the big-KM and the klog so that they both add value to each other.

Just an idea....

TrackBack and RSS

Fri, 09 Aug 2002 21:58:47 +0100

Staying on theme...

I see MT's TrackBack has the ability to generate some RSS output, but it doesn't actually look like RSS (as in, I don't think you could just subscribe your aggregator to it).

Here's what I have in mind. I want to know whenever someone pings any of my pages. I register with the TrackBack server a pattern like http://radio.weblogs.com/0107808/*

Whenever someone pings a page under that URL it gets added to a dynamic RSS feed with a URL generated for me. I can subscribe to that feed in Radio and see all the trackback pings appear as news items.

If I'm interested in who is writing about things Dave is saying I might add a pattern http://www.scripting.com/* and get his TrackBack results as well.

Does MT's TrackBack do this already? I couldn't really tell what it's RSS was doing...

Radio news handling

Sun, 11 Aug 2002 16:30:07 +0100

More Flexible News Scanning Needed. This really hits the mark -- I've been traveling for two weeks with very limited connectivity. [Blunt Force Trauma]

» I too wish that Radio would handle news more flexibly. The idea of a "poll now" button would be genuinely useful as would a way to adjust the frequency with which different RSS feeds are polled.

However when you get to the point where you want to poll every 3 minutes I think you may be reaching the breaking point for the medium. I would be thinking about moving to Groove, Instant Messenger, Shared/Instant outlining or something like that to handle a real-time interaction. The results of that interaction could then be published klog style for everyone to share.

Getting referrers via RSS.

Thu, 15 Aug 2002 10:52:12 +0100

Radio Wishlist - RCS Referers: RSS feed and rolling 24 hours..

Can I get my referer lists as "RSS" feeds from the "Radio Community Server"?

Can we make the list a rolling 24 or 25 hours instead of a clean sweep at midnight?

[aka Blue Sky Radio]

[a klog apart]

» Now this would be cool.

TrackBack information should also appear this way (that's how I'm implementing it).

Aggregating IA/ is pretty sucky

Tue, 20 Aug 2002 13:18:45 +0100

What is it about Radio's new aggregator and the IA/ feed?

I seem to get all their articles over and over, no matter how many times I delete them. I've even got three copies of the whole set in the aggregator at the moment.

This sucks!

Active reading with liveTopics

Thu, 22 Aug 2002 11:05:40 +0100

liveTopics to create virtual weblog channels.

Here's an idea I've been thinking about for the use of liveTopics.

What I'd like to do is offer the reader is the chance to create their own categories and here's how I think it would work:

The next time the reader visits the page they only get posts that match the selected "virtual channel." along with a drop-down to change channel and the customise button.

Anybody else think this is an interesting idea? [Curiouser and curiouser!]

It is interesting, but in my view it would be even better if we could subscribe to a RSS feed for the shared categories we like. This is a generalization of KMPings. The following step would be to document and interlink the shared categories in a (shared) wiki.

[Seb's Open Research]

» Agreed. For a while now the ability to create dynamic channels from the RSS feed has been part of my plan. RSS 0.92 already has the necessary tag that can be co-opted to take topics instead / as well. I've patched the Radio RSS generator to do this, but need to do some more work in terms of turning this patch into something that can be distributed in Radio.

A key advantage would be the ability to have multiple RSS feeds aggregated together into a channel when they reference the same topics.

However I still think there is a role for the browser view. Despite the growing popularity of RSS I think people are still going to want to read some sites and at the moment they don't work for the reader.

The way I see it is that this is a control thing in the same way that blogging itself is a control thing. As a blogger I write what I like, that's where I get my control. But as a reader I have no control. I see what the writer intends (or not -- consider the font sizing issue that has been hot recently).

I was just musing that it would be interesting for the reader of a site to be able to re-frame the content in a way that suited them. But RSS does come first...

The quest for the holy RSS

Fri, 06 Sep 2002 10:30:33 +0100

Babble out in simile....

Dave Winer opines:

I've been posting this question on various weblogs and in private emails. The question is this. Why not add language to the spec that says it's okay for an RSS feed to include elements not defined in the spec, and leave it at that.

Please. God. No.

Pretty much every element of RSS0.94 is optional, and can be left out. If you also allow arbitrary elements to be added, do you have a specification any more?
In the RSS 0.9x series, a valid version 0.9x document is also a valid 0.9x+1 document. If you allow arbitrary elements you break this contract, or at least make it impossible to add new elements to the specification. Each added element becomes a redefining of a previously permitted but undefined element.
It would discourage the adoption of RSS modules (for which there are already a good set of defined standards, and which can be mixed and matched) in favour of a hundred splintered dialects of the core RSS, which may be mutually incompatible.

Oh, and ten points for someone who can tell me where this post's title (Babble out in simile, since the titles are only visible in my RSS feed) comes from, without using Google.

[The Desktop Fishbowl]

» I think Charles is spot on with all three of his arguments.

It weakens the specification.
It breaks an implied contract.
It fails to set a standard.

The background to this appears to be a quest to "nail RSS once and for all." I don't understand that. What's wrong with it evolving over time and as requirements change?

(Oh and zero points for me!)

You got it right with 0.94

Mon, 09 Sep 2002 19:21:16 +0100

DanBri. Dan Brickley has written, imho, an excellent posting on the value of RDF with RSS. All those bothered by this,... [Content Syndication with XML and RSS]

» Dan Brickley's arguments for RDF (something else I'm not an expert in) sound very cogent but, more than that, they evoke some of the same sensations as I got from reading an earlier piece from McGee's musings about innovation:

"When Hollywood asks us to enumerate the uses we'll lose if it gets its way, we can't. That's innovation for you. If we could predict the future uses of new technology, they wouldn't be innovative. That's innovation. It's the force that drives our civilization. It's the force that drives our culture. It's the force that makes us human ("the tool-using animal"). I'm not willing to give it up, even if I don't know what it is."

Dan writes:

"Just use namespaces" doesn't address the problem of one task, multiplenamespaces: people, events, music, documents, concerts, prices,locations... If we're interested in applying a variety of descriptivevocabularies to a single task, we'll need to use vocabularies developedoutside of RSS-DEV. RDF apps focus on just this, whereas many XML appsfocus on a single monolithic DTD or Schema that captures a specific task.

Dave's rush to cast RSS 2.0 seems more and more premature. I think it would be much better if he went back to calling it 0.94 and published it. Then we can get to using it straight away without the hieghtened tension of all the RDF people grinding their teeth (and surely that's a good thing right Dave?)

In the meantime we can all take the necessary time to thrash out the issues about RDF and RSS2.0, maybe even what RSS stands for. It's obvious that there are a lot of issues and just unilaterally declaring it so isn't going to put them to rest.

Here's hoping that there isn't an RSS3.0

Tue, 10 Sep 2002 08:12:14 +0100

Questions for Dave. In the comments to Entry below, Dave Winer says: "To everyone, I believe the new spec is leaps and bounds... [Content Syndication with XML and RSS]

» Depressing, depressing, depressing.

One of the 7 habits (of Stephen Covey fame) is "Seek first to understand, then to be understood." The tension in this discussion is rising fast and empathic listening seems to be out of the window. It's all sounding more and more like a debate in the house.

I think Joe Gregorio makes an interesting point. If the RSS1.0 folks could stand to walk away from the name RSS then much of the tension disappears. You can't stop Dave issuing his own 2.0 and trumping him with an RDF 3.0 is just going to escalate things. Sounds a bit like the middle east doesn't it?

Bill Kearney has already proved that any new standard, even one that Userland doesn't support, can introduced into Radio.

I vote for taking the creative energy in this discussion into a new TLA and a new forum. The worst that suggests itself to me about this route is having to support two specifications. But isn't that the case already with RSS1.0?

JIRA does RSS, so cool!

Tue, 24 Sep 2002 23:05:56 +0100

Oh man this so rocks!

I'm now have Radio subscribed to a feed coming out of my liveTopics JIRA project. I get an RSS item for each change that happens, i.e. someone adds a new issue, someone adds a comment, it's all there. This is *so great* for project visibility.

Unfortunately it does kinda point out the weaknesses in the Radio aggregator interface. It was probably fine when you only had a dozen or so feeds but I'm up to 58 and some of them are really big. I need a tabbed interface that lets me organize feeds the way I want. I need keyword prioritzation, I need the ability to delete all items from a specific feed at once. I'd like them outlined to save screen real-estate. [Actually John Robb's just pointed out that I can delete all the items at once, using the magnifying glass icon - must check that out]

Damnit I want aggregator Mark 2.

liveTopics RSS Feed now available

Wed, 25 Sep 2002 12:33:28 +0100

For anyone interested in liveTopics I have added an RSS feed which will be used to announce new releases and important events.

Language logging

Fri, 18 Oct 2002 10:28:02 +0100

Blogs for developing language skills.

One of my friends has commented on my not perfect English and suggested her help for improving it. She is going to start Radio blog, so I think about the following:

She makes a special category, which is not visible in her blog, but has an RSS feed.
Then she uses this category to comment on my posts pointing to errors and suggesting improvements.
I subscribe to this RSS, get my personal feedback and correct posts.

Implications:

For me it should be more effective than any course or private lesson: no stupid exercises, but just-in-time feedback to improve my writing.

This could be a service that someone can provide for bloggers. Personally, I wouldnt mind to pay a bit for it.

For multinational companies this could be a solution to help their employees developing language skills and overcoming fears of writing in foreign language.

[Mathemagenic]

» Excellent idea. This would be a good adjunct to any language teaching course too. Although managing the 300 or so categories required for each student could be a challenge...

Fetch delivers Enterprise Streaming

Wed, 23 Oct 2002 10:13:47 +0100

Have you seen Fetchserver.com.

Fetch delivers on the enterprise streaming side of klogging.

Server pulls SQL data from sources, on schedule, outputs RSS feeds.
Clients grab feeds, scroll news in a task bar UI.

Note the variable query frequency: pulled more often for rapidly changing data, presented more prominently for more important data.

Why isn't this database bridge part of Radio or Manila?

[a klog apart klogs]

[a klog apart]

» An interesting light weight RSS client that scrolls headlines and gives a click through to more info. Reminds me of PointCast (who now seem to be InfoGate, "enabling leading media companies to offer turnkey premium subscription services to their clients").

As Phil mentions there is no reason why Radio/Frontier couldn't act as the FetchServer part of a Radio based enterprise RSS streaming network. Although the Radio client would need a lot of work to be as functional in that context.

The strength (and weakness) of the Fetch client is that it is always visible, docked to the taskbar. With headlines scrolling continuously. However I can imagine that it also becomes a distraction, or an annoyance, especially if you are subscribed to a lot of channels.

To be free of Outlook?

Thu, 07 Nov 2002 15:55:41 +0000

W a r m b r a i n: Mail from Aggregator. The spaces buzz is building. Can't wait to try spaces out.

But what if you could take it a step further though? Wouldn't it be easier to just have the news aggregator incorporated into your email client? Merge the notions of 'get new mail' and 'get new news' into a single application that already has the read-offline paradigm build in. This would avoid the hassle of having to configure the news aggregator to retrieve the RSS feed once, parse it, repackage it and then send it back onto the network to be retrieved by email. Wouldn't it rock if there was an imminent release of just such a product? But there is! Diego Doval's spaces is exactly that product and it can do all I mentioned above and a whole lot more.
[Roland Tanglao's Weblog]

» Finally, a chance to be free of Outlook? Beyond my wildest dreams!

liveTopics in RSS2.0

Wed, 27 Nov 2002 09:52:54 +0000

Is this the first valid RSS feed with topic metadata?

Today I've finished the experimental RSS generator for Radio that exports the associated liveTopics with each post in the RSS feed. At the moment topics are contained in a "liveTopics" XML namespace.

This will enable a smart aggregator to use the topic's for filtering & combining feeds together.

liveTopics in RSS2.0 #2

Thu, 28 Nov 2002 15:55:44 +0000

Matt Mower's Knowledge Log - ( liveTopics, k-log, radio, blogging, RSS ): "This will enable a smart aggregator to use the topic's for filtering & combining feeds together." + filtering is in the pipeline for myRadio, on dates, keywords, and now topics. will be tricky to devise a UI.
+ filter a single feed, or multiple feeds. multiple feeds would require agreement on a common pool of topics, i think.
+ Syndication, with meta-data, gathered by smart aggregators, has a lot of possibilities. It would be cool to hear more about usage scenarios. [Brain Off]

Mikel picks up on my post yesterday regarding adding topics to Radio RSS. I've got a few things in mind for this, but I'm sure others will really lead the way. Let's just address one point first.

When trying to handle feeds from multiple blogs, inevitably, as Mikel points out, we will reach the situation where people using different words to mean the same topic. This will be a problem, but hopefully not as a big of a problem as it could be. It is for this reason that I have been tracking XFML so carefully. With XFML we have the ability to say "A's topic X is the same as B's topic Y". liveTopics already does XFML.

So I think the first and simplest usage scenario will be within the type of aggregators that we have now as a way of filtering a feed to get rid of posts we deem irrelevant. This will allow us to subscribe to many, many more feeds since we don't have to weed out so much chaff. Although I think we'll need to be careful as it may make it more difficult to experience serendipitous moments.

The next scenario I can imagine is as a way of producing a consolidated "on-topic" feed from a number of other feeds. Combined with technology to scrape RSS from sites and databases and with a little automagic to add topics where they don't exist this could be very powerful.

My imagination runs out here, maybe someone else..?

BlogChannels with liveTopics + RSS

Thu, 28 Nov 2002 20:06:46 +0000

BlogChannels for loosely joining webloggers?.

Sebastien reports on a very cool project. BlogChannels could be a powerful tool... and it might save me a lot of manual work that I need to do for Seblogging. If people simply "pinged" a common channel I would not need to sift through all the stuff that is getting published on the individual Weblogs. This is especially the case with less focused Weblogs where only now and then a posts refers to educational applications of personal Webpublishing and Weblogging...

[Seblogging News]

The BlogChannels idea presented here merges very neatly with the work I am doing with liveTopics + RSS2.0 and one of the suggested applications of that work, i.e. consolidating multiple RSS feeds based upon topic information. The nice thing is that this work can be easily duplicated for any blogging system.

liveTopics in RSS2.0 #3

Thu, 28 Nov 2002 20:43:06 +0000

My RSS2.0 feed is now live using a liveTopics namespace to add topic information to each post. If your aggregator gets smart you could filter out all the stuff I say that you're not interested in.... err... hello? Where did everyone go?

liveTopics RSS, RDF and the Dublin Core

Fri, 29 Nov 2002 08:55:46 +0000

liveTopics RSS2.0 feeds now use a vendor neutral XML namespace:

xmlns:rsstopics=http://purl.oclc.org/NET/rss-topics/

which is currently pointed at http://www.novissio.com/resources/rsstopics/

So each now comes with an definition for every topic it is associated with. These tags will soon be pointing back to their ToC entries and optionally to their definition within the XFML version of the weblog.

I have taken a quick look at the work done on the RSS1.0 taxonomy module with defines an RDF syntax for specifying topics and advises the use of Dublin Core metadata for adding information. I'll certainly be persuing the use of DC tags but does anyone think I should be trying to re-use the RDF module & syntax?

BlogBrowsers

Fri, 29 Nov 2002 09:24:14 +0000

Okay I admit it, I'm hearing lots of people hopping up and down about BlogBrowsers and I can't help but ask; What is all the fuss about? Indeed more than that I can't help but ask the questions; Why? Where does this get us?

So a blog-browser reads & renders RSS, big deal. Wheres the value?

I hear Dave and others talking about "routing around Microsoft" but I don't see it. I'm a fan of blogs but I fail to see how a niche blogging application is going to route around MS. There's a whole lot of HTML web out there, are you planning to just leave it behind? And haven't we bought something pretty valuable in having a single platform (the browser) for delivering web applications? Isn't this what DHTML is working towards? If you want to innovate around MS then it makes more sense to me to start contributing to Mozilla's success.

Could someone give me the obviously missing pieces that will make this fit together for me...?

Technorati as RSS

Mon, 02 Dec 2002 18:47:14 +0000

I first saw Technorati show up in my referers early last week, but I'm just now getting a chance to play with it. It will take me some time to tour my own cosmos!

My first question is can I get additions to my list of inbound blogs and inbound links as an RSS feed?...

[The Shifted Librarian]

Technorati does have watchlists and this was my first question too. So far the answer appears to be no, but even if it is I doubt that will last long. These guys are too smart not to spot the opportunities that RSS presents.

Paying for RSS

Fri, 06 Dec 2002 13:14:51 +0000

How many feeds would you pay for? I have just subscribed to technorati.com to get my Link Cosmos as an RSS feed. Interesting service, but I still have to understand if it's worth $10 per year. Anyway, I think that the real news here is that for the very first time I'm paying to get an RSS feed into my aggregator. Here's an interesting excercise: go trough the feeds you are currently subscribed to, how many would you pay for? [Paolo Valdemarin: Paolo's Weblog]

Yeah this is my feeling too. I suggest to them that I would be unlikely to pay $5/yr when we were talking about e-mail. Not because I think that it is too much individually, but that if I like it I don't want to be paying $275/yr when I hit 75 feeds. I think something $25/yr for unlimited feeds would be more the mark.

Paolo makes a good point though. He's ponying up some dough for an RSS feed. This is a milestone.

RssDistiller: vital tool for Radio Users

Sun, 08 Dec 2002 10:43:53 +0000

Last time I checked out RssDistiller from evectors I wasn't really into RSS very much. It was just after I started using Radio and, frankly, I was more interested in messing around with it and what it could do. What did I care about feeds? Also creating patterns to distill sites is a bit of an art, who has the time?

Well of course I'm a little older and wiser now. RSS has grown to be very important to my thinking and to how I think business should be done. So important that tools to get non-RSS delivered content into feeds are vitally important. Of course, this is exactly what RssDistiller does.

To create a feed from a website you point RssDistiller at the site and specify patterns marking the start and end of the areas RssDistiller should look at, and then the start and end of each "item" it should create. RssDistiller will then turn that into a valid feed.

For example the following patterns are how i configure a feed for a website that I use:

ignore text before: ignore text after: start pattern:
end pattern:
item template: ##text##

fiddly, but worth it.

RssDistiller is definitely worth checking out.

Filtered feeds - see it yet?

Thu, 12 Dec 2002 11:19:44 +0000

I've been going on about filtering RSS feeds with topics and apart from the "!paolo" and "!mikel" I've had little in the way of feedback. Have I been preaching to the converted? Or do people not see value in this idea?

I have become quite religious about adding topics like "humour", "politics" and "culture" to posts that I consider off-topic for my k-log. If my views on these things aren't your cup of tea I want you to be able to say:

Matt's feed - { "humour", "politics", "culture" }

to just knock all that clutter straight out of the feed.

Alternatively you may find that there are one or two interesting posts in a number of different feeds that share a common theme. I want to enable an aggregator to make a consolidated feed out of those. java.blogs is a great example of where this kind of thing is going. However in order to be useful to a wider community I think the tools have to come to the users, like RssDistiller

So who is working on the smart aggregators? Who is interested?

Testing RSS liveTopics

Sun, 15 Dec 2002 22:59:04 +0000

Test RSS topics.

Knowledge maps for k-logs

Tue, 17 Dec 2002 08:35:18 +0000

OPML directories I agree on the fact that "OPML" directories are a very interesting part of the development of content/knowledge management systems. I think that a particulary interesting application of this technology are self-building directories. This is what we are working on with Matt and with his LiveTopics Radio tool. It's somehow similar to what Dave describe as "timeless weblog", but instead of routing each post to one node, it will do it to several nodes in different categories and, most of all, it will be based on a server-side RSS 2.0 parser, so it will be able to organize contents from several k-logs. What we are working on is the automatic creation of a directory containing knowledge maps based on the topics attached to each post. The main use will be k-logging, but we are already seeing other intresting applications. Stay tuned. [Paolo Valdemarin: Paolo's Weblog]

Right on!

Nose back to the grindstone

Mon, 30 Dec 2002 20:04:39 +0000

The christmas RSS backlog is now reduced to managable proportions. Back to work on the liveTopics user guide!

Looking for my next aggregator

Wed, 08 Jan 2003 23:15:36 +0000

Okay this is a real problem with the Radio news aggregator. There is an RSS item that I know is in there somewhere, I saw it this afternoon, but I can't find it.

It has been buried by a mountain of new items. As far as I can tell the only way to find it again is to delete enough items that Radio will show it on the news page again. Oh I guess that I can find the feed URL and fake a URL for the Zoom feature to display all items from that feed... (since the feed doesn't appear on the news page at all right now).

As I come to depend more and more upon RSS to keep track of the life & work of the people I am becoming friends and colleagues with I am increasingly finding that the Radio aggregator doesn't cut it. This is just one of a number of items that are bugging me and I don't have time to address, even if I thought it was a worthwhile exercise.

I'd really like to try out some of the alternatives. Can anyone recommend any aggregator software? It must run on Windows (so don't suggest iNews) and ideally it should be written in Java (but that's just a nice to have).

RSS topics in the mix

Tue, 21 Jan 2003 18:17:40 +0000

Dave Winer. Callbacks for RSS-writing released [radio-dev]

This is good stuff. liveTopics was using my own witches brew of code to inject topic information into the RSS feed. Now it can do it with a proper set of callbacks. And this will work across all categories as well. Neat, thank you Dave.

Googlert does no RSS

Wed, 22 Jan 2003 14:34:39 +0000

Interesting: Googlert.

Interesting: Googlert

This looks neat. [_Go_]

Note: Currently untried by me. If I could remember where I stored down my Google key, I'd probably even try it. Thanks to Andy for pointing it out.

[The FuzzyBlog!]

It looks interesting, but you know the first thing that came to my mind?

Where is the RSS feed?

For this kind of service e-mail should now be a fallback!

Some people just don't get it...

Medscape RSS

Wed, 19 Feb 2003 11:28:01 +0000

Medscape Makes RSS Feeds Available!.

Medscape Jumps into RSS

"I am really happy to announce that Medscape, the leading news, information, research and CME site for Physicians on the Internet has joined the RSS revolution. We are now publishing RSS feeds of our headlines in each specialty for which we have a home page. And if you really want the full picture of what's going on in medicine, you can subscribe to a full site feed that syndicates just about every article, news story and CME program that is published.

This is a direct result of my experiments here in weblogs, as well as interactions I've had with the 'Doc Bloggers' in the column to the right. Admittedly, we are one of the few sites that rely on getting people to look at our content on our site launching this feature (and outside of technology-oriented sites, you can probably count the major media participants on one hand), and are certainly the first serious medical resource to do so.

So why aren't we afraid that publishing an RSS feed will actually lead to less traffic on our site? It comes down to this...we believe in the quality of our content. We know there is nowhere else on the Internet where you can get the same timeliness, focus and professional quality of medical information. If you are a doctor (or you are interested in medical information), our RSS feed is the best way to stay up to date on what we are publishing, and you will invariably want to visit our site to see the whole story." [Tales of Hoffman]

Add Mescape to the "ClueTrained-In" column!

Remember - RSS doesn't have to be a supplement for site visitors; it can easily be a complementary channel. Congratulations to Steve and Medscape for taking the long view! I truly believe this will benefit them in the long run, and I hope Steve will be able to provide us with periodic updates.

[The Shifted Librarian]

This is important for a number of reasons not least of which, from my perspective, is that in the medical professional we have a group of people for whom the classification of knowledge feeds isn't going to be a nice-to-have but a must-have.

These people are going to have so much information pushed at them that it is going to be essential that they can intelligently organise the streams into useful knowledge bases.

Socialising with the SocialText crew

Tue, 25 Feb 2003 20:28:11 +0000

Good conference call today between Ross and Adina of SocialText and Paolo and I. We all shared of our vision and our plans for the future which seem to be quite complementary. It's good to talk to other people doing interesting things in this space and I hope that we get an opportunity to work together. Certainly we both believe in the importance of supporting standards which, for us, means a big focus on RSS2.0 and XTM.

100th subscription

Wed, 26 Feb 2003 10:33:45 +0000

It's official:

Rogue Semiotics is now my 100th RSS subscription.

And to think I was complaining back when I only had about 30!

Do I get a telegram from anyone now that I've reached 100?

A different kind of reader

Tue, 04 Mar 2003 15:03:34 +0000

RSS Shell Integration.

RSS Shell Integration

Chris has some neat ideas on RSS integration right into the Windows shell:

Imagine this: a "News" submenu sitting at the top of your Start Menu, right above the Programs submenu. It cascades out into an organized list of feeds (each feed has its own folder). [_Go_]

Definitely interesting. My only concern would be performance. I have a quite large Start menu without this and its already slow. I can just imagine would it would be like with feeds streaming in all the time. Microsoft's whole shortcut based shell concept is cool and all but its damn slow.

[The FuzzyBlog!]

I haven't read Chris' post so maybe I've missed some nuance but, as described, I think it sounds like an awful idea for a serious reader - maybe it would work for someone with only a couple of feeds.

As Scott says I really don't want my Start menu to slow down any more. I think having 100's (I have 103 and counting) of feeds folders, containing lots of items will do that. It will also clutter it further. When I'm trying to start a program I don't want the shell to halt while it processes 2000 posts because I accidentally ran the mouse over the wrong sub-menu.

Also I think it's the wrong interface. Reading news isn't like starting a program or opening a document (at least, to me it isn't). I would rather have it contained in an application which gives. Also putting feeds in folders (the one advantage this route offers) is hardly a new idea.

What I am looking for is an application designed to handle at least 250 busy feeds. I want an engaging user interface and an approach that acknowledges interests and values as being central to my reading experience.

Outlined RSS in the browser

Tue, 04 Mar 2003 21:28:17 +0000

Outlined RSS Comes to the Browser. [img] I have finally released activeRenderer vs 1.4. The new version packs 3 new features:

activeRenderer now renders RSS format files (news feeds) in active outlined form,
with activeRenderer installed in Radio, you can now visualize both OPML and RSS local files in the new outline browser,
activeRenderer's rendering engine is now accessible as a web service, via both a local URL and a public one at services.activeRenderer.com/activerenderer/render.

Here are some more screenshots of the outline browser: win/mozilla - win/msie - mac/msie - mac/safari). [read more] [s l a m]

This is really cool. Marc's pretty close to realising his vision.

Roogle

Fri, 07 Mar 2003 22:17:58 +0000

What 10 odd Hours of Hacking Can Produce: An RSS Search Engine.

What 10 odd Hours of Hacking Can Produce: An RSS Search Engine

further ado, I give you:

R O O G L E

(yeah that's RSS google)

And the UI is a total Google ripoff. Thank you to Google for the time being before I get around to changing it.

Want to see what feeds are indexed? Check out this out.

[The FuzzyBlog!]

This is pretty cool.

Exploring topics in RSS2.0

Sat, 05 Apr 2003 10:38:33 +0100

I've been doing some thinking about how to encode topic information into RSS2.0 feeds. As a simple test of the Radio callback facility I have implemented a very simplistic protocol. Within each is a tag

<topic id="topic_id" type="topic-type" source="url">topic nametopic>

for each topic associated with the item (post). A concrete example (using the rsstopics namespace):

<rsstopics:topic rsstopics:id="the_state" rsstopics:source="http://matt.blogs.it/topics/topicsT.html#the_state" rsstopics:type="generic">the statersstopics:topic>

Whilst this does have the advantage that it's simple and direct it's also a bit silly to invent a new format for topic information when we have two standard culprits available already:

RDF is a general format for describing resources. A resource in RDF terms is anything which can be uniquely identified by a URI. An RDF statement (utilizing Dublin Core metadata) that asserts me as the owner of my weblog might look something like:

<rdf:Description rdf:about="http://matt.blogs.it">
<dc:Creator>Matt Mowerdc:Creator>
rdf:Description>

If you cut away the syntactic fluff what this says is:

Matt Mower is the Creator of http://matt.blogs.it

Referring back to the problem at hand, describing what a post (expressed as an RSS item) is about we could come up with something like:

<item rdf:about="permalink">
<topic id="topic_id" type="topic-type" source="url">topic nametopic>
item>

Which is more or less exactly where we started -- using RDF hasn't altered the solution but it has added some framework around it (in this case adding rdf:about to signal the presence of RDF data within the item). However we can go a step further. A useful article by Eric van der Vlist discusses this very subject and refers to the RSS1.0 taxonomy module.

Somewhat counter to what you would expect RSS2.0 does not follow on from RSS1.0, nor does RSS1.0 follow on from the popular RSS0.9x formats. RSS1.0 is, depending upon your point of view, a step forward or an aberation. RSS1.0 uses a modular set of RDF based tags to describe items in the RSS feed. One such module is the Taxonomy module which is intended to allow classification of RSS channels & items.

Using the taxonomy module you create something like:

<item rdf:about="permalink">
    <taxo:topics>
        <rdf:Bag>
            <rdf:li resource="topic-uri-1"/>
            <rdf:li resource="topic-uri-2"/>
        rdf:Bag>
    taxo:topics>
item>

Here the element contains a list (using the RDF defined Bag - or unorderer list - container element) of resources indicating topics that describe the item. Each resource then has a element that describes the topic. It might look something like:

<taxo:topic rdf:about="http://matt.blogs.it/topics/topicsT.html#the_state">
    <taxo:link>http://matt.blogs.it/topics/topicsT.html#the_state<taxo:link>
    <rsstopics:type>genericrsstopics:type>
    <dc:title>The Statedc:title>
taxo:topic>

Although it's a jumble of RDF, the RSS1.0 taxonomy module, Dublic Core, and, a custom rsstopics schema this says exactly the same thing as the original:

<topic id="topic_id" type="topic-type" source="url">topic nametopic>

But do we have to deal with such an ugly mess? Perhaps not. Our original choices included the XML Topic Maps format. This is a complete specification for exchanging topic information. An example of a topic in XTM format might look something like:

<topic id="the_state">
    <instanceOf>
        <topicRef xlink:href="http://www.purl.org/rss-topics/rss-topics#generic">
    instanceOf>
    <baseName>
        <baseNameString>The StatebaseNameString>
    <occurence id="the-state-item">
        <instanceOf>
            <topicRef xlink:href="http://www.purl.org/rss-topics/rss-topics#story">
        instanceOf>
        <resourceRef xlink:href="">
    occurence>
topic>

Again this encodes the same information, using a standard format and only one required namespace (that of XTM itself). A URI such as http://www.purl.org/rss-topics/rss-topics#generic points at a topic in another map (in this case a topic describing the topic-type generic).

The use of XTM comes with a number of advantages with the main one being that there are an increasing number of tools available to process & manipulate it (for example, see topicmap.com). However there also a number of problems with this representation when you attempt to embed it within another XML format such as RSS.

It's not clear whether an XTM fragment such as this is valid when used in this way
Each time a topic is used we will be duplicating it's details, bloating the markup & potentially creating invalid entries
The relation within the element is technically redundant. The enclosing indicates the occurrence.

One way to avoid these problems would be to embed the topics within the RSS definition and refer to them from each . However we still need a way to refer to the topic and XTM doesn't provide this. If we had a good way to reference topics then we could either embed mini topic map within the RSS file, or just have the in an external file and point to it. What could we use? One possibility is RDF.

Using a combination of RDF and XTM would mean something like:

<item rdf:about="">
<rsstopics:topic>http://www.example.org/myTopicMap.xtm#topic-idrsstopics:topic>
item>

<item rdf:about="">
<rsstopics:topic>#topic-idrsstopics:topic>
item>

In this example the item now refers to an XTM defined topic either elsewhere in the RSS feed (contained within a valid element) or within an external topic map. The referenced element can further describe the topic (names, types and so on) using all the expressiveness of XTM. It's also efficient since there is no duplicated information within the feed.

I have described approaches using RDF, XTM and a hybrid of the two. Each has advantages and disadvantages although I believe the hybrid makes the best use of both formats.

I'd welcome comments and or opinions from interested parties.

Is RSS1.0 Taxonomy module dead?

Mon, 07 Apr 2003 16:22:46 +0100

I'm really interested in finding out whether anyone actually uses (either produces or consumes) the the RSS1.0 Taxonomy module (hereafter <taxo:topics>). If you do, I would be grateful if you could add a comment to this post along with the URL of your feed and/or application.

As I posted on Saturday I am investigating methods for incorporating topic metadata into RSS feeds. As a Radio developer I'm mainly interested in RSS2.0 feeds but I don't want to duplicate any efforts / re-invent any wheels unnecessarily. So I'm considering <taxo:topics> very carefully.

However my view is that almost nobody is using it. I've spent a good bit of time this afternoon examining the feeds of various people in the RSS community (obviously trying to home in on those using RSS1.0) and there is no sign.

Phil Ringnalda points me at Syndic8's feed information where you can see that even while 25% of feeds there are in the 1.0 RDF format, less than 2.5% actually declare the <taxo:topics> namespace (and we have no way yet to verify whether they actually go ahead and use it).

If it really is the case that nobody is using it then I have to ask why?

Announcing: ENT v1.0 Easy News Topics for RSS2.0

Fri, 11 Apr 2003 13:59:53 +0100

Paolo and I are pleased to announce the release of the first public draft of the Easy News Topics (ENT) specification. ENT1.0 is an RSS2.0 module designed to make it really easy to incorporate topics into RSS feeds. Why would you want to do that? Because it will help to enable a raft of new, smarter, aggregator products.

RSS has become very important to a lot of us and we are starting to see its penetration into the business world as well. We think that integrating topics will help aggregators applications to scale to meet the future needs of users as well as delivering some very powerful applications. I've spoken before about the kinds of thing I want my aggregator to do:

group posts from many feeds by interest.
filtering posts I don't want to see
scoring & promote posts
recombine different feeds dynamically.

I hope that ENT might help bring all these things a little closer. We also see a role for classification in bringing new ways to order, view, and, search weblog data.

We are offering ENT1.0 to the community (under a Creative Commons License) in the hope that we can foster these applications and many more, that we haven't even begun to think of yet.

I will soon be releasing to the public the next version of liveTopics which will be ENT compliant. At that point any Radio user will be able to easily add topic metadata to their RSS feed. We hope that there will soon be many applications available to make use of it.

We look forward to your comments.

liveTopics 1.1.3 now supports ENT

Tue, 15 Apr 2003 11:38:34 +0100

Pretty soon I will be displaying the new ENT 1.0 logo <%radio.macros.imageRef( "/images/rssent.gif" )%> (courtesy of Paolo) on this page as my feed is now compliant.

Yahoo and ENT?

Wed, 16 Apr 2003 10:04:04 +0100

ask.yahoo.com RSS beta. Ask Yahoo!, a daily column that features Q&A with Yahoo!'s expert team of Surfers, is now syndicating its content via RSS. Here's the link to the RSS file:

[Michael J. Radwin's blog]

Hey hey -- some RSS action from Yahoo -- subscribe now so they know we're out here! [Brain Off]

I've subscribed, sight unseen. Now wouldn't this be a great feed to have topics? So you could see the Q&A that really mattered to you. And Yahoo! already have a taxonomy.

Relocating an RSS feed

Wed, 16 Apr 2003 15:18:29 +0100

A while back I came across instructions for what you need to put into an RSS feed to tell readers that it has moved to a new location (URI). But, for the life of me, I can't find it now. Does anyone know what I'm talking about? I'm reasonable sure I didn't imagine it.

Update: Terry Frazier came up with the answer. Thanks Terry.

TopicExchange does ENT

Wed, 16 Apr 2003 17:25:04 +0100

New RSS features on topicexchange.com. A minor (but very handy) change to the Internet Topic Exchange today: it now supports ENT (spec), which means suitably equipped aggregators will be able to pull topic information straight out of the RSS feeds.

Making this actually useful is a new RSS feed: all posts on the site. If you want to keep track of everything, subscribe to that one (traffic on the Exchange is still not awfully high, so you won't find yourself overwhelmed). An aggregator which understands topics will be able to just pull down this one RSS feed instead of heaps of individual topic channel feeds.

The most interesting bit is yet to come: I've been contacted by Scott Johnston and Greg Gershman, who both seem interested in using Topic Exchange information to do some sort of classification of search results. Sort of like the way Google uses dmoz to give you links to relevant categories when you search. This functionality is yet to come, but the hooks are there in the Topic Exchange, so any developers are welcome to start using them from now on!

For people who are interested in using this, I've written a page to explain how to handle the data. Enjoy!

Comment

[Second p0st]

This is fantastic news. Well done Phil!

MyRadio gets ENT

Mon, 21 Apr 2003 09:45:18 +0100

myRadio supports ENT 1.0. My supports

Announcing the release of ENT 1.0 (Easy News Topics) support in myRadio. One of the stated goals of ENT is to "represent topics sufficiently that they be useful in enabling smart aggregators (e.g. filtering, recombining feeds, etc...)". RSS+ENT feeds can be filtered in myRadio, by selecting topics of interest.
Available topics for a feed are those seen by the aggregator, in the RSS feed. That list will grow in time. Later, myRadio will support topicRolls for this purpose. Future features may also include recombining feeds according to topic.
Update myRadio.root in RU, or download the latest here. Configure using the "Edit Topics" link in the myRadio navigation bar. Please contact me with any feedback, suggestions, and bug reports.
Currently, the only known feeds supporting ENT 1.0 are Curiouser and curiouser and Topic Exchange. ENT enabled feeds should increase greatly when liveTopics 1.3.3 is released. [Brain Off]

Fantastic news. Well done Mikel.

This will be the first application in the hands of users that will let them get the benefit of topics in their feeds. liveTopics 1.1.3 is in beta at the moment and should be available soon. Once that happens there will be a small cluster of feeds that do support ENT. But we need to do more.

Specifically we need to find a way to get at the hordes of MovableType users and get them in the game.

David Sifry on ENT

Mon, 21 Apr 2003 10:01:27 +0100

Easy News Topics. Last week, Paolo Valdermarin and Matt Mower released their specification of Easy News Topics 1.0 (ENT), which is designed as an RSS 2.0 module that can add topic and categorization information to an RSS feed. I committed to get back to them (and others) with a review and some commentary on the approach. The good news: As a format, ENT is easy to understand, easy for application developers to implement, and pretty easy to parse. Kudos to Matt and Paolo for coming up with a design that is simple but extensible. Now the bad news: I'm worried about two issues. First is the problem of self-categorization. ENT presupposes that authors can successfully create microcontent with the following properties: It can be placed in one or more categories the author is qualified to categorize the content correctly the author's categories have meaning to the reader In addition, we then run into a larger problem with self-categorization, which is the question of categorization across feeds. In other words, we have a problem of definitions - one person's rebel is another person's revolutionary. Even with ENT's inclusion of clouds, which are (potentially) external topic maps that create self-consistent maps of the world, we still have the problem of intentional or unintentional misunderstanding and misreading of metadata like categories, which leads me to think that the entire concept of self-categorization is extremely difficult to work on a large scale. A good example of this failure to scale is the history of web page... [Sifry's Alerts]

David presents a good analysis of ENT 1.0 and, as you would expect from David, some of the wider issues around self-categorization of data. In particular he compares a future of users adding topics to their RSS feeds to the abuse of META tags in HTML. It's a point worth discussing:

Off the cuff I can see two potential arguments to suggest that this won't be a big problem.

ENT is not designed only for self-categorization of feeds. Yes that's how we intend to use it, and certainly I think a lot of people will use it that way. But ENT could just as easily be used by a categorizer bot that sucked in feeds and annotated them (using heuristics) with topics from it's own cloud. This thought has lead me to wonder if there is some need for authorizing the use of a cloud. Would you trust Googlebot to add topics to RSS feeds? Or Feedster bot?
As David points out, the solution to the META problem, as wrought by Google, was to bring an element of the social into the mix. He rightly, I think, indicates that a solution to the eventual problems of metadata in RSS will probably be social also. There has been a lot of talk recently about identity and reputation systems. Blogging tends to be very much more personable than ever web publishing was before. I read sites because they are meaningful to me. If your categorization isn't, I probably won't read you for long.

ENT goes Movable!

Tue, 22 Apr 2003 23:03:08 +0100

ENT enabled RSS. ENT enabled RSS: No idea if its correctly implemented. Maybe a ENT validator is in order. Update: Some errors corrected.... [Daily Bytes]

Congratulations to André for having the first ENT enabled MovableType feed!

This is really, really, fantastic news.

Making aggregators more useful

Thu, 24 Apr 2003 12:34:15 +0100

Aggregator as archives, not only as instant readers..

Dave Winer: RSS readers that work like Usenet readers are a waste of time, imho. Aggregators should not organize news by where items came from, just present the news in reverse chronologic order.

I totally agree, this is why after trying other aggregators I came back to "Radio": it's just like reading a very large weblog updated by several people every hour, it's an activity we are familiar with and the ability that we have to scan titles and relevant information in a page makes this approach much more efficient than having to move from source to source. If an aggregator is meant as a way to take a snapshot of what's going on on hundreds of sources and quicky present it to us, I believe that presenting news in reverse chronological order is the way to go. But I also think that aggregators could be an interesting way to archive content, to let somebody quickly retireve something wrote sometime in the past. Archiving by author, again, does not make sense: most weblogging applications already do that, if I'm looking for something and I know who wrote it, I can simply look on the author's site. There are search engines, which are of course a good way to find information, but not always very efficient. There are cases when a directory might be more useful. We believe that archiving by topic in a directory could be a solution, and this is what we are trying to do. It's not for daily instant reading, it's to archive content. [Paolo Valdemarin: Paolo's Weblog]

I agree with Dave that organisation based upon source isn't a particularly useful innovation on it's own. But if he means that aggregators shouldn't attempt to organise posts at all (other than by order of arrival) then I totally disagree.

I still use Radio as my aggregator but I am not happy with it. It's just the most convenient reader I have found to date. Why? As I have said before, those readers actually made things less convenient. For example one, I forget which, offered me the ability to group feeds into an outline structure. This was good it meant I could keep my general news feeds aside from the feeds of people I know and so on. But when I clicked on a folder of feeds, it didn't present me with a view of all the items from each of the feeds in the folder. Now, why the hell not? This feature alone would have allowed me to dump the Radio aggregator.

Also we are working on k-collector which is a server based aggregator that will group items together from a number of different feeds, but it does that using ENT topics. This is integrated with our topic roll concept so that if you and I use topics from the same topic roll, our posts will appear together. Once you see this working (and we hope to show you soon) you can see how powerful this becomes.

Licensed to aggregate (pt #2)

Tue, 10 Feb 2004 08:00:08 +0000

I have received a couple of comments to my earlier piece about licensing & RSS. Among others Phil Ringnalda pointed me at the Creative Commons RSS Module authored by Dave Winer in Dec 2002.

The CC module is pretty much the same solution as I was proposing in that it seems to be a start in the right direction. However neither it, nor my own suggestion, answer all the questions in a way I can appreciate (e.g. as Phil points out, what is covered by the license: text only? images? etc...), so i'm going to keep chipping at this.

A commentor, Lorenzo, has suggested that

Copyright is a "state" and licensing is an "action" made possible by the state.

I kind of disagree with this statement but mainly on terminological grounds. I hope we can cut through that by agreeing that the central point is rights. Who has them? What uses do they permit?

It seems to me that a Copyright statement reserves all rights relating to making copies (& derivative works) to the author who can then make exceptions on a case-by-case basis. As in the case where an author grants their publisher the right to make copies for sale.

Hence an RSS feed with a copyright notice shouldn't be read (copied) unless you consider the act of offering feed to be an implicit agreement by the author to do so. Of course from a software perspective implicit rights can be problematic, especially when they are not immutable or well understood. For an RSS feed with a copyright statement what rights are actually on offer? People commonly republish content from posts they have aggregated either whole or in part. How do they know they have the right to do so?

Then you have some interesting anomalies when software gets in the way. For example, my weblog has a Creative Commons license, yet my feed is Copyright. I guess Radio is automatically adding the copyright notice, I don't know how to make it stop. What am I telling you?

Alternatively, and I guess this may be common, Phil Ringnalda's blog doesn't have any kind of license at all, neither does his feed. Can I assume Phil intends all his material to be public domain? If i'm not clear that this is the intension how can my software be?

Creative Commons offer a range of licenses which offer the right to copy, or make derivative works, with certain restrictions such as share alike and non commercial use only. Primarily Public Domain takes this a step further in granting unlimited rights with exceptions being, if you'll excuse the pun, the exception.

Maybe it's my developer "tunnel vision" at work but this looks very similar to the common model of permissions adopted in software everywhere:

include * except A, B, C,...
exclude * except R, S, T,...

where A, B, etc.. are not users or hosts but specific actions, by identified individuals, in specific circumstances.

For most personal news aggregators I guess none of this matters much. If someone publishes a feed, and you're just reading it, then whats the harm? Unless of course you weren't supposed to have the URL to the feed. But that's a different issue.

However for aggregators like K-Collector and users who are reposting content it's a different matter. For example K-Collector doesn't mess with the content of posts, but it does republish them in a new context. If K-Collector has a better Google page rank than the author then we even begin to suck traffic away from them based upon their own material!

(To be clear I am only talking about the public K-Collector portal W4. This is an untypical use of K-Collector which is designed for use within organisations. But the point still stands. And what about Feedster?)

As ambitious, semi-automated, software like K-Collector become more common then a reasonable, dependable, system of rights is going to be required. To my way of thinking the Creative Commons RSS Module, whilst a start in the right direction, addresses a necessary, but not sufficient, subset of the goals. What do we do to take it further?

Licensed to aggregate (pt #3)

Tue, 10 Feb 2004 11:10:14 +0000

Just musing out loud...

The HTTP protocol specifies headers which clients can supply in a request to control what a server can return to them. One in particular is the accept header. Which looks something like:

accept:text/xml,application/xml,application/xhtml+xml,text/html;q=0.9,text/plain;q=0.8,image/png,image/jpeg,image/gif;q=0.2,*/*;q=0.1

so there is already a simple model available for delineating content types. I am thinking of something along the lines of:

I'm not sure whether this is something that should be embedded in an RSS feed or referenced from another location (since it is potentially applicable to the blog as well). Other questions are:

Should it be extended to handle specific identifiable resources?
To whom are these rights granted?
Do we have the right set of rights to be granted? (cf my use of read above.)
Is the creative commons URL per scheme approach the right one?
Can we come up with a similar scheme for copyright & public domain?

It occurs to me that it might be possible to link a scheme like this to FOAF and use a service like PeopleAggregator to create the publishing/sharing relationships. This moves much closer to Ted Nelsons Transclusion publishing concept.

UpComing events

Thu, 26 Feb 2004 09:25:29 +0000

One event standard - please!.
Here's Matt Mower's thoughts on adding ESF events to the When category of the k-collector.

I'm gonna rely upon Matt and Paolo to grok all this stuff. I just hope they're making sure:

that all these events standards can work together

that they've gone back and checked out the earlier standardization efforts that happened in the blogosphere (I seem to remember an rdf schema from Danny Ayers)

that they're talking to Andrew Baio of UpComing.org.

Schedule and Calendar aggregation is key to the overall digital lifestyle aggregation scenario.
[Marc's Voice]

I think we're likely to support ESF straight away because:

it's there
it's simple
it's RSS2.0

I notice that there is also an events module for RSS1.0. Does anyone use it?

Based upon Marc's suggest I have taken a look at UpComing.org however it doesn't seem to support event metadata in it's feeds yet. Clearly we're interested in what such services will do in this area.

It didn't

Thu, 04 Mar 2004 06:23:18 +0000

Google Goes Atom.
Google spurns RSS for rising blog format | CNET News.com. I typically don't pay much attention to the syndication standards wars but this is kind of interesting. Google has gone with the Atom standard instead of RSS for syndication on its Blogger network. Existing Blogger Pro users with RSS feeds can keep them but all other Blogger customers will only be able to deploy Atom feeds.
[High Context]

Sigh

When Atom (ne Echo) was first proposed I said:

"If it means an end to the madness, I'm for it."

It didn't.

A thought about nested facets

Sat, 01 May 2004 11:01:52 +0100

Quick thought about nested facets while I was in the shower. Paolo and I have chatted numerous times about typed RSS feeds. The idea being that when s don't come from blogs we may get different information about them. For example an item corresponding to an event will have useful metadata which might correspond to some nested facet. An item from a database feed of helpdesk tickets could have different (or possibility similar) metadata.

Hmmm...

We built a better mousetrap, where are the mice?

Sat, 08 May 2004 22:53:24 +0100

Wouldn't it be cool?.
Wourldn't it be cool if otehr people started parsing the ENT tags embedded in this post?

That way if I talked about FOAF - for instance - someone like danbri could scarf JUST the FOAF posts and do anything he wanted with them!

This would create an incredible two-way kind of capability because systems could then communictae back to me based upon what I said. I know I know -it's RSS2.0 but that becomes a really nice gateway to a world that has 75% market share of feeds(maybe even more.)
[Marc's Voice]

I think it would be pretty cool -- I've been hoping other would start grokking this for a while now. Yes it's RSS2.0, yes it's not perfect, but it's here, now and I think that by the 80/20 rule it's good enough. If there's anything we can do to help get ENT support included in other applications please let us know and we will do our best to help.

I'd also like to see applications start using the SGUID information that Paolo and I have had in our feeds for about the same length of time as we've been doing ENT. Using SGUID one post in an RSS feed can refer directly to the permalink of the post it is quoting from. Standard RSS2.0 only allows you to refer to the feed. So for example, tags like:

http://www.theobviousblog.net/blog/archives/000506.html

should allow a clever aggregator to thread posts.

An aggregator that did topics and sguid-based threading, that would be nice...

K-Collector client demystified

Thu, 20 May 2004 08:49:02 +0100

More on Topic-Sharing Community.
There's already been a great response to my post last night (see the comments to previous entry). Greg suggested his aggregator Blogdigger could be included in this - I agree! Matt and Andrew also posted very thoughtful responses.

Here's some of my feedback (copied from the comments - I must get these enabled inline...):

Overnight while pondering my post (which I regard as just a 'starter for 10' btw, not a final solution by any means), I did conclude that KC essentially already does what I describe - polls registered RSS feeds with ENT in them and aggregates them. It would be great if TE also had that functionality.

It's the client ping that I think is unnecessary and possibly holding back community uptake - with TE the ping is a manual process for the blogger, and with KC you need to install an add-on tool to enable the pinging. Both require too much manual effort for the blogger (IMHO of course). eg Bloglines does all its aggregation automatically (every hour I think), with no pinging required from the blogger.

Although Andrew I take your point about bandwidth utilization. But if Bloglines (and Blogdigger) can do it, why not KC and TE?
[Read/Write Web]

There seems to be some confusion about how K-Collector server works and the role of the K-Collector client so I thought I would try and give an explanation of how things fit together.

The first thing to understand is that you absolutely do not need the K-Collector client for your blog to be part of a K-Collector site. The client offers a set of benefits aimed at improving the experience for the user, but they are entirely optional. We currently aggregate many feeds to the W4 site which are not using one of our clients.

There are three reasons why we think using the client is beneficial:

1) The ping (it's the least important, but seems most misunderstood so I'll cover it first)

The K-Collector server contains an aggregator which reads all feeds on a rotating basis. It aims to read each feed more or less once per hour but this isn't guaranteed. It collects posts from feeds and assigns them to topics using either ENT metadata supplied in the feed or choices which are auto-discovered using various word-stemming and matching techniques.

This all happens entirely independent of the client ping. All the ping does is to move your feed up the list so that new posts you have written are likely to be collected sooner. If you don't ping the server just reads your feed automatically a little later on, that's all.

2) The topic manager

Through the client, the topic manager is integrated into the blog editing process and gives authors the ability to assign community topics to their posts as well as being able to create new topics. The topic manager also attempts to suggest topics which may be relevant to the content of the post to make choosing topics easier.

Without the client you have no way to decide which topics should be assigned to each post. In this case the server will, when it reads the feed, use it's own automatcher to automatically assign those topics it thinks are relevant.

3) ENT feeds

Where the author has choosen topics in the topic manager the client adds the appropriate ENT metadata to the outgoing RSS feed. K-Collector can then use this metadata to accurately assign posts to topics.

In summary the K-Collector client offers what we think are very useful benefits to weblog authors, however it is entirely optional and you do not need it for you weblog to be part of a K-Collector site. Equivalently K-Collector itself only cares about RSS. It doesn't care whether ENT metadata was created by our client or some other application, we're completely agnostic about that.

I hope this goes some way to clearing up how the K-Collector system works.

All formats lead to Rome (and back again)

Thu, 17 Jun 2004 22:04:17 +0100

I'm doing some work with Rome which is an ambitious open source Java RSS toolkit by Alejandro Abdelnur, Patrick Chanezon, and Elaine Chien of Sun.

Rome does not attempt to be all things to all men, choosing (correctly imo) to concentrate on providing low-level feed parsing & generation services. It can read & write RSS in all it's flavours 0.9x, 1.0, 2.0 as well as Atom 0.3. It can also convert feeds between formats and provide an abstract syndication feed layer over them. Crucially they have good support for modular extension & I am bashing out an ENT 1.0 module which I'll contribute back to the project.

It's still alpha but already looks impressively useable. Good stuff.

Experimental Friday

Fri, 06 May 2005 18:58:52 +0100

It's like casual Friday only with fall-out!

At one point, when I was using liveTopics, there were topics all over this weblog. Then I decided to go as minimal as possible and stripped a lot of it out. Now, on a whim, I'm bringing some of it back.

In the right hand pane I now have a Zeitgiest listing which I shamelessly tried (and failed) to pinch from Joi Ito. Alas my CSS skillz were not sharp enough to work out why the layout goes to hell if I put display: inline on the list elements. So I have to have 'em in a drab list for now (Can anyone tell me why it doesn't work? The list items end up at the bottom of the page)

However I wasn't done stealing there. Looking at a FeedBurner rendered RSS feed gave me an idea. Rather than rendering a table of contents as HTML why not just publish each topic as a separate RSS feed and style it? Then it will work either programmatically or in the browser.

Turns out it works quite well (although it takes a devil of a time to upstream all 534 feeds). The only problem is that the tags in the field are poking through and not getting rendered into HTML. Apparently my XSLT skillz are no better than my CSS ones. Can anyone tell me what I did wrong?

However I think the end result is promising and it means that you can now subscribe to individual topics that I write about. Which is a probably boon if you think I blather on about, for example, politics way too much! :-)

Also I have a quick view of all topics (a blatant ripoff of Joi's format) which highlights those that have been used most recently.

Firefox not XSLT standards compliant?

Sat, 07 May 2005 09:46:51 +0100

As I wrote yesterday, I'm now rendering individual topic feeds. Each is a (slightly bare) RSS feed of posts on a particular topic. Taking slighly more than a leaf out of the FeedBurner playbook each feed has an XSLT stylesheet which renders it for display in the browser. That way the same file that you can subscribe to in your aggregator can also do dual purpose as a cross-reference index of my blog. So far, so good.

What sucks is that, when you view a feed in FireFox, the HTML pokes through instead of being rendered as part of the page. I've spent some time this morning trying to figure it out and just couldn't seem to make it work. I called on Don Park who gave me some useful pointers.

Then I made an interesting discovery. In both IE and XML-SPY the output from the StyleSheet appeared to be correct, i.e. the entities were decoded and the HTML took it's rightful place. Some hunting turned up a FAQ on Mozilla XSLT which seems to hold the answer: Mozilla's XSLT engine does not respect the disable-output-escaping option.

They claim it causes trouble and I'm not in a position to disagree, however it also appears that this means Mozilla and FireFox aren't XSLT standards compliant which, if true, is both irritating and hypocritical because, as far as I can see, it's not an optional part of the standard.

How is the Mozilla team picking and choosing which bits of the XSLT standard they choose to implement any different to situations in which other vendors do it, and get called on it?

I'm not in a position to contribute to the Mozilla codebase or to look at this problem so it's unfair of me to gripe. Nevertheless I will. The reasons given are that it causes crashes and can slow down rendering. Both are reasonable first responses when pushing to get a 1.0 product out the door. But neither Mozilla nor FireFox are 1.0 products any more and these reasons don't stand up. Crashes can be solved (write better quality code) and, ultimately, judgements of rendering speed should be down to the user, not the developer.

In short, I think it sucks.

On a more practical note I really do want to see if I can figure out a way around this. I am a FireFox user and I care more about FireFox users than I do about IE users (for whom the feed renders properly, damnit!)

Since I am generating the XML for the feeds myself is there perhaps another way of solving this? Some way of getting the HTML in the field through XML validation but without requiring the entities to be unescaped in the XSLT?

Any help you could give me would be much appreciated.

Go go Styled RSS!

Sun, 08 May 2005 23:33:53 +0100

Big thanks to Phil Ringnalda for providing the answer to getting RSS styled properly via XSL. It's so obvious once you know. Add JavaScript to the page to detect whether disable-output-escaping has been honoured and, if not, fix up the HTML. Now my topic pages work fine in FireFox and I'm a happy man!

Thanks Phil!

l.m. hacks RSS and Atom while you wait

Wed, 14 Sep 2005 08:19:54 +0100

l.m.orchard has written a book about hacking RSS and Atom. I'm doing a lot of stuff with RSS parsing at the moment (right now I'm experimenting with Lucas Carlson's excellent SimpleRSS parser) so I might go pick up a copy, even if he does use Python ;-)

Simple Sharing Extensions for ENT

Tue, 22 Nov 2005 09:45:00 +0000

Some time back when Paolo and I were developing the Easy News Topics specification for including topic metadata in RSS2.0 feeds I was considering the ways in which a weblog taxonomy should be considered a living entity subject to change with new topics being born, topics getting married and using the same name, old topics withering away and dying, and so on.

At the time I was playing with the notion of publishing a changelog for an ENT cloud that could be used to keep track of these changes. This morning I started reading the Simple Sharing Extensions for RSS and OPML specification by Ray Ozzie and George Moromisato of Microsoft.

It immediately hit me that SSE was a great mechanism for distributed taxonomy without requiring a central respository. By allowing the user to keep control it also solves the problem of what to do about conflicts/unwanted topics: Let each user decide what their outcome should be.

As a first step in exploring this I am will be attempting to build support for SSE into the ENT support within Squib.

I don't like ads in feeds

Mon, 16 Jan 2006 17:25:57 +0000

I just loaded the eWeek Macintosh feed and noticed something odd looking about it. It took me a moment to realise that each post had an advert. The total effect is to make it look fugly. An advert-per-item RSS feed is never going to work with me. I won't put up with that level of noise to signal ratio. What's even worse is that this is an exercpted feed so I get one line of the article and a 4-line advert. Dumb!

Combining OPML and RSS to create an export format for a blog

Mon, 10 Apr 2006 09:30:46 +0100

Marc Canter links to Joe Brockmeier's post about weblogs having a shared format. Timely.

I've been thinking about this myself because I want such a format too. Although I have written a tool to serve my own needs I won't be using it forever and I (probably) want to take my blog with me. I've also been thinking about how to do backup and restore. The two problems appear to be the same to me.

I think we already have the answer: RSS. It's already a natural format for holding the essential data of a weblog and namespacing is an easy way to store the tool-specific data. A tool that understands another tools metadata (e.g. ENT topics) can import it, a tool that cannot can safely ignore it. Actually why are we even discussing this?

The real question seems to me to be: how best to use RSS for this purpose? Do we have one gigantic RSS feed for a weblog? In my case with about 2100 posts it would be pretty big and unwieldy. Back in 2004 Paolo and I were talking about how to do weblog archives.

I was messing with an approach that combined RSS and OPML to create a weblog archive. For each post/day/month (pick your granularity) create a corresponding RSS feed of weblog entries. These feeds are then referenced from an OPML file that defines the overall structure of the archived weblog. In this way you can quickly narrow down to find an individual post, or suck up the whole thing (useful for tools like Sigmund).

For convenience the whole lot could be wrapped up in a .tar.gz. It might be helpful to include some kind of (optional?) metadata file at the top-level that describes the contents (ala JAR archives).

I'm not sure why I stopped working on that, maybe it just got shoved aside by other things. I might have a go at adding this feature to Squib since we need a backup format anyway.

Atom is better than RSS

Wed, 12 Apr 2006 23:16:10 +0100

I think I can hear a sterile argument brewing about using OPML and RSS to create weblog archives. For example: Wouldn't Atom be a better choice? Maybe hAtom? And how about XOXO? Or perhaps our old friend RDF?

I don't know. Frankly I don't much care either. I have a hard time working up any enthusiasm for such questions.

I'm not building a CAT scanner or putting a man into orbit. This is an archive of my weblog entries. Is OPML good enough? Is RSS good enough? Only time will tell. But if they're not, and you can do a better job, then please lead the way.

An experimental OPML+RSS archive for C&C

Wed, 12 Apr 2006 19:48:29 +0100

Over the last couple of days I've hacked together experimental support for OPML+RSS archives in Squib as I described a couple of days ago. You can grab my entire archive from here either directly or as a .tar.gz archive.

The structure of the archive looks like this:

The weblog.opml file is an outline that contains the date-based structure of Curiouser and Curiouser. There is a branch for each year, and each month of each year. At the leaves are pointers to daily RSS files and the ID & title of entries.

It occurred to me that I could just put the entire entry data directly into the OPML file and cut out the RSS. However, with over 2,100 entries, I felt that would lead to a very big and unwieldy file. Being just a file of pointers means it can still be sensibly opened in an OPML editor.

Another reason for using RSS is to ensure that users of the archive can take advantage of all the software out there to parse RSS. Once you've figured out which days entries you want, you can hand the corresponding RSS file to a standard parser and get back the entries.

Update: I've added <link rel="archive" type="application/opml+xml" href=".../archive/data/weblog.opml" /> to my home page to allow archive auto-discovery. I did a minimum amount of research before doing this so please correct me if that's a gross misuse of a link tag or there is some established way of doing this already.