Curiouser and Curiouser!

Curiouser and Curiouser! on aggregators http://matt.blogs.it/ RSS feed for topic aggregators Copyright 2006 Matt Mower Squib/0.1 self@mattmower.com self@mattmower.com en-gb JIRA does RSS, so cool! Tue, 24 Sep 2002 23:05:56 +0100 Oh man <A href="http://opensource.atlassian.com/novissio/secure/IssueNavigator.jspa?pid=10000&tempMax=25&view=rss&reset=true">this</A> so rocks! I'm now have Radio subscribed to a <A href="http://opensource.atlassian.com/novissio/secure/IssueNavigator.jspa?pid=10000&tempMax=25&view=rss&reset=true">feed</A> coming out of my liveTopics <A href="http://opensource.atlassian.com/novissio/secure/Dashboard.jspa">JIRA project</A>.  I get an RSS item for each change that happens, i.e. someone adds a new issue, someone adds a comment, it's all there.  This is *so great* for project visibility. Unfortunately it does kinda point out the weaknesses in the Radio aggregator interface.  It was probably fine when you only had a dozen or so feeds but I'm up to 58 and some of them are really big.  I need a tabbed interface that lets me organize feeds the way I want.  I need keyword prioritzation, I need the ability to delete all items from a specific feed at once.  I'd like them outlined to save screen real-estate.  [Actually John Robb's just pointed out that I can delete all the items at once, using the magnifying glass icon - must check that out] Damnit I want aggregator Mark 2.   http://matt.blogs.it/entries/00000582.html liveTopics in RSS2.0 #2 Thu, 28 Nov 2002 15:55:44 +0000 <BLOCKQUOTE dir=ltr style="MARGIN-RIGHT: 0px"> <A href="http://radio.weblogs.com/0107808/2002/11/27.html#a566">Matt Mower's Knowledge Log - ( liveTopics, k-log, radio, blogging, RSS )</A>: "This will enable a smart aggregator to use the topic's for filtering & combining feeds together." + filtering is in the pipeline for myRadio, on dates, keywords, and now topics. will be tricky to devise a UI. + filter a single feed, or multiple feeds. multiple feeds would require agreement on a common pool of topics, i think. + Syndication, with meta-data, gathered by smart aggregators, has a lot of possibilities. It would be cool to hear more about usage scenarios. [<A href="http://radio.weblogs.com/0100875/">Brain Off</A>]</BLOCKQUOTE> Mikel picks up on my post yesterday regarding adding topics to Radio RSS.  I've got a few things in mind for this, but I'm sure others will really lead the way.  Let's just address one point first. When trying to handle feeds from multiple blogs, inevitably, as Mikel points out, we will reach the situation where people using different words to mean the same topic.  This will be a problem, but hopefully not as a big of a problem as it could be.  It is for this reason that I have been tracking <A href="http://xfml.org/">XFML</A> so carefully.  With XFML we have the ability to say "A's topic X is the same as B's topic Y".  liveTopics already does XFML. So I think the first and simplest usage scenario will be within the type of aggregators that we have now as a way of filtering a feed to get rid of posts we deem irrelevant.  This will allow us to subscribe to many, many more feeds since we don't have to weed out so much chaff.  Although I think we'll need to be careful as it may make it more difficult to experience serendipitous moments. The next scenario I can imagine is as a way of producing a consolidated "on-topic" feed from a number of other feeds.  Combined with technology to scrape RSS from sites and databases and with a little automagic to add topics where they don't exist this could be very powerful. My imagination runs out here, maybe someone else..? http://matt.blogs.it/entries/00000886.html Filtered feeds - see it yet? Thu, 12 Dec 2002 11:19:44 +0000 I've been going on about filtering RSS feeds with topics and apart from the "!paolo" and "!mikel" I've had little in the way of feedback.  Have I been preaching to the converted?  Or do people not see value in this idea? I have become quite religious about adding topics like "humour", "politics" and "culture" to posts that I consider off-topic for my k-log.  If my views on these things aren't your cup of tea I want you to be able to say: <BLOCKQUOTE dir=ltr style="MARGIN-RIGHT: 0px"> Matt's feed - { "humour", "politics", "culture" }</BLOCKQUOTE> to just knock all that clutter straight out of the feed. Alternatively you may find that there are one or two interesting posts in a number of different feeds that share a common theme.  I want to enable an aggregator to make a consolidated feed out of those.  <A href="http://www.javablogs.com/Welcome.jspa">java.blogs</A> is a great example of where this kind of thing is going.  However in order to be useful to a wider community I think the tools have to come to the users, like <A href="http://store.evectors.com/itproducts/story$num=1&sec=1&data=products">RssDistiller</A> So who is working on the smart aggregators?  Who is interested? http://matt.blogs.it/entries/00000988.html MetaBlogging *requires* topics Fri, 31 Jan 2003 10:04:59 +0000 <BLOCKQUOTE dir=ltr style="MARGIN-RIGHT: 0px"> <A href="http://radio.weblogs.com/0103807/2003/01/30.html#a1279">MetaBlogging and Human Moderation</A>. The JavaBlogs site is one that aggregates different blogs on Java into one convenient holding place.  And now Kasia is finding that the JavaBlogs site just isn't working for her: <BLOCKQUOTE dir=ltr style="MARGIN-RIGHT: 0px"> I really like the concept behind <A href="http://www.javablogs.com/">JavaBlogs.</A>.. and for a while it really worked well for me. I've read many excellent entries on Java I may have (actually very likely would have) missed otherwise. Unfortunately, as is true with any growing website, the signal to noise ratio is becoming worse and worse.. and not in a good way. When I first discovered JavaBlogs I read nearly every entry posted and most were great.. agreed with some, disagreed with others.. typical blog-reading experience. These days I find myself skipping more and more entries.. why? They're not about Java.. and many of the ones that are simply reiterate or link to previously posted entries. <A href="http://www.unix-girl.com/blog/archives/000670.html">[_Go_]</A></BLOCKQUOTE> I see her point.  Take a look at what I just posted in my PHPblog (the entry below on Bunnies).  Yes it was php related but the relationship was tangential at best.  It is almost like we need a "Related But Not Very" type of categorization.  Or a scalable value.  Of course the problem is that if it is human done we'll all just configure it wrong.  Perhaps we need "Latent Semantic Indexing" (this is a really excellent paper if you care about low level search engine issues).  <A href="http://javelina.cet.middlebury.edu/lsa/out/cover_page.htm">[_Go_]</A> I think what needs to ultimately happen is that our aggregators need to get smarter in the background i.e. they still gather everything but show us things that we are more "likely" to be interested in.  They'll have to do this via some kind of background analysis (probably) of links followed out of the aggregator.  And then some kind of periodic training.  And this is a) difficult and b) far, far from perfect. [<A href="http://radio.weblogs.com/0103807/">The FuzzyBlog!</A>]</BLOCKQUOTE> I could see this problem raising it's head back when JavaBlogs first took wing.   People don't stay on-topic.  For example, this week I have posted far more about politics, government surveillance, crime and terrorism than about KM, topic mapping, organisational development, etc... It's quite likely that usual readers of what I'm saying have gotten fed up with yet another preachy diatribe ;-) Of course if their news aggregator was smart enough to pick up the topics in my posts (I appreciate they aren't actually in here at the moment, I'm re-working some of the code, they'll be back soon though) they could filter out all the posts I mark with topics like: <BLOCKQUOTE dir=ltr style="MARGIN-RIGHT: 0px"> law, politics, privacy, terrorism</BLOCKQUOTE> and so forth. It'll happen. http://matt.blogs.it/entries/00001117.html Topic Rolls, Controlled vocabulary and the ITE Mon, 07 Apr 2003 16:51:49 +0100 <BLOCKQUOTE> <A href="http://matt.blogs.it/2003/03/27.html#a836">Topic Rolls near reality</A>. <BLOCKQUOTE cite=http://matt.blogs.it/2003/03/27.html#a836> The next step is to allow liveTopics to import topicrolls from other locations. [<A href="http://radio.weblogs.com/0107808/">Matt Mower</A>]</BLOCKQUOTE> Matt expands his work on topic information for Weblog entries. He adds another hierarchical layer by introducing "topic types". This basically allows you to group your containers (topics) into another layer of containers (topic types), thus adding more modelling power to one's own topic structure. Without knowing what really is going on in Italy I would be very cautious with attempts to "define a control language for topics rather than allowing users to make up their own." This is where most projects in this are seem to go the wrong way. At some point someone steps in and wants to take control instead of developing tools that would allow the negotiation of shared topic meanings... [<A href="http://Seblogging.cognitivearchitects.com/SebastianFiedler">Sebastian Fiedler</A>] [<A href="http://Seblogging.cognitivearchitects.com/">Seblogging News</A>]</BLOCKQUOTE> Since Paolo's currently out of <A href="http://paolo.evectors.it/2003/04/07.html#a1559">range</A> I guess I should add something here. <A href="http://www.bookcafe.net/blog/aggregator/">Blog Notes</A> (Giuseppe Granieri's blog aggegrator project) is a structured news feed viewer.  It allows you to attach certain pre-defined topics to a post to indicate such things as: <UL> <LI>the post is an ironic comment</LI> <LI>the post is about politics</LI></UL> In order to keep the structure under control the range of topics must necessarily be restricted.  So in essence the range of topics in Blog Notes forms a controlled vocabulary and liveTopics will, to some extent, support this. However that won't mean that people will be unable to create their own topics or interact with other applications that do permit is such as Phil Pearson's <A href="http://topicexchange.com/">Internet Topic Exchange</A>.  As a sidenote I am very conscious that the work in integrating liveTopics and TopicExchange never got finished.  I have it in mind to get in touch with Phil again and see where we are with the outstanding issues.  (Phil if you see this and have a moment, please ping me via IM). http://matt.blogs.it/entries/00001362.html Announcing: ENT v1.0 Easy News Topics for RSS2.0 Fri, 11 Apr 2003 13:59:53 +0100 <A href="http://www.purl.org/NET/ENT/1.0/"><IMG alt="Easy News Topics" src="http://matt.blogs.it/specs/ENT/1.0/ENT10.gif" border=0></A> <A href="http://paolo.evectors.it/2003/04/11.html#a1567">Paolo</A> and I are pleased to announce the release of the first public draft of the <A href="http://www.purl.org/NET/ENT/1.0/">Easy News Topics (ENT) specification</A>.  ENT1.0 is an RSS2.0 module designed to make it really easy to incorporate topics into RSS feeds.  Why would you want to do that?  Because it will help to enable a raft of new, smarter, aggregator products. RSS has become very important to a lot of us and we are starting to see its penetration into the business world as well.  We think that integrating topics will help aggregators applications to scale to meet the future needs of users as well as delivering some very powerful applications.  I've spoken before about the kinds of thing I want my aggregator to do: <UL> <LI>group posts from many feeds by interest. <LI>filtering posts I don't want to see <LI>scoring & promote posts <LI>recombine different feeds dynamically.</LI></UL> I hope that ENT might help bring all these things a little closer.  We also see a role for classification in bringing new ways to order, view, and, search weblog data. We are offering ENT1.0 to the community (under a <A href="http://creativecommons.org/licenses/by-sa/1.0/">Creative Commons License</A>) in the hope that we can foster these applications and many more, that we haven't even begun to think of yet. I will soon be releasing to the public the next version of liveTopics which will be ENT compliant.  At that point any Radio user will be able to easily add topic metadata to their RSS feed.  We hope that there will soon be many applications available to make use of it. We look forward to your comments. http://matt.blogs.it/entries/00001374.html MyRadio gets ENT Mon, 21 Apr 2003 09:45:18 +0100 <BLOCKQUOTE dir=ltr style="MARGIN-RIGHT: 0px"> <A href="http://radio.weblogs.com/0100875/2003/04/17.html#a932">myRadio supports ENT 1.0</A>. My<IMG src="http://radio.weblogs.com/0100875/images/radioBadge.gif"> supports <A href="http://matt.blogs.it/specs/ENT/1.0/"><IMG src="http://matt.blogs.it/specs/ENT/1.0/ENT10-small.gif" border=0></A> Announcing the release of <A href="http://matt.blogs.it/specs/ENT/1.0/ENT10-small.gif">ENT 1.0</A> (Easy News Topics) support in <A href="http://radio.weblogs.com/0100875/outlines/myRadio/">myRadio</A>. One of the stated goals of ENT is to "represent topics sufficiently that they be useful in enabling smart aggregators (e.g. filtering, recombining feeds, etc...)". RSS+ENT feeds can be filtered in myRadio, by selecting topics of interest. Available topics for a feed are those seen by the aggregator, in the RSS feed. That list will grow in time. Later, myRadio will support topicRolls for this purpose. Future features may also include recombining feeds according to topic. Update myRadio.root in RU, or download the latest <A href="http://radio.weblogs.com/0100875/gems/myRadio.root">here</A>. Configure using the "Edit Topics" link in the myRadio navigation bar. Please contact me with any feedback, suggestions, and bug reports. Currently, the only known feeds supporting ENT 1.0 are <A href="http://matt.blogs.it/rss.xml">Curiouser and curiouser</A> and <A href="http://topicexchange.com/rss">Topic Exchange</A>. ENT enabled feeds should increase greatly when liveTopics 1.3.3 is released. [<A href="http://radio.weblogs.com/0100875/">Brain Off</A>]</BLOCKQUOTE> Fantastic news.  Well done Mikel. This will be the first application in the hands of users that will let them get the benefit of topics in their feeds.  liveTopics 1.1.3 is in beta at the moment and should be available soon.  Once that happens there will be a small cluster of feeds that do support ENT.  But we need to do more. Specifically we need to find a way to get at the hordes of MovableType users and get them in the game.   http://matt.blogs.it/entries/00001409.html Making aggregators more useful Thu, 24 Apr 2003 12:34:15 +0100 <BLOCKQUOTE dir=ltr style="MARGIN-RIGHT: 0px"> Aggregator as archives, not only as instant readers.. <BLOCKQUOTE><A href="http://scriptingnews.userland.com/2003/04/23#When:7:44:43PM">Dave Winer</A>: RSS readers that work like Usenet readers are a waste of time, imho. Aggregators should not organize news by where items came from, just present the news in reverse chronologic order.</BLOCKQUOTE> I totally agree, this is why after trying other aggregators I came back to "Radio": it's just like reading a very large weblog updated by several people every hour, it's an activity we are familiar with and the ability that we have to scan titles and relevant information in a page makes this approach much more efficient than having to move from source to source. If an aggregator is meant as a way to take a snapshot of what's going on on hundreds of sources and quicky present it to us, I believe that presenting news in reverse chronological order is the way to go. But I also think that aggregators could be an interesting way to archive content, to let somebody quickly retireve something wrote sometime in the past. Archiving by author, again, does not make sense: most weblogging applications already do that, if I'm looking for something and I know who wrote it, I can simply look on the author's site. There are search engines, which are of course a good way to find information, but not always very efficient. There are cases when a directory might be more useful. We believe that archiving by topic in a directory could be a solution, and this is what we are trying to do. It's not for daily instant reading, it's to archive content. [<A href="http://paolo.evectors.it/">Paolo Valdemarin: Paolo's Weblog</A>]</BLOCKQUOTE> I agree with Dave that organisation based upon source isn't a particularly useful innovation on it's own.  But if he means that aggregators shouldn't attempt to organise posts at all (other than by order of arrival) then I totally disagree. I still use Radio as my aggregator but I am not happy with it.  It's just the most convenient reader I have found to date.  Why?  As I have said before, those readers actually made things less convenient.  For example one, I forget which, offered me the ability to group feeds into an outline structure.  This was good it meant I could keep my general news feeds aside from the feeds of people I know and so on.  But when I clicked on a folder of feeds, it didn't present me with a view of all the items from each of the feeds in the folder.  Now, why the hell not?  This feature alone would have allowed me to dump the Radio aggregator. Also we are working on k-collector which is a server based aggregator that will group items together from a number of different feeds, but it does that using <A href="http://www.purl.org/NET/ENT/1.0/">ENT</A> topics.  This is integrated with our topic roll concept so that if you and I use topics from the same topic roll, our posts will appear together.  Once you see this working (and we hope to show you soon) you can see how powerful this becomes.   http://matt.blogs.it/entries/00001431.html The petunia's had it Tue, 24 Jun 2003 07:25:13 +0100 <BLOCKQUOTE dir=ltr style="MARGIN-RIGHT: 0px"> <A href="http://www.myelin.co.nz/post/2003/6/24/#200306241">Replacing RSS?</A>. It looks like various people are <A href="http://intertwingly.net/wiki/pie/RoadMap">working on a replacement for RSS</A>, the ubiquitous <A href="http://backend.userland.com/rss">weblog syndication format</A>. I look at this and think one thing:    why? Why do you need to replace RSS? Why do all blogging tools and aggregators need a new format? Do they really need it, or can you do it with namespaced additions to the RSS 2 spec? <Sigh> <A title="Click here to comment on this post." href="http://www.myelin.co.nz/phil/pss/comments.php?u=2&p=200306241&link=http://www.myelin.co.nz/post/2003/6/24/#200306241">Comment</A> [<A href="http://www.myelin.co.nz/post/">Second p0st</A>]</BLOCKQUOTE> Curiously enough, the only thing that went through the mind of the bowl of petunias as it fell was "Oh no, not again."  Many people have speculated that if we knew exactly why the bowl of petunias had thought that we would know a lot more about the nature of the universe than we do now. http://matt.blogs.it/entries/00001502.html aggregator:// Tue, 15 Jul 2003 17:45:53 +0100 <BLOCKQUOTE dir=ltr style="MARGIN-RIGHT: 0px"> Aggregator url. <BLOCKQUOTE><A href="http://radio.weblogs.com/0103807/2003/07/15.html#a1885">Scott Johnson</A>: what I'd recommend to aggregator vendors is that they standardize on an <a href="aggregator://"><a href="aggregator://">aggregator://</A></A> protocol so that other tools which produce RSS can easily embed that into applications.  I'd gladly add a generic Aggregator button to Feedster in a heartbeat so that this could work with any tool that handles the <a href="aggregator://"><a href="aggregator://">aggregator://</A></A> protocol.  [<A href="http://radio.weblogs.com/0103807/">The FuzzyBlog!</A>]</BLOCKQUOTE> Sounds like a good idea to me. We'd happily support it too within our current and future applications. [<A href="http://paolo.evectors.it/">Paolo Valdemarin: Paolo's Weblog</A>]</BLOCKQUOTE> I don't quite understand the idea, what is processing this aggragator:// protocol scheme?  And what is it telling you? http://matt.blogs.it/entries/00001515.html Leave a little leaway Thu, 17 Jul 2003 19:20:23 +0100 <BLOCKQUOTE dir=ltr style="MARGIN-RIGHT: 0px"> <A href="http://radio.weblogs.com/0103807/2003/07/17.html#a1892">What If I Winer Watched You?</A>. <H3>What If I Winer Watched You?</H3> Although the recent controversy over the "Winer Watcher" (ww) has now ceased due to its shutdown, I'd like to raise the issues again.  But before I do, here's a recap. <OL> <LI><A href="http://www.scripting.com/">Dave Winer</A>, the subject of the WW is a prominent blogger who, like other people, sometimes changes his blog posts. <LI><A href="http://diveintomark.org/">Mark Pilgrim</A> is another prominent blogger who put up an application which tracks every post and shows you all versions of the same post. <LI>Dave has called this stalking while Mark maintains that it is useful since Dave changes his posts and often to Mark's detriment.</LI></OL> Now to start you might be wondering why I'm bringing this up again.  Well a Feedster user, one I respect quite a lot, has asked for us to implement this type of WW functionality -- but for anyone.  So the question becomes: Evil or Not Evil?  And I'm really not going to hash out the rights and wrongs here.  That's not my job 'mon so to speak.  What I am going to point out is this: <UL> <LI>What felt unfair to me is that Mark didn't subject himself to the same microscope.  He put up the application for Dave and Dave alone.  Why didn't he subject his own posts to the same scrutiny?  If Mark never changes his posts then we'll all know that once and for all.  And if he does then isn't it just plain "more fair" to know that? <LI>It would be technically trivial for Feedster to implement Winer Watcher.  All we need to do is add a version bit to our database since we track updated posts anyway.  What we currently do is update our posts table when a post is updated.  It would be easy to just store the updated version, only index the latest version and then show on our cached posts page all versions of a post sorted by newest to oldest. <LI>How would you feel if we did this not to just Dave but to anyone?  How would you feel if we did it to you ?</LI></UL> Please understand that I'm not saying that we are going to do this or we aren't.  Just because technology makes something possible, that doesn't mean that it should be done.  My biggest objection to Mark's WW was the lack of fairness -- he subjected one person and one person only to the Microscope and didn't reverse the tool onto himself.  Quite honestly we saw the death of privacy a long time ago and now all we have is an illusion.  Perhaps that illusion is a good one.  I'm not certain.  I do think there is a lot to be said for making people accountable for what they said.  And if there is no record then there is no accountability.  None.  I personally think that one of the things that keeps debate int he blog world more civil than it is in other cyberspace forums is that there is a good record.  Mailing lists and chatrooms, while they may have records, are never as easy to navigate as are blogs.  Does a global WW actually lead to a better, more civilized blogspace ?  Is being accountable for what you say a bad thing? <BLOCKQUOTE dir=ltr style="MARGIN-RIGHT: 0px"> Now let's think of this in another way.  What about our elected officials?  I mean <A href="http://blog.deanforamerica.com/">Howard Dean</A> is now a blogger.  How good would it be to have a WW that tracked what Dean says.  Or a WW that tracked what any politician said?  Or is it that we only want that WW technology when its applied to people we don't like?</BLOCKQUOTE> Personally I come down on the accountability is good side of the argument.  But that's me.  I will admit, however, that I love the thought of a WW for our politicians. Comments requested please.  I really do think that this is an important issue. References <UL> <LI><A href="http://weblog.infoworld.com/udell/2003/07/16.html">Jon Udell</A> <LI><A href="http://weblog.burningbird.net/fires/001349.htm">Burning Bird</A> <LI><A href='http://feedster.com/search.php?hl=en&ie=ISO-8859-1&q="Winer%20Watcher"&sort=date'>Feedster Search</A></LI></UL> [<A href="http://radio.weblogs.com/0103807/">The FuzzyBlog!</A>]</BLOCKQUOTE> I would be quite happy for such a technology to exist, provided there was a common framework of rules about how it should work.  For example I quite often edit a post within a few minutes of writing it when I wonder if I've been too snippy or got something wrong.  I accept that, quite often, someone has polled my feed in those few minutes.  That's the name of the game. What I would like to see is perhaps a value added to the RSS/ATOM feed that specified "do not keep before X minutes".  The way this should work is that if the engine sees an update within those X minutes, it discards the previous version.  Otherwise it can keep it. Now this is no more enforceable than any other rule, but if followed widely it would allow for the useful ability to track changes, whilst still allowing authors some leaway in making & correcting mistakes of grammar, content or judgement. http://matt.blogs.it/entries/00001516.html Unwanted-content-encoded: Wed, 30 Jul 2003 14:55:25 +0100 <BLOCKQUOTE> <A href="http://radio.weblogs.com/0103807/2003/07/30.html#a1928">Atom: The Fatal Flaw No One Has Yet Noticed</A>. <H4>Atom: The Fatal Flaw No One Has Yet Noticed</H4> For the answer to the 1st part of the question, you'll have wait for the interview.  But the flaw to me was here "Atom supports content".  Now I don't have any problem with content being supported -- but -- Atom supports encoded content.  And that, to me, is a fatal flaw. ... Now if you notice the "base64" I'd have to assume here that they actually intend to support an image file within the RSS feed.  Now what happens when people start to actually use this.  You'll see issues like this: <UL> <LI>Every single user has to download that image.  Even if they don't want it. This both increases bandwidth and removes control from the reader.  It makes syndication more like email.  Oh that's a good idea !  Sheesh. <LI>Bandwidth usage with increase for everyone -- provider and reader.  <LI>There's no way to control who downloads that image (or video or audio) -- its all in the one syndication file and you have to download the file to see it <LI>If users are disabled (blind or hearing impaired) they still have to get the media.  That makes sense.  Yeah sure it does.  What was <A href="http://diveintomark.org/">Mark Pilgrim</A> thinking?  He even <A href="http://diveintoaccessibility.org/">understands</A> these issues. <LI>What happens when someone embeds something illegal or a pirated audio or video clip into their syndication file?  Who's liable?  At least when its linked you can choose not to follow it.  Now you could end up with stuff on your hard disc you have no idea was there.  And don't think it won't happen.  What about pr0n in the workplace? <LI>When you're a blog author do you have to choose every time that you want a media item to go in your feed or as a linked item?  Sure there can be defaults but what you really want is your tool to say something like this: This jpeg is 1.2 megs.  Your feed is downloaded on average 2500 times per day and this will cause X megabytes of bandwidth to be used costing you $154.37.  Do you want this in the feed where everyone will see it or as a link where only 25% (expected probability) will click on it. Shame no one will ever write that dialog box.  Good for hosting companies though.  Sure I want to give them more bandwidth. <LI>Let's play "Crash that Aggregator"!  Just wait until someone starts fiddling with encoding options and your aggregator is told to expect a jpg file and gets an EXE instead.  Think email is a security minefield now?  Guess what -- your aggregator is headed the same way if it supports encoded content.  </LI></UL> Atom support for encoded content.  What were they thinking ?  Sounds good in theory but in practice?  Yikes.  Just because you can do something technically doesn't mean you should. [<A href="http://radio.weblogs.com/0103807/">The FuzzyBlog!</A>]</BLOCKQUOTE> It seems to me that Scott is making some very good points here.  I remember reading the stuff about content encoding and thinking "good idea" so I guess it has some valid uses, but perhaps these are good enough reasons to find another way around... http://matt.blogs.it/entries/00001533.html When aggregators attack Mon, 18 Aug 2003 14:28:23 +0100 <BLOCKQUOTE dir=ltr style="MARGIN-RIGHT: 0px"> <A href="http://scriptingnews.userland.com/2003/08/18#wiredOnAggregators">Comments here</A> on Wired's <A href="http://www.wired.com/news/infostructure/0,1377,60053,00.html">article</A> on news aggregators. [<A href="http://www.scripting.com/">Scripting News</A>]</BLOCKQUOTE> The title of the article  Aggregators Attack Info Overload  suggested to me a slightly more in depth piece looking at the value of using RSS and aggregators to communicate and share.  In this, I think it fell short. I'm also disappointed because it meant that <A href="http://w4.evectors.it/">K-Collector</A> didn't get a mention.  We may be the new kid on the block but with RSS+ENT I think we're doing something really interesting in this space. http://matt.blogs.it/entries/00001550.html Aggregator work Wed, 27 Aug 2003 14:24:51 +0100 We're rewriting part of the K-Collector aggregator using a new design. http://matt.blogs.it/entries/00001592.html Test#2 Wed, 27 Aug 2003 14:34:49 +0100 Continuing to test the aggregator. This may take some time & many posts. http://matt.blogs.it/entries/00001594.html Aggregator as hub Fri, 05 Sep 2003 10:40:17 +0100 <div style="margin-left: 40px;">Intranet aggregators. I spent most of the week visiting small and large companies. The more I talk with "real people working in real companies" (meaning: not nerds spending their whole days hacking), the more convinced I am that a news aggregator is the ideal center for any Intranet. The basic idea is merge to the same server contents coming from: </div> <ul style="margin-left: 40px;"> <li>internal sources (accounting, trouble ticketing, exiting document management applications, other data bases: we should be able to get a feed from any internal app)</li> <li>k-logs (every member of the group has one)</li> <li>external news sources (general news, weblogs, specialized sources, scraped pages)</li> </ul> <div style="margin-left: 40px;"> </div> <div style="text-align: center; margin-left: 40px;"><img src="http://paolo.evectors.it/myImages/intranetAggregator.gif"> <div style="text-align: left;">The output of the aggregator should be both html that people can browser with their browser and more feeds which could end up in personal aggregators or funneled in other applications. Centralized aggregators should not necessarily mean that every user has to read all feeds. There should be both the kind of personalization allowed by personal aggregators (deciding which feeds to subscribe to) but also added vaue services that would allow users to discover additional sources of information and anyway give different relevance to different kind of information snippets that are displayed on the page. </div> </div> <div style="margin-left: 40px;"> [<a href="http://paolo.evectors.it/">Paolo Valdemarin: Paolo's Weblog</a>] </div> Paolo has put it succinctly. The aggregator becomes the organisational hub gathering information from all sources and using personalisation and intelligence to filter & recombine information in useful ways before presenting it either as views or as feeds for other consumers. http://matt.blogs.it/entries/00001610.html NewsBayes Thu, 20 Nov 2003 16:23:49 +0000 <div style="margin-left: 40px;"><a href="http://weblog.infoworld.com/udell/2003/11/20.html#a851">Working with Bayesian categorizers</a>. </div> <blockquote cite="O'Reilly Network" style="margin-left: 40px;"> There's been some discussion in the blog world about using a Bayesian categorizer to enable a person to discriminate along various interest/non-interest axes. I took a run at this recently and, although my experiments haven't been wildly successful, I want to report them because I think the idea may have merit. [Full story: <a href="http://www.xml.com/pub/a/2003/11/19/udell.html">O'Reilly Network: Working with Bayesian Categorizers</a>] </blockquote> <div style="margin-left: 40px;">This month's O'Reilly Network column was a struggle because categorization itself is a struggle. I remain convinced that the automated classifiers that are doing such a good job beating back the tide of spam will also turn out to be more generally useful. But finding the right synergy between an automated assistant and a human overseer is a subtle and tricky thing. ... [<a href="http://weblog.infoworld.com/udell/">Jon's Radio</a>] </div> Great write-up by Jon of his experiments trying to hotwire Bayesian categorizers for auto-classifying blog posts. I think his conclusions demonstrate that there is value in the approach, certainly as Jon notes, for auto-classification of incoming items as in a news aggregator.  Of course this is exactly what we are doing with <a href="http://w4.evectors.it/">K-Collector</a> and I have mused for a while about whether we could use a Bayesian classifier to improve how we do that (at the moment we are using some pretty simple keyword analysis). I think one of the biggest obstactles is the training.  We have somewhere around 400 topics and many thousands of posts.  How hard would it be to train a classifier?  How long would it take?  Could you sell it? http://matt.blogs.it/entries/00001743.html What comes after changeblogs? Thu, 20 Nov 2003 16:37:16 +0000 <div style="margin-left: 40px;"><a href="http://www.decafbad.com/blog/links/boocjaajai.html">Quick Links</a>. </div> <ul style="margin-left: 40px;"><li><a href="http://jeremy.zawodny.com/blog/archives/001075.html">CVS Commit + Weblog = Changeblog </a></li></ul> <div style="margin-left: 40px;"> [<a href="http://www.decafbad.com/blog">0xDECAFBAD</a>] </div> <a href="http://www.dijest.com/aka/">Phil Wolff</a> and I were <a href="http://matt.blogs.it/2003/07/21.html#a1042">talking</a> about this a little while ago and now <a href="http://www.multiply.org/notebook/">Jason Gessner</a> has it working.  That's great! The next stage for me would be to start creating RSS feeds with <a href="http://matt.blogs.it/spects/ENT/1.0/">ENT</a> metadata from those changeblogs. At a very basic level I can imagine a <a href="http://w4.evectors.it/">K-Collector</a> topic corresponding to each project.  Then each topic view could aggregated not only what developers were writing in their own blogs but also CVS messages corresponding to the status of the project. But why stop there.  Your junit based daily smoke & build test could be generating a similar feed, your project management system, issue database and so on.  All these feeds could be flowing into project topics giving you an uptodate and holistic view of what is happening in those projects. Who'd ever be caught on the hop in a project meeting again? http://matt.blogs.it/entries/00001746.html Whose personality do you want today? Fri, 21 Nov 2003 17:47:08 +0000 <a href="http://www.decafbad.com/blog/">l.m.orchard</a> <a href="http://radiocomments.userland.com/comments?u=0107808&p=1243">commented</a> regarding the <a href="http://matt.blogs.it/2003/11/20.html#a1243">using Bayesian analysis on news</a>. In fact, as soon as I saw it I remembered, I had read his <a href="http://www.decafbad.com/blog/geek/bayes_agg_one.html">piece</a> already. It was probably his writing that triggered my initial interest in using a Bayesian classifier in <a href="http://w4.evectors.it/">K-Collector</a>. Re-reading that piece I got an interesting different angle since his approach was to blend a Bayesian classifier with his news aggregator to try and have it prioritize news he would find interesting and not to categorize it by topic. I think this is a much more scalable task, from a K-Collector perspective, than what <a href="http://weblog.infoworld.com/udell/2003/11/20.html#a851">Jon</a> is experimenting with. I think the efforts of training a system-wide recognizer to differentiate between topics would be too much for most users of the product to bear. Our product roadmap for K-Collector already includes allowing users to personalize the system. For example we think that people should be able to say which feeds they think are relevant on different topics. Notice that this is a much very granular relationship since it means that I can say "Matt Mower is a real expert on the topic sock puppets" but that this says nothing about how relevant I am on "dating." or any other topic. Indeed each user might rate the exact same sources differently over a wide range of topics. What might be interesting is if people could "share" and "subscribe to" preference maps. As a new user of the system you might not really know who is relevant on any particular topic. But imagine you worked with <a href="http://www.hyperorg.com/blogger/">David Weinberger</a>, <a href="http://www.dijest.com/aka/">Phil Wolff</a>, or <a href="http://weblog.siliconvalley.com/column/dangillmor/">Dan Gillmor</a>. If you knew them and trusted their judgement you could pick one of their preference maps as a starting point and immediately gain a usseful insight into the data as it is structured by topic. You might even switch between personalities to get more perspective! Thanks to l.m.'s piece I am now wondering also about whether a Bayesian classifier might be more use in helping users to establish their own preference maps about which content is most relevant to them. http://matt.blogs.it/entries/00001755.html I am not a vector. I am a free man! Mon, 24 Nov 2003 15:22:16 +0000 We've posted an <A href="http://www.evectors.com/itkcollector/story$num=4&sec=1&data=kcollector">essay</A> which attempts to highlight where we think organisations can benefit from blogging. <%softShadow("http://paolo.evectors.it/stories/vectorWorkers/images/img6.gif")%> It continues our effective organsiations theme and uses what we think is a very strong visual metaphor for understanding how different organisations can apply similar amounts of effort but get wildy different results. http://matt.blogs.it/entries/00001769.html Licensed to aggregate Sat, 07 Feb 2004 23:41:47 +0000 I have been encouraged to think about how copyright content works with RSS. For example I publish my weblog under a <a href="http://creativecommons.org/">Creative Commons</a> license however you couldn't tell that from my <a href="http://matt.blogs.it/rss.xml">RSS feed</a>. Does that mean that my feed is not under a CC license? I don't, but I guess that it's confusing at best. What should an application reading my feed and not my blog do? What rules should it apply? I didn't find any solutions to this question searching Google just now so I've started thinking of one myself. Obviously if anyone knows of solution already in use I'd be grateful to hear about it. My first attempt at a solution is to propose a simple new RSS 2.0 extension for Licensing. This extension would add just one new element <code><license></code> which can be applied at both <code><channel></code> and <code><item></code> level. Using such a mechanism a license can be applied to the feed and overriden for specific items if required (although this would require more control in the editing environment). For example one might apply a <a href="http://www.primarilypublicdomain.org/">Primarily Public Domain</a> license to their feed, but override this on a specific item for which they wished to retain the copyright. The next question is what the content of such an element should be. Creative commons licenses have a useful URL which it would be helpful to include. Other (or future) types of license may also have a similar arrangement. Copyright notices on the other hand do not generally have an addressable resource. Therefore I propose the use of an optional <code>src</code> attribute which can point at any addressable resource related to the license. Since copyright is (as far as I understand it) a binary concept I propose another attribute, <code>copyright</code> which has a default value of <code>true</code>. To remove the copyright the attribute should be specified with a value of <code>false</code>. The content of the element then can be an arbitrary string. A copyright notice in the case of a copyright work, or some other useful descriptive string. Some examples: <ul><li><code><license>Copyright (c) 2004 Matt Mower</license></code></li> <li><code><license copyright="false" src="http://creativecommons.org/licenses/by-nd-nc/1.0">Creative Commons - By Attribution, Non-Commerical, No Derivative works</license></code></li> </ul> Any takers? http://matt.blogs.it/entries/00001853.html Permission based posting Sun, 08 Feb 2004 00:13:18 +0000 Just a couple of follow-on thoughts... <ol><li>If a feed is copyright, does that mean an aggregator should not collect it? Is this reproducing the work? (How is it different, for example, from downloading pirated software?)</li> <li>What does it say when an author asserts copyright of material, but publishes it in an RSS feed? The Creative Commons licenses grant rights to the recipient. Copyright (AFAIK) does not.</li> <li>Does it matter what type of aggregator it is? For example <a href="http://www.evectors.com/itkcollector/">K-Collector</a> aggregates in a different way (and for a different purpose) to, for example, <a href="http://radio.userland.com/">Radio Userland</a> or <a href="http://ranchero.com/netnewswire/">NetNewsWire</a>.</li> <li>If it aggregates a feed, should it prevent you from re-posting aggregated content? In Radio this is very easy, I can click a button on a post in the aggregator to use it as the basis for a post to my own weblog. If an item is copyright should it stop me doing that?</li> </ol> I think this draws us awfully close to the kinds of things Ted Nelson was thinking about when he outlined his vision for transclusion based publishing. I can almost envisage a system where, when I press the post button on a copyright item, my aggregator goes off to check with the original authors system for permission to publish. You could imagine using the data from a social network to decide who can republish your content with your permission. http://matt.blogs.it/entries/00001857.html Licensed to aggregate (pt #2) Tue, 10 Feb 2004 08:00:08 +0000 I have received a couple of comments to my earlier <a href="http://matt.blogs.it/2004/02/08.html#a1315">piece</a> about licensing & RSS. Among others Phil Ringnalda pointed me at the <a href="http://backend.userland.com/creativeCommonsRssModule">Creative Commons RSS Module</a> authored by <a href="http://www.scripting.com/">Dave Winer</a> in Dec 2002. The CC module is pretty much the same solution as I was proposing in that it seems to be a start in the right direction. However neither it, nor my own suggestion, answer all the questions in a way I can appreciate (e.g. as Phil points out, what is covered by the license: text only? images? etc...), so i'm going to keep chipping at this. A commentor, <a href="http://www.lulop.com/">Lorenzo</a>, has suggested that <blockquote>Copyright is a "state" and licensing is an "action" made possible by the state.</blockquote>I kind of disagree with this statement but mainly on terminological grounds. I hope we can cut through that by agreeing that the central point is rights. Who has them? What uses do they permit? It seems to me that a Copyright statement reserves all rights relating to making copies (& derivative works) to the author who can then make exceptions on a case-by-case basis. As in the case where an author grants their publisher the right to make copies for sale. Hence an RSS feed with a copyright notice shouldn't be read (copied) unless you consider the act of offering feed to be an implicit agreement by the author to do so. Of course from a software perspective implicit rights can be problematic, especially when they are not immutable or well understood. For an RSS feed with a copyright statement what rights are actually on offer? People commonly republish content from posts they have aggregated either whole or in part. How do they know they have the right to do so? Then you have some interesting anomalies when software gets in the way. For example, my <a href="http://matt.blogs.it/">weblog</a> has a <a href="http://creativecommons.org/">Creative Commons</a> <a href="http://creativecommons.org/licenses/by-nd-nc/1.0">license</a>, yet my <a href="http://matt.blogs.it/rss.xml">feed</a> is Copyright. I guess Radio is automatically adding the copyright notice, I don't know how to make it stop. What am I telling you? Alternatively, and I guess this may be common, Phil Ringnalda's blog doesn't have any kind of license at all, neither does his feed. Can I assume Phil intends all his material to be public domain? If i'm not clear that this is the intension how can my software be? Creative Commons offer a range of licenses which offer the right to copy, or make derivative works, with certain restrictions such as share alike and non commercial use only. Primarily Public Domain takes this a step further in granting unlimited rights with exceptions being, if you'll excuse the pun, the exception. Maybe it's my developer "tunnel vision" at work but this looks very similar to the common model of permissions adopted in software everywhere: <ul> <li>include * except A, B, C,...</li> <li>exclude * except R, S, T,...</li> </ul> where A, B, etc.. are not users or hosts but specific actions, by identified individuals, in specific circumstances. For most personal news aggregators I guess none of this matters much. If someone publishes a feed, and you're just reading it, then whats the harm? Unless of course you weren't supposed to have the URL to the feed. But that's a different issue. However for aggregators like <a href="http://w4.evectors.it/">K-Collector</a> and users who are reposting content it's a different matter. For example K-Collector doesn't mess with the content of posts, but it does republish them in a new context. If K-Collector has a better Google page rank than the author then we even begin to suck traffic away from them based upon their own material! (To be clear I am only talking about the public K-Collector portal <a href="http://w4.evectors.it/">W4</a>. This is an untypical use of <a href="http://www.evectors.com/itkcollector/">K-Collector</a> which is designed for use within organisations. But the point still stands. And what about <a href="http://www.feedster.com/">Feedster</a>?) As ambitious, semi-automated, software like K-Collector become more common then a reasonable, dependable, system of rights is going to be required. To my way of thinking the <a href="http://backend.userland.com/creativeCommonsRssModule">Creative Commons RSS Module</a>, whilst a start in the right direction, addresses a necessary, but not sufficient, subset of the goals. What do we do to take it further? http://matt.blogs.it/entries/00001862.html Enhancing Radio's aggregator Wed, 25 Feb 2004 21:49:25 +0000 <blockquote><a href="http://radio.weblogs.com/0104487/outlines/radio/enhancedAggregator.html">Introducing the enhancedAggregator</a>. <a href="http://radio.weblogs.com/0104487/images/my/aggregator.png" target="_blank" title="view hires"><img src="http://radio.weblogs.com/0104487/images/my/aggregatorLR.gif" alt="aggregator topics" align="right" border="0"></a>I've recently spent some time investigating Radio's aggregator code, looking for an easy way to support additional RSS modules in general, and ENT 1.0 topics in particular. The <a href="http://updates.prec-it.com/download/enhancedAggregator.root" title="click to download">enhancedAggregator tool</a> is the -provisional- result of this investigation. It comes with full <a href="http://matt.blogs.it/specs/ENT/1.0/" target="_blank">ENT 1.0 topics</a> support for Radio's aggregator, and skeletons for aggregating <a href="http://www.intertwingly.net/wiki/pie/FrontPage" target="_blank">Atom 0.3 feeds</a> and <a href="http://www.esfstandard.org/" target="_blank">ESF 1.0 events</a> for RSS 2.0. I'd like the enhancedAggregator to become a community driven project, allowing Frontier/Radio developers to easily test aggregation of new syndication formats and extensions, without mobilizing Userland scarse resources. So I've added some intal/uninstal/update/prefs ancillary functions to the tool, and provided <a href="http://radio.weblogs.com/0104487/outlines/radio/enhancedAggregator.html" target="_blank">guidelines</a> for updating the current drivers and adding new ones, with pointers to the available online documentation. I hope <a href="http://matt.blogs.it/" target="_blank">Matt</a> will copy the <a href="http://matt.blogs.it/specs/ENT/1.0/" target="_blank">ENT</a> module driver and paste it into the <a href="http://groups.yahoo.com/group/k-collector-support/" target="_blank">k-collector</a> client for Radio, and <a href="http://paolo.evectors.it/2004/02/24.html#a2062" target="_blank">Paolo</a>'s eVector crew will build upon the <a href="http://www.esfstandard.org/" target="_blank">ESF</a> module driver skeleton, copying the result to their new tool when it's stable enough. [<a href="http://radio.weblogs.com/0104487/">s l a m</a>]</blockquote> Marc's doing sterling work. http://matt.blogs.it/entries/00001915.html We built a better mousetrap, where are the mice? Sat, 08 May 2004 22:53:24 +0100 <blockquote><a href="http://matt.blogs.it/specs/ENT/1.0/">Wouldn't it be cool?</a>. Wourldn't it be cool if otehr people started parsing the ENT tags embedded in this post? That way if I talked about FOAF - for instance - someone like danbri could scarf JUST the FOAF posts and do anything he wanted with them! This would create an incredible two-way kind of capability because systems could then communictae back to me based upon what I said. I know I know -it's RSS2.0 but that becomes a really nice gateway to a world that has 75% market share of feeds(maybe even more.) [<a href="http://blogs.it/0100198/">Marc's Voice</a>] </blockquote> I think it would be pretty cool -- I've been hoping other would start grokking this for a while now.  Yes it's RSS2.0, yes it's not perfect, but it's here, now and I think that by the 80/20 rule it's good enough.  If there's anything we can do to help get ENT support included in other applications please let us know and we will do our best to help. I'd also like to see applications start using the <a href="http://matt.blogs.it/specs/SGUID/1.0/">SGUID</a> information that Paolo and I have had in our feeds for about the same length of time as we've been doing ENT.   Using SGUID one post in an RSS feed can refer directly to the permalink of the post it is quoting from.  Standard RSS2.0 only allows you to refer to the feed.  So for example, tags like: <blockquote><sguid:sourceRef>http://www.theobviousblog.net/blog/archives/000506.html</sguid:sourceRef> </blockquote> should allow a clever aggregator to thread posts. An aggregator that did topics and sguid-based threading, that would be nice... http://matt.blogs.it/entries/00002203.html K-Collector client demystified Thu, 20 May 2004 08:49:02 +0100 <blockquote><a href="http://www.readwriteweb.com/archives/001877.html">More on Topic-Sharing Community</a>. There's already been a great response to <a href="http://www.readwriteweb.com/archives/001876.html">my post last night</a> (see the comments to previous entry). <a href="http://www.blogdigger.com/blog/index.html">Greg</a> suggested his aggregator <a href="http://www.blogdigger.com/">Blogdigger</a> could be included in this - I agree! <a href="http://matt.blogs.it/">Matt</a> and <a href="http://www.andrewsw.com/news/index.php">Andrew</a> also posted very thoughtful responses. Here's some of my feedback (copied from the comments - I must get these enabled inline...): Overnight while pondering my post (which I regard as just a 'starter for 10' btw, not a final solution by any means), I did conclude that KC essentially already does what I describe - polls registered RSS feeds with ENT in them and aggregates them. It would be great if TE also had that functionality. It's the client ping that I think is unnecessary and possibly holding back community uptake - with TE the ping is a manual process for the blogger, and with KC you need to install an add-on tool to enable the pinging. Both require too much manual effort for the blogger (IMHO of course). eg Bloglines does all its aggregation automatically (every hour I think), with no pinging required from the blogger. Although Andrew I take your point about bandwidth utilization. But if Bloglines (and Blogdigger) can do it, why not KC and TE? [<a href="http://www.readwriteweb.com/">Read/Write Web</a>] </blockquote>There seems to be some confusion about how <a href="http://www.evectors.it/itkcollector/">K-Collector</a> server works and the role of the K-Collector client so I thought I would try and give an explanation of how things fit together. The first thing to understand is that you absolutely do not need the K-Collector client for your blog to be part of a K-Collector site. The client offers a set of benefits aimed at improving the experience for the user, but they are entirely optional. We currently aggregate many feeds to the <a href="http://w4.evectors.it/">W4</a> site which are not using one of our clients. There are three reasons why we think using the client is beneficial: 1) The ping (it's the least important, but seems most misunderstood so I'll cover it first) The K-Collector server contains an aggregator which reads all feeds on a rotating basis. It aims to read each feed more or less once per hour but this isn't guaranteed. It collects posts from feeds and assigns them to topics using either <a href="http://www.purl.org/NET/ENT/1.0/">ENT</a> metadata supplied in the feed or choices which are auto-discovered using various word-stemming and matching techniques. This all happens entirely independent of the client ping. All the ping does is to move your feed up the list so that new posts you have written are likely to be collected sooner. If you don't ping the server just reads your feed automatically a little later on, that's all. 2) The topic manager Through the client, the topic manager is integrated into the blog editing process and gives authors the ability to assign community topics to their posts as well as being able to create new topics. The topic manager also attempts to suggest topics which may be relevant to the content of the post to make choosing topics easier. Without the client you have no way to decide which topics should be assigned to each post. In this case the server will, when it reads the feed, use it's own automatcher to automatically assign those topics it thinks are relevant. 3) ENT feeds Where the author has choosen topics in the topic manager the client adds the appropriate ENT metadata to the outgoing RSS feed. K-Collector can then use this metadata to accurately assign posts to topics. In summary the K-Collector client offers what we think are very useful benefits to weblog authors, however it is entirely optional and you do not need it for you weblog to be part of a K-Collector site. Equivalently K-Collector itself only cares about RSS.  It doesn't care whether ENT metadata was created by our client or some other application, we're completely agnostic about that. I hope this goes some way to clearing up how the K-Collector system works. http://matt.blogs.it/entries/00002254.html Who aggregates the aggregators Thu, 19 May 2005 16:17:48 +0100 I've been reading <a href="http://feeds.feedburner.com/corante/getreal?m=38">Stowe talking about</a> following Jon Udell's lead and unsubscribing from feeds. The reasoning being that anything good will bubble to the surface anyway.<blockquote>Jon's [Udell] choice is to withdraw the feed tube on a blogger-by-blogger basis. Bloglines and de.licio.us have helped cull the wheat from most chaff feeds, so Jon is willing to forego the main feed and wait the additional few minutes it takes for other filters to bubble up the occasional gem to the surface. But multiply this effect by thousands, as Bloglines reports indirectly via its public subscription data, and a power law begins to emerge. When thought leaders like Udell stop subscribing, thought readers follow suit.</blockquote> It's hard to argue with on one level since blogs are often about finding stuff other people are doing and reporting on it. People posting what they find that's relevant. This is added new layers. But I think it takes care to select the channel from which you are going to receive these gems. What's most relevant to you is down to how content gets analysed. Do you always get the right analysis? As in every case where you appoint an intermediary you have to trust they will report what you need to hear. Of course at a certain point 80/20 is good enough I suppose. For myself I'd rather find ways of scaling better. http://matt.blogs.it/entries/00002628.html Hunting for Squib Thu, 05 Jan 2006 13:08:35 +0000 I just committed some changes that <a href="http://radio.weblogs.com/00001017">Andy</a> sent me last night that update the Fragen3.14 theme for Squib. I've switched Curiouser and Curiouser! to use Andy's theme and I think it looks pretty spiffy. Something else that's spiffy is the way the sidebar is now completely dynamic -- driven by a simple <a href="http://www.yaml.org/">YAML</a> which aggregates chunks of user written HTML, OPML, Textile, or Markdown content. This means the user can re-structure and add to the sidebar content without having to change the theme at all. We're making pretty good progress towards the 0.4 release in all areas but one: Multiple Weblog Support. I need this right now but I'm struggling to get it done because it's a pretty major change (curse me for not anticipating this requirement when I started Squib). Worse yet I will have to merge it with the huge number of improvements that we've made since I branched. Subversion makes this possible without going insane, but it's still not pretty. After that though it should be plain sailing to the 0.4 release and I feel like I'm still on track for mid-January. I've also dusted off the MemeScope aggregator code I was messing with last year. It's almost unrecognizable now (being written for the, now ancient history, Rails 0.11 and on Windows no less!) but it should be reasonably easy to get it working again. In 0.5 I want to merge the aggregator into Squib. Again, this is something I really want. Blogging for me was never the same after I stopped using Radio's <a href="http://www.reallysimplesyndication.com/riverOfNews">River of News</a> aggregator. Although the aggregator was basic and faulty the model was just right. Nothing has replaced my ability to see a post in my aggregator, click a button marked Post, and have it neatly formatted as a new entry ready to blog about. I want that back and Squib 0.5 will give it to me. 0.5 will also see MetaWeblog API support (courtesy of code from Typo) and some other neat bits and pieces. 0.5 will be good. So good progress being made to 0.4 and an exciting roadmap for 0.5! Now I just have to find time to do it! http://matt.blogs.it/entries/00003142.html More on my Google implosion Thu, 12 Jan 2006 08:51:01 +0000 I'm working on the theory that <a href="http://weblog.philringnalda.com/">Phil Ringalda</a> is right about Google hating duplicated content. I'm going to try and get the aggregators with copies of my content to stop publishing it and see if that helps to resuscitate me. I never understood the issue until now but I guess this may be an anti-splogging defence. What really ticks me off though is that I am completely guessing. Google offers me no way to understand what has happened. They may be punishing me for something completely beyond my control and I have no way to tell. The more I learn about Google the more it feels like some kind of <a href="http://en.wikipedia.org/wiki/Skinner_box">Skinner Box</a> gone awry. So now I am faced with the prospect of attempting to get any aggregators republishing my content to stop. I've the good folks at <a href="http://www.iaslash.org/">iaslash.org</a> a polite request that I hope they'll act upon. I'm also trying to get the test blog Phil noticed shut down. I notice that <a href="http://www.google.com/search?num=30&hl=en&lr=lang_en&safe=off&client=safari&rls=en-us&q=%22phil+ringnalda%22&btnG=Search">Phil has been resusciated</a> so maybe, if I can de-duplicate myself, I have the answer. http://matt.blogs.it/entries/00003180.html