A few weeks ago I suggested merging Topic Exchange and K-Collector together, or at least bring the two sets of functionality closer together. I figure I'll take a leaf out of Marc Canter's book and try and rally the community together on this project. I'm hoping the respective developers of Topic Exchange and K-Collector can get together and figure out some easy solutions. I'm happy to set up a mailing list or wiki to co-ordinate this?
It has to start somewhere and where better than from a humble user's perspective - i.e. me. So here's what I've done so far to get the ball rolling and I want to challenge others to build on this (or if its wrongheaded, suggest other ways to merge KC and TE's functionality).
When I was using Radio Userland, I had my site hooked up to K-Collector using their add-on tool for Radio Userland. When I moved to Movable Type a couple of weeks ago, I had to cease my connection to K-Collector because as far as I know it's a Radio Userland-only tool (although there was talk of an MT add-on being developed?). So tonight I did some tinkering with my blog, to
1) try and re-enable community topic sharing for my weblog; and
2) to resume my project to get an internal topic navigation system running (I had done some initial work with XSLT at the beginning of the year, but that fell by the wayside...).
Now, there is one major thing that Topic Exchange and K-Collector have in common: they both use ENT (Easy News Topics), which is an extension of RSS. Paulo and Matt created ENT as a simple form of topic mapping - and recently they've been talking of upgrading it. When I used Radio, whenever I published a post I also selected some topics from the K-Collector Radio add-on tool. This added the topics to my RSS feed, like so:
<ent:cloud ent:href="http://w4.evectors.it/itEntDirectory/topicRoll.opml">
<ent:topic ent:classification="what"
ent:href="http://w4.evectors.it/itEntDirectory/topic?topic=strategy"
ent:id="strategy">Strategy</ent:topic>
</ent:cloud>
As I understand it, the K-Collector website then aggregated my post to whatever KC topic was specified in the ENT tags in my RSS file ("Strategy" in the example above).
Because I'm using Movable Type now, I needed to find another way to specify topics. The great thing about MT is that it has spare fields where you can basically add whatever you like. So I decided to use the previously unused "Keyword" field to hold all my Topic data. This will serve two purposes: 1) I can hopefully get Topic Exchange and K-Collector to aggregate my posts based on the topic data I enter using this field (more on that in a minute); and 2) I will use it also for my own internal topic navigation. So this allows me to match my own topic navigation to the community topics (in TE and KC).
The next thing I did was alter my RSS template. Firstly I declared the ENT namespace in the <rss> element, like so:
<rss version="2.0" xmlns:ent="http://www.purl.org/NET/ENT/1.0/">
Next, I added the ENT tags. Now for this I decided to start off using Topic Exchange, because it happens to be a perfect fit with the <$MTEntryKeywords$> tag which I'm using for my topics. If you look at the Topic Exchange RSS feeds for each channel, it makes it easy to swap in my MTEntryKeyword. e.g. this is from the TE "Technology" topic:
<ent:cloud ent:href="http://topicexchange.com/topics">
<ent:topic ent:id="technology" ent:href="http://topicexchange.com/t/technology/">
technology</ent:topic>
</ent:cloud>
Because all the TE URL's are in the format "http://topicexchange.com/t/" and the id perfectly matches the topic name, it's easy for me to swap in my MTEntryKeyword. Here's what I added to my RSS template in MT:
<ent:cloud ent:href="http://topicexchange.com/topics">
<ent:topic ent:id="<$MTEntryKeywords$>"
ent:href="http://topicexchange.com/t/<$MTEntryKeywords$>/">
<$MTEntryKeywords$></ent:topic>
</ent:cloud>
I added this just above the ending <item> tag.
NB: it may be just as easy to do this with KC, I haven't checked that yet. However KC has the additonal component of the ent:classification tag, so I decided to tackle TE first as it didn't have that extra tag.
So that's my RSS file ENT-enabled. Now whenever I add a new entry to my weblog, I simply enter the Topic Exchange topic name into my "Keywords" field. When published, this adds it to my RSS file in the ENT tags specified above - which at the moment map exactly to Topic Exchange's format. But...I still have to manually send a ping to TE under the current TE system.
So here's my suggestion (and I apologise if this has been suggested before - I'm sure it has!). What if we, the users, could register our RSS files with Topic Exchange and K-Collector, and both of those services then regularly poll all registered RSS files - say hourly - and pick up any new posts that have ENT tags in them? So in effect Topic Exchange and K-Collector act like RSS Aggregators (e.g. Bloglines) and poll users RSS files, rather than the users having to manually ping the TE and KC websites. It makes more sense for the Aggregator to do the work, rather than the person/blogger.
The beauty of this idea is that we - the users - can use ENT as our 'Topic Central'. That is, we specify our topics in the RSS - and that's the end of our part of the job. It's then up to the TE and KC aggregators to poll registered RSS files and collect all the new ENT tagged data. If e.g. both TE and KC have a topic named "technology", then great they won't have to do any work to add them to their respective clouds. There is a bit of work of course when the topic names and/or ids don't match up - but that's solvable surely? e.g. some form of topic mapping between TE and KC.
So what do you all think? I'd really like to see some action to merge the functionality of TE and KC together - for the good of the community (rah rah!).
TrackBack URL for this entry: http://www.readwriteweb.com/cgi-bin/mt/mt-tb.cgi/2363
Comments
Subscribe to comments for this post OR Subscribe to comments for all Read/WriteWeb posts
Just out of curiosity, why not use the Categories that are supported by Movable Type? Then MT can generate category-based archives for your posts. You can display these in yoru RSS using either dc:subject or the RSS 2.0 category tag. I've been struggling to understand what ENT provides over the basic category functionality, other than reference to some external hierarchy.
In terms of aggregation, Blogdigger does just what you mention - and searching can be done on "topic" (or subjects, as I call them) by querying for something like subject:technology (http://www.blogdigger.com/search.jsp?q=subject:technology)
New posts get added into our "subject cloud" automatically.
Posted by: Greg Gershman | May 19, 2004 11:11 AM
Hi Richard,
We've had a client for MovableType (v2.5+) for some time. It's in the files section of the Yahoo group and includes the topic manager & an ENT enabled RSS template.
The way K-Collector works is broadly as you describe, routinely polling registered feeds. The client ping is simply a way to have your feed read immediately.
Regards,
Matt
Posted by: Matt Mower | May 19, 2004 11:43 AM
Requesting blogging tools to ping upon changes, especially to a central location, is quite reasonable, IMHO. Depending on frequency of updates, ping upon change is quite likely far better bandwidth utilization because otherwise, the other tool might be checking for changes far too much.
However, that central location, until it is a de-facto standard, can have issues getting everybody to ping it - hence the "go and check often" approach is used by those that aren't de-facto standards (yet).
Posted by: Andrew | May 19, 2004 12:10 PM
In no way was I trying to imply that anyone was/was-not a de-facto standard yet. Sorry if I came across that way. :(
Posted by: Andrew | May 19, 2004 12:12 PM
Greg, yes I did consider using MT "category" and it may be something I end up doing. But initially I didn't do it that way because I think of "topic" as being like a subset of "category" - there will be multiple topics and they are more on-the-fly than categories (to me anyway). But as I say, I may yet revert to using MT Categories...
Also the main benefit I see to ENT is that it enables community topic-sharing - esp TE and KC. But then you make a very good point about BlogDigger doing much the same functionality. I'll take a closer look at Blogdigger again - maybe we should consider it just like TE and KC in regards to a topic community?
Matt, thanks for your comment and pointing out there is an MT tool for KC. Overnight while pondering my post (which I regard as just a 'starter for 10' btw, not a final solution by any means), I did conclude that KC essentially already does what I describe - polls registered RSS feeds with ENT in them and aggregates them. It would be great if TE also had that functionality.
It's the client ping that I think is unnecessary and possibly holding back community uptake - with TE the ping is a manual process for the blogger, and with KC you need to install an add-on tool to enable the pinging. Both require too much manual effort for the blogger (IMHO of course). eg Bloglines does all its aggregation automatically (every hour I think), with no pinging required from the blogger.
Although Andrew I take your point about bandwidth utilization. But if Bloglines (and Blogdigger) can do it, why not KC and TE?
Posted by: Richard MacManus | May 19, 2004 1:33 PM
I think that you're on the right track here, i.e. that finding ways to get rid of the client ping will create a lot more uptake.
Posted by: Lucas Gonze | May 19, 2004 1:57 PM
Summary:
KC works by polling RSS files and looking for ENT tags, then pulling the posts out from there.
TE works by listening for pings, and saving them as posts.
As Matt says, you don't actually _need_ to ping KC. I've been thinking about getting TE to read RSS for a while, but haven't done anything about it. TE gets informed when blogs update via blo.gs anyway, so if you ping blo.gs, the TE will find out, and could read your RSS after that.
Posted by: Phillip Pearson | May 19, 2004 2:52 PM
OK, so we've established that KC doesn't actually require the ping process. And I asked Matt in an email whether KC will aggregate my posts if I just include the KC cloud ref in my RSS: the answer is yes (so I don't actually need the KC add-on for MT). That's great.
Phil or other TE techies, how easy would it be to enable TE to read RSS and add posts to TE based on that?
Posted by: Richard MacManus | May 19, 2004 3:09 PM