Using P2P for RSS Distribution
As more and more people wake up to the idea that a fundamentally polling-based RSS distribution model might have some "scaling challenges", different ideas to handle this come to the fore. Here's an interesting model: using the BitTorrent peer-to-peer infrastructure to distribute RSS feeds.
BitTorrent is a cool bit of engineering that solves the "Tragedy of the Commons" problem that plagues many p2p protocols: you gotta pay (i.e., offer some upstream bandwidth) to play (download). The Java-based client Azureus is my client of choice, but it's really the protocol that's the smart stuff here.
Getting back to how this works with RSS, I personally think this is overkill for the scaling issues at present: I think getting the clients and server-side aggregators/publishers to agree on a push protocol and transport is a more pressing concern. I think the P2P idea is a good thing to keep in our pocket when RSS starts getting bulkier.
Comments
I also think the BitTorrent angle for feeds is not (yet) necessary. But what you guys could do: change the syndication frequency dynamically.
E.g. if your polling of a source feed shows it only changes twice a week on average, you could add an updatePeriod/updateFrequency to the burned feed that tells RSS aggregators to update daily instead of hourly. http://web.resource.org/rss/1.0/modules/syndication/
This would drop the traffic you guys are getting (to 1/24). You could also decide to only poll the source feed once per X hours instead of every 60 minutes.
Posted by: Peter | October 20, 2004 02:51 PM
Oh! Oh! RSS is broken again!
Posted by: Randy Charles Morin | March 10, 2005 07:24 AM