So, why is the so-called Semantic Web (caps absolutely Required) doomed to fail? While there might be lots of technical reasons, all you really need to know is that one of the core specs for the Semantic Web has the acronym of GRDDL. I'm sure they want me to pronounce that "griddle", but I can't help but pronounce it "girdle". What's even worse is what is stands for: Gleaning Resource Descriptions from Dialects of Languages. Gleaning? Really ... you're going with gleaning? I can understand picking lame-ass words to fit a bitchin' acronym, but you chose gleaning so you could nail down the girdle acronym? I really don't need to know any more about these people.
This has become a useful filtering mechanism, actually. I get contacted by lots of different companies about all sorts of things (note to tech recruiters: I will never ever answer my phone, so stop calling). Now, when I check the company's web page, if the words "Semantic Web" show up I know I can ignore them, as they will be neither competitive nor complementary to anything I'm interested in doing.
One of the key parts of FeedBurner's stats processing is trying to determine if a request for a feed represents a casual, drive-by browse or an intentioned subscriber. It has gotten a little bit more complicated lately as some clients serve double-duty. I wanted to share with you how we handle the requests from the two most popular browsers, Firefox and Internet Explorer 7.
The Firefox browser can also be used as a feed-reading client with the Live Bookmarks feature. So, the key for FeedBurner is to determine if a request for a feed is coming from Live Bookmarks (where we can count it as a subscriber) or from a visitor that just happened to click on the feed chicklet (where we just report it as a browser hit). Up until Firefox version 18.104.22.168, we really have to guess, since the requests for the most part look identical: they both have a User-Agent that looks like Mozilla/.*Firefox/.*. So, what we do is that we look at a couple of other headers: X-Moz and Referer. So the logic tree looks like this:
If version < 22.214.171.124 and (X-Moz: prefetch or Referer is not empty), then it's a browser.
Otherwise, it's a Live Bookmarks request.
That's not ideal, because if someone just types in the feed URL in the location bar or launches the feed URL from a different app, we'll count it as a Live Bookmarks hit because the Referer will be empty. But we have nothing else to hang onto.
Firefox 126.96.36.199 has a wonderful new addition that makes this tracking much more accurate. Now, if the request is coming from Live Bookmarks, there will be an X-Moz: livebookmarks header. We can detect that and we don't have to do the referrer guessing game.
If version >= 188.8.131.52 and X-Moz: livebookmarks, then it's a Live Bookmarks request.
Otherwise, it's a browser request.
Internet Explorer 7
The latest version of Internet Explorer adds feed reading capabilities by leveraging the Windows RSS Platform. So, on the surface, things seem really straight-forward, since the Windows RSS Platform has its own User-Agent that's distinct from the IE7 User-Agent.
If User-Agent matches Windows[- ]RSS[- ]Platform/\S+ .*, then it's a "Windows RSS Platform" subscription.
At this point, however, things get complicated. Outlook 2007 has a cool feed reading capability. Unfortunately, the Microsoft Office team didn't get the memo and identifies itself the same as IE7 instead of leveraging the Windows RSS Platform, which would have made much more sense. So how do we distinguish between IE7 browser hits and Outlook 2007 subscriptions? We use the old referrer trick: if there's no referrer, assume it came from the automated poller fueling Outlook.
If User-Agent matches Mozilla/4\.0 \(compatible; MSIE 7.* and Referer is empty, then it's an Outlook 2007 subscription.
But wait ... there's more! It turns out that some Microsoft Vista Gadgets also identify themselves as IE7, and we think it's more appropriate to treat those requests as subscriptions rather than browser hits. Fortunately, there's a hook: we can look at the Referer, and if it starts with x-gadget:///, then the request is coming from a Gadget.
If User-Agent matches Mozilla/4\.0 \(compatible; MSIE 7.* and Referer starts with x-gadget:///, then it's a Vista Gadget subscription.
Finally, if none of the other rules match, we treat it as an IE7 browser hit.
So, those are the kinds of decisions that we make when evaluating each of the over 300 million feed requests we get each day. We're constantly reviewing the list of User-Agents we get in those requests in an effort to make these stats as accurate they can be. What really makes our lives easier is when we can definitively discern through request headers if the request is for an intentioned subscription vs. "other". With developments like a distinct User-Agent header for the Windows RSS Platform and the new X-Moz header, we're getting closer!
I recently checked my email archive to find out when, exactly, we launched the "pre-alpha" of FeedBurner. Turns out it was February 25, 2004 ... three years ago today. Happy birthday, FeedBurner!
Here was the original email that we sent out to friends and family on that day. I'm amazed with how we've pretty much stayed on target with our original vision.
We are very excited to announce a pre-alpha release of our new service, FeedBurner.com ( http://www.feedburner.com ). FeedBurner is an RSS/Atom post-processing service that enables publishers of syndicated content to enhance their feeds in a variety of interesting and powerful ways. By "pre-alpha", we mean that the software still contains a number of bugs, and while we wouldn't release something that was "unstable", we also wouldn't go throwing around the term "high availability" just yet either.
Our pre-alpha release contains a small subset of the services we will be rolling out over the coming weeks. Look for additional services like authenticated feeds (enabling premium content to be syndicated), "future-proofing" to eliminate the market's current debate over feed formats, and even content namespace enhancements to facilitate the broad syndication and feed-splicing of rich content types (eg, think syndicated music meta-data and associated news and purchase information).
As syndication and non-browser content aggregation/display become rapidly more popular, we believe our "syndication clearinghouse" model will provide a large collection of publishers with an enourmous [sic] amount of leverage in the market. For example, today even the most popular bloggers have little to no control over how frequently their content is polled, what newsreaders do with their content layout/style, or how many advertisements are layered into the display at the fringes of syndication. Nor are they afforded any revenue opportunities for their syndicated content. By channeling large numbers of publishers through our post-processing facility, we can begin to help these publishers effect changes to their benefit in a number of interesting ways ( poorly coded newsreaders that excessively drain bandwidth can be shutout, significant opportunities to provide revenue channels to the individual publisher emerge, and much much more).
If you'd like to stay up to speed on the service and the company, we are going to maintain a weblog at http://www.burningdoor.com/feedburner that tracks the progress of the company and service, and we'll also use this space to discuss publisher services and business issues.
Burning Door Syndication Services, Inc.
First off, let me just say that I think Yahoo! Pipes is very cool and that it has the potential to be an important building block for the next phase of the web (see "A More Personalized Internet?" for an overview). It's the logical next step for this ecosystem that is made possible because of the standard content interchange format called a "feed". Feeds first allowed there to be a loose coupling between content publishers and content consumers and let each evolve separately. Then, FeedBurner came around and showed that this loose coupling also enabled value-add middleware that respected and in some cases even strengthened the "content contract" between the producers and clients. Pipes is a logical next step which does a very cool thing: it allows external parties to construct content workflows and, most importantly, gives them a sharable URL. FeedBurner and Pipes actually complement each other very well, and I've been having a lot of fun over the past week demonstrating that.
There are some very interesting directions that Pipes can take as it evolves, and I'll be curious to see what Yahoo! does with it. One of the first things I wanted to do when I started working with Pipes is that I wanted to be able to construct and share new modules. I hope that it's something they would consider exposing, because man would that be tight! From personal experience, though, I know that it's probably not going to happen -- it's really hard to lock-down any kind of code that would have to execute in your process space, so that's probably out. But maybe if they could just expand the existing "Fetch" module so that one could POST the current state of the stream to an external URL that I host on a server somewhere and I could return the transformed content, and you could wrap that up in a sub-pipe that expects additional user inputs as the config parameters ... something like that could work.
Which brings me to the meat of this post: wonderful things could happen if you marry Pipes to the Atom Publishing Protocol (APP). What if the pipe output, rather than just being XML that spills on the floor when the URL is requested, could instead be hooked up to a module that speaks APP? Now you've got a really cool content routing mechanism. The "Fetch" module already really handles the input end of things, but being able to channel the output to a different destination could open up some amazing possibilities.
One detail to be worked out is the triggering mechanism for the workflow. Currently, a request to the resultant URL serves as the triggering mechanism that the workflow should executed. This is how FeedBurner works as well -- there's no master cronjob that ticking away and retrieving all the source feeds every 30 minutes. Instead, when a request for the burned version of the feed comes and the source feed is stale (i.e., hasn't been checked in the last 30 minutes), then go refresh the source feed. That way, you don't waste cycles updating dormant feeds. Pipes works the same way.
So, if there isn't a request URL, how would you "run" the workflow? Probably the most appropriate thing to do would be to use something like a ping mechanism, so that if the pipe is pinged and the content has been modified since the previous run, you run the pipe. That could probably work.
In the end, if you take the promise of Pipes, the potential of Google Base, and add some of the stuff that you'll see from FeedBurner in the next few months, you'll have some wicked tools to start rewiring the next version of the web. I think it's going to be quite a trip.
Last Thursday we had another hackathon at FeedBurner -- our fourth -- which is always an event to look forward to. This one was a lot of fun and there are some really cool hacks that are live right now. My hack was an "Event Feed" service, and I thought I'd write up a bit more information about how it works and how I hope publishers might use it.
First of all, like most good ideas, the idea for this one came from one of our publishers. This suggestion came from Jon Morris (Matis). He keeps his upcoming performances in upcoming.org and wanted a way to display the events on different web sites.
At first, it seems like this should be easy. Upcoming.org has feeds, and FeedBurner has BuzzBoost (not to mention several other widgety tools). Should be able to just burn the feed and choose BuzzBoost, right? Well, not quite. For some arcane reasons, the dates associated with the items are not the event dates, but instead represent when the event was added to upcoming.org. Turns out, that's not terribly useful for most feed consumers. Fortunately, they include other elements in the feed using the xCal namespace that includes information like the event date and location.
Enter the "Event Feed" service at FeedBurner. When you activate this service on the "Optimize" tab at FeedBurner, you're saying that the source feed represents a series of events and that you want them treated that way. So, what FeedBurner does for each item is:
- Find the event date and make that the item date
- Put the items in ascending date order (optionally only including future events), and
- Optionally prepend the event date in the item title
So far, so good. But I don't use upcoming.org ... I happen to use Google Calendar. And what good is a hack if it's not useful to the author?!? So, the Event Feed service will also work with public and private Google Calendars. You just have to find the RSS feed for the particular calendar you want, which can be a bit of a challenge sometimes. Hint: it's somewhere on the Calendar Settings page.
This will work for both shared and private calendars ... let's see how it works. Here's a public calendar: Chicago Bulls schedule. If you look at the feed, you can see it's not terribly helpful: events are totally unordered and it includes past games as well as future games. So, let's burn it and add the Event Feed service. Ah, much better. Here's a nice little headline animator for it as well:
Finally, after staring at those nice upcoming.org feeds complete with event location information, I couldn't resist tossing a FeedFlare into the mix. This was literally cobbled together in 10 minutes right before the end of the day -- I love how easy it is to create these things. Anyway, here it is: Map Event FeedFlare. Take a look: it's just examining the item and looking for the lat and long elements and the venue name, then composing a URL to Google Maps with that information in it. This is actually a good example of how you can use XPath to pull arbitrary information from other namespaces out of a feed item and use them in your FeedFlare. Here's an example of a feed that's using that FeedFlare: source feed, burned feed. And here's what it looks like in as a SpringWidget:
I hope you enjoyed this behind the scenes look at this hack. I had a great time building it and I can't wait to see what people do with it!
Looks like my new T60 widescreen will be waiting for me when I get home. I'm shivering with anticipation From Kowloon Bay to Arlington Heights in less than a week. That's cool.
We introduced a really powerful new capability at FeedBurner last week: custom Headline Animators. We're already seeing a ton of cool skins that people have created, and I thought I'd have a little mash-up fun with it ... introducing, the "Dream Date Dock Worker's Word of the Day" widget, brought to you by the OnDemand Action Figure Builder Tool, TheFreeDictionary.com, and FeedBurner.
In case you didn't notice, we've been growing our core catalog of FeedFlare at FeedBurner. We now have three very cool dynamic FeedFlare units that leverage other web services to help bring interactivity to your feeds and sites.
- Technorati Cosmos Links. Example: Technorati: 8 links to this item
- Scape This. Example: Vote at Netscape. 231 votes, 548 comments
- Digg This! Example: Digg This! (87 Diggs, 55 comments)
You can add one or all of these FeedFlare to your feed or site -- and remember that the information displayed is always current, without marking old items in your feed as modified. Just go to the "Optimize" tab of your feed and select "FeedFlare". Or, you can choose other FeedFlare from the growing, chaotic Flare Catalog ... or build your own!
The RSS Advisory Board approved a specification revision ... ahhh! What does it mean? How do I stay up-to-date??? Here you go, this is all you have to do. If you have a <docs> element at the channel level, just change it from this:
All set! As you can see, it's not a big deal.
"The migration to the service must be seamless for my current subscribers"
Answer: Yes ... see "Can I keep my original feed URL but still use FeedBurner?" and "Pointing an existing feed URL at your new FeedBurner URL".
"The service should provide a suite of metrics that allow me better understand who, how often, when, how long, how many times (You know what I mean, the numbers)."
Answer: That's the foundation of FeedBurner. I think you'll be pleased with the stats available to you.
"In the case that I choose to leave your service, I must be able to migrate my feeds transparently and also be able to migrate my usage data."
Answer: To leave, see "Ciao, FeedBurner". To export your usage data, see "Awareness API and Subscriber Numbers".
"The service must have tight wordpress integration."
Answer: Please see Steve Smith's excellent WordPress FeedBurner Plugin.
"Integration with analytical packages say like Google analytics"
Answer: We don't have an integration with Google analytics, but how about Mint? And wait until you see what we have cooked up after we ingest Blogbeat!
"feeds must be human readable"
Answer: Again, see our Browser Friendly service.
"I would like it to support leave comment from the feed."
Answer: Use our "Add a comment" FeedFlare ... it's somewhere on this list of "101 FeedFlares for a Better Tomorrow".
"Configurable support for social network services like Digg or del.icio.us (Sometimes I want it on, sometimes off)"
Answer: FeedFlare to the rescue. You choose which services you'd like to add to your items in your feed.
"Support for the widest set of RSS extensions"
Answer: Yep. We support them. And all feed formats.
"A strong record of availability, reliability and scalability"
Answer: This is the most important quality of whatever you choose for feed management, and it's our number one priority. I think our availability track record shows that. We also have a number of contingency plans in place that we hope we never have to enact, like our "Feed Insurance" capability that all of our publishers get for free.
"Should support multiple feeds per site, no limit."
Answer: We support multiple feeds per site, no limit.
"Integration to see where my feed is bookmarked and subscribed"
Answer: Part of of our stats package includes the ability to see where the items in your feed have been read and possibly resyndicated. See "And There Was Much Rejoicing: New FeedBurner Statistics" for more info.
"Advertising, I prefer none."
Answer: No advertising, unless you want it and have a certain threshold of subscribers. See "Does FeedBurner put ads in my feed without my consent?"
I hope that gives you enough confidence to give FeedBurner a try, Tom!
I wanted to clear a couple of things up about the Redirecting Guy post. First, FeedBurner feeds never "expire": once you have a FeedBurner URL, it's yours until you explicitly delete it. Second, if you do decide to delete your FeedBurner feed, you can select the "30-day Redirect" option, which goes through the following steps:
Day 1-10: Any requests for the FeedBurner feed are sent an HTTP 301 "Permanent Redirect" response back to your source feed. This will cause most feed readers to forget the FeedBurner URL and use the new URL from that point on. Your subscribers don't feel a thing.
Day 11-20: If your FeedBurner feed is still getting requests at this point, it probably means that your feed reader is treating that "Permanent Redirect" as a "Temporary Redirect". That's actually pretty common, so now we enter "Phase 2". Now, any requests for your FeedBurner feed will receive a "redirect document". What is a redirect document? Dave Winer displayed foresight by anticipating this need back in 2002 and provided this specification so that a publisher could keep control of their feed location. We strongly encourage more feed readers to support this specification, and we are going to be widely campaigning for this capability.
Day 21-30: You're still here? Well, at this point we return a valid feed that contains a single item that says "This feed has moved to (feed URL here)". So even though all of the transparent mechanisms to redirect the subscription have failed, there's still a trail for your subscribers to follow.
After Day 30, your feed is deleted and that URL is up for grabs. If you would like to extend this time period beyond 30 days, however, just drop us a note to feedback [at] feedburner.com and we'd be happy to extend this redirect period indefinitely. We've done that for lots of publishers.
Mike's point is valid: as a publisher, you should be aware that if you delete your FeedBurner feed you are also releasing your claim to that particular URL, just like what happens if you give up a domain name or a delete a blog on a hosted service. We have provided the tools (through redirection, MyBrand, and the 30-day Delete process) to ensure that publishers maintain complete control over their feed, but we always welcome feedback and suggestions for improvement!
Bloglines recently announced much improved support for Atom 1.0 feeds. Therefore, at FeedBurner, we flipped the bit on the SmartFeed service so that Atom 1.0 feeds will be delivered as-is to Bloglines instead of being down-converted to Atom 0.3.
Very soon, we're going to be switching SmartFeed from the current "assume a reader doesn't support Atom 1.0 unless we have recorded otherwise" logic to the "assume a reader does support Atom 1.0 unless we have recorded otherwise". That might cause a few problems with some readers that we haven't identified, but we should be able to chase those down quickly.
Below the fold is a simple walkthrough for how this tool works. I hope it helps some enterprising FeedFlare developers out there!
Michael and the Six Apart team did an awesome job with this integration with FeedBurner. Really nice work!
There's been a lot of chatter lately about FeedBurner, which is fantastic. There have also been some misunderstandings about how a publisher can keep control over their feed. I think we at FeedBurner can be a lot clearer about this, but until that happens I would like to point people to an older post I made called "FeedBurner on Your Terms". Basically, it details four different ways to hold onto your feed:
- Use an HTTP temporary redirect
- Use a PURL
- Use partner domain mapping (now called "MyBrand")
- Embrace the burn
Since that post, we have also added our Feed Insurance capability, which has been chugging along since we announced it, ready to spring into action if it's ever needed. Hopefully, it will never be needed.
FeedBlitz recently announced a very clever service called the FeedBlitz AutoResponder. The name doesn't quite describe what it does, imho ... it's really a cool way to offer a serial version of content to interested subscribers that is delivered over the course of several days or weeks. You basically set up a blog somewhere with the dates of the posts spaced out how you want them. Then, when someone subscribes, FeedBlitz delivers the posts via email on that schedule. Smart and elegant. At FeedBurner, we did something kind of similar with Tom Evslin to deliver his hackoff.com novel via a feed, but that was a bit hacky and hasn't been generalized yet. Nice job, Phil.
There is a very useful RSS extension that I've been using on this site for a long time that allows a "feed of comments" to be associated with each entry in a feed. I've been calling these "microfeeds", and the extension that allows this to happen is called the Well-Formed Web namespace elements. There are all sorts of cool things you can do with this element, including using FeedBurner's Comment Count FeedFlare, and this element has made it into the default WordPress templates.
So, what's the problem? The person who maintains the landing page for the namespace URI at http://wellformedweb.org/CommentAPI/ changed the element name from CommentRSS to CommentRss without really telling anyone ... and didn't change the date of the page from 2003-10-10! It's like the history eraser button. This is lame. Here's the reasoning behind the change from Joe Gregorio on the feedvalidator mailing list:
I changed the Well-Formed Web site after it was pointedAck! Every time I've run into this element it has always been capital RSS ... because that's what the namespace's URI landing page said. Sure, that's not binding, but now this is really becoming a problem for publishers that want to validate their feeds ... and now all downstream consumers of feeds are realistically going to have to look for both variations. We've had to change our FeedFlare to look for both variations, for example.
out to me that it disagreed with Chris Sell's Specification,
after all he is the author of the spec. I would have made a
bigger deal of it had I realized how many implementations
of commentRss there were.
So, for those following along at home, that is *not* the way to care for a namespace.
Today, TypePad introduced TypePad Widgets, which is a elegant handshake framework for being able to add things to the sidebar of your TypePad blog. You can test it out at FeedBurner by going to the "Chicklet Chooser" service on the "Publicize" tab of your feed ... make your selection and scroll to the bottom. We're planning on adding this functionality to every place in FeedBurner that generates an HTML snippet.
I really like the simplicity of this framework, and I hope it's the first of many steps from TypePad to open things up! I'm not sure why they have the API behind an application form, but it's probably just until all the kinks get worked out.
Today we announced the second phase of FeedFlare, which allows you to put the FeedFlare back on your site. To see an example, I've added it to this blog ... just look right below this item. The "Add to del.icio.us" and "View CC license" are generated by FeedFlare. They showed up because I put the following code (as directed by our help guides) in my Movable Type template:
There's one other very powerful thing about FeedFlare on the site: we are using all of the metadata and expressive power of the feed to inform the FeedFlare on the site. Since FeedBurner is processing your feed, all of the structured information is available to help generate the FeedFlare on the site. No having to scrape an HTML page like other scripts are forced to do -- we can use the upstream information that simply isn't available in the final, rendered page.
I'm really excited about this new development with FeedFlare, but I'm really really excited about what's coming next. The API is going to be a lot of fun, and I can't wait to see the kinds of FeedFlares that will be created. So, back to work for me!
I sure miss Mark's blog, but his mailing list posts almost make up for it!
Source: Sam Ruby: Photocasting Hyperbole
Even though the title of the post is "FeedBurner's Dick Costolo on RSS Ads", this is a good Q&A about more than ads. Nice job, Dick!
Apple, what are you thinking? First off, the good: movie trailers are a great idea for the new iPods. They even have a nice page that lists a bunch of trailers: cool.
Check it out -- an RSS feed for this page. Alrighty, they must be podcasting the trailers so you can subscribe to it right in iTunes, complete with the iTunes namespace extensions. Great idea, Apple.
Then I looked at the feed:
<title>Apple Movie Trailers</title>
<description>Popular movie trailers presented in QuickTime.</description>
<copyright>Copyright 2002, apple.com.</copyright>
<title>Harry Potter and the Goblet of Fire (Trailer 2)</title>
RSS 0.91? No enclosures? No itunes elements? Everything linking to the same URL? There's not even enough information to run this thing through SmartCast.
Apple movie trailer page people, if you'd like to enable this feed for podcasting, please let me know and I'd be happy to help. As it is now, it's ... well, a little shameful.
I understand that geeknews doesn't like FeedBurner, and I can respect that. But I want to make sure that people's opinions of FeedBurner are informed by accurate information, so I just wanted to respond to a few of the comments on the post "Why would 100,000 people trust their RSS feed to FeedBurner".
"It is well known that an update on a site does not necessarily mean that feedburner is going to know about it immediately."
That's true unless you ping us. We will poll your feed every 30 minutes, so it's possible that your burned feed could be up to 30 minutes stale. To update your burned feed instantly, you have a number of options:
- Configure your blogging software to ping FeedBurner at http://ping.feedburner.com. Here are instructions to ping FeedBurner in Movable Type, but other blogging platforms have a similar capability.
- Use our FeedBurner ping form to let us know. You can also bookmark the result page to ping us by visiting the bookmark.
- You can also use Ping-o-Matic with FeedBurner.
"...you have no idea if they go offline and they don't graph any outages"
We actually haven't had any downtime since June 25 when our colocation center got hit with a power outage. We pride ourselves on our availability, and we're always seeking to eliminate points of failure like this.
We will, however, notify you if there's ever an outage with your feed. Every publisher gets their own personal feed (called "FeedBulletin") where we will proactively notify publishers if their source feed is ever invalid or unreachable. You can visit your "My Account" page on FeedBurner for more information on this cool feature.
"...the statistics are never truly accurate."
Much like web stats, feed stats are an inexact science. We think we have the best understanding of all the different aggregators and their unique polling characteristics, but we're constantly learning and improving our algorithms.
One place we definitely want to improve is our podcast statistics. Currently, we don't track downloads to the enclosures themselves. That's something we're working on.
"I tired it and never got accurate Stats on podcasts listened to, not only that your statistics are public. I got nothing to hide but My stats are not everyone elses bussiness"
Point taken on the podcast stats (see above). But your stats are most definitely not public unless you as a publisher choose to make them public. By default, your stats are private. If you choose to enable the Awareness API, then your stats are made available to feed and podcasting directory services, which has been a boon to many podcasters.
"...but the question remains why would 100,000 people trust a third party to update and maintain the lifeblood of the one single document that connects most sites to their readership."
There are a number of ways that the publisher can keep complete control over their feed URL, so if we ever disappoint we can be cut out of the loop instantly. That just means we have to work hard to keep publishers very, very happy.
That's all for now. We're always open to suggestions for how we can improve our service. I'm going to put the cape away and return to fight FUD another day.
I'll write more about this in a later post, but if you're a FeedBurner user there's one thing you have to go do right now. Go subscribe to your personal feed. Go to your account page on FeedBurner, find the URL of your personal feed, and add it to your reader. You won't regret it. It will tell you if your feed is ever invalid or unreachable by FeedBurner.
It our way of communicating with you without you having to give us your email address. You control the conversation. But we have some good things to say, so please subscribe!
I had the opportunity to talk with Brian Livingston recently about RSS, and he just wrote up his article RSS Readers: Narrowing Down Your Choices. Brian was a very interesting guy to talk with and he wrote one of my all-time favorite computer books, Windows 95 Secrets.
I was looking through some of the FeedBurner stats the other day and noticed a cool feed that's quite timely: the USGS M>2.5 Earthquakes feed. Those of you in California might want add that feed to your reader!
Reminds me of one of the popular Spyonit spies we had in the summary of 2001: the CA Rolling Blackout Spy. You left-coasters have all the fun.
First thing you might be thinking is "Why is FeedBurner rewriting links?" Well, one of the service that we offer to publishers is "clickthrough tracking", where we report the number of clickthroughs on a per-item basis. To do this, we rewrite the URL associated with the item and when it's clicked on, we log the click and then forward it on to the original item URL.
The second thing you need to know is that we never rewrite the <guid> element in an RSS 2.0 feed, although we may switch the isPermaLink attribute from "true" to "false" in some cases. The guid element is used in many cases to unique identify a feed element, and we don't want to mess with that.
Last thing to know is that if we ever rewrite the link to an item, we'll stick the original value of the link in a <feedburner:origLink> element, which is useful for some search engines and other consumers of the feed to reconcile items that may be syndicated into multiple endpoints.
If you want more details on exactly how the rewriting works, it's in the extended entry.
I'll be at the Syndicate conference in NY next Tuesday and Wednesday. Hope to see you there!
Dare has a great post that talks about all the great things that can happen when sites agree upon a standard content interchange format: RSS. A nice companion post to Jon Udell's from about a month ago, previously mentioned.
I still think there's a pretty decent chance that Atom will supplant RSS in 12-18 months as the interchange format of choice, simply because the standard nouns (the Atom Format) are coupled with standard verbs (the Atom API), but you never can tell in this business!
I tend to listen very carefully when Adam Bosworth speaks (like those E.F. Hutton ads for you old timers). He has the unique combination of a wide perspective and a deep understanding of what it takes to realize a vision.
Looks like My Web is Yahoo!'s answer to del.icio.us. Gather links and search results, publish your My Web links via RSS, and use the API for My Web. It's a beta and it shows ... del.icio.us is still miles ahead.
But here's some specific feedback on the RSS feed that is available for the shared folder.
I think it's funny when people say that feed formats are really irrelevant because all of the readers support all of the formats anyway. While that happens to be mostly true right now, the new, non-backwards compatible Atom Format spec is going to be released one of these days, and we'll be right back to the chicken-and-egg problem of publishers not wanting to convert their feed to the new version until it's widely supported in the readers, and the feed readers having to prioritize adding support for the new format and managing a release.
I think SmartFeed is going to be a big help in helping publishers manage that transition. When the spec is released, we'll make sure that SmartFeed "down converts" the new spec to the existing 0.3 spec for readers that haven't been upgraded yet.
What's in store for Format Wars, Episode 4 you think? I'm thinking it's going to be in reconciling different namespace extensions for rich media types. Different music namespaces, photo namespaces, video namespaces ... and how about enclosures vs. Y! RSS Media extension for the next generation of podcasting? All coming soon.
We've been patiently waiting for the Google AdSense in RSS news to leak out, and it looks like it has. Google's approach is straight-forward: you edit your source feed template and modify it so that you include a linked img, the src of which includes the URL of the permalink associated with that item. That piece of markup gets generated once, and the img that returns can change over time. That prevents the feed from being marked as modified if they want to change the ad that gets slotted for an item.
FeedBurner has had a framework like this in place for quite a while, and we're excited to be able to add support for this program, which we detail in the post Google AdSense Support. Our offering will open up this program to user's that can't edit their feed template, but we can add a bunch of value to all publishers. Using FeedBurner to add your AdSense ads will give you a number of additional configuration options, like:
- only including ads in posts of a certain length (at least 100 words, for example)
- including in alternating posts instead of every post (if you're worried about advertising overload!)
- choosing which categories/tags to add ads too (include ads in my "venture capital" posts, but not my "family" posts)
The most exciting thing we can do, though, is integrate your ad performance statistics with your existing FeedBurner stats. Wait until you see our new Total Stats PRO paid offering -- you're gonna like what you see. Ad performance on a per item basis anyone?
Anyway, as John Motson likes to say, "things are hotting up now!" It's so much fun to be in this space right now.
Jon, as usual, gets it. Increasingly, companies will have to defend their choice to not represent their content as RSS as it becomes the standard content interchange format. OpenSearch, as mentioned before, is an excellent example.
So, this "item" shows up in my Feedster result feed for FeedBurner:
That seems pretty slimy to me. I have no issue with these SimpleFeed people buying the "Feedburner" adword ... in fact, that's kind of flattering. But to have "Feedburner" displayed within the item itself is misleading, and I hope people don't get the impression that we have anything to do with them.
Amazon introduced OpenSearch at ETech yesterday. Awesome. It just goes to show you that RSS is becoming the standard transport for content, and this is a great example of how RSS is getting more and more valuable as it gets more and more invisible.
I wanted to dig into OpenSearch a little bit, so that's just what I did. If you have a blog run by Movable Type, here's what you have to do to have your own A9 column. Just follow these steps.
- You have to create an alternate search results template. This is the one part of MT that isn't exposed via their UI. Just download this file and put it in your search_templates directory with the name opensearch.tmpl. (You can do this by going to your MT install, cd search_templates, then wget http://www.burningdoor.com/eric/files/opensearch.tmpl)
- You have to add one line to your mt.cfg file:
AltTemplate opensearch opensearch.tmpl
- Enough of the command line. Now you can use the UI to create a new index template. Call it "OpenSearch Description", make the output file be opensearch.xml, and paste the contents of this file into the text box. You might want to tweak a few of the tags to be appropriate for your blog. Make sure you build the template.
- You're done! Now go to Create New A9.com Column and enter the URL of the opensearch.xml file you created. Share and trade with your friends.
If you want to see an example for this blog, you can see my opensearch.xml file, and a sample search results page.
I'm not sure how useful this would really be for us small publishers, but I imagine this could be nice for some larger publishers out there. FeedBurner doesn't quite work with this yet because we can't pass through parameters to the source feed, but that would be a nice service we could offer (stats on search queries maybe?).
Nice job, Amazon.com!
It has been a while since my last post on microfeeds, and not a lot has really changed -- except that they would be more useful than ever if we could figure out a way to consume them appropriately.
I continue to offer microfeeds attached to each post that contains any comments for that post, but I'm not aware of any newsreader that recognizes the wfw:commentRSS element (each item in my feed uses that element to point to its microfeed). We've also seen interesting uses of RSS for things like FedEx package tracking, and there's an interesting proposal for using feeds to help deal with phishing scams. I'd love to have microfeeds for order tracking from a bunch of different online retailers.
The problem is, of course, that the traditional feed reading clients don't really deal with microfeeds that well. Microfeeds might have a lifecycle associated with them ("Kill thyself after the package has been delivered"), they might not deserve the status of "channel" (toss a bunch of them into a single "order confirmations" channel), or they might just serve to support existing content that I've found interesting (like comments on an entry). "Feeds" are too bulky a concept in today's readers to handle these scenarios.
I don't have an answer, but I feel the need!
[F]or some, the feed is becoming a first-class citizen in the world of content, and FeedBurner wants to help. Feeds pretty much started out being a lightweight content transport, or a notification mechanism. This remains a completely valid and appropriate use of feeds: as an extension of your web site that draws users to visit your web-based content. But as publishing tools begin to specialize ("I keep my photos on Flickr, my blog's with Blogger, I use del.icio.us to store my links, and then I've got my calendar, NetFlix queue, Amazon wish list, and so on and so forth"), it starts to get harder and harder to point to a single URL and say "This is me." The one thing that all of these content sources have in common, however, is the ability to provide a standardized representation of themselves as a feed. A FeedBurner goal is to help you collect and manage the "scattered pieces of you" into a single resource that can easily be shared with others. The feed starts to become much greater than the sum of its parts.
Now, that talks about a feed being a distinct entity composed of a number of point sources, but it's not hard to imagine directing this flow of information at some endpoint that, itself, could be some kind of CMS. There are a number of very interesting implications of being able to decouple the points of content creation with its management and hosting, but that's for another post.
Erik Benson, of All Consuming and 43 Things fame, recently did some cool personal content consolidation using Bloglines as the integration conduit. Check out his posts "Outsourcing the blog" and "Using Bloglines to manage my online presence".
This meme has been floating around for a while (see also "more thoughts on similarities between software packages" and "Interweb connectivity through ping-/trackback"), but Erik's duct tape implementation is the first real implementation I've seen outside of MT plug-ins. Nice job!
If you read this via my feed (the only sane way to keep track of things, imho), let me apologize in advance. I'm going to be messing with it a little bit, so some old items might show up as being modified. Sorry about that.
Dear Comedy Central,
I have an idea for you. Something that would expand your reach, raise your audience in a desirable demographic, and, well, let's be honest: really improve my life. Here's the idea: make every evening's The Daily Show available online, distributed via a "video podcast". If that doesn't make sense, maybe some explanation is in order.
The latest draft version (05) of the Atom format specification is out, and it has really shown an amazing amount of improvement since the 03 version. The document itself is very well organized, with the welcome addition of Relax NG fragments throughout the document. Makes it much more readable.
We're staying on top of this at FeedBurner, so hopefully we can support the revised spec right after it is approved.
Source: The Atom Syndication Format
The past couple of weeks have been pretty eventful in the world of content syndication. We've seen a new feed format proposal from a group of frustrated users called RSS 1.1, and we have a new round of edits on the latest Atom format proposal. All of these efforts are well-intentioned, and even attractive from an engineering perspective, but it goes to show you that this space just isn't getting any simpler.
And that's just the feed formats. What's really going to be interesting are the namespace extensions. You think Microsoft will adopt the Apple ITMS namespace (hmmm ... can't get to this right now -- did Apple take this down?) in its feeds? Or Amazon.com will start using Six Apart's book namespace? Of course, a major part of FeedBurner's mandate is to sit in the middle and help negotiate these formats and namespaces between the publishers and clients, but it's going to be fascinating to see how clients will grow and adapt to take advantage of the additional metadata that will start flowing through these feeds.
The other interesting "debate" that has flared up again recently is the "auto-subscription" issue: if you click on a little XML chicklet, what should happen? How does that end up communicating to your aggregator-of-choice that you'd like to subscribe to that feed? People have generally fallen into two camps: the "feed:" group ("Register a protocol handler on your machine!") and the MIME-type group ("Just send down the right MIME type and configure your browser to use the right application!"). The MIME-type approach has found a recent champion in Randy Charles Morin, with his Universal Subscription Mechanism proposal. Both solutions have interesting technical challenges, yet neither approach helps a brand new user that doesn't yet have a client ... nor is it possible to really handle multiple clients ("I want most feeds to go to NewsGator, but my podcasts I want to go to iPodder ... oh, and Azureus should handle my television feeds"). That's fine: we have to start somewhere. I like our Browser-Friendly approach (look at my feed in a browser, for example), but I think we could do a better job helping people to subscribe to the feed by offering both feed:// and a USM approaches, as well as a wider varienty of online aggregators, so we can leverage the work that's going on to solve this problem. I think we'll work on that as soon as we get the time.
All of this is to say that the world of syndication is very active and exciting right now. Another indicator of the activity is the number of clients out there. As Dick mentioned in the recent Burning Questions posting entitled "RSS Market Share":
RSS Client market is not yet consolidating, it's expanding. There were 409 different clients polling the top 800 FeedBurner feeds in September and now there are 719 different clients. FeedBurner actively catalogs the behavior and specifications for hundreds of these user-agents.That's awesome. I just hope that the confusion surrounding the different formats doesn't deter the after-hours hacker from creating the next great syndication client.
There are a number of blog search engines out there, and most of them allow you to create a "watch list", which is an RSS feed specifically for your search terms. This is very useful for things like egosurfing and competitive intelligence.
I've started noticing that the results feed of one search service are starting to show up as a source in another: so I'm beginning to see PubSub results in my Blogdigger feed, and Technorati results in my Feedster feed. This might be a tough problem to solve, as some of the services actually transform the feed as it flows through their system, so a search engine can't do a simple item compare to detect duplicates.
Ah, one of the interesting challenges of a loosely coupled environment.
I was going to post a HOWTO on how to use the power of RSS and BitTorrent to be able to download television shows, but Phillip Torrone at Engadget did a better job than I ever could have, so check his guide out.
Now, I'm a huge Tivo fan with a number of hacked DirecTV-Tivo units throughout the house. I've even got a Series 1 unit that's nicely set up for video extraction, but the BitTorrent solution is really superior imho.
I have a 35-minute commute on the train two times a day ... perfect for catching up on some shows that I watch but Christine does not watch (The West Wing, South Park, and pretty much the entire Sunday night lineup on Fox). Azureus with the RSS plug-in dumps them straight to the laptop. Even better, most of the shows are captured in HD. Sweet!
This even came in handy last Thursday night when there was a DirecTV local channels outage across the country. Screwed up Survivor, CSI, and The Apprentice, which are three shows we watch. Bummer. But, a visit to "that tv torrents" site the next day, a few downloads, and a DVD burn, and we were set to watch them by Friday night. That sure was better than having to read about the episodes.
Over at FeedBurner we've started to offer publishers the ability to include links to Amazon.com items and (on a limited trial basis) advertisements in their feed. While this is sure to be a contentious issue, I personally think it's great to give publishers the option if they are looking for ways to monetize their hard work and effort. But that's neither here nor there -- I want to talk about some of the technical issues we've worked through to associate dynamic content within the unique constraints of feeds.
The number one big concern when you're dealing with feeds is: don't modify old items! There are few things more annoying than having an old entry show up as modified in your newsreader only to discover that the reason it's modified is that now there's an ad in the item. Lame. That said, we still give publishers the option to put ads in old items, but they are strongly discouraged from doing so:
Hmm ... maybe that should be stronger. Let me go talk to creative.
There's of course a bunch more stuff that goes into managing all of this, but I just wanted to let everyone know that we're really trying to respect the nature of this wonderful medium of feeds and not screw things up. There are going to be the people out there that cry "you've ruined feeds", but I really view this as filling a need that publishers have. It's been said before but I feel I need to reiterate: FeedBurner NEVER puts anything in the feed that the publisher doesn't want. If you see an ad in a feed, that's at the request of the publisher. We're going to be doing quite a bit of tweaking and tuning over the next few weeks before we open up some of the capabilities to a wider audience, but we all value your feedback.
What's the true promise of RSS? That it fades into the background, becoming as ubiquitous and invisible as HTTP. Producers, consumers, intermediaries ... all speaking this language of streaming information, shuttling all kinds of data.
Here's a great example: Halo 2 uses RSS to send stats about the online games's you've played. Awesome.
There's a new draft of the Atom Syndication Format specification draft-ietf-atompub-format-03 that incorporates a lot of the suggested changes that have been flying around on the atom-syntax mailing list the past few months.
On the whole, I like it. We probably won't put support for the revised format at FeedBurner until it goes beyond draft status, but we'll see. I especially like the Text construct, as well as the clarification of the dates on entries.
In a previous post, I talked about how FeedBurner will "propagate, respect, and enforce" Creative Commons licenses in feeds. Here, I'd like to present the "Law of License Propagation", which details how licenses should be handled when multiple feeds are merged or spliced together.
Over at FeedBurner we recently announced a service that lets you, the publisher of a feed, associate a Creative Commons license with your feed. Is this a big deal? It’s a HUGE deal! I'd like to take a look at how FeedBurner might propagate, respect, and enforce these licenses both now and in the future. In a subsequent post, I’d like to offer up a "Law of License Propagation" as it applies to syndication feeds.
As far as RSS readers go, I'd have to say that Bloglines is number one out there. Personally, I use both Bloglines and FeedDemon. FeedDemon is more polished, but I really like the "access from anywhere" nature of a server-side application like Bloglines: no matter where I reading my feeds, Bloglines always remembers my subscriptions and what I've seen.
But what might the future hold for Bloglines? Well, if the past is any indication, Mark Fletcher, the talented developer behind Bloglines, might be aiming for an acquisition ... and let's face it: as syndication takes off, the big players are going to want to add this capability to their list of services. And time is of the essence -- Bloglines is the big fish in the server-side pond right now, but word on the street is that Rojo (why-o-why does that have to rhyme with "mojo"?!?) could put some serious pressure on that dominance.
Okay, so who would acquire Bloglines? You might think that Mark would go back to the well and talk to Yahoo! (who acquired eGroups in 2000), but hmm, they just bought Oddpost. So, they've probably got something brewing there in the RSS space.
Google? Nah, not with the team that did Gmail on staff. They could whip up a kick-ass DHTML aggregator all by themselves, and if they're not, shame on them. Of course, they did acquire Neotonic in 2003, some old eGroups buddies, but I just think they'll do something themselves, though.
MSN? AOL? Still too early for them. In fact, I think the whole syndication space will probably have to go through another iteration (pushing the feeds down to the protocol level) before this becomes appropriate for the masses.
So who? I'll tell you who: Ask Jeeves. They're still hanging in there, but jumping into this space could be the shot in the arm the old butler needs. They've got some good technology (they're behind more sites than you might think), some decent revenues, but they need some buzz. Bloglines could be just the thing for them.
Of course, Mark might just want to go it alone, to which I say: good luck, and thanks for the great app!
A good article on Salon that talks about using RSS as a mechanism to stream BitTorrent feeds. We have talked about this before (here, here, and here), but it's nice to see the community growing before our eyes.
What I really like about this is that it's using each technology appropriately. Use RSS as the "notification" mechanism, with enough meta-data around the content to be able to discriminate and select items that are desired. Use BitTorrent as the large file distribution system. Create a custom client (in this case, Buttress) that uses both of these technologies, but pushes them down to the transport layer. Very nice.
In the effort to deal with spam, almost everybody has some kind of spam filter either built-in to their email client or provided by a service. I know, personally, that once I've trained my spam engine, I never look in the "Spam" folder and only give a cursory glance to the 20 or so emails that end up in my "Maybe" folder every day to see if any emails were misclassified as spam.
How sure are you that your emails are getting through to your intended recipient? Answer: you can't be sure. Send an e-greeting card to a friend? Might have been shuttled to their spam folder. What about sending a quick link to some colleagues? If you don't provide some supporting text, there's a good chance your co-workers will never see it.
It's an even bigger problem for permission marketers who depend on email to maintain a non-spam relationship with their customers. Entire companies like Return Path have sprung up to ensure that "the email must go through".
Mr. Permission Marketing himself has found himself in this trap and sees RSS as the way out. All I can say is: yes, Seth ... hold on, we're working as fast as we can!
Source: Seth's Blog: Seth Godin: Spammer
I like the recently introduced new look and feel for Bloglines, but it seems like it's been acting a little funky the past day or so. It's not remembering items I've read, and sometimes an entire feed will just mark itself as unread. Is this happening to anyone else?
Between this and the "Brad Bug" on My Yahoo!, it's been a rough week for online aggregators. Or maybe it's just my account.
Do you like my new sig file?
You can have one for your blog, too. We haven't announced it yet, but this capability exists on FeedBurner as of tonight. You, loyal blog reader, get a little sneak peek.
Yes, the syndication community is abuzz with the announcement that the next version of Apple's browser, Safari, will have RSS (and Atom) support built-in. There's a little Flash demo of how the feature works.
While things like the "article length" slider and the local search across feeds demo well, I don't think they'll be terribly useful. What is very useful, however, is the "RSS" icon that pops up in the location bar when the browser detects that a page has a feed associated with it. I hope that particular feature "plays well with others" like NetNewsWire, so you can enable other handlers for the feeds that have been detected.
Even though Apple has a relatively small user base, this has the potential to really elevate the awareness of syndication to the masses. Plus, now we get to see what IE will look like in 18-24 months.
RSS + BitTorrent + MythTV = Torrentocracy. It's a way to share rich media (that's the BitTorrent part) using a feed as the transport mechanism (that's the RSS part), fully integrated into your PVR (that's the MythTV part). It's all coming together now.
Source: torrentocracy - home
Just when you thought that the RSS 2.0 spec was a dead document, we get this nice little present late on a Friday: a clarification for encoding HTML in the <description> tag in an item. While it used to be ambiguous, the proposed clarification now states that the <description> tag will always contain entity-encoded HTML.
This one little change will help out a lot. Thanks to the group of people that contributed to this clarification.
(Yes, it was done behind closed doors, renders some number of previously valid feeds "invalid", and opens the door to other changes to a supposedly frozen spec, but I still welcome the change.)
As a follow-up to my earlier posting "Using P2P for RSS Distribution", there is now a plug-in for Azureus (the BitTorrent client of choice) that watches an RSS feed and slurps up any torrent files that are mentioned, subject to filters that you establish. I can't wait to see how people use this capability.
Good post from my colleague Dick on how there will always be a need for reconciling different feed representations even if everyone agrees on a common syndication format.
For those of you who aren't living this syndication stuff every day (and I hope you're not!), the story of Atom (a syndication format) has taken an interesting turn recently. There seems to be a serious possibility that the W3C might charter a Working Group to standardize Atom. The article referenced here talks a little bit about what some of the advantages and disadvantages might be if such a "marriage" were to take place.
A few words about the article. Just so you understand the author's biases, he's definitely in the RSS 1.0/RDF camp. I personally disagree with a few of his statements: I wouldn't call Atom "insanely successful" quite yet, even given the author's definition of success. I also wouldn't call RDF as an extension mechanism "infinitely preferable" to RSS 2.0's extension mechanism, as I've mentioned before. Tone down the hyperbole a little, please.
Those reservations aside, however, this is a good article to read to get a handle on what's going on in the Atom universe these days.
Source: Intelliseek's BlogPulse
This is actually a big deal, IMHO. Reuters is viewing its RSS feeds as a direct distribution channel to the end consumer -- something it really hasn't had to date. If one envisions a world where a "feed reader" is not just a three-panel glorified mail client but instead a pervasive part of one's desktop and mobile work environment, why wouldn't you go directly to Reuters for your news?
Things are hotting up now!
It's nice to see Wired really paying some attention to the growing adoption of newsreaders. This particular article talks about an issue that I'm sure many of you are familiar with: what happens when all this stuff really takes off and we've got millions of polling newsreaders?
Here's someone who was fed up enough with the RSS/Atom format war that he created his own syndication spec: SDF (Streaming Document Format). I admire his moxie, and the spec is a good example of a readable specification document.
Potential adoptors (publishers, consumers, and everyone in between) are going to look at the cost/benefit for supporting yet another format. Are the benefits compelling enough? At first glance, I'd have to say no, but I won't dismiss it out of hand. It's something to keep an eye on, but it's got a loooong road ahead of it.
If the effort does start to pick up steam, it would be easy enough to extend FeedBurner's SmartFeed service to make SDF feeds work with legacy readers.
For the love of God, please don't put RSS on the cover of Wired! Actually, truth be told, I kind of wish I had a Pointcast application appropriate for the times, where I could configure the feed sources.
Source: The Return of Push!
I continue to be impressed with the server-based newsreader Bloglines and the thoughtful way they implement new features. The latest feature is "References", where they provide links to other threads that mention the item that you are reading. Very well-done and appropriate in the context.
Does anyone else notice how Bloglines is getting in the middle of a lot of traffic much more effectively than companies like Technorati? Bloglines is getting all sorts of great usage information as a by-product of providing a genuine service to users (i.e., the news reader). Technorati, on the other hand, is explicitly about meta-information, without really providing a compelling experience outside the compilation of that meta-information. I'm guessing that the quality of statistics that Bloglines gathers as a side-effect is much richer than what Technorati gathers with its crawlers.
Here's an interesting thought: Bloglines is a subscriber-facing proxy service, while FeedBurner is a publisher-facing proxy service. I bet there are some cool things we could do together. Hmm, note to self.
Source: Bloglines References
I guess the "homebrew coder" represents the evolution of the "shareware" programmer. Interesting that two of the four examples that this Economist article gives are of RSS readers -- and two of the most popular RSS readers at that.
Source: Return of the homebrew coder
A nice post here on trying to figure out the right way to make syndication more broadly appealing. One thing that is discussed is the use of a style sheet associated with the XML document so that if a user clicks on a feed link in a browser, they at least get something legible instead of a bunch of angle brackets.
That's what we're doing with the "Browser Friendly" service in FeedBurner -- that uses a CSS style sheet, although we could get even more sophisticated if we moved to an XSLT style sheet. All that is well and good, but is it enough to drive adoption? We're thinking of some other ways to make push syndication almost down to the protocol level to make the whole experience better for both publishers and subscribers, but I like seeing discussions (well, no comments on this item, so I guess it's not much of a discussion unfortunately) on this topic.
We all laugh at Feedster's business model, right? "Feedster is the Google of RSS". Well, they might have some issues when Google decides to become the Google of RSS, but until then I've found a few things that Feedster is really good at that Google is not: product reviews and travel destinations.
Have you ever tried to use the web to do some research on a product before you buy it? Try typing it into Google and you usually get hundreds of places you can buy the product or some big media reviews of the product. Epinions.com might have some information, but it's usually a ghost town there. Often, though, you can type your query into Feedster and get a number of individual perspectives and mini-reviews on the product.
Same thing for travel destinations. Searching on Google results in hundreds of travel agents that are trying to sell you a trip, so they won't say anything bad about the place. Searching through Feedster will often result in interesting stories and anecdotes about the specific town and hotel or restaurant that you're thinking about visiting. Cool.
Thought for the future: figure out how to incorporate a standard "review" namespace like RVW into the feed and watch a distributed review marketplace spring into existence.
Whew! After a lot of hard work and long nights, we have a pre-alpha of "FeedBurner" up and running. If you publish RSS or Atom feeds, please check it out. There's also a blog (everything's a blog, right?) that we're kind of treating as our support and news channel at Burn This! that you can follow along with.
I've burned my existing content and comments feeds to be "Browser-Friendly" and track item stats. I used the Apache redirect trick, so you should just pick up the new feeds automatically.
Back to business!
So if you're thinking about syndicating your content, should you care which "syndication format" you support? At this point, I'd have to say "not really", because there's no meaningful difference in the feature sets of the various formats for the majority of content producers out there.
That doesn't mean it's always going to stay that way as clients and publishers become more sophisticated (iTunes reader anyone?), but for now this article by Dare (who also did the nice character encoding article the other day and is active in the Atom syntax community) gives some pragmatic advice to publishers considering which formats to support.
Wouldn't it be nice if you as a publisher could be "future-proofed" against these format wars?
Good timing for this discussion -- I've been struggling with the right way to do character encoding just recently. The interaction between the HTTP Content-Type header and the encoding attribute of the XML declaration (and even the machine's default locale!) creates a confusing environment.
Wouldn't it be nice if there was no ambiguity and the HTTP charset parameter always matched the encoding of the XML feed? Stay tuned ... (evil grin!)
Just a germ of an idea right now, but another innovative use of RSS: using it to syndicate PVR suggestions. This could be cool as TiVo and TiVo-like PVRs become increasingly networked. The author, Andrew Grumet, proposes some elements in a separate namespace.
Again, very early in the process, but I'm going to keep my eye on this. And even though the answer to "Are copyrighted TV programs exchanged over the Net?" is "No", you don't have to stretch your imagination too far to envision a MythTV install pumping out an RSS 2.0 feed with the <enclosure> element pointing to a captured copy somewhere.
Source: RSSTV: Syndication for your PVR
Trying to make sense of the different versions of RSS can be, uh ... confusing. That's being generous. Mark does a good job here of showing how all the different versions relate to each other and where the incompatibilities lie.
Now, just so you know, Mark's one of the main guys behind Atom, which is not a bad thing at all (it's a good thing in fact), but you can see that these RSS incompatibilities have chafed him over time like burlap jogging shorts. You have to take the doom-and-gloom tone with a grain of salt. Yes, these incompatibilites are annoying, but if a client misinterprets the
skipHours element and doesn't check the site at 8 PM instead of 9PM (if it even respects the element in the first place), it's not the end of the world. I think the most damaging ambiguity is the content-type of the
description element, but even with that you can usually make an intelligent guess.
Greg has shown how NewsGator 2.0 handles RSS extensions in this posting. At this point it really only appeals to geeks, but that's fine because the geeks will write the extensions that the public at large can then leverage. It's nice to see the clients thinking about this.
An excellent example of using RSS to carry content other than news headlines or blog entries can be found at the iTunes Music Store RSS Generator. You fill out the form (sure wish I could deselect all those checkboxes quickly from Mozilla) and it constructs a URL you can use to retreive a nicely formed RSS 2.0 feed.
So that's cool ... the feed URL is really encapsulating a query that returns its results in RSS. The really nice thing, though, is that Apple has defined a namespace for the iTunes-specific meta-information. If we look at the root element, you can see the reference to the namespace with a prefix "itms":
And then on the individual items you can see how this namespace is used:
If you're a content publisher or are responsible for coding a content retrieval engine, you should know about how web content is cached. Especially if you publish RSS/Atom feeds, and ESPECIALLY if you are writing an RSS client. Unfortunatley, really understanding the interactions between the server, client, and proxy servers/web caches can be confusing. Fortunately, Mark Nottingham (he's also very active in the Atom community) has written an excellent document called "Caching Tutorial for Web Authors and Webmasters" that is required reading for any feed aggregator coders or blog hosting sites out there.
Having written a few web caching engines before, I'd just like to add a few suggestions that I've found useful:
- The "last modified" date should always reflect the server's clock, while a "last fetched" (for secondary cache expiration) should always use the client's clock. Don't get lazy: if you're requesting a remote resource, hold onto the last modified date that the server returns to you and store it so you can use it to set your If-Modified-Since header next time around. Don't use your local clock.
- If you're on the server side of things and need to provide a Last-Modified header, truncate the date value you store (and send down) to the second level. I picked that tip up from Jason Hunter's book Java Servlet Programming. That means if you've just grabbed the local system time to mark when a page or item has been modified, you should do one of these tricks before you set the value and send it down in the HTTP header:
long modifiedTime = System.currentTimeMillis() / 1000 * 1000;
- There's nothing wrong with setting the ETag header on a response to be the same as the Last-Modified header in quotes.
One of the smart things that most of the blogging engines do (thank you Blogger for establishing the precedent) is they provide the illusion of dynamic content by generating static pages, and then let the smart folks who wrote the HTTP server figure out the caching parameters to set. That doesn't mean that the server configuration can't be tweaked, though, especially with the Cache-Control header.
Oh, don't forget about supporting gzip compression.
Because RSS-compatible products are based on open Internet standards, anybody can produce them. So there'll be no RSS media empire or RSS billionaires.That's cute. The article also has a nice shout-out to FeedDemon and, best of all, recalls the infamous Wired "PUSH!" cover.
Source: 'Push' technology gets a nudge
Here's a nice list of blog search engines and aggregators. I hope this evolves into a larger "rss ecosystem" overview, because that's really what he's got a nice start on here. Thanks Ari!
I'm not sure how useful this would really be in practice, but here's an article that examines "feed splicing": combining multiple RSS feeds in interesting ways to create a new feed.
It's certainly a valid operation, but I'm wondering (and maybe you can help me out) what value consumers and/or publishers would derive from these capabilities. Would you like to be able to define these "spliced" feeds that can be represented by some globally accessible unique URL, or would you just prefer to see those filtering capabilities in your feed reader or aggregator?
Sam Ruby gave this presenation at the XML Conference last week, and it details the motivation for wanting to rev another syndication format. It more or less boils down to tightening up the spec so there's less ambiguity in what is allowed where in a feed. The presentation does a great job of comparing and contrasting how the existing specs (and psuedo-specs and spec interpretations) handle the different feed elements.
One other thing I wanted to call attention to is the format of the presentation itself. The slides look beautiful in Mozilla, less so in IE 6.0, but do a View Source on a couple of those slides. Isn't that fabulous? Snaps for Sam Ruby for the awesome and appropriate use of CSS.
Source: Sam Ruby: Atom in Depth
Some more tantalizing hints as to how a syndication client on Longhorn might take over the world.
Prediction: in a couple of years, RSS clients (or whatever syndication format emerges as the standard) will look a whole lot more like Headline News than Microsoft Outlook. If there's any client that's going to get there before Longhorn is released it'll most likely be a killer-app developed on Macromedia Central. It's a race.
Source: The Scobleizer Weblog.
As more and more people wake up to the idea that a fundamentally polling-based RSS distribution model might have some "scaling challenges", different ideas to handle this come to the fore. Here's an interesting model: using the BitTorrent peer-to-peer infrastructure to distribute RSS feeds.
BitTorrent is a cool bit of engineering that solves the "Tragedy of the Commons" problem that plagues many p2p protocols: you gotta pay (i.e., offer some upstream bandwidth) to play (download). The Java-based client Azureus is my client of choice, but it's really the protocol that's the smart stuff here.
Getting back to how this works with RSS, I personally think this is overkill for the scaling issues at present: I think getting the clients and server-side aggregators/publishers to agree on a push protocol and transport is a more pressing concern. I think the P2P idea is a good thing to keep in our pocket when RSS starts getting bulkier.
A very nice article from salon.com that talks about the building wave of RSS and content syndication. It's funny, around the office we've been making the exact same analogy: that RSS feels a whole lot like HTML did in 1994.
It's worth sitting through the ad to get to the article.
The template that Movable Type comes with to represent an RSS 2.0 feed, while valid, can be construed as a little bit "funky" because it uses some Dublin Core elements instead of just core RSS 2.0. That's fine, but I prefer to stick to the spec if I can. Here's the template I use that seems to validate at Feed Validator:
Just copy and paste it into your "RSS 2.0 Index" template in MT. Remember to edit the
<webMaster> elements, as well as change the timezone from "CDT" in the various
MTEntryDate tags (haven't figured out how to generate that from the format parameter).
The well-intentioned Atom Feed syntax effort is at a critical point right now. Many developers are fed up with the contentious turn that the discussion has taken lately and want some kind of resolution sooner rather than later. This frustration has prompted Don Park's proposal (see also the corresponding Wiki entry) that Atom just start with RSS 2.0 as the base syntax and "fix" it. A totally reasonable proposal that was exactly Dave Winer's intention when he finalized the RSS 2.0 spec.
A very good summary of the situation is given by Diego Doval in his blog posting, which spurred an interesting series of comments. For the record, here's where I sit:
- Diego: right on. I feel your pain. Also, this community seems unnaturally predisposed to resort to aggression and pettiness quicker than most.
- Zoe: namespaces are not overrated, and in fact represent the salvation of content syndication by letting us define a very simple core model, which happens to be the speed bump that Atom syntax can't seem to get over right now.
- Yoz: good posting ... the dates are kind of screwy, but most clients will deal with both
<dc:date>. I personally go with RSS 2.0 with
- Dave: very well-stated points. Especially like the comment "I would like it if people started with the RSS spec, evolved it into something that wasn't called RSS. Read the Roadmap. That's what it says."
This will be good to keep following, that's for sure!
I've been asked a few times (no really, I have!) where I stand with the whole RSS-RDF-Atom religous war. There's a very contentious history there, with some strong personalities that have made the domain of content syndication more polarized than perhaps it could have been. I'd really like to stay above the politics and look at the different specs from a technologist's viewpoint... so that's what I'm going to try to do here.
Nice piece here that shows that RSS is starting to peek into the mainstream. It's kind of funny that he alludes to Pointcast as a true "push" technology -- it really wasn't any more pushy than current RSS clients, and in fact polled just like the RSS clients do. The problem was with its popularity, the inability to customize what was being retrieved, and the general network capacities at the time. If naive RSS clients take off the way Pointcast did, we could have a lot of the same problems if we don't starting thinking about millions of clients polling millions of feeds every five minutes.
Sorry, got off track there. This article still serves as a nice introduction to feeds and explains their benefits with respect to email distribution.
Well, the entry the other day about microfeeds got me thinking that I should be eating my own dog food. If they're such a good idea, why don't I have any? So, I've done a little feed maintenance and added an RSS 2.0 "Microfeed" to each entry. This microfeed contains the comments and trackbacks made on that entry, so if there's some entry you want to keep track of you can go ahead and subscribe to its microfeed.
As I mentioned in the post the other day, if you're using a traditional 3-pane RSS reader you'll probably find all of these microfeeds more trouble than they're worth. But who knows what other clients might show up that can deal with these in an almost transparent fashion. I think what I'd like to do is define an extension (in its own namespace, of course) that allows a microfeed to point to its parent feed, which could be a valuable hint for a client that knew how to deal with a parent-child relationship between feeds. I'm assuming this doesn't exist right now -- please correct me if I'm wrong.
I also created a blog-wide comments feed in case you're interested.
So there's an example or two of companies using RSS for press releases: Dick talks about it here and Jason talks about it here. I think the big idea here is that we can transform a static piece of information into an ongoing conversation with potential customers. By representing the information as a syndication feed, it becomes a "living press release". Potential customers can "subscribe" to that press release and the company can issue news and updates related the release as new content items. Cool, eh?
Following the life of a press release is an example of what we might call a "microfeed". Think of a piece of content as anchoring a discussion. The other obvious example of a microfeed would be the comments on a blog item. One approach is to have a feed associated with each item ... in fact, something like that is proposed with the includedComments module.
The problem that emerges isn't that there are so many feeds to generate -- that's a piece of cake on the publisher's side, as it's just another page. The problem is that the traditional three-pane nntp^H^H^H^H RSS reader isn't the right model for dealing with potentially hundreds of microfeeds, many of which might never be heard from again. I'm not sure what the corrent interaction model is, but it probably has to do with recognizing these microfeeds in the context of the original parent, treating them like "child feeds". The list of feeds that I'm following probably looks a lot more like a big threaded discussion than a serial list of feeds.
Then again, the right answer is probably something completely different. What other ideas are out there?
Ken brings to light an issue that will become more and more important as we introduce addition content types (namespace extensions) to our syndicated feeds. Since "feeds" are temporal entities that really serve the purpose of notification, putting information in the feed that will wink out of existence once enough new items have been pushed on the stack is not the way to go if that information has value in its permanence. Put another way, don't put stuff exclusively in the feed that has value outside the feed.
That makes sense if you equate RSS (and its varients) with a "notification feed", and in fact Ken provides some really good suggestions for creating "microcontent" and linking to that from the different syncdication formats.
I think what we're seeing, however, is that RSS evolving beyond being simply a feed format, and is instead becoming more of a standard for content representation. You can argue that this is a misuse of the format, but there seems to be a great deal of innovation going on related to the <item> element even taken out of the context of the <channel> element. Why would that be? My guess is because it's an easy starting point, with common meta-information that almost any kind of content would require, AND because RSS has promoted the use of XML namespaces and provided enough examples to make extensions accessible.
So anyway, I don't think there's a right or wrong answer here, but take Ken's advice and make sure that any information you're publishing that has value in its permanence doesn't get washed away in the feed.
This viewpoint has been around for a while, but it's usually (unfortunately) characterized as "RSS will replace email", which is just silly. But there is a real idea in here: RSS feeds could offer a cure for all of those sites that you register for where you have to give your email address and then uncheck 20 little boxes that try to get you to sign-up for their mailing lists.
I personally would be much more likely to "try out" some of those information feeds if they were presented as RSS becuase I have control. I can unsubscribe without having to rely on the content providers mailing list server, I don't have to give out my email address, and I might find something useful in their feed. I probably won't, but at least I would be exposed to their information instead of never receiving it in the first place.
Here's a nice article that summarizes why this could be a good thing.
This account was taken from a "Birds of a Feather" session on blogging at the Microsoft Professional Developers conference. I think he's talking about the people who care about the infrastructure behind blogging more than "bloggers", but still the sentiment is depressing. There are few things more irritating than socially insecure semi-intelligent grown men (not so much women - it's a testosterone thing) getting in a group and trying to prove how semi-intelligent they are by discussing a contentious topic like children. The staggering lack of effort some people have to put forth to be an asshole is truly breathtaking.
I think there's a lot of good work being done in the content syndication community, provided you can evade the energy-sucking deinzens that float around like Inky, Blinky, Pinky and Clyde. Keep your eye on the prize.
This posting by Sean Neville (Macromedia guy) talks about a lot of the things that I find exciting right now. Atom/RSS! Formalized content model for things you can by with Atom/RSS namespaces! Wireless! Macromedia Central! Mystery! Intrigue!
There's some convergence of a lot of these ideas that's just waiting to emerge ... what it's going to be and who's going to find it is anyone's guess right now. But it's going to be a Big Idea.
There are few people that have a deeper history with XML than Robin Cover. I remember the first time I heard the acronym "XML" was when I visited the Summer Institues of Linguistics with Marc Rettig back in 1997. We were researching content object models, and I met Robin briefly (not that he'd remember) and came away with the impression that XML was the future. Now that's the kind of guy you want to pay attention to!
Here Robin gives an overview of the content syndication format Atom. Personally, I don't really care if Atom takes off or if RSS 2.0 takes off or even if we have competing standards for a while ... the underlying content model is similar enough that transforming one into another (with sufficient namespace extensions possibly) should be feasible for a while. There's still some innovation to foster, I think, and it's only two schemas (famous last words, right?)
This posting from Davic Galbraith touches on another emerging trend in RSS ... moving beyond publishing just "text items". RSS 2.0 has support for namespaces, and RSS 1.0 has modules, but they're both trying to capture the same thing: the ability to extend RSS to capture data (and meta-data) about disparate types without breaking what's great about RSS to begin with: the standardization of the base syndication format. Yes, that was a sentence with two colons in it. Sorry about that.
Once a community can agree upon the accepted namespace extensions to represent the desired content, all sorts of great things can happen. In this case, the community has to include both the feed publishers and (at this point non-existent) extension-aware client. At this stage, I think the key is to not over-engineer things ... you don't have to have some dublin core-esque schema to realize value in constrained domains. Just be good about versioning your schemas!
I was flying back home today after a nice weekend vacation, staring at the GTE Airfone (or whatever it's called these days) in the seatback in front of me and thought (not for the first time) "what a waste". Talk about your captive audience. They really could use that LCD panel to provide valuable information. I know that some of them give news updates and update stock market information, but even that stuff is on a really tight loop.
Same thing with my wireless phone. I have a Samsung I-500 ... a pretty advanced phone, on the whole, that can poll my email every 10 minutes or so with the pretty speedy Sprint Vision service. So why can I have a cool RSS client or some kind of "Pointcast for the Phone" as my screensaver instead of (or in addition to) a clock? I think all the pieces are there.
This guy's thinking along the same lines I think.
This article talks about one person's experience with Google's Adsense textads. Another thing to put on the "List of Things Google Does Right". I also like Matthew's advice for blog writers: keep it focussed and write your ass off.
The other cool thing about this article that now I've got another feed I'm interested that I can start tracking: PVRBlog.
Jeremy Allaire proposed in his blog today an extension of RSS to accomodate more data-oriented applications that he's calling RSS-Data. It's great to see how thought-leaders like Jeremy and Dave Winer are approaching the evolution of RSS to be the standard "information interchange" vehicle.
My concern with this proposal probably has more to do with me not quite understanding the goal of incorporating the XML-RPC data serialization elements right into the steam. With RSS 2.0, we have the ability to include arbitrary descriptive elements without breaking any kind of compatability with parsers that are not aware of the particular domain. See the RSS 2.0 spec and this nice description by our good friend Morbus for a quick rundown of how RSS 2.0 namespaces work.
With XML-to-Object (de-)serialization libraries so common, it seems that Jeremy's assertion that we'd have to write a new RSS parser for each namespace doesn't seem any more arduous than having to map the data serialization elements to the correct object definitions.
But then again, maybe the whole point here is that there's no way to say that an element contains anything other than a String unless there's additional semantic meaning imposed on the element that is not captured in the document itself (like the date elements having to conform to the Date and Time Specification of RFC 822, or the ttl element having to represent an integer). If that's the case, I'm wondering why we can't associate an XML Schema with a particular namespace. Jeremy mentions XML Schema earlier in his post, so I'm sure he's thought of this! It seems like that would be a more powerful and elegant mechanism to represent these datatypes.
Well, whatever happens here, it can only mean good things for the evolving world of content syndication!
So what's the difference between an "Enterprise Weblog" and an "Enterprise Portal"? You got me ... I'm guessing it's the ease of being able to publish new information in pursuit of the Holy Grail of Effective Knowledge Management within the Enterprise.
Several years back, we were all with a company called "DKA" (Digital Knowledge Assets), and we tried to tackle the problem of KM in much the same way: by making it really easy to share and publish information into an "ecosystem" of information. When we realized that selling KM into enterprises was a long haul, the app morphed into a personal publishing tool called "SceneServer", which would look awful familiar to most bloggers today.
But as it turns out, Blogger added a word to the common vernacular and was bought by Google (I'm still so psyched for Evan), while the URL for DKA points to an X-Box cheats site. Ah well, no regrets. But Traction, you've got quite a load to tow ... good luck.
We usually associate RSS with blogs and news sites, which is fine, but it's also a narrow way to look at things. It might be more helpful to think of RSS as a description of a "channel of information", where each piece of information ("item") has a title, URL, and a short description. There's a little more to it, but that's pretty much it. You can see how it lends itself to news-like text-based items, but that's not a built-in constraint of the format.
One guy who's almost always thinking about things in the right way is Micheal Sippey, and in his blog he mentions the ability in Feed Demon to represent some kind of filtered aggregation as its own discrete RSS feed. You can do a similar kind of thing with Feedster searches, where the results of an RSS search can themselves be expressed as an RSS feed.
Once you've agreed upon the relatively simple structure for information exchange, it opens up the whole food chain to innovation: producers (sources of RSS), consumers (RSS clients), and, perhaps most interesting, the middleware (aggregators and filters like Feedster, BlogStreet, and many others).
When we developed Spyonit, we had a number of ways to be notified when your "spies" fired: you could get an email, an SMS, an instant message, or it could be reported on your "My Spy Reports" web page. If I were to build that today, I would most certainly add "as an RSS feed" as a notification option (not too mention the ability to monitor RSS feeds for alerting purposes ... but that's obvious [smile]), so that your spies could report to you through your favorite RSS client or aggregator.
Stay tuned to this channel ... I think the RSS world is going to get more and more interesting.
Not crazy about the name: "geeplogs" is supposed to be "global positioning" logs ... the merging of GPS and blogs. This article is not too terribly interesting, but the final paragraph thinks about layering information on a physical space using RSS as a means to represent that information. I'm not sure that's an appropriate use of the RSS, but it's a different way of thinking about things.