Abstract

For many information sources on the Web, it is useful to have some standardized way of subscribing to information updates. Syndication formats such as RSS and Atom can be used by these information sources to publish a feed of updated information items. The basic syndication mechanism describes a data format for publishing entry metadata and entries. In addition to the basic format, services might also support additional functionality, such as the ability to indicate that clients may page through a feed, if it contains more entries than published on the first page.

Content Feeds

Early Web content was static or updated very infrequently
- there was not yet the requirement to reuse content in different contexts
Frequently updated Web content quickly became a very common scenario
- as commercial interests took over, users should have a reason to re-visit a site
- presenting a steady stream of new content creates the image of a live Web site
There are two major use cases where HTML is not sufficient
1. users want an efficient way to get the updated content from a site
2. sites want to aggregate updated content from other sites and re-publish it
Syndication Formats are designed to support these two use cases
- container formats for updated items
- a small amount of metadata about these items for automated processing

Outline (Syndication Formats)

Syndication Formats [19]
1. RSS [11]
2. Atom [8]
Syndication Aggregation [2]
Atom Add-Ons [4]
Conclusions [1]

Outline (RSS)

Syndication Formats [19]
1. RSS [11]
2. Atom [8]
Syndication Aggregation [2]
Atom Add-Ons [4]
Conclusions [1]

RSS History

The Myth of RSS Compatibility provides a good overview
RSS is a schoolbook example for why standards are a good thing
- RSS 0.9x was created for the My Netscape portal in March 1999
- RSS 0.91 (a simplification) was introduced in July 1999 (as an interim solution)
- the AOL/Netscape merger removed the format from the company's portal
- RSS was without an owner, and different parties claimed/denied ownership
- RSS 1.0 was created by an informal developer group
- RSS 0.92 (and 0.93 and 0.94) were published without acknowledging RSS 1.0
- finally, RSS 2.0 was released as a follow-up to the RSS 0.9x versions
Using RSS has become an exercise in managing a menagerie of versions

RSS 0.9x

RSS means RDF Site Summary (or Rich Site Summary?)
- based on an RDF draft and not compatible with the final RDF specification
- RDF was considered too cumbersome and unstable
- 0.90 (proto-RDF) was quickly replaced by the non-RDF 0.91 version
RSS 0.92+ versions were developed as unilateral specifications
- starting with RSS 0.91, RSS means Rich Site Summary
- it is no longer built on RDF, instead it simply uses XML
- the 0.9x branch eventually was renamed to RSS 2.0

RSS 0.91 Example

<rss version="0.91">
 <channel>
  <title>XML.com</title>
  <link>http://www.xml.com/</link>
  <description>XML.com features a rich mix of information and services for the XML community.</description>
  <language>en-us</language>
  <item>
   <title>Normalizing XML, Part 2</title>
   <link>http://www.xml.com/pub/a/2002/12/04/normalizing.html</link>
   <description>In this second and final look at applying relational normalization techniques to W3C XML Schema data modeling, Will Provost discusses when not to normalize, the scope of uniqueness and the fourth and fifth normal forms.</description>
  </item>

RSS 1.0

RSS means RDF Site Summary (this time for real)
- based on the final RDF specification and thus incompatible with any RSS 0.9x
- developed when the Semantic Web and RDF were first heavily marketed (1999)
- RDF was expected to become the format for metadata on the Web
RSS 1.0 makes heavy use of XML Namespaces
RSS 1.0 introduces features which were not present in 0.91
- date information for published items (very relevant for news feeds)
- individual authors for various items in a feed
RSS 1.0 is the latest version of RDF-based RSS
- the Semantic Web wave is not over yet, but RDF has lost its novelty appeal
- for a more XML-oriented encoding, RSS 0.9x provides a better foundation

RSS 1.0 Example

<rdf:RDF xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#" xmlns="http://purl.org/rss/1.0/" xmlns:dc="http://purl.org/dc/elements/1.1/">
 <channel rdf:about="http://www.xml.com/cs/xml/query/q/19">
  <title>XML.com</title>
  <link>http://www.xml.com/</link>
  <description>XML.com features a rich mix of information and services for the XML community.</description>
  <language>en-us</language>
  <items>
   <rdf:Seq>
    <rdf:li rdf:resource="http://www.xml.com/pub/a/2002/12/04/normalizing.html"/>
    <rdf:li rdf:resource="http://www.xml.com/pub/a/2002/12/04/som.html"/>
    <rdf:li rdf:resource="http://www.xml.com/pub/a/2002/12/04/svg.html"/>
   </rdf:Seq>
  </items>
 </channel>
 <item rdf:about="http://www.xml.com/pub/a/2002/12/04/normalizing.html">
  <title>Normalizing XML, Part 2</title>
  <link>http://www.xml.com/pub/a/2002/12/04/normalizing.html</link>
  <description>In this second and final look at applying relational normalization techniques to W3C XML Schema data modeling, Will Provost discusses when not to normalize, the scope of uniqueness and the fourth and fifth normal forms.</description>
  <dc:creator>Will Provost</dc:creator>
  <dc:date>2002-12-04</dc:date>
 </item>

RSS 2.0

RSS now means Really Simple Syndication
- RSS 2.0 is the continuation of the 0.91 branch (which dropped RDF)
- together with RSS 1.0 it is the most popular version of RSS
- migration from 0.91 to 2.0 is easily possible
RSS 2.0 tries to avoid the use of XML Namespaces
RSS 2.0 is increasingly used with extensions for vendor-specific information
- the RSS core is minimal, so many applications need extensions
- many extensions have overlapping functionality
- most extensions have unclear semantics and unclear versioning policies
- Apple's podcasts are probably among the most popular extensions

RSS 2.0 Example

<rss version="2.0" xmlns:dc="http://purl.org/dc/elements/1.1/">
 <channel>
  <title>XML.com</title>
  <link>http://www.xml.com/</link>
  <description>XML.com features a rich mix of information and services for the XML community.</description>
  <language>en-us</language>
  <item>
   <title>Normalizing XML, Part 2</title>
   <link>http://www.xml.com/pub/a/2002/12/04/normalizing.html</link>
   <description>In this second and final look at applying relational normalization techniques to W3C XML Schema data modeling, Will Provost discusses when not to normalize, the scope of uniqueness and the fourth and fifth normal forms.</description>
   <dc:creator>Will Provost</dc:creator>
   <dc:date>2002-12-04</dc:date>
  </item>

The Case for Content Management

RSS is rarely produced by hand
- by definition, RSS contains redundant information for a specific purpose
If a Content Management System (CMS) is used, RSS can be generated
- basic metadata can be generated by the CMS (title, author, date)
- better tagging of content results in better tagging of feeds
- well-tagged feeds are better foundations for large-scale reuse of feed items
Blogging is simply a specialized case of a CMS
- Web-based interface for controlling everything
- strictly time-ordered sequenced of published items
- navigation features primarily based on the time-specific facets of the blog (maybe tags)
- all blogging tools include feed support

Consuming RSS

RSS feeds often have quality problems
- surprisingly often feeds do not even deliver well-formed XML
- the use of embedded markup in RSS is not well-defined
Writing an RSS reader from scratch is not a good idea
There are three major tasks which RSS readers must do
1. accept non-XML RSS feeds and fix them to be XML
2. look at the feed contents and bring them into a unified form
3. produce a unified view of feeds regardless of the RSS version

RSS Technical Problems

What to put into an item's description

the fundamental question is whether a description is text or HTML
HTML is often used in a non-XML way (omitted end-tags)

interpretation is client-specific if there is no well-defined indication

<description>This is a post<br>on two lines</description>

<description>This is a post&lt;br>on two lines</description>

<description>This is a post about <br> in RSS feeds</description>

<description>This is a post about &lt;br> in RSS feeds</description>

<description>This is a post about &amp;lt;br> in RSS feeds</description>

Underspecified and not very robust in various other areas
- broken RSS is accepted by most readers (and fixing it can change the interpretation)
- the interpretation of relative URIs is not mentioned in the specifications
- some minimal semantics (classification) for items would be very useful

RSS Political Problems

Multiple and incompatible RSS versions are still in widespread use
- RSS 1.0 and RSS 2.0 are incompatible by design (RDF vs. non-RDF)
None of the specifications is being updated or fixed
- none of the RSS versions is maintained by a universally accepted standards body
- some of the lessons learned by RSS deployment are not used in a new version
- it is unlikely that a new version will be produced which merges the RSS landscape
Invent something new instead of trying to fix RSS
- Atom started in 2003 (called Echo at first)
- W3C or IETF would have been promising candidates for a new RSS
- W3C is more formal, IETF is more developer-centered
- IETF was chosen over W3C because the of Atom community's preferences

Outline (Atom)

Syndication Formats [19]
1. RSS [11]
2. Atom [8]
Syndication Aggregation [2]
Atom Add-Ons [4]
Conclusions [1]

Atom History

RSS's shortcomings were very apparent and could not be fixed
In mid-2003, discussions started about an improved format
It also became apparent that the format should have a protocol
Atom 0.3 was released in December 2003 but had no formal home
IETF was chosen as the new home with a working group in June 2004
RFC 4287 (Atom) was published in December 2005
AtomPub has been published as RFC 5023 in October 2007

Atom vs. RSS

Standardized by the IETF (well-defined process)
Classification of entries (user-defined categories)
More XML-like markup design (more nesting)
Namespaces are used and supported as standard mechanism
Atom feeds must be well-formed XML (there even is a schema)
Interpretation of content is well-defined (various content types)
Support for xml:lang and xml:base

Atom Example

<feed xmlns="http://www.w3.org/2005/Atom" xml:lang="en-us">
 <title>ongoing</title>
 <id>http://www.tbray.org/ongoing/</id>
 <link rel='self' href="http://www.tbray.org/ongoing/ongoing.atom"/>
 <updated>2007-04-11T12:55:09-07:00</updated>
 <author>
  <name>Tim Bray</name>
 </author>
 <subtitle>ongoing fragmented essay by Tim Bray</subtitle>
 <entry xml:base="When/200x/2007/04/02/">
  <title>Atom Publishing Protocol Interop!</title>
  <id>http://www.tbray.org/ongoing/When/200x/2007/04/02/APP-Interop</id>
  <published>2007-04-02T13:00:00-07:00</published>
  <updated>2007-04-10T14:24:00-07:00</updated>
  <category scheme="http://www.tbray.org/ongoing/What/" term="Technology/Atom"/>
  <category scheme="http://www.tbray.org/ongoing/What/" term="Technology"/>
  <category scheme="http://www.tbray.org/ongoing/What/" term="Atom"/>
  <content type="xhtml">
   <div xmlns="http://www.w3.org/1999/xhtml">
    <p>Mark your calendar: <a href="http://www.intertwingly.net/wiki/pie/April2007Interop">April 16-17 at Google</a>. <em>Everybody</em> is invited, provided they bring along an APP implementation, client or server. This was just announced a couple of days ago, and as I write this there are already <s>six</s> twelve client and <s>seven</s> fourteen server implementations signed up to be there and try to <a href="http://www.intertwingly.net/wiki/pie/InteropGrid">fill in the grid</a>. Let’s drop some names, in alphabetical order: AOL, Flock, Google, IBM, Lotus, Microsoft, Oracle, O’Reilly, Six Apart, Sun, WordPress. Um, have I mentioned that the APP is going to be huge?</p>
   </div>
  </content>
 </entry>
</feed>

Atom Content

RSS had no well-defined way of finding out what an entry's content is
- implementations use smart ways to figure out what the RSS author really wanted
- one of Atom's main goals was to improve this in a well-defined way
- Atom allows escaped markup (the only way to include non-XML HTML in an XML format)
Each content element should have a type (the default is text)
Atom's content interpretation algorithm (use first applicable rule):
1. if type is text, no child elements are allowed (plain text content)
2. if type is html then RSS's method of escaped markup is used
3. if type is xhtml then there must be an div containing XHTML markup
4. if type is an XML MIME type then the content should be treated as this type
5. if type starts with text/ then no child elements are allowed
6. for all other values, the content must be an base64-encoded entity of the specified MIME type

Atom Content Examples

<content type="xhtml">
  <div xmlns="http://www.w3.org/1999/xhtml">
    One <strong>bold</strong> foot forward
  </div>
</content>

<content>The "atom:content" element either contains or links to the content of the entry. The content of atom:content is Language-Sensitive.</content>

<content type="html">The &lt;code>atom:content&lt;/code> element either contains or links to the content of the entry. The content of &lt;code>atom:content&lt;/code> is &lt;a href="http://www.ietf.org/rfc/rfc3066.txt">Language-Sensitive&lt;/a>.</content>

<content type="image/png">
iVBORw0KGgoA … TAAAAAElFTkSuQmCC
</content>

<content src="image.png" type="image/png"/>

Atom Categories

Atom allows to assign categories to entries
- each category element must have a term attribute for the category
- an optional scheme identifies the categorization scheme (by URI)
- an optional label attribute provides a human-readable label for the category
AtomPub defines a document format for Category Documents
Three different cases of categorization can be distinguished
1. use a well-known scheme (such as Dublin Core)
2. use a private but well-designed scheme (which has a URI and can be reused reliably)
3. use tags without schemes
Tags can be handled in different ways
- leave the scheme empty which indicates that there is no scheme
- use some special scheme for referring to a tagging scheme

Switching from RSS to Atom

Generate both feeds but serve RSS with a HTTP redirect (301)
- old subscribers with broken clients can still use the RSS feed
- old subscribers with correct clients will start using the Atom feed
Atom exposes more information than RSS (category for tags)
- the mapping of publishing info to the feed has to be changed/extended
- for standard metadata use Atom's built-in metadata elements
- for application-specific metadata consider reusing an existing metadata schema
Atom can be used to publish snippets as well as full content
- content allows any type of content to be used and may contain a complete entry
- summary allows only text and should provide a condensed version of an entry
- some Atom sources publish two feeds (full content and some excerpt longer than the summary)
Generate good Atom and downgrade it to RSS 1.0 & 2.0

Outline (Syndication Aggregation)

Syndication Formats [19]
1. RSS [11]
2. Atom [8]
Syndication Aggregation [2]
Atom Add-Ons [4]
Conclusions [1]

End-User Aggregation

Users often have a small number of preferred Web sites
- news can be tracked by subscribing to their feeds
- this requires an RSS-aware client (Firefox is not a good RSS reader)
Ajax-based RSS could be implemented as a client-side application
- retrieve the feed, transform it to HTML, and render the result
- but the security restriction of XMLHttpRequest gets into the way
- there is a way around this as used in the Ajax RSS reader
How can users find a feed?
- feeds have URIs (they are Web resources) and can be referenced from within HTML

<link rel="alternate" type="application/rdf+xml" title="…" href="…" />
<link rel="alternate" type="application/rss+xml" title="…" href="…" />

<link rel="alternate" type="application/atom+xml" title="…" href="…" />

Aggregation Intermediaries

RSS is a frequently used format between content providers
- news agencies want to distribute news items as easy as possible
- by using a single format, producers and consumers can more easily interoperate
The RSS History caused publishers to look for something better
- RSS had a head start and still is widely used
- Atom is a much better format and will eventually replace RSS
User-configured portals are the merger between both scenarios
1. users select a number of feeds as their preferred information sources
2. the feeds are presented on a single personalized Web page
3. by sharing and re-publishing items, users are becoming aggregation intermediaries

Outline (Atom Add-Ons)

Syndication Formats [19]
1. RSS [11]
2. Atom [8]
Syndication Aggregation [2]
Atom Add-Ons [4]
Conclusions [1]

Extensibility

Atom's goal is to support the core of syndication metadata
The Atom feed format is extremely simple
- feed, entry, and content are the container elements
Atom's metadata model is a bit more exhaustive
- author, category, contributor, generator, icon, id, link, logo, published, rights, source, subtitle, summary, title, updated
Additional semantics can use Atom's extension model
- the challenge is to identify useful units of functionality
- some extensions have already been standardized
A starting point for searching for extensions is IANA's Atom Link Relations registry

Atom Threading Extensions

Mailing lists and newsgroups are collections of resources
Threaded discussions create resource relationships
- resources may be in reply to other resources
- replies to a resource may be available elsewhere (maybe in a feed)
- it might be interesting to know the count and updated timestamp of those replies
- it might be interesting to know the total number of replies to an entry
RFC 4685 defines Atom extensions for threaded discussions
Reading mailing lists archives with RFC 4685 support is much easier
- it is not necessary to access the complete set of entries
- building an Atom-based reader for news or email is greatly simplified
- the approximations of counts and times is not a serious problem

Atom License Extension

Atom's rights element is a text-based element
- requires the text to be repeated in every single entry
- explicitly states it SHOULD NOT be used to convey machine-readable licensing information
Machine-readable licenses need identity of licenses
- adding a link to a license uses a URI and allows to recognize the URI
- Atom's basic link relations do not include license semantics
RFC 4946 defines a new link relationship for links to licenses
License links can be added to Atom feeds or entries
- a new license link relationship is defined
- licenses can be recognized by comparing links to well-known URIs
- there is no universally standardized format for a license description format

Feed Paging and Archiving

Feeds often are used to publish varying or changing collections of entries
- complete feeds contain a list of all entries and may change completely
- pages feeds are split across multiple documents and are not necessarily stable
- archived feeds are split but are stable over time and allow repeated access
RFC 5005 defines Atom extensions for defining these types of feeds
- by specifying the type of a feed, clients can better adapt their behavior
- plain Atom clients are still able to read these feeds, but have less information about them
Complete feeds contain all entries of a feed (top twenty blog posts)
Paged feeds are a set of linked feeds that constitute the logical feed
- first, last, previous, and next link relationships
- there must be at least one of these relationships in each feed document
Archived feeds are stable feed documents available from a subscription document
- prev-archive, next-archive, and current link relationships
- subscription documents must link to archives to make archived entries available

Outline (Conclusions)

Syndication Formats [19]
1. RSS [11]
2. Atom [8]
Syndication Aggregation [2]
Atom Add-Ons [4]
Conclusions [1]

Semantic Web Light

Syndication creates representations for a few universal concepts
Atom adds some concepts to RSS's model
Syndication revolves around the idea of interacting with items
Atom-based interaction is one way of implementing REST
For more semantics, Atom is only the foundation

Atom Syndication Format

Web-Based Services (INFO 290-4)

Erik Wilde, UC Berkeley School of Information
2008-02-26

Abstract

Content Feeds

Outline (Syndication Formats)

Outline (RSS)

RSS History

RSS 0.9x

RSS 0.91 Example

RSS 1.0

RSS 1.0 Example

RSS 2.0

RSS 2.0 Example

The Case for Content Management

Consuming RSS

RSS Technical Problems

RSS Political Problems

Outline (Atom)

Atom History

Atom vs. RSS

Atom Example

Atom Data Model

Atom Content

Atom Content Examples

Atom Categories

Switching from RSS to Atom

Outline (Syndication Aggregation)

End-User Aggregation

Aggregation Intermediaries

Outline (Atom Add-Ons)

Extensibility

Atom Threading Extensions

Atom License Extension

Feed Paging and Archiving

Outline (Conclusions)

Semantic Web Light

Atom Syndication Format

Web-Based Services (INFO 290-4)

Erik Wilde, UC Berkeley School of Information2008-02-26

Abstract

Content Feeds

Outline (Syndication Formats)

Outline (RSS)

RSS History

RSS 0.9x

RSS 0.91 Example

RSS 1.0

RSS 1.0 Example

RSS 2.0

RSS 2.0 Example

The Case for Content Management

Consuming RSS

RSS Technical Problems

RSS Political Problems

Outline (Atom)

Atom History

Atom vs. RSS

Atom Example

Atom Data Model

Atom Content

Atom Content Examples

Atom Categories

Switching from RSS to Atom

Outline (Syndication Aggregation)

End-User Aggregation

Aggregation Intermediaries

Outline (Atom Add-Ons)

Extensibility

Atom Threading Extensions

Atom License Extension

Feed Paging and Archiving

Outline (Conclusions)

Semantic Web Light

Erik Wilde, UC Berkeley School of Information
2008-02-26