Creative Commons License

This work is licensed under a CC
Attribution 3.0 Unported License

 <a href="./" title="Tutorial Homepage">From SOA to REST:<br/>Designing and Implementing RESTful Services</a><br/>Tutorial at <a href="http://icwe2009.webengineering.org/">ICWE 2009</a> (San Sebastián, Spain) Cesare Pautasso Faculty of Informatics, University of Lugano Erik Wilde School of Information, UC Berkeley June 22, 2009 2009 Cesare Pautasso and Erik Wilde
Presenter Subject Slides Additional Resources

:
Introduction Cesare Pautasso Faculty of Informatics, University of Lugano Erik Wilde School of Information, UC Berkeley (PDF) Loose Coupling This introduction presents the schedule, the tutorial presenters, and some background for the tutorial. Specifically, we briefly mention all the *OA terms that have been invented in recent years, such as SOA (Services), ROA (Resources), WOA (Web), SynOA (Syndication), and EOA (Event), and set them into context. Our main goal is to explain our notion of SOA for the purpose of this tutorial, and what we perceive as the core tasks when moving from SOA to REST. Abstract

Schedule
  • 9.00-10.30: Intro &
  • 11.00-12.30:
  • 14.30-16.00:
  • 16.30-18.00:
Presenters Cesare Pautasso Erik Wilde *OA Overload SOA and REST
  • Service-Oriented Architecture (SOA)
    • Service: How to define a service
    • Architecture: How to apply SOA in the design/implementation process
  • Representational State Transfer (REST)
    • Representation: Resources (not functions) are the primary abstraction
    • State: Trying to push state to the edges (client or resource)
    • Transfer: Focus on the exchange of representations (uniform interface)
 What is SOA?
  • What is Service-Oriented Architecture?
  1. Alignment of business objectives and IT
    • can be implemented with any architecture, technology, products
    • SOA explained like this is more for business-oriented people
    • this definition is not within the realm of technical terms
  2. Technical architecture (interfaces are exposing services)
    • focus on IT and the idea of services as the main abstraction
    • still little guidance on how exactly a service is identified and exposed
    • this definition is in the right space, but often highly underspecified
  3. SOA as the high-level explanation for WS-* Web Services
    • this is how SOA as a buzzword started (SOA as architecture for Web Services)
    • most SOA products are focusing on this (overly narrow) view of Web Services
    • this definition has too many implicit decisions built-in
 From SOA to REST
  • Starting from the second definition of SOA
    • a technical architecture for structuring/implementing a service landscape
    • many crucial aspects are undefined/underspecified and need clarification
  • Services can come in a variety of flavors
    • IT services often are guided by middleware/RPC views of the world
    • Web architecture does not expose middleware/RPC views of services
    • Web services (not the WS-* kind) are exposed as resources and links
    • aligning Services and the Web means redefining services
  • From SOA to REST is about SOA and Web architecture
    • SOA with the goal of designing services for the Web
    • the most important part is to get the service part right
    • RESTful SOA means properly designing/implementing Web services
 What are Web Services?
Definition: A Web service is a software system designed to support interoperable machine-to-machine interaction over a network. It has an interface described in a machine-processable format (specifically WSDL). Other systems interact with the Web service in a manner prescribed by its description using SOAP messages, typically conveyed using HTTP with an XML serialization in conjunction with other Web-related standards.

Web Services Architecture, W3C Working Group Note, February 11, 2004

 Resource Oriented Architecture (ROA)
  • More concrete guidelines for Web-based implementations
  • Taking services and turning them into RESTful Web services
    • Query string parameters are appropriate if they are inputs to a Resource which is an algorithm
    • Prefer pragmatic uses of putting data into URI, instead of using HTTP Headers
    • RPC-style APIs are avoided in favor of Resources and protocols
    • A representation of a resource should have many links to the other Resources in the application, so that a client can discover state transitions
    • URI templates provide the technology behind specifying families of URI to clients
  • Not really an architecture, more a set of engineering principles
 Web Oriented Architecture (WOA)
  • WOA: Putting the Web back in Web Services
  • Unclear distinction from ROA (maybe no HTTP extensions?)
  • Questionable conceptual landscape
    • REST is the protocol most preferred since it's a natural extension of HTTP for the purposes of sharing self-describing information and state.
 Syndication Oriented Architecture (SynOA)
  • Syndication can be regarded as a pattern for information dissemination
  • and are existing Web standards for syndication
  • SynOA is a pattern for building a SOA architecture
    • services are defined around collections
    • interactions are centered around reading and updating feeds
 Event Oriented Architecture (EOA)
  • Riding the SOA wave for event-oriented systems
  • Now trademarked by eClient
 Conclusions
  • *OA is hard to define and very hype-sensitive
  • SOA lacks well-defined ways of how to define a service
  • Business-level SOA tends to be implemented RPC-oriented/WS-*
  • RESTful SOA is a better route for achieving loose coupling
 Schedule
  • 9.00-10.30: Intro &
  • 11.00-12.30:
  • 14.30-16.00:
  • 16.30-18.00:
What is REST? Erik Wilde UC Berkeley School of Information (PDF) Fielding Dissertation · ACM Transactions Representational State Transfer (REST) is defined as an architectural style, which means that it is not a concrete systems architecture, but instead a set of constraints that are applied when designing a systems architecture. We briefly discuss these constraints, but then focus on explaining how the Web is one such systems architecture that implements REST. In particular, the mechanisms of the Uniform Resource Identifiers (URIs), the Hypertext Transfer Protocol (HTTP), media types, and markup languages such as the Hypertext Markup Language (HTML) and the Extensible Markup Language (XML). We also introduce Atom and the Atom Publishing Protocol (AtomPub) as two established ways on how RESTful services are already provided and used on today's Web. Abstract

Abstraction Layers What is REST?
  • Defining Representational State Transfer: 3 popular definitions
  1. An architectural style for building loosely coupled systems
    • defined by a set of very general constraints (principles)
    • the Web (URI/HTTP/HTML/XML) is an instance of this style
  2. The Web used correctly (i.e., not using the Web as transport)
    • HTTP is built according to RESTful principles
    • services are built on top of Web standards without misusing them
    • most importantly, HTTP is an application protocol (not a transport protocol)
  3. Anything that uses HTTP and XML (XML without SOAP)
    • XML-RPC was the first approach for this
    • violates REST because there is no uniform interface
 What is Architecture?
  • What is the A in SOA?
  • Architecture is constraint-based design
    • constraints are derived from requirements (contextualized requirements)
    • design without constraints probably is art
  • Constraints can be derived from a wide variety of sources
    • technical infrastructure (current landscape and expected developments)
    • business considerations (current landscape and expected developments)
    • time horizon (short-term vs. long-term requirements)
    • existing architecture
    • scalability
    • performance (based on performance requirements and definitions)
    • cost (development, deployment, maintenance)
 Architecture Examples
Architecture vs. Design Architectural Styles
  • Architectural Style vs. Architecture
    • Architectural Style: General principles informing the creation of an architecture
    • Architecture: Designing a solution to a problem according to given constraints
    • Architectural styles inform and guide the creation of architectures
 REST is not an Architecture
  • REST is an architectural style
    • distilled from the Web a posteriori
    • some of the Web's standards and practices are not perfectly RESTful
  • The Web is an information system following REST
  • It is possible to design other RESTful information systems
    • different uniform interfaces (not using HTTP's methods)
    • different representations (not using HTML or XML)
    • different identification (not using URIs)
 SOA is not an Architecture
  • SOA is more a style than an architecture
  • SOA's biggest problem: What is a service?
    • is a service something that is described by RPC-like custom functions?
    • is a service exposed through a uniform interface?
  • OASIS has a SOA Reference Model TC
    • the Reference Model defines a service as a mechanism to enable access to one or more capabilities, where the access is provided using a prescribed interface and is exercised consistent with constraints and policies as specified by the service description.
    • the Reference Architecture describes a WS-* oriented world view
  • SOA can be done RESTfully or not
    • whether a RESTful approach makes sense depends on the constraints
    • if the constraints allow REST, there should be a good reason for ignoring REST
 REST: The Definition The REST Architectural Style
  • A set of constraints that inform an architecture
  • Claims: scalability, mashup-ability, usability, accessibility
 Resource Identification
  • Name everything that you want to talk about
  • Thing in this case should refer to anything
    • products in an online shop
    • categories that are used for grouping products
    • customers that are expected to buy products
    • shopping carts where customers collect products
  • Application state also is represented as a resource
    • next links on multi-page submission processes
    • paged results with URIs identifying following pages
 Uniform Interface
  • The same small set of operations applies to everything
  • A small set of verbs applied to a large set of nouns
  • verbs are universal and not invented on a per-application base
  • if many applications need new verbs, the uniform interface can be extended
  • natural language works in the same way (new verbs rarely enter language)
  • Identify operations that are candidates for optimization
    • GET and HEAD are safe operations
    • PUT and DELETE are idempotent operations
    • POST is the catch-all and can have side-effects
  • Build functionality based on useful properties of these operations
 Self-Describing Messages
  • Resources are abstract entities (they cannot be used per se)
    • guarantees that they are clearly identified
    • they are accessed through a
  • Resources are accessed using resource representations
    • resource representations are sufficient to represent a resource
    • it is communicated which kind of representation is used
    • representation formats can be negotiated between peers
  • Resource representations can be based on different constraints
    • XML and JSON can represent the same model for different users
    • whatever the representation is, it must support links
 Hypermedia Driving Application State
  • Resource representations contain links to identified resources
  • Resources and state can be used by navigating links
    • links make interconnected resources navigable
    • without navigation, identifying new resources is service-specific
  • RESTful applications navigate instead of calling
    • representations contain information about possible traversals
    • the application navigates to the next resource depending on link semantics
    • navigation can be delegated since all links use identifiers
 Stateless Interactions
  • This constraint does not say Stateless Applications!
    • for many RESTful applications, state is an essential part
    • the idea of REST is to avoid long-lasting transactions in applications
  • Statelessness means to move state to clients or resources
    • the most important consequence: avoid state in server-side applications
  • Resource state is managed on the server
    • it is the same for every client working with the service
    • when a client changes resource state other clients see this change as well
  • Client state is managed on the client
    • it is specific for a client and thus has to be maintained by each client
    • it may affect access to server resources, but not the resources themselves
  • Security issues usually are important with client state
    • clients can (try to) cheat by lying about their state
    • keeping client state on the server is expensive (but may be worth the price)
 Web Architecture What is the Web?
  • Web = URI + HTTP + ( HTML | XML )
  • RESTful Web uses HTTP methods as the uniform interface
    • non-RESTful Web uses GET/POST and tunneled RPC calls
    • a different RESTful Web uses Web Distributed Authoring and Versioning (WebDAV)
  • Imagine your application being used in 10 browsers
    • resources to interact with should be identified and linked
    • a user's preferred font size could be modeled as client state
    • what about an access count associated with an API key?
  • Imagine your application being used in 10 browser tabs
    • no difference as long as client state is representation-based
    • cookies are shared across browser windows (different client scope)
 Uniform Resource Identifier (URI) Identifying Resources on the Web
  • Essential for implementing a
  • URIs are human-readable universal identifiers for stuff
    • many identification schemes are not human-readable (binary or hex strings)
    • many RPC-based systems do not have universally identified objects
  • Making every thing a universally unique identified thing is important
    • it removes the necessity to scope non-universal identifiers
    • it allows to talk about all things in exactly the same way
 URI Schemes 
URI = scheme ":" hier-part [ "?" query ] [ "#" fragment ]
  • URIs in their general case are very simple
    • the scheme identifies how resources are identified
    • the identification may be hierarchical or non-hierarchical
  • Many URI schemes are hierarchical
    • it is then possible to use relative URIs such as in a href="../"
    • the slash character is not just a character, in URIs it has semantics
[…] the URI syntax is a federated and extensible naming system wherein each scheme's specification may further restrict the syntax and semantics of identifiers using that scheme.

Uniform Resource Identifier (URI): Generic Syntax, RFC 3986, January 2005

 Query Information
  • Query components specify additional information
    • it is non-hierarchical information further identifying the resource
    • in most cases, it can be regarded as input to the resource
  • Query components often influence caching
    • successful GET/HEAD requests may be cached
    • only cache query string URIs when explicitly requested (Expires/Cache-Control)
The query component contains non-hierarchical data that, along with data in the path component […], serves to identify a resource within the scope of the URI's scheme and naming authority […].

Uniform Resource Identifier (URI): Generic Syntax, RFC 3986, January 2005

 Processing URIs
  • Processing URIs is not as trivial as it may seem
    • escaping and normalization rules are non-trivial
    • many implementations are broken
    • complain about broken implementations
    • even more complicated when processing an Internationalized Resource Identifier (IRI)
  • URIs are not just strings
    • URIs are strings with a considerable set of rules attached to them
    • implementing all these rules is non-trivial
    • implementing all these rules is crucial
    • application development environments provide functions for URI handling
 Hypertext Transfer Protocol (HTTP) How RESTful Applications Talk
  • Essential for implementing a
    • HTTP defines a small set of methods for acting on URI-identified resources
  • Misusing HTTP turns application into non-RESTful applications
    • they lose the capability to be used just by adhering to REST principles
    • it's a bad sign when you think you need an interface description language
  • Extending HTTP turns applications into more specialized RESTful applications
    • may be appropriate when more operations are required
    • seriously reduces the number of potential clients
 HTTP Methods
  • Safe methods can be ignored or repeated without side-effects
    • arithmetically safe: 41 × 1 × 1 × 1 × 1 …
    • in practice, without side-effects means without relevant side-effects
  • Idempotent methods can be repeated without side-effects
    • arithmetically idempotent: 41 × 0 × 0 × 0 × 0 …
    • in practice, without side-effects means without relevant side-effects
  • Unsafe and non-idempotent methods should be treated with care
  • HTTP has two main safe methods: GET HEAD
  • HTTP has two main idempotent methods: PUT DELETE
  • HTTP has one main overload method: POST
 Cookies
  • Cookies are client site state bound to a domain
    • they are convenient because they work without having to use a representation
    • they are inconvenient because they are not embedded in representations
  • Cookies are managed by the client
    • they are shared across browser tabs
    • they are not shared across browsers used by the same user
    • essentially, the client model of cookies is a bit outdated
  • Two major things to look out for when using cookies:
    1. session IDs are application state (i.e., non-resource state)
    2. cookies break the back button (requests contain a URI/cookie combo)
  • The ideal RESTful cookie is never sent to the server
    • cookies as persistent data storage on the client
    • interactions with the server are only using URIs and representations
 Representations Structured Documents What is identified by a URI?
  • Essential for implementing
    • also should provide support for
  • only talks about an abstract resource
    • resources are never exchanged or otherwise processed directly
    • all interactions use resource representations
  • Representations depend on various factors
    • the nature of the resource
    • the capabilities of the server
    • the capabilities or the communications medium
    • the capabilities of the client
    • requirements and constraints from the application scenario
    • negotiations to figure out the best representation
  • Representations are identified by media type (sometimes called MIME type)
 Extensible Markup Language (XML)
  • The language that started it all
    • created as a streamlined version of SGML
    • took over as the first universal language for structured data
  • XML is a metalanguage (a language for representing languages)
    • many domain-specific languages are defined as XML vocabularies
    • some metalanguages use XML syntax (RDF is a popular example)
  • XML is only syntax and has almost zero semantics
    • very minimal built-in semantics (language identification, IDs, relative URIs)
    • semantics are entirely left to the XML vocabularies
  • XML is built around a tree model
    • each XML document is a tree and thus limited in structure
    • RESTful XML introduces hypermedia to turn XML data into a graph
 JavaScript Object Notation (JSON)
  • The XMLHttpRequest API has been built for requesting XML via HTTP
    • this is useful because XML is the most popular data format
    • all requested data has to be processed by using XML access methods in JavaScript
  • JavaScript does not have XML as its internal data model
    • the XML received via XMLHttpRequest has to be parsed into a DOM tree
    • DOM access in JavaScript is inconvenient for complex operations
    • alternatively, the XML can be mapped to JavaScript objects (also requires parsing)
  • JavaScript Object Notation (JSON) encodes data as JavaScript objects
    • more efficient for the consumer if the consumer is written in JavaScript
    • this turns the generally usable XML service into a JavaScript-oriented service
    • for large-scale applications, it might make sense to provide XML and JSON
    • this can be negotiated with HTTP content negotiation
 JSON Example Resource Description Framework (RDF)
  • Developed around the same time as XML was developed
    • based on the idea of machine-readable/understandable semantics
    • builds the Semantic Web as a parallel universe on top of the Web
  • RDF uses URIs for naming things
    • RDF's data model is based on (URI, property, value) triples
    • triples are combined and inference is used to produce a graph
  • RDF is a metalanguage built on the triple-based data model
    • RDF has a number of syntaxes (one of them is XML-based)
    • RDF introduces a number of schema languages (often referred to as ontology languages)
 Atom
  • A language for representing syndication feeds
  • Much more modest in its goal than XML or RDF
    • models feeds as a sets of entries with associated metadata
    • uses an XML vocabulary for representing the data model
    • uses links for expressing relationships in the data model
  • Will be discussed in detail as a good foundation for REST
 Linked Documents Making Resources Navigable
  • Essential for using
  • RPC-oriented systems need to expose the available functions
    • functions are essential for interacting with a service
    • introspection or interface descriptions make functions discoverable
  • RESTful systems use a
    • no need to learn about functions
    • but how to find resources?
    1. find them by following links from other resources
    2. learn about them by using
    3. understand them by recognizing representations
 URI Templates
  • REST does not care about URI details
  • Apart from the scheme, URIs should be semantically opaque
    • media types should not guessed by URI (breaks content negotiation)
    • semantics should not be inferred from inspecting URIs
    • URIs should not be guessed based on previously encountered URIs
  • URI hacking on the Web works and can be useful
    • Firefox Go Up allows easy navigation up one level
    • good URIs and bad UIs sometimes turn the address bar into a useful UI
  • Technically speaking, URI templates are not required by REST
    • practically speaking, URI templates are a useful best practice
    • all URI navigable resources should also be navigable using representations
 State State Management on the Web
  • Essential for supporting
  • are a frequently used mechanism for managing state
    • in many cases used for maintaining session state (login/logout)
    • more convenient than having to embed the state in every representation
    • some Web frameworks switch automatically between cookies and URI rewriting
  • Cookies have two interesting client-side side-effects
    • they are stored persistently independent from any representation
    • they are shared state within the context of one browser
  • Session ID cookies require expensive server-side tracking
    • not associated with any resource and thus potentially global
    • load-balancing must be cookie-sensitive or cookies must be global
  • Resource-based state allows RESTful service extensions
 State in HTML or HTTP State in the Server Application State as a Resource Stateless Shopping
  • Typical session scenarios can be mapped to resources
    • Client: Show me your products
    • Server: Here's a list of all the products
    • Client: I'd like to buy 1 of http://ex.org/product/X, I am "John"/"Password"
    • Server: I've added 1 of http://ex.org/product/X to http://ex.org/users/john/basket
    • Client: I'd like to buy 1 of http://ex.org/product/Y, I am "John"/"Password"
    • Server: I've added 1 of http://ex.org/product/Y to http://ex.org/users/john/basket
    • Client: I don't want http://ex.org/product/X, remove it, I am "John"/"Password"
    • Server: I've removed http://ex.org/product/X to http://ex.org/users/john/basket
    • Client: Okay I'm done, username/password is "John"/"Password"
    • Server: Here is the total cost of the items in http://ex.org/users/john/basket
  • This is more than just renaming session to resource
    • all relevant data is stored persistently on the server
    • the shopping cart's URI can be used by other services for working with its contents
    • instead of hiding the cart in the session, it is exposed as a resource
 Reusing Resources Conclusions
  • REST is simple to learn and use
  • Unlearning RPC in most cases is the hardest part
    • OO is all about identifying classes and methods
    • distributed systems very often are built around RPC models
    • many classical IT architectures are RPC-centric by design
  • REST and RPC do not mix
    • resource orientation ↔ function orientation
    • cooperation ↔ integration
    • openly distributed ↔ hiding distribution
    • coarse-grained ↔ fine-grained
    • complexity in resources formats ↔ complexity in function set
 RESTful Service Design Cesare Pautasso Faculty of Informatics, University of Lugano RESTful Web Services REST is simple to define, but understanding how to apply it to design RESTful services is more difficult. The goal of this part of the tutorial is to present the main design elements of a RESTful architecture and introduce a design methodology for RESTful services. A selection of useful patterns and anti-patterns will be discussed with some examples that will be further developed in the practical part. REST vs. WS-* Comparison Cesare Pautasso Faculty of Informatics, University of Lugano WWW2008 REST vs. WS-* In this part we summarize the WS-* vs. REST debate by giving a quantitative technical comparison based on architectural principles and decisions. We show that the two approaches differ in the number of architectural decisions that must be made and in the number of available alternatives. This discrepancy between freedom-from-choice and freedom-of-choice explains the complexity difference perceived. However, we also show that there are significant differences in the consequences of certain decisions in terms of resulting development and maintenance costs. The comparison helps technical decision makers to assess the two technologies more objectively and select the one that best fits their needs: REST is well suited for basic, ad hoc integration scenarios, WS-* is more flexible and addresses advanced quality of service requirements commonly occurring in enterprise computing. REST in Practice Erik Wilde UC Berkeley School of Information (PDF) Atom · AtomPub · Feed Validator REST's level of abstraction and its simplicity as a small set of constraints can make it hard to get a grasp on how it can be applied for real-world projects. This presentations introduces real-world REST by looking at how REST can be used by reusing existing RESTful designs in terms of representations and interaction protocols; Atom and the Atom Publishing Protocol (AtomPub) are used as examples for existing RESTful designs. In addition, we take a brief look at how to go beyond using these canned REST approaches, and how existing programming framework provide support for designing and implementing RESTful services. Abstract

Precious Snowflakes
  • Many applications are not as unique as you might think
    • reusing design patterns and even technologies is a good idea
    • avoiding reuse usually should be justified by good reasons
  • Take the Web's HTML as an inspiring example
    • HTML is not all that great as a document format
    • it was good enough as a container for a lot of useful content
    • the network effect far outweighs its functional shortcomings
    • could you imagine a Web based on PDF or Word?
  • Simplicity and wide applicability are valuable
    • simplicity means a lower barrier to entry
    • wide applicability means higher chances to create network effects
 RESTful <q>Hello World</q>
  • Internet connectivity establishes HTTP URI addressibility
    • host must be reachable and identifyable (DNS name or at least IP address)
  • HTTP server provides the uniform interface (GET only)
    • by default listens to incoming HTTP requests on port 80
    • in simple scenarios often is configured to (also) serve static files
  • XML and application/xml provide a machine-readable representation
  • Dropping the XML into the file system creates a RESTful Hello World service
    • hello-world.xml
  • Accessible via any HTTP- and XML-capable client
 Feeds!
  • Information dissemination is a complex and varied problem
  • Feeds look at all these problems with a simple approach
    • the simple approach lacks some sophistication/specialization
    • less specialization means wider applicability
    • wider applicability means bigger network effects
    • pull-based syndication is loosely coupled and scalable
  • Some claim that pull publishing is bad and Publish/Subscribe (PubSub) is good
    • there is little proof that this is more than just a claim
    • scalable PubSub is expensive to implement (but still may make sense)
    • PubSub may be a good idea if the requirements rule our REST
 Syndication
  • Atom is a format and also an evolving landscape
    • , the feed format
    • , write access to Atom-oriented collections
    • support for threaded discussions in feeds
    • feed licensing
    • feed paging and archiving
    • many other ongoing developments
  • A landscape of RESTful interactions with collections of things
 Syndication Formats RSS RSS History
  • The Myth of RSS Compatibility provides a good overview
  • RSS is a schoolbook example for why standards are a good thing
    • was created for the My Netscape portal in March 1999
    • RSS 0.91 (a simplification) was introduced in July 1999 (as an interim solution)
    • the AOL/Netscape merger removed the format from the company's portal
    • RSS was without an owner, and different parties claimed/denied ownership
    • was created by an informal developer group
    • RSS 0.92 (and 0.93 and 0.94) were published without acknowledging RSS 1.0
    • finally, was released as a follow-up to the RSS 0.9x versions
  • Using RSS has become an exercise in managing a menagerie of versions
 RSS 2.0
  • RSS now means Really Simple Syndication
    • RSS 2.0 is the continuation of the 0.91 branch (which dropped RDF)
    • together with it is the most popular version of RSS
    • migration from 0.91 to 2.0 is easily possible
  • RSS 2.0 tries to avoid the use of XML Namespaces
  • RSS 2.0 is increasingly used with extensions for vendor-specific information
    • the RSS core is minimal, so many applications need extensions
    • many extensions have overlapping functionality
    • most extensions have unclear semantics and unclear versioning policies
 RSS 2.0 Example Consuming RSS
  • RSS feeds often have quality problems
    • surprisingly often feeds do not even deliver well-formed XML
    • the use of embedded markup in RSS is not well-defined (parse and pray)
  • Writing an RSS reader from scratch is not a good idea
  • There are three major tasks which RSS readers must do
    1. accept non-XML RSS feeds and fix them to be XML
    2. look at the feed contents and bring them into a unified form
    3. produce a unified view of feeds regardless of the RSS version
 RSS Political Problems
  • Multiple and incompatible are still in widespread use
    • and are incompatible by design (RDF vs. non-RDF)
    • none of the RSS versions is maintained by a universally accepted standards body
  • None of the specifications is being updated or fixed
    • some of the lessons learned by RSS deployment are not used in a new version
    • it is unlikely that a new version will be produced which merges the RSS landscape
  • Invent something new instead of trying to fix RSS
    • started in 2003 (called Echo at first)
    • W3C or IETF would have been promising candidates for a new RSS
    • W3C is more formal, IETF is more developer-centered
    • IETF was chosen over W3C because the of Atom community's preferences
 Atom Atom History
  • RSS's shortcomings were very apparent and could not be fixed
  • In mid-2003, discussions started about an improved format
  • It also became apparent that the format should have a protocol
  • Atom 0.3 was released in December 2003 but had no formal home
  • IETF was chosen as the new home with a working group in June 2004
  • RFC 4287 was published in December 2005
  • AtomPub has been published as RFC 5032 in October 2007
 Atom vs. RSS
  • Standardized by the IETF (well-defined process)
  • Classification of entries (user-defined categories)
  • More XML-like markup design (more nesting)
  • Namespaces are used and supported as standard mechanism
  • Atom feeds must be well-formed XML (there even is a schema)
  • Interpretation of content is well-defined (various content types)
  • Support for xml:lang and xml:base
 Atom Example Atom Content
  • RSS had no safe way of finding out what an entry's content is
    • this led to different implementations being smart about what the RSS author really wanted
    • one of Atom's main goals was to improve this in a well-defined way
    • Atom allows escaped markup (the only way to include non-XML HTML in an XML format)
  • Each content element should have a type (the default is text)
  • Atom's content interpretation algorithm (use first applicable rule):
    1. if type is text, no child elements are allowed (plain text content)
    2. if type is html then RSS's method of escaped markup is used
    3. if type is xhtml then there must be an div containing XHTML markup
    4. if type is an XML media type then the content should be treated as this type
    5. if type starts with text/ then no child elements are allowed
    6. for all other values, the content must be an base64-encoded entity of the specified MIME type
 Atom Content Examples 

  

    One bold foot forward
  

]]>
The "atom:content" element either contains or links to the content of the entry. The content of atom:content is Language-Sensitive.]]>
The <code>atom:content</code> element either contains or links to the content of the entry. The content of <code>atom:content</code> is <a href="http://www.ietf.org/rfc/rfc3066.txt">Language-Sensitive</a>.]]>

iVBORw0KGgoA … TAAAAAElFTkSuQmCC
]]>
]]>
 Atom Categories
  • Atom allows to assign categories to entries
    • each category element must have a term attribute for the category
    • an optional scheme identifies the categorization scheme (ontology, taxonomy, …)
    • an optional label attribute provides a human-readable label for the category
  • AtomPub defines a document format for
  • Three different cases of categorization can be distinguished
    1. use a well-known scheme (such as Dublin Core)
    2. use a private but well-designed scheme (which has a URI and can be reused reliably)
    3. use tags without schemes, which then are little more than content labels
 Switching from RSS to Atom
  • Generate both feeds but serve RSS with an HTTP redirect (301)
    • old subscribers with broken clients can still use the RSS feed
    • old subscribers with correct clients will use the Atom feed
  • Atom exposes more information than RSS (category for tags)
    • the mapping of publishing info to the feed has to be changed/extended
    • for standard metadata use Atom's built-in metadata elements
    • for application-specific metadata consider reusing an existing metadata schema
  • Atom can be used to publish snippets as well as full content
    • content allows any type of content to be used and may contain a complete entry
    • summary allows only text and should provide a condensed version of an entry
    • some Atom sources publish two feeds for summaries and content
  • Generate good Atom and downgrade it to RSS 1.0 & 2.0
 Syndication Aggregation End-User Aggregation
  • Users often have a small number of preferred Web sites
    • news can be tracked by subscribing to their feeds
    • this requires an RSS-aware client (Firefox is not a good RSS reader)
  • How can users find a feed?
    • feeds have URIs (they are Web resources) and can be referenced from within HTML
<link rel="alternate" type="application/rdf+xml" title="…" href="…" />
<link rel="alternate" type="application/rss+xml" title="…" href="…" />
<link rel="alternate" type="application/atom+xml" title="…" href="…" />
 Aggregation Intermediaries
  • RSS is a frequently used format between content providers
    • news agencies want to distribute news items as easy as possible
    • by using a single format, producers and consumers can more easily interoperate
  • The caused publishers to look for something better
    • had a head start and still is widely used
    • is a much better format and will eventually replace RSS
  • User-configured portals are the merger between both scenarios
    1. users select a number of feeds as their preferred information sources
    2. the feeds are presented on a single personalized Web page
    3. by sharing and re-publishing items, users are becoming aggregation intermediaries
 FeedBurner Fixing Feeds Load Balancing Statistics/Analytics Query Capabilities
  • Feed technology is still evolving
  • Feeds are mostly viewed as ordered by time
    • allows optimization for accesses and caching
    • makes it hard to use feeds for non-timed information
  • Feeds could be ordered by any sort key
    • makes server-side feed processing much more expensive
    • enables customized feeds that are processed on the server-side
 Supporting Queryable Feeds Atom Publishing Protocol (AtomPub) RESTful Syndication
  • Atom is a format for retrieving a set of entries as a feed document
    • feeds often are time-based and are refreshed periodically or whenever needed
    • feeds can use any other strategy for deciding what to publish
  • Read-only access to feeds should be complemented by full access
    • full access needs the CUD out of the CRUD set of operations
    • many Web-centric technologies try to build on the Web's REST model of interaction
  • AtomPub builds on Atom and adds a RESTful protocol on top of it
    • POST for creating new entries (sending the request to the collection)
    • PUT for updating existing entries (overwriting the existing entry)
    • DELETE for deleting entries from a collection
 Collections, Members, Entries, Media
  • AtomPub's top-level concept is a collection
    • collections are used for managing and organizing members
    • Atom feed documents are the representation of collections
    • collections just exist; there is no way of interacting with them
  • Members of a collection can be entry and media resources
    • entry resources represent metadata and are represented as Atom entries
    • media resources can have any media type and are the data described by entries
    • a media link entry is an entry associated with a member
 Atom/AtomPub Data Model Protocol Summary
Resource HTTP Method Representation Description
Introspection GET Atom Service Document Enumerates a set of collections and lists their URIs and other information about the collections
Collection GET Atom Feed A list of member of the collection (this may be a subset of all entries in the collection)
Collection POST Atom Entry Create a new entry in the collection
Member GET Atom Entry Get the Atom Entry
Member PUT Atom Entry Update the Atom Entry
Member DELETE n/a Delete the Atom Entry from the collection
Collection Management
  • Managing collections is outside the scope of AtomPub
    • AtomPub allows interactions with existing collections
    • Collection management is inaccessible or using a proprietary protocol
  • Metacollections (collections of collections) can extend AtomPub
    • each member in a metacollection corresponds with a collection
    • accessing a metacollection means managing collections
    • AtomPub's existing features can be reused for collection management
  • Ongoing discussion in the Atom community
    • currently there is no widely established method
 Service Documents
Service Documents represent server-defined groups of Collections, and are used to initialize the process of creating and editing resources.
  • The real top-level construct of AtomPub is the workspace
    • collections on a server are organized into different workspaces
    • workspaces have no AtomPub semantics and no operations can be performed on them
  • Service documents list constraints on the members of collections
    • accept specifies a comma-separated list of media ranges (with entry as special value)
    • categories defines the list of categories that can be applied to members (can be fixed)
    • AtomPub servers are likely to reject operations not satisfying these constraints
 Service Document Example Category Documents
  • Categories are important for creating and reading entries
    • they may contain metadata using any classification scheme
  • contain a list of allowed categories
  • AtomPub defines a document format for standalone category documents
    • a useful interface between AtomPub systems and other systems using classification schemes
    • REST makes it important to make relevant resources accessible
 Category Document Example Extending Atom/AtomPub Atom/AtomPub as Foundation
  • Atom/AtomPub can be used as a foundation
    • apply the methodology of
    • identify missing representations/links/states
    • add these to the Atom/AtomPub model
  • Atom has a well-defined extension model (sort of)
    • foreign markup must be silently ignored by Atom processors
    • foreign markup is not allowed to change the semantics of Atom markup
    • every Atom extension must be well-behaving with plain Atom processors
  • Atom can serve as a good example for extensible markup
 RESTful Web Service Design
  • URIs are used for resource identification
  • HTTP's methods are used as the uniform interface
  • Representations can use any syntax and structure
  • Hypermedia is based on links in representations
  • Use transaction resources to create stateless interactions
 RESTful Web Service Design Procedure
  1. Figure out the data set
  2. Split the data set into resources
  3. Name the resources with URIs
  4. Expose a subset of the uniform interface
  5. Design the representation(s) accepted from the client
  6. Design the representation(s) served to the client
  7. Design hypermedia integration with other resources
  8. Figure out the normal control flow
  9. Figure out error conditions
  • (This is the ROA methodology)
 Watch Caching
  • Decent HTTP libraries have built-in caching
    • caching can also be implemented by proxies or reverse proxies
  • HTTP has various ways to control caching
    • Expires set an expiration data on a response
    • Cache-Control for various cache control directives
    • ETag for uniquely identifying a resource version
  • Caching allows Conditional GETs
    • If-Modified-Since is conditional based on the Last-Modified date
    • If-None-Match is conditional based on the ETag label
 REST Tools Toolboxes and Frameworks
  • Toolboxes contain a number of low-level tools
    • tools for essential functionality (HTTP, URI, XML)
    • add-ons for frequent tasks (HTML tidying, XML validation, language tag parsing)
    • diagnostic and test tools (HTTP monitoring, protocol validators)
  • Frameworks provide programming/structuring support
    • frameworks encourage and support a certain coding style and structure
    • REST focuses on exposing uniform interfaces and various media types
    • code should be structured by representation, not by function
 REST Frameworks
  • JAX-RS
  • RESTlet (covers any REST, not just RESTful HTTP)
  • Ruby on Rails
  • Django
  • GData
 Poster