3 http://dret.net/lectures/services-spring08/ SIMS INFO 290-4 2006 F 2008-01-22 2008-05-06 Professor EW Erik Wilde dret@berkeley.edu Office +1-510-6432253 http://dret.net/netdret/ LG

Web-Based Services are services which provide access to information resources based on Web technologies. Web-based services are an ideal platform on which Web-Based Publishing can build, using the information resource access to provide interfaces for various platforms. The more common term Web Services is often used for the rather restricted set of technologies based on WSDL, SOAP and UDDI. This course focuses on a less middleware-oriented approach to Web-based services, and instead focuses on foundations such as the Hypertext Transfer Protocol (HTTP) for communications and Representational State Transfer (REST) as the architectural principle. The goal of this course is to look at Web-based services with a truly Web-like perspective, emphasizing simple and accessible technologies.

LEC Tu 08:30:00 10:30:00 202 South Hall 2008-01-01 dret 2008-01-22 XML · Web Web-Based Services are services which provide access to information resources based on Web technologies. Web-based services are an ideal platform on which Web-Based Publishing can build, using the information resource access to provide interfaces for various platforms. The more common term Web Services is often used for the rather restricted set of technologies based on WSDL, SOAP and UDDI. This course focuses on a less middleware-oriented approach to Web-based services, and instead focuses on foundations such as the Hypertext Transfer Protocol (HTTP) for communications and Representational State Transfer (REST) as the architectural principle. The goal of this course is to look at Web-based services with a truly Web-like perspective, emphasizing simple and accessible technologies.

• Projects in the fields , , and
• Regular meeting time reserved for project work
• presentations about related work (research, projects, products)
• presentations about project plans and issues
• discussion of project plans and issues
• One fixed meeting slot per group per week
• discussion of project plans and issues
• assignment of presentations
• Overall workload should be 6-8h per week

• Final
• presentations which briefly present the project results
• reports which describe the project and its results in greater detail
• Grade is based on:
• 30% presentations during the semester
• 30% final project presentation & report
• 40% project results

• Place Markup Language (PlaceML) for describing locations
• places can be described geographically (coordinates)
• places can be described with metadata (descriptions)
• place descriptions can be structured
• What services can use place descriptions, and how?
• meeting services (alert me when friends are nearby)
• location annotations (leave messages at places)
• Some of these services don't even need locations
• if users share a location vocabulary, they know where a place is
• the evolution of places from coordinates to social constructs

• Data linking on the Web is rudimentary
• HTML has some primitive support in the document head
• microformats describe special-purpose links (e.g., to a license)
• The Semantic Web has stifled the development of the plain Web
• anything that can be done on the Web can be done on the Semantic Web
• adoption is still non-existent outside of closed communities
• Links are good, following links is good
• XLink is the Web's generalized linking language
• XLink is badly designed, underspecified, and unpopular
• a new link format must be accompanied by an access protocol

• Feeds are widely used to publish and reuse itemized information
• RSS and Atom provide formats for publishing feeds
• AtomPub is a format for providing write access to a feed
• Mapping itemized services to AtomPub is useful
• existing protocol with interactions for remote management
• easy reuse of deployed technologies such as Podcasts
• Internet Archive currently has idiosyncratic upload support
• rsync for syncing complete file systems
• FTP for uploading items in a one-by-one process
• HTML form for upload which also only allows one-by-one upload

• Applications should ship data, not code
• REST-based back-ends can support any kind of interface
• shipping code is very client-specific and should be avoided
• Applications should focus on the service perspective
• how to deliver a service that can be easily used by others
• how to describe the service's data model for potential users
• how to control access and authenticate service users
• If possible, other services should be integrated
• if you need authentication, why not use OpenID?
• if you need social networking, why not use OpenSocial?
• if you need location, why not use Fireeagle?

• Rich Internet Applications are based on the idea of a smarter client
• Ajax applications ship JavaScript code for modern browsers
• Flash applications ship ActionScript code for flash players
• ActiveX applications ship Windows components for IE
• RIAs should be built on top of a RESTful service
• standard services will provide XML-based interactions
• Ajax-specific services may provide JSON representations
• Service and RIA should be completely separated
• if required, the service can be provided as an API
• if required, alternative UIs (e.g., purely Web-based) can be implemented

• Computer Science at Technical University of Berlin (TUB) (88-91)
• Ph.D. at ETH Zürich (92-97)
• Post-Doc at ICSI, Berkeley (97/98)
• book on Technical Foundations of the World Wide Web
• Various activities back in Switzerland (98-06)
• teaching at ETH Zürich and FHNW
• working as independent consultant (training, courses, consulting)
• research in various XML-related areas
• Visiting Assistant Professor at the School of Information (since fall 2006)
• Web and XML courses in fall 2007
• technical director of the Information and Service Design (ISD) program

• Course Web page: http://dret.net/lectures/services-spring08/
• Course mailing list: subscribe at majordomo@ischool.berkeley.edu
• no subject (leave blank)
• body of message: subscribe wbs
• Letter grade based on presentations and course project (results and presentation)

• Generated from XSLidy XML
• Designed for online presentation and use (lots of links!)
• for printing, use a (all slides), and then s (smaller font) a couple of times
• A good real-world example for Web-based publishing
• Slidy is useful, but there is no support for structures and hyperlinking
• XSLidy adds these features by adding an XSLT transformation
• XSLidy is useful, but there is no interface (XML editing only)

• My Online Glossary at http://dret.net/glossary/
• suggestions, updates, corrections are very welcome (really!)
• another exercise in how to use XML and XSLT for information management
• My bibliography at http://dret.net/biblio/
• suggestions, updates, corrections are very welcome (really!)
• produced by an XML-centric system for managing bibliography data
• The World Wide Web Consortium (W3C)
• headed by Tim-Berners Lee, inventor of the Web (with Robert Cailliau)
• The Internet Engineering Task Force (IETF)
• mainly Internet standards, but also responsible for URIs and HTTP

• Create a project description until next week
1. context, goals, and non-goals of the project
2. deliverables in one-month phases (i.e., 3 phases)
3. your project will be graded based on these goals!
• Next week we will discuss and assign projects
• If you don't have a project idea, just ask me …

2008-01-29 Projects are being assigned this week. The three project areas are Location, Linked Data, and Feed Technology. Each of these areas has data model and service components. In all project areas this week is dedicated to learning more about the project environment, and existing technologies and approaches. This week's work will allow students to give initial presentations about their project next week, serving as a starting point for further work and possible collaborations among projects.

• Locations and location-based services are popular themes
• Most applications are based on closed scenarios
• location information is made available only to special providers (mobile scenario)
• location information is only usable within one silo
• Tasks for first week:
• develop scenarios for sharing location information across services
• how good is Fire Eagle for this? (centralized)
• start collecting map URIs as a test dataset for mapping map URIs
• Next tuesday:
• present the PlaceML data model and possible services around it
• present some first ideas about location services and social networking
• Next goals:
• Services for locations and places without being restricted to geolocation

• Map services support map bookmarks by using URI UIs
• What about using these as URI APIs?
• There is no standard for map URIs
• there is a pretty small common set of parameters for controlling a map
• creating a data model for these parameters formalizes the API
• Tasks for first week:
• take the existing Map Mapper Map and add new map services
• maybe add new parameters to the data model (if required)
• maybe improve the mapping process to be more flexible and powerful
• Next tuesday:
• present the updated map and its model
• point to challenges and problems which need to be solved
• Next goals:
• collecting enough URIs and services to define a robust mapping scheme

• Many resources on the Web are actually compound resources
• HTML has rudimentary support for expressing resource relationships
• some connections are explicit (img, object, CSS URIs …)
• others are implicit and require understanding (a)
• many are hidden deep in scripting code and maybe even dynamically loaded data (Ajax)
• link sometimes helps but is not very popular
• Various vocabularies have been developed to represent links
• XLink, METS, DIDL, OAI-ORE, …
• so far, none of these has been successful on the Web
• Tasks for this week:
• compile a complete dictionary of all HTML linking constructs
• work on a list of issues which are hard or impossible to solve
• Next tuesday:
• present all ways in which HTML & CSS connect resources
• present a first rough outline of an algorithm for identifying compound resources
• Next goals:
• tweak the algorithm to be configurable so that it can be customized as a harvester

• Links (such as information about ) are discrete resources
• Atom and AtomPub have been designed to manage collections
• Instead of just serving content, a link feed could serve context
• Tasks for this week:
• read AtomPub and look at the Atom landscape
• think about how AtomPub could be extended to be timeless
• Next tuesday:
• brief presentation of the issues with AtomPub in this scenarios
• making the connection between and link feeds
• Next goals:
• more research in AtomPub extensibility

• Archiving of (more or less) any kind of data
• rsync for syncing complete file systems
• FTP for uploading items in a one-by-one process
• HTML form for upload which also only allows one-by-one upload
• Tasks for the first week:
• inventory of currently supported interactions for content providers
• inventory of currently supported interactions for content consumers
• start work on lists of possible improvements (focusing on content, not UI)
• read AtomPub and look at the Atom landscape
• Next tuesday:
• presentation of data flows in and out of the Internet Archive
• brief overlook of how Atom and AtomPub fit into that picture
• Next goals:
• would it be possible to build a mod_atom interface for the archive?

• Introductory reading and some first experiments
• Your (last) chance to choose the direction in which you are going
• My goals and interests are not necessarily yours …

2008-02-05 This week has a short introduction to Mantis, the issue tracking system that will be used for the course projects. Following this brief introduction, course projects will be discussed.

• Often used as a bug tracking system for software development
• Issues are anything which needs resolution
• bugs in an existing implementation
• requests for a feature which should be added
• ideas for future features which should be captured
• Issue tracking is not chatting
• please do not have conversations using the system
• each issue and each note should have substance to it
• Try to resolve issues to the point where they can be closed

• Users get a login and have access to the application
• Each installation can have multiple projects
• users have access to the projects which are public or open to them
• Each project has a list of issues
• the category describes which part of a project an issue is for
• the severity describes how important the issue is
• the status tracks an issue through its lifecycle (until it is closed)
• Categories identify well-defined parts of a project
• issues entered for a category are assigned to a default user
• issue assignment can be changed manually at any time
• Adding notes to an issue can be done in two ways
• just adding a note without changing its status
• changing its status and then adding the note which describes the status change
• Issues can be monitored to get emails about activities

2008-02-26 Identifying Atom Spec · Validator · Atom Landscape For many information sources on the Web, it is useful to have some standardized way of subscribing to information updates. Syndication formats such as RSS and Atom can be used by these information sources to publish a feed of updated information items. The basic syndication mechanism describes a data format for publishing entry metadata and entries. In addition to the basic format, services might also support additional functionality, such as the ability to indicate that clients may page through a feed, if it contains more entries than published on the first page.

• Early Web content was static or updated very infrequently
• there was not yet the requirement to reuse content in different contexts
• Frequently updated Web content quickly became a very common scenario
• as commercial interests took over, users should have a reason to re-visit a site
• presenting a steady stream of new content creates the image of a live Web site
• There are two major use cases where HTML is not sufficient
1. users want an efficient way to get the updated content from a site
2. sites want to aggregate updated content from other sites and re-publish it
• are designed to support these two use cases
• container formats for updated items
• a small amount of metadata about these items for automated processing

• The Myth of RSS Compatibility provides a good overview
• RSS is a schoolbook example for why standards are a good thing
• was created for the My Netscape portal in March 1999
• RSS 0.91 (a simplification) was introduced in July 1999 (as an interim solution)
• the AOL/Netscape merger removed the format from the company's portal
• RSS was without an owner, and different parties claimed/denied ownership
• was created by an informal developer group
• RSS 0.92 (and 0.93 and 0.94) were published without acknowledging RSS 1.0
• finally, was released as a follow-up to the RSS 0.9x versions
• Using RSS has become an exercise in managing a menagerie of versions

• RSS means RDF Site Summary (or Rich Site Summary?)
• based on an RDF draft and not compatible with the final RDF specification
• RDF was considered too cumbersome and unstable
• 0.90 (proto-RDF) was quickly replaced by the non-RDF 0.91 version
• RSS 0.92+ versions were developed as unilateral specifications
• starting with RSS 0.91, RSS means Rich Site Summary
• it is no longer built on RDF, instead it simply uses XML
• the 0.9x branch eventually was renamed to

• RSS means RDF Site Summary (this time for real)
• based on the final RDF specification and thus incompatible with any
• developed when the Semantic Web and RDF were first heavily marketed (1999)
• RDF was expected to become the format for metadata on the Web
• RSS 1.0 makes heavy use of XML Namespaces
• RSS 1.0 introduces features which were not present in 0.91
• date information for published items (very relevant for news feeds)
• individual authors for various items in a feed
• RSS 1.0 is the latest version of RDF-based RSS
• the Semantic Web wave is not over yet, but RDF has lost its novelty appeal
• for a more XML-oriented encoding, provides a better foundation

• RSS now means Really Simple Syndication
• RSS 2.0 is the continuation of the 0.91 branch (which dropped RDF)
• together with it is the most popular version of RSS
• migration from 0.91 to 2.0 is easily possible
• RSS 2.0 tries to avoid the use of XML Namespaces
• RSS 2.0 is increasingly used with extensions for vendor-specific information
• the RSS core is minimal, so many applications need extensions
• many extensions have overlapping functionality
• most extensions have unclear semantics and unclear versioning policies
• Apple's podcasts are probably among the most popular extensions

• RSS is rarely produced by hand
• by definition, RSS contains redundant information for a specific purpose
• If a Content Management System (CMS) is used, RSS can be generated
• basic metadata can be generated by the CMS (title, author, date)
• better tagging of content results in better tagging of feeds
• well-tagged feeds are better foundations for large-scale reuse of feed items
• Blogging is simply a specialized case of a CMS
• Web-based interface for controlling everything
• strictly time-ordered sequenced of published items
• navigation features primarily based on the time-specific facets of the blog (maybe tags)
• all blogging tools include feed support

• RSS feeds often have quality problems
• surprisingly often feeds do not even deliver well-formed XML
• the use of embedded markup in RSS is not well-defined
• Writing an RSS reader from scratch is not a good idea
• There are three major tasks which RSS readers must do
1. accept non-XML RSS feeds and fix them to be XML
2. look at the feed contents and bring them into a unified form
3. produce a unified view of feeds regardless of the RSS version

• What to put into an item's description
• the fundamental question is whether a description is text or HTML
• HTML is often used in a non-XML way (omitted end-tags)
• interpretation is client-specific if there is no well-defined indication

<description>This is a post<br>on two lines</description>

<description>This is a post&lt;br>on two lines</description>

<description>This is a post about <br> in RSS feeds</description>

<description>This is a post about &lt;br> in RSS feeds</description>

<description>This is a post about &amp;lt;br> in RSS feeds</description>

• Underspecified and not very robust in various other areas
• broken RSS is accepted by most readers (and fixing it can change the interpretation)
• the interpretation of relative URIs is not mentioned in the specifications
• some minimal semantics (classification) for items would be very useful

• Multiple and incompatible RSS versions are still in widespread use
• and are incompatible by design (RDF vs. non-RDF)
• None of the specifications is being updated or fixed
• none of the RSS versions is maintained by a universally accepted standards body
• some of the lessons learned by RSS deployment are not used in a new version
• it is unlikely that a new version will be produced which merges the RSS landscape
• Invent something new instead of trying to fix RSS
• Atom started in 2003 (called Echo at first)
• W3C or IETF would have been promising candidates for a new RSS
• W3C is more formal, IETF is more developer-centered
• IETF was chosen over W3C because the of Atom community's preferences

• RSS's shortcomings were very apparent and could not be fixed
• In mid-2003, discussions started about an improved format
• It also became apparent that the format should have a protocol
• Atom 0.3 was released in December 2003 but had no formal home
• IETF was chosen as the new home with a working group in June 2004
• RFC 4287 (Atom) was published in December 2005
• AtomPub has been published as RFC 5023 in October 2007

• Standardized by the IETF (well-defined process)
• Classification of entries (user-defined categories)
• More XML-like markup design (more nesting)
• Namespaces are used and supported as standard mechanism
• Atom feeds must be well-formed XML (there even is a schema)
• Interpretation of content is well-defined (various content types)
• Support for xml:lang and xml:base

• RSS had no well-defined way of finding out what an entry's content is
• implementations use smart ways to figure out what the RSS author really wanted
• one of Atom's main goals was to improve this in a well-defined way
• Atom allows escaped markup (the only way to include non-XML HTML in an XML format)
• Each content element should have a type (the default is text)
• Atom's content interpretation algorithm (use first applicable rule):
1. if type is text, no child elements are allowed (plain text content)
2. if type is html then RSS's method of escaped markup is used
3. if type is xhtml then there must be an div containing XHTML markup
4. if type is an XML MIME type then the content should be treated as this type
5. if type starts with text/ then no child elements are allowed
6. for all other values, the content must be an base64-encoded entity of the specified MIME type

  

    One bold foot forward
  
]]>

The "atom:content" element either contains or links to the content of the entry. The content of atom:content is Language-Sensitive.]]>

The <code>atom:content</code> element either contains or links to the content of the entry. The content of <code>atom:content</code> is <a href="http://www.ietf.org/rfc/rfc3066.txt">Language-Sensitive</a>.]]>

iVBORw0KGgoA … TAAAAAElFTkSuQmCC
]]>

]]>

• Atom allows to assign categories to entries
• each category element must have a term attribute for the category
• an optional scheme identifies the categorization scheme (by URI)
• an optional label attribute provides a human-readable label for the category
• AtomPub defines a document format for Category Documents
• Three different cases of categorization can be distinguished
1. use a well-known scheme (such as Dublin Core)
2. use a private but well-designed scheme (which has a URI and can be reused reliably)
3. use tags without schemes
• Tags can be handled in different ways
• leave the scheme empty which indicates that there is no scheme
• use some special scheme for referring to a tagging scheme

• Generate both feeds but serve RSS with a HTTP redirect (301)
• old subscribers with broken clients can still use the RSS feed
• old subscribers with correct clients will start using the Atom feed
• Atom exposes more information than RSS (category for tags)
• the mapping of publishing info to the feed has to be changed/extended
• for standard metadata use Atom's built-in metadata elements
• for application-specific metadata consider reusing an existing metadata schema
• Atom can be used to publish snippets as well as full content
• content allows any type of content to be used and may contain a complete entry
• summary allows only text and should provide a condensed version of an entry
• some Atom sources publish two feeds (full content and some excerpt longer than the summary)
• Generate good Atom and downgrade it to RSS 1.0 & 2.0

• Users often have a small number of preferred Web sites
• news can be tracked by subscribing to their feeds
• this requires an RSS-aware client (Firefox is not a good RSS reader)
• Ajax-based RSS could be implemented as a client-side application
• retrieve the feed, transform it to HTML, and render the result
• but the security restriction of XMLHttpRequest gets into the way
• there is a way around this as used in the Ajax RSS reader
• How can users find a feed?
• feeds have URIs (they are Web resources) and can be referenced from within HTML

<link rel="alternate" type="application/rdf+xml" title="…" href="…" />
<link rel="alternate" type="application/rss+xml" title="…" href="…" />

<link rel="alternate" type="application/atom+xml" title="…" href="…" />

• RSS is a frequently used format between content providers
• news agencies want to distribute news items as easy as possible
• by using a single format, producers and consumers can more easily interoperate
• The caused publishers to look for something better
• had a head start and still is widely used
• Atom is a much better format and will eventually replace RSS
• User-configured portals are the merger between both scenarios
1. users select a number of feeds as their preferred information sources
2. the feeds are presented on a single personalized Web page
3. by sharing and re-publishing items, users are becoming aggregation intermediaries

• Atom's goal is to support the core of syndication metadata
• The Atom feed format is extremely simple
• feed, entry, and content are the container elements
• Atom's metadata model is a bit more exhaustive
• author, category, contributor, generator, icon, id, link, logo, published, rights, source, subtitle, summary, title, updated
• Additional semantics can use Atom's extension model
• the challenge is to identify useful units of functionality
• some extensions have already been standardized
• A starting point for searching for extensions is IANA's Atom Link Relations registry

• Mailing lists and newsgroups are collections of resources
• Threaded discussions create resource relationships
• resources may be in reply to other resources
• replies to a resource may be available elsewhere (maybe in a feed)
• it might be interesting to know the count and updated timestamp of those replies
• it might be interesting to know the total number of replies to an entry
• RFC 4685 defines Atom extensions for threaded discussions
• Reading mailing lists archives with RFC 4685 support is much easier
• it is not necessary to access the complete set of entries
• building an Atom-based reader for news or email is greatly simplified
• the approximations of counts and times is not a serious problem

• Atom's rights element is a text-based element
• requires the text to be repeated in every single entry
• explicitly states it SHOULD NOT be used to convey machine-readable licensing information
• Machine-readable licenses need identity of licenses
• adding a link to a license uses a URI and allows to recognize the URI
• Atom's basic link relations do not include license semantics
• RFC 4946 defines a new link relationship for links to licenses
• License links can be added to Atom feeds or entries
• a new license link relationship is defined
• licenses can be recognized by comparing links to well-known URIs
• there is no universally standardized format for a license description format

• Feeds often are used to publish varying or changing collections of entries
• complete feeds contain a list of all entries and may change completely
• paged feeds are split across multiple documents and are not necessarily stable
• archived feeds are split but are stable over time and allow repeated access
• RFC 5005 defines Atom extensions for defining these types of feeds
• by specifying the type of a feed, clients can better adapt their behavior
• plain Atom clients are still able to read these feeds, but have less information about them
• Complete feeds contain all entries of a feed (top twenty blog posts)
• Paged feeds are a set of linked feeds that constitute the logical feed
• first, last, previous, and next link relationships
• there must be at least one of these relationships in each feed document
• Archived feeds are stable feed documents available from a subscription document
• prev-archive, next-archive, and current link relationships
• subscription documents must link to archives to make archived entries available

• Syndication creates representations for a few universal concepts
• Atom adds some concepts to RSS's model
• Syndication revolves around the idea of interacting with items
• Atom-based interaction is one way of implementing REST
• For more semantics, Atom is only the foundation

2008-05-06 ...