Creative Commons License

This work is licensed under a CC
Attribution 3.0 Unported License

 <a href="./" title="Course Homepage">Web Architecture</a><br/>Fall 2009 — INFO 290 (CCN 42593) Erik Wilde UC Berkeley School of Information Fall Semester 2009 2009 Erik Wilde
Date Subject Slides Required Reading Additional Resources Assignments
: (* Slides)
Overview and Introduction 2009-08-27 This introductory lecture gives the motivation for the course, some information about the people involved and the organization of the course, a high-level overview of the course's topics, and an overview of the assignments which are an important part of the course program. Abstract

Building Things That Actually Work
  • Building things …
    • building is designing (with a lot of implicit design decisions)
    • building things uncovers an essential set of constraints
    • changing things should not be not too hard (if they are well-designed)
  • … that actually work
    • work means more than look good for the final demo
    • how adaptable is an application to a changing environment?
    • how easy is it to integrate new input and output channels?
    • how easy can it be extended to meet new requirements?
  • System design is (an important) part of design
  • Start building things as early as possible
 What is Architecture?
Architecture vs. Design What is an Architect?
  • Star Architects are not typical
    • they sell brand names and deliver high profile results
    • most architects are more modest and less visible
  • Architects must understand how things work
    • a reasonable understanding of the disciplines involved
    • an excellent understanding of how disciplines interact
    • negotiating between specialists for a good overall design
  • Architects are guides
    • they provide guidance for going in the right direction
    • they can tell why this direction is the right direction
    • they can explain why a wrong direction is wrong
 How to become a Web Architect?
  • Understand Web technologies and their dependencies
    • no need to become an expert in all of the areas
    • the important part is understanding the dependencies
  • Understand how to compare application architectures
    • there is no best solution for any given problem
    • every solution must be evaluated in terms of various constraints
  • Next steps for your career in Web architecture
 Course Setup
  • Broad overview of core Web technologies
    • this is not a Web design or Web programming course
  • Assignments working with various Web technologies
    • how to deliver client-specific content
    • how to design client-specific styles
    • using Ajax for creating more dynamic Web pages
    • repurposing existing content for syndication
  • Grading is based on assignments and final exam (30min oral)
    • the assignments cover most of the topics in the course
    • all assignments have an optional part for extra credit
 Motivation Closed World Assumption
If the only tool you have is a hammer, you tend to see every problem as a nail.

Abraham Maslow

  • People, such as content creators, typically are lazy
    • developing content and code for diverse users and clients is hard
    • by making assumptions, this job can become considerably easier
  • Tools often hide complexity and/or take away freedom
    • they are good if tool users know what they are doing
    • tool users should know alternatives and when to switch tools
  • Tool makers provide support for lazy people
    • built-in simplifications of the tool's target technology
    • pre-packaged excuses why it is appropriate to use the tool
 Your Tax $ @ Work
  • How to apply for NSF grants
  • Over $400 billion in grants each year
  • PureEdge is required as the technology to fill out grant forms
  • All of this probably looked nice for the final demo …
  • Web forms do not provide all the features required by the specification
    • offline editing of applications is not possible, Web forms work online only
    • there is no built-in signing of form contents, but there are technologies for it
 Version 1: Go IE or Go Home
  • Well, but not if you are using Vista …
  • PureEdge is an IE plug-in for filling out forms online and offline
    • plug-ins are specific for the browser for which they are developed
    • plug-ins are specific for the OS on which they run
 Version 2: Buy a Virtual Computer
  • Government authorities are (usually) concerned about accessibility
    • restricting $400 billion of grant money to IE users only seems a bit restrictive
    • is there a reasonable argument to be made for this restriction
  • Grants.gov recommended to get a virtual PC to access the portal
    • users have to buy virtual PC software
    • users have to buy Windows to run on the virtual PC
 Version 3: Use a Virtual Computer
  • Grants.gov set up a Citrix server for grant applicants
    • Citrix server licenses are not cheap to buy
    • applicants still have to install the Citrix client (which is free)
    • running a Citrix server farm is pretty expensive
 Version 4: Crash your Computer
  • After some time, PureEdge for Mac was released, features include:
    • occasional crashes and subsequent loss of any unsaved data
    • inability to run on Mac OS version prior to 10.4.6
 Classical Lock-In
  • Companies usually sell products, not just solutions
  • Lock-in happens quickly and is hard to escape from later
  • Lock-in usually carries a pretty high price tag
  • Lock-in solutions can be good, but it is an important decision
  • Standards-based solutions may lack some sophistication
    • but often they may still be good enough to solve a problem
    • being able to change the platform easily is a valuable asset
 Varia About Me About this Course About these Slides
  • Generated from Hotspot XML
  • Designed for online presentation and use (lots of links!)
    • Firefox Go Up allows easy navigation up one level
    • Firefox Site Navigation Bar supports navigation of links
    • Firefox Link Widgets requires a bit more configuration (more flexibility)
    • for printing, use a (all slides), and then s (smaller font) a couple of times
  • A good real-world example for Web-based publishing
    • Slidy/Kilauea is useful, but there is no support for structures and hyperlinking
    • Hotspot adds these features by adding an XSLT transformation
    • Hotspot is useful, but there is no interface (XML editing only)
 Additional Resources Web Browsers 2009-09-01 Wikipedia · History · YouTube Firefox · Safari · IE · Chrome · Opera This lecture looks at Web browsers and how they work. It introduces the basic functionalities of a browser; retrieval and rendering of Web pages. Any modern browser needs to support more than just HTTP and HTML; it must support CSS for stylesheets, JavaScript for scripted Web pages, various image formats, and popular applications such as Flash. In addition, browsers can support additional functionality such as off-line operation, or in general more application-oriented features such as AIR or Silverlight. Abstract

Browser Basics What is a Web Browser?
  • Network access (HTTP, HTTPS, FTP, file system, …)
  • Rendering HTML layout (a subset of CSS layout)
    • CSS specifies many more features
  • Handling special HTML in the required way
    • images (in various formats) must be downloaded and embedded
    • forms must be rendered and form data must be submitted
  • Running scripts and providing them access to the page
    • re-rendering when scripts change the page (DHTML)
    • providing scripts with network access (Ajax)
  • Utility functions to make the browser more usable
    • tabs and bookmarks for more organized browsing
    • security policies for safer browsing
    • additional content types may be supported (by external software)
    • the browser may be extended (add-ons)
 One Minute in the Life of a Browser
  1. Analyze URI and connect to server to retrieve resource
    • recursively repeat until all required resources are retrieved
  2. Analyze HTML, correct errors, and compute a DOM tree
    • DOM is a memory representation of the HTML markup
  3. Apply CSS and compute the layout of the styled DOM tree
    • compute CSS decorated DOM and apply formatting algorithm to it
  4. Start executing code and change the DOM as required
    • scripting may have initial phase and user interaction phase
  5. Continue executing scripting code in response to user interactions
    • for many dynamic Web pages, this is a continuous activity
  6. If the user clicks on a link, start all over again
 Browsers, Apps, Operating Systems
  • Traditionally, a browser is an application for an OS
    • loading page descriptions and rendering the pages
    • the first browsers did not execute any code
  • and Plug-ins changed the browser into a platform
    • browsers are code (the browser code) executing downloaded code
  • Browsers are becoming increasingly feature-rich
    • XMLHttpRequest allows script/server communications
    • HTML5 adds WebStorage, WebSockets, and more OS-like features
  • Browsers could become the only app to run on hardware
    • Google's ChromeOS may become the first browser OS
    • Palm is going a different route with webOS (JavaScript apps)
    • WebOS and ChromeOS are essentially the same (rich JavaScript runtimes)
 Browser Usage

██ Internet Explorer (69.80%)

██ Mozilla Firefox (20.66%)

██ Safari (7.18%)

██ Chrome (0.87%)

██ Opera (0.72%)

██ Netscape (0.52%)

██ Other (0.25%)

 Browsers and CSS
  • Browsers have their own built-in CSS code
    • HTML pages with no CSS are still formatted in some way
    • HTML pages can provide their own CSS to change defaults
    • users can change the browser's default to their own preferences
  • CSS has a well-defined way of how stylesheets are combined
    1. browser defaults
    2. user declarations
    3. page declarations
    4. page important declarations
    5. user important declarations
  • Rendering of HTML/CSS depends on a variety of factors
    • default settings of the browser
    • preferences set by the user
    • CSS code provided by the page author
    • HTML/CSS capabilities of the browser
 Browsers and the Internet

Before retrieving the Web page http://www.berkeley.edu/, the browser first has to find out the IP address of the www.berkeley.edu server. Using this address, it can then open an HTTP connection. The lookup service used by the browser is the .

 Supported URI Schemes
  • Most Web pages are available over HTTP
    • one popular exception are pages available over HTTPS
  • Most browsers support more than just the HTTP and HTTP
    • http: and https: are necessary (these are the Web protocols)
    • file: allows the browser to load local files
    • ftp: is useful because many documents are available on FTP servers
    • mailto: usually is not built into the browser (the mail tool is started)
    • tel: is a useful scheme for devices with telephone functionality
  • Firefox 3 allows the registration of protocol handlers
 Caching
  • Browsers retrieve resources for rendering Web pages
  • In a typical user session, many resources are used repeatedly
    • using the browser's back button
    • accessing pages reusing the same CSS or images
  • Caching is a frequently used optimization in computer systems
    1. store retrieved data locally
    2. reuse that data when it is used again instead of fetching it again
    3. the hard (and important) part is cache invalidation
  • Prefetching allows browsers to load pages in advance
    • predicting user behavior usually is hard or impossible
    • unnecessary prefetching generates unnecessary load on servers and the network
    • faster networks make load delays less painful
      • <link rel="prefetch" href="http://www.example.com/">
  • Google Web Accelerator was a Google-specific approach
    • using prefetching and compression (connecting to a Google cache)
    • serious privacy implications (all traffic is routed through Google)
 Security and Privacy
  • Browsers store a lot of security-sensitive data
    • data entered in forms is stored for future visits
    • authentication credentials () are stored on behalf of servers
    • the browsing history of visited pages is stored
    • passwords are stored in password managers
  • Connecting to HTTPS Web sites requires a certificate validity check
    • browsers come with a large set of pre-installed certification authorities
    • users implicitly trust this list of pre-installed authorities
  • Browsers provide control over these features in complicated settings
  • Browsers start providing more user-friendly private modes
    • Safari calls the feature private browsing
    • IE8 has an InPrivate mode
    • Firefox 3.1 includes Private Browsing
  • Security/Privacy (as always) is a trade-off with convenience
 Browsers and Scripting
  • is essential for most modern Web pages
    • well-designed Web pages also work when scripting is turned off
    • many Web pages are not designed all that well
    • when scripting is turned on, behavior should be predictable and consistent
  • Scripting problems plagued Web developers for a long time
    • major parts of Web development go into ensuring compatibility
    • ill-behaving browsers (such as IE) make it impossible to develop simple code
    • provide compatibility layers on top of browsers
  • Browsers can morph into runtime environments
    • using Google Docs has little to do with Web browsing
    • some essential features are missing (offline capabilities, local storage)
    • is Google's attempt to morph the Web into an application platform
 Content Type Handling Content Types
  • Browsers retrieve resources and render them
    1. URI identifies a resource to be retrieved
    2. HTTP request is sent to the server requesting the resource
    3. HTTP response is received containing and describing the resource
    4. the media type determines how the browser handles the resource
  • Browsers can handle resource in four different ways
    1. allows the browser to handle the resource by itself
    2. extend the browser with additional capabilities
    3. are platform-specific extensions
    4. are programs to which the browser passes the resource
    5. for unknown media types most browsers prompt users to save them locally
 Built-In Support Built into the Browser
  • The Web is built on few universal media types
  • Variety on the Web is achieved through two major factors:
    1. the established media types are not application-specific
    2. advanced content can be based on browser-based runtime environments
  • Examples of built-in media types:
    • HTML+CSS for rendering formatted Web pages
    • popular (GIF, JPEG, PNG, ICO)
 Advantages/Disadvantages
  • Advantages of built-in media types:
    • fast and seamless rendering
    • should be supported by any browser (safe choice for developers)
  • Disadvantages of built-in media types:
    • cannot be added by users (not extensible)
    • browser must be upgraded to add new types
 Add-Ons Browser-Specific Additions
  • Supported by the browser itself (built into the browser framework)
  • Usually support additional functionality and not new media types
  • Add-ons are productivity enhancements for users
  • Examples of add-ons:
 Advantages/Disadvantages
  • Advantages of add-ons:
    • only browser-dependent (i.e., not OS dependent)
    • can be installed individually and specific for users
    • allow browser specific behavior (e.g., for Firefox debugging)
  • Disadvantages of add-ons:
    • cannot be used across browsers
    • conflicts between add-ons can lead to instabilities
    • Web applications cannot count on them being available (e.g., Google Gears and )
 Plug-Ins Platform Code in the Browser
  • Platform-specific code running inside the browser
    • window created by an object with given window dimensions
    • the window displays whatever the plug-in code generates as display
  • Examples of plug-ins:
 Advantages/Disadvantages
  • Advantages of plug-ins:
    • high performance (OS-specific code)
    • reasonably easy to implement if OS specific code already exists
  • Disadvantages of plug-ins:
    • hard to implement for a new OS (needs OS-specific code)
    • no easy fallback if not supported by a browser
    • plug-in internals are invisible to the browser
 External Viewers Browser and Applications
  • Applications often are not integrated with a browser
    • designed as standalone applications
    • capable of handling certain media types
    • registered with the OS and/or with a browser
  • External applications can do anything they like with a resource
    • the browser downloads the resource to a temporary file
    • the external viewer is started and handed the file's location
  • Examples of external viewers:
 Advantages/Disadvantages
  • Advantages of external viewers:
    • almost no integration effort with a browser (just registering)
    • sophisticated and highly optimized applications
    • can be taken offline and used for non-Web activities
  • Disadvantages of external viewers:
    • high dependency on platform and configuration of a user's browser
    • completely breaks the workflow of working with a browser
 Others Greasemonkey
  • Greasemonkey is an add-on for add-ons
  • Runtime environment for user scripts
    • specifically addressing a Web page or a Web site
    • locally changing the Web page (in the browser)
    • support for changing a Web site's display according to my user needs
  • User scripts analyze/extract/update a Web page
    • when Web pages are redesigned, user scripts often break
    • a fragile way of repurposing the information from a Web site
 Extended Browsers Chrome
  • Google's goal is to move more information online
  • Today's browsers lack good offline support
  • Chrome is built around Google's strategic goals
    • robust and high quality rendering based on WebKit
    • efficient and fast execution of scripting code with V8
    • includes Google Gears for offline mode
    • minimal user interface to look more like an application
 Accessibility Browsers
  • Browsers for blind users depend on well-designed Web pages
    • some browsers read aloud the textual information on a page
    • Braille readers are an alternative to read-aloud browsers
  • Only well-designed Web content is accessible to these browsers
    • HTML and CSS content can be analyzed in the browser
    • content (such as Flash) cannot be handled at all
    • may have their own accessibility features
  • Usability should play an important role in Web design
    • Section 508 requires public information to be made accessible
 Conclusions
  • Browsers are more than just HTML viewers
  • Users can customize their browsing experience
  • Information providers should be aware of browser issues
 Hypertext Markup Language (HTML) 2009-09-03 Getting started with HTML HTML Tutorial · HTML Reference · HTML Validator A1 assigned (due date: 9/20) The Hypertext Markup Language (HTML) is the most important content type on the Web. This lecture covers a basic overview of how to use HTML markup in general. In particular, we look at page titles, meta tags, inserting text and images, using lists, and creating simple tables. Attributes can be used for more layout control in the HTML tags, but most layout issues are deferred until the CSS lecture. Abstract

Structured Documents on the Web
  • Hypertext Markup Language (HTML) is the language of the Web
    • Hypertext because the Web is a hypermedia system
    • Markup because documents are encoded using text
    • Language because HTML is used for communications
  • Markup Languages are different from most file formats
    • many computer formats are binary encoded and not just text
    • markup allows structured documents to be encoded as just text
  • Web data formats use markup as well as other encodings
    • HTML and XML are markup languages
    • JavaScript is also exchanged textually (but it's not markup)
    • images and other multimedia content is encoded as binary files
 HTML Validation Checking for Correctness
  • HTML's structure is defined by a Document Type Definition (DTD)
    • formally speaking, a DTD defines the grammar of the HTML language
    • (and if you really want to know, SGML defines the syntax)
    • colloquially speaking, a DTD defines how to combine elements and attributes
Tool-Based Validation
  • Testing HTML makes sure that it is well-defined
    • any errors have to be corrected by the browser
    • the results of browser-side corrections are hard to predict
  • HTML editors allow validation within the tool
  • In theory, HTML editors should always produce valid HTML
    • things today are not as bad as they used to be
    • creating valid HTML can be a challenge for complex Web pages
 Web-Based Validation Firebug
  • Browser-based inspection (much better than view source)
  • Learning Web design by looking at Web design
    • Firefox's View → Page Source provides access to a page's source
    • Firefox's Tools → Page Info provides access to all ancillary files
  • Understanding how complex HTML works is hard
    • looking at the source requires brain-based rendering
    • looking at a rendered document makes it hard to see the source
    • Firebug provides a convenient inspection feature for Web pages
  • Inspection allows both directions of understanding HTML
    • inspecting the rendered page and looking at the source part
    • inspecting the source and seeing how it is being rendered
  • Firebug also displays the associated CSS code
 HTML and WYSIWYG
  • Thinking of HTML as a page-description language is wrong
  • HTML has been designed as a structure-description language
    • structured contents can be reformatted and reflowed
  • HTML rendering depends on a many client properties
    • screen/window size, resolution, and color depth
    • different available fonts
    • fonts with the same family name but different metrics
    • different hyphenation algorithms
    • hyphenation setting defaults
    • hyphenation dictionaries
    • different size spaces
    • different line-breaking algorithms
    • different widow/orphan/keeptogether rules
 Web Browsers

██ Internet Explorer (69.80%)

██ Mozilla Firefox (20.66%)

██ Safari (7.18%)

██ Chrome (0.87%)

██ Opera (0.72%)

██ Netscape (0.52%)

██ Other (0.25%)

 HTML and Structure Text
  • h1-h6 are different levels of headings
  • p contains paragraph text
    • whitespace and line wrapping are ignored
    • paragraphs are set as boxes containing a number of lines
  • Text inside paragraphs can use additional markup (phrase markup)
    • em for emphasized text
    • strong for text with a strong emphasis
    • sub for subscript text
    • sup for superscript text
    • q for quoted text (try nesting quotes)
    • code for code examples
  • rendering of all these elements is built into the browser
 More Advanced Text
  • Quotations can be explicitly marked up as such
    • blockquote for block-level quotations
    • q for inline quotations (part of a block)
    • cite provides support for pointing to the source
  • Preformatted text allows text formatting in the HTML source
    • pre leaves whitespace intact and usually uses monospaced fonts
    • word wrapping may be turned off by default
 Lists
  • HTML supports three kinds of lists
    • ul for unordered lists containing li
    • ol for ordered lists containing li
    • dl for definition lists containing dt/dd
Tables
  • Tables are the most complex visual structure in HTML
    • table represents a table as a sequence of rows
    • tr represents a table row as a sequence of cells
    • td represents a table cell containing table data
    • th is a special cell containing header data
Merging Table Cells Merging Rows
  • Table cells can span more than table row
    • rowspan specifies how many rows a cell is spanning
    • subsequent table rows must leave that space empty
Merging Columns
  • Table cells can span more than a single table column
    • colspan specifies how many columns a cell is spanning
    • the following cells in the same row must be left empty
Images
  • The Web is an open hypermedia system
    • hyper refers to the term hypertext for linked content
    • media refers to the fact that multiple media types are supported
  • For a long time, the Web only supported text and images
    • images can be used in a variety of formats (GIF, JPEG, PNG)
    • audio and video are possible today, but not part of the Web
  • Images are not part of a Web page, they are included by markup
    • img is an empty element for including images
    • src is a URI pointing to the image (often a relative URI)
<img src="../img/portrait.png" alt="Portrait">
 Image Details
  • Images must use a format supported by the browser
    • GIF, JPEG, and PNG are pretty safe choices
    • HTTP allows the browser to understand the image format
    • limited browsers might have size/complexity restrictions
  • Image information increases accessibility of a page
    • alt contains a short description of the image
    • for icons it is essential to provide this information
  • Image dimensions and image rendering
    • width/height specify the dimensions of the image
    • allows the browser to start rendering the page before the images are received
    • will be used to resize the image if the real image size is different
    • browser-based scaling of images often is not a good idea
 Links
  • Links are the most important feature of the Web
    • conceptually, the Web is one large hypermedia document
    • links are based on Web identifiers, the Uniform Resource Identifier (URI)
  • a is a link anchor and links to a URI (the link target)
    • <a href="http://www.berkeley.edu" title="UC Berkeley">UCB</a>
  • URIs can have various forms
    • http: points to resources available on Web servers
    • https: is the same but uses encrypted connections
    • URIs can use a variety of other
    • URIs can be relative (in the same was as file names)
    • relative URIs are evaluated relative to the URI of their occurrence
    • relative URIs can use path segments such as / and ..
 Conclusions HTML Matters
  • HTML is not just getting text displayed
  • Good HTML allows better browsing
  • First represent as much as possible in HTML
  • Then add what is missing as CSS and/or microformats
  • Graceful degradation is important
 Advanced HTML 2009-09-08 Advanced HTML Online Image Map Editor This lecture covers linking in general and in header information, and a more general view of HTML layout based on the box model used by browsers. The concept of frames is introduced, which can be used in a combination of framesets and pages, or as inline frames. Finally, image maps are introduced as a way of how images can be turned not only into links, but into a set of various linked areas overlayed over the image. Abstract

Other Links
  • Links using a are the most important links on the Web
    • href points to the link target
    • most of the time, the link anchor is text or an image
  • HTML has many more element linking to other resources
    • q/blockquote point to the source of quotations
    • img specifies an image and embeds this image into the page
    • form points to a URI to which the contents of a form are submitted
    • object embeds an object in a web page (such as a Flash app)
    • frame loads a Web page into a frame
    • iframe embeds a Web page into a Web page
    • link connects a page to ancillary resources
    • script specifies the location of scripting code
    • this list is not complete (but close)
 Header Information HTML Document Structure HTML Document Type
  • HTML pages have to declare their document format
    • browsers/clients should know which version of HTML they are dealing with
    • HTML uses the same element names in different version
  • A Document Type Declaration officially declares the document type
  • HTML has three different document types:
    1. Transitional for backwards-compatibility
    2. Strict for HTML as it should be (more restricted than Transitional)
    3. Frameset for using
  • The public identifier specifies HTML version information
    • <!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN">
    <!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01//EN">
    <!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Frameset//EN">
Document Metadata
  • All document content is specified in the HTML body
    • this is what a browser renders in the document window
    • rendering may need additional information such as scripts and style
  • Information about the document is contained in its head
Essential Metadata
  • Page titles are used in various places
    • in the title bar of the browser window and/or the browser tab
    • when creating bookmarks
    • in the results of search engines
  • Style information can be embedded in style (not reusable)
  • HTML pages can link to external resources
    • a number of link types define relationships
    • some relationships are in widespread use but are not standardized (e.g., icon)
    • one possible link type is stylesheet for pointing to external styles
  • External styles need three pieces of information
    • href specifies the URI of the external stylesheet
    • rel specifies the link type stylesheet
    • type specifies the type of the stylesheet as a media type (text/css)
 Additional Metadata
  • base sets the base URI for all relative URIs
    • can be useful if the page contains many references to a different site
      • <base href="http://en.wikipedia.org/wiki/">
      <a href="HTML" title="Wikipedia: HTML">HTML</a>
  • meta specifies general metadata for a page
    • keywords and description from the early days of search engines, largely ignored these days
    • additional metadata schemes have been defined (Dublin Core)
    • metadata is a wide field and depends on usage and users
 Creating Content All-Purpose Elements
  • HTML elements are supposed to convey structural semantics
  • HTML content should represent structural information
    • not all content can be mapped to HTML elements
    • in many cases HTML elements are available (there are even diff elements)
  • HTML also has all-purpose elements
    • these elements have no semantics and are just containers
    • span is used as an inline container
    • div is used as a block container
    • all-purpose elements should only be used if no HTML element is available
 Retain Content Structures
  • HTML should represent content structures
    • CSS can be used to tweak the formatting (if required)
  • Rich content should be mapped to rich Web pages
    • use HTML elements if available
    • augment HTML elements with CSS classes for more specific semantics
    • use for capturing more semantics
  • HTML is just one possible representation of a resource
    • the data model of resources should not be limited by HTML
    • richer representations may become available in the future
    • why are there so few Web pages with tel: URIs?
 HTML/CSS Box Model Structure and Layout Box Structure Floating Boxes Layout Floating Boxes Markup Frames Combining Documents in the Browser
  • HTML pages usually are one document loaded by the browser
  • Frames were created to be able to combine documents
  • Frames were created when server-side frameworks were primitive
    • building site navigation with frames is rather simple (HTML only)
    • building site navigation without frames is harder (server support required)
  • More modern applications combine content differently
    • assembled on the server side and delivered as one document
    • assembled in the browser via and one logical document
 Problems with Frames
  • Frame-based sites are hard to use from the Web point of view
    • it is hard or impossible to link to pages
    • user's have a hard time creating bookmarks
  • Search engines have problems pointing users to results
    • pointing to the frameset might not even contain the result
    • pointing to individual pages can result in unusable pages
  • Printing frame-based pages usually does not work very well
    • most browsers support print functionality for one frame at a time
  • Frames are not considered good practice anymore
    • they can still be useful for internal or limited audiences
    • they can be useful for rapid prototyping
    • pages for a general Web audience should not use frames
 Regular Frames Framesets and Frames
  • HTML pages can be HTML content or framesets
  • For framesets, the page only defines a frameset skeleton
    • the frameset described the structure of the page
    • individual frames point to actual HTML content
  • The browser retrieves the frameset and all frame contents
    • rendering a frameset results in a compound document
  • Links in the frameset can load content into individual frames
    • a frame's name identifies a frame with a name
    • a's target instructs the browser to load content into that frame
 Frameset and Frame Content IFrames Embedding HTML into HTML
  • Inline frames embed HTML pages in HTML pages
    • segment the window and then load HTML into the parts
    • iframe is a box somewhere in an HTML page and contains an HTML page
  • IFrames have the same usability/accessibility issues as frames
    • printing is a problem (scrolled content in scrolled content)
    • navigation is a problem (complex navigation in the context of one page)
    • addressing (bookmarks/search) is a problem
 IFrame Example Image Maps Clickable Images
  • Links on Web pages are often text or images
    • almost anything can be turned into a link by wrapping it in a
    • if an image turns into a link, any part of the image can be clicked
  • Images may convey structural information
    • an organizational chart of a company
    • a world map with countries or regions
    • photographs with marked parts (buildings, persons, …)
  • HTML image maps turn an image into a structured link
    • the older are simpler for the browser
    • the newer are much more user-friendly
 Server-Side Image Maps 
<a href="server-side-program"><img src="grid.png" ismap="ismap"/></a>
 Client-Side Image Maps 
<img src="grid.png" usemap="#gridmap">
<map name="gridmap">
	<a title="Square 2/2" href="square22" shape="rect" coords="60,60,96,96"/>
	<a title="Square 3/6" href="square36" shape="rect" coords="106,244,142,282"/>
</map>
  • Various shapes are supported
    • rect for rectangles (x1,y1,x2,y2)
    • circle for circles (x,y,radius)
    • poly for polygons (x1,y1,…,xn,yn)
 Cascading Style Sheets (CSS) 2009-09-15 Adding a Touch of Style CSS Spec · Properties · CSS Tutorial · CSS Validator Cascading Stylesheets (CSS) have been designed as a language for better separating presentation-specific issues from the structuring of documents as provided by HTML. CSS uses a simple model of selectors and declarations. Selectors specify to which elements of a document a set of declarations (each being a value assigned to a property) apply; in addition there is a model of how property values are inherited and cascaded. The biggest limitation of CSS is that it cannot change the structure of the displayed document. Abstract

Why CSS? Structure vs. Layout
  • HTML started as very simple layout-oriented structures
    • more layout control was introduced as attributes (align, color)
    • HTML became increasingly polluted by layout information
  • CSS was introduced as a format for layout information
    • the HTML can be kept simple, containing only the structures
    • layout information can be reused by using separate CSS files
  • CSS has been designed for HTML
    • it has been generalized to also cover XML
    • the HTML heritage is still very visible in CSS
 What's Still Missing?
  1. Restructuring content
    • CSS assigns formatting properties to elements
    • the document tree which is formatted cannot be restructured
    • parts can be ignored or new parts can be inserted
  2. Interpreting content
    • img has a lot of special meanings attached
    • form elements have special semantics (such as creating input fields)
  • can be used for implementing some of these issues
  • CSS is simply for more sophisticated styling of HTML pages
 How CSS Works CSS in Action HTML and CSS
  • CSS specifies how HTML elements are formatted
    1. formatting can be attached to every element (redundant inside document)
    2. formatting can be included in document (redundant across documents)
    3. separate CSS files describe the formatting (best reuse)
  • Any combination of these methods is possible
Formatting Model
  • are central to the CSS formatting model
    1. create a document tree
    2. identify the media type (e.g., screen or print)
    3. retrieve all stylesheets required for the media type
    4. assign values to all properties in the document tree
    5. generate a formatting structure (a different tree)
    6. render the formatting structure on the target medium
  • Properties control the rendering of elements
  • Styling in CSS means assigning values to properties
 CSS Strategies Use Classes & Containers
  • CSS code should never show up in your HTML
  • Classes should reflect content or formatting
  • Containers can restrict styles to a context
  • Context can be nested
    • orthogonal concepts should be represented as nested classes
    • for example, pages for staff and faculty and current and past as classification
    • different levels of formatting sophistication can be implemented with CSS only
  • Avoid redundant CSS code
    • Whenever appropriate, inherit properties
    • for invisible links use a { color : inherit ; }
 Properties Formatting Instructions
  • Properties define how elements are formatted
    • they define a specific facet of formatting
    • they may have interdependencies with other properties
    • they can be assigned explicitly
    • they may be defined through or
  • A property has a name and is used in a name/value-pair
    • the name identifies the property that is being set
    • the value space depends on the property
    • some properties accept complex values
    • sets of values: p { font : bold italic large Palatino }
    • sequences of values: p { font-family : "Segoe UI", verdana, helvetica, arial, sans-serif }
  • Property specifications can be grouped
    • .thinboxed { border-width : 1px ; padding : 10px ; margin : 5px }
 CSS1 Properties Factoring out HTML
  • CSS1 was published in 1996 and revised in 1999
  • HTML suffered from too many attributes
    • layout information was specified as CSS
    • style attributes in HTML were marked as deprecated
  • A small set of formatting features as CSS properties
    • font: p { font : 80% sans-serif }
    • color and background: body { background : url(logo.jpeg) right top }
    • text: h1 { text-transform : uppercase }
    • box: p.quote { border-style : solid dotted }
    • classification: img { display : none }
 CSS2 Properties CSS2
  • CSS2 was published in 1998 and is still being revised (CSS21)
  • CSS21 is what you can expect from modern browsers
    • with IE (even IE7) being the exception
  • CSS2 is a single and coherent specification
    • CSS3 is a jungle of concurrent module development
    • CSS3 will never be finished (some modules will, though)
 Generated Content
  • CSS1 had no way of adding information to the document
    • by using display, parts of the document could be ignored
  • Generated content allows content to come from the CSS
    • only possible with :before and :after pseudo-elements
    • static strings: p.abstract:before { content : "Abstract: " }
    • special effects like quotes: q:before { content : open-quote }
    • counters: h1:before { content: "Chapter " counter(chapter) ". " ; counter-increment : chapter }
  • Quotes can be defined as being language dependent
    • q:lang(en) { quotes : '"' '"' "'" "'" }
    • q:lang(no) { quotes : "«" "»" '"' '"' }
 Tables
  • CSS1 does not address table formatting
    • table layout still had to be done using HTML attributes
    • a lot of redundant code specifying cell alignment and borders
  • CSS2 introduces tables on the CSS level
table    { display: table }
tr       { display: table-row }
thead    { display: table-header-group }
tbody    { display: table-row-group }
tfoot    { display: table-footer-group }
col      { display: table-column }
colgroup { display: table-column-group }
td, th   { display: table-cell }
caption  { display: table-caption }
 Fixed vs. Automatic Table Layout
  • HTML defines a complex table rendering algorithm
    • tables are rendered incrementally
    • table layout is determined by looking at the complete table
Automatic Fixed
col 1 row 1 col 2 row 1 col 2 row 1 col 3 row 1 col 3 row 1 col 3 row 1
col 1 row 2 col 2 row 2 col 2 row 2 col 3 row 2 col 3 row 2 col 3 row 2
col 1 row 1 col 2 row 1 col 2 row 1 col 3 row 1 col 3 row 1 col 3 row 1
col 1 row 2 col 2 row 2 col 2 row 2 col 3 row 2 col 3 row 2 col 3 row 2
Selectors Select and Style
  • are applied to elements
    • properties can be directly applied in an element's style attribute
    • in all other cases, selectors are used to select the styled elements
  • Selectors are good for reusable CSS code
    • identifying the most appropriate formatting classes is not easy
    • planning for CSS for a larger site is a difficult task
  • CSS project management should separate selectors and properties
    1. selectors are about which things should be identified and styled
    2. properties are about how this styling is implemented
 CSS1 Selectors CSS for Dummies
  • Very small set of selectors
    • selecting elements by name: h1 { font-size : large }
    • selecting elements by their id: #author { font-weight : bold }
    • selecting elements by their class: .abstract { font-size : small }
    • combining these mechanisms: p.warning { color : red }
  • Pseudo-classes and -elements allow interesting effects
    • a links have state: a:visited and a:active
    • selection without markup: p:first-letter and p:first-line
 CSS2 Selectors More Selectors
  • are available
    • element name, id, class, and combinations of these
  • CSS2 introduced many new selectors
    • descendants: ul li { font : italic }
    • children: ul > li { font : italic }
    • adjacent siblings: h1 + h2 { margin-top : 0.5em }
    • attribute matching: h1[lang=nl] { color : orange }
  • CSS2 selectors are sufficient for most tasks
  • Setting class attributes is very important
 CSS2 Pseudo Classes
  • CSS1's pseudo-elements are available
    • link states and first letter and line of content
  • CSS2 adds more qualifications for elements
    • first child: p:first-child { text-indent : 0 }
    • dynamic behavior: a:hover { … } a:active { … } a:focus { … }
    • language: :lang(de) { quotes: '»' '«' '‹' '›' }
    • : q:before { content : open-quote } q:after { content : close-quote }
 CSS Mechanics Cascading
  • Stylesheets may have three different origins
    1. page authors associate CSS with their pages
    2. users configure their browser to use some CSS
    3. user agents (browsers) have built-in CSS how to style content
  • Conflicts must be resolved using the following algorithm
    1. find all matching declarations (matching media type and selector)
    2. sort according to importance (browser < user < author)
    3. same importance must be sorted by specificity (more specific selectors)
    4. finally, sort by order in which they were specified
  • !important rules can influence the algorithm
    • they are interpreted in step 2 (sorting by importance)
    • browser < user < author < author(important) < user(important)
 Inheritance
  • Properties often are inherited by children
    • setting a table's color sets the color for all contents
    • without inheritance, CSS stylesheets would have to be very large
  • Inheritance is mostly intuitive
    • in reality, it is a bit more complicated
    1. specified value: what the property specified (, inheritance, or initial)
    2. computed value: computed based on the environment (e.g., ex → px)
    3. used value: converted to an absolute value (e.g., percentage widths)
    4. actual value: specific for the environment (e.g., borders with pixel fractions)
 Structuring Stylesheets
  • Stylesheets may need to be structured
    • importing CSS code is supported: @import "/dretnet.css" ;
    • modules of CSS code can be reused in different contexts
  • Stylesheets may be specific for a media type
    • braille, embossed, handheld, print, projection, screen, speech, tty, tv
    • specified in HTML: link rel="stylesheet" type="text/css" media="print" href="/print.css"
    • specified in CSS: @media print { … }
    • media-dependent import: @import "/print.css" print ;
 CSS Conclusions
  • Appropriate for HTML
    • Flexible selection of elements using
    • Powerful formatting of elements using
    • Interesting interface design with pseudo-classes and -elements
  • Inappropriate for general publishing
    • documents often need to be restructured
    • XML → HTML+CSS is a popular Web publishing setup
 HTML Forms 2009-09-16 HTML Forms FAQ Style Guide · Forms Spec This lecture introduces HTML Forms, a way how an HTML page can provide input fields, so that users can provide data to a Web-based application. HTML forms are regular HTML pages (i.e., using regular HTML structures), but they also contain special HTML elements for data entry. Most importantly, each form contains instructions on how to submit the entered data, and the browser will use that information to compose a request containing all the data of the form submission. Abstract

Forms Basics HTTP Web Services
  • Services can be provided through URI/HTTP
    • URI-based services need input as a query string
    • the question is how the user gets information into the URI
  • HTML forms provide an interface for assembling query strings
    • users fill out a form providing several fields
    • the browser submits the entered information by HTTP to a URI
    • the result of the request is displayed to the user
  • HTTP has two different methods for submitting data
    • GET encodes the data as a URI query string
    • POST encodes the data as HTTP request entity
 Forms Mechanics
  • HTML forms are normal Web pages (using form elements)
  • The process receiving the form data produces a result page
 Forms Markup
  • All form elements must be inside a form element
    • specifies the URI for submitting the form values (action="URI")
    • specifies the method for submitting the form values (method="GET|POST")
  • form contains regular HTML markup and form elements
    • the regular HTML markup creates the form's layout (table, list, texts)
    • the form elements create the controls for acquiring input data
  • Each form should have a submit button
    • when pressing this button, the form values are sent to the action URI
    • without such a button, the form values cannot be submitted
 Forms Elements (User View)
  • HTML provides a small set of form controls
  • Sufficient for many applications
Text:
Password:
Checkbox:
Radio Button:
Text Areas:
Selection:[ not supported ]
Multiple Selection:
Markup for Disabled/Readonly Controls Conclusions
  • HTML forms allow data entry on Web pages
  • Only a small number of form controls are available
  • Form submission is just another HTTP request
  • Forms should be structured to be more accessible
 Microformats 2009-09-22 Wikipedia Microformats · Tutorials HTML pages are for human users and describe a resource in structural terms (headings, lists, tables, …). For machine-based interaction, it is often necessary to have more information about the application concepts. XML is a popular language for representing application structures, but is targeted at machine-based processing alone. Microformats and more formal approaches such as the Resource Description Format (RDF), RDF in Attributes (RDFa), and Web Ontology Language (OWL) often are used to describe Web content semantically. Abstract

HTML vs. XML
  • HTML describes structures in a very general way
    • HTML elements describe logical page structures such as headings, lists, tables, …
    • useful for dynamic and adaptive page rendering, but not for understanding contents
  • Good HTML may have more information available
    • classes in HTML elements may represent underlying concepts (CSS may use this)
    • HTML containers may represent aggregation of some basic information items
  • Very good HTML
    • some guidelines/rules/methods for understanding class names
    • some model for the underlying schema (what may appear in which combination)
  • Excellent HTML is dynamically generated from XML
    • the model is exposed as structured XML data that is available to the client
    • there is a stylesheet for producing the HTML version of the XML
    • but even XML does not provide semantics (it is just a structured syntax)
 Plain HTML Good HTML Excellent HTML XML → HTML Stylesheet Graceful Degradation
  • XML was designed as a language for Web content
    • the idea was that XML documents would be delivered to the browser
    • stylesheets (CSS/XSL) would take care of the client-side rendering
  • CSS is good at supporting graceful degradation
    • viewing an HTML page with CSS turned off most of the time works fine
  • XSLT is not good at supporting graceful degradation
    • the browser just displays the raw XML when XSLT is not supported
  • Serving XML on the Web is not a good idea
    • in closed scenarios (intranet applications) this is a viable solution
    • in open scenarios, HTML should be served as the default representation
    • alternate versions can be provided by supporting
Excellent HTML From Information, Knowledge
  • XML is often said to be self-describing
    • many people think this is the same as self-explanatory
    • the catch is what exactly it is you refer to by describing
  • Database data cannot live without a database
    • database data is simply content, the structure is provided by a DBMS
    • XML documents have their structure encoded within them
    • compared to database data, XML in fact is self-describing
  • What is the gap between self-describing and self-explanatory?
    • it is impossible to find out how the document could be modified
    • there are no semantics associated with structure or content
    • so self-describing means, you can guess a lot, but you maybe wrong
 The Semantic Web Hype
1965, H. A. Simon: machines will be capable, within twenty years, of doing any work a man can do
1967, Marvin Minsky: Within a generation [ … ] the problem of creating artificial intelligence will substantially be solved.
  • How to get past the limitations of HTML?
    • a machine-friendly Web must make Web resources machine-processable
    • XML solved the problem on the syntax level
    • how could the problem be solved on the level of semantics?
  • As in the 1970's, description logic was declared as being the solution
    • there was a need for the Web to move towards semantics
    • there was a community of AI researchers with a long history
    • the Semantic Web was born and is currently repeating AI history
 Semantic Web Layer Cake Microformats Islands of Semantics
  • Microformats solve very specific problems in a very specific way
    • encoding address information on a Web page
    • encoding a location of something represented by a Web resource
  • Microformats can be compared to tagging
    • a very simple mechanism with a minimal barrier-to-entry
    • little flexibility in adapting the mechanism to slightly other uses
    • often underspecified and interpretation implementation-dependent
    • no unified rules across different platforms which makes processing hard
    • nice and easy to start with, but questionable for robust long-term solutions
  • Currently there are about 10 reasonably popular microformats
 Microformat Syntax
  • HTML has some underspecified and underused elements
    • dfn, code, samp, kbd, var, cite, abbr, acronym
    • they can be reused and augmented with additional information
  • HTML allows non-HTML content in HTML pages
    • unknown elements and attributes must be ignored
  • HTML allows class attributes to carry semantics
  • HTML has a head which contains page metadata
    • for example, the link element specifies connections to other resources
 Magic Names
  • A syntax defines where and how to embed information
    • what is embedded and how well is it defined semantically?
    • is there an underlying model for specifying dependencies?
    • how many assumptions does it take to implement a microformat?
  • Names are never self-explanatory, they always represent concepts
    • nothing can remove the burden of defining a conceptual model
    • if this is not done, models evolve and there will be more than one
  • Microformats and tagging share the same folklore
    • define simple things and good things will happen
    • this works by supporting a quickly growing ecosystem of diverging semantics
    • semantics are most useful when they are well-defined
    • loose semantics also have some utility
 Microformats on the Web
  • Easy to embed for generated content
    • some of the very basic formats may even appear in browsers one day
    • combining well-designed URIs with document relationships is better than every site map
  • Hard to rely on for applications that need dependable semantics
    • useful as a hint and as a starting point
    • microformats are not a good idea for complex information management tasks
  • Use as foundation for representing common concepts
    • when formatting addresses, use adr class names
    • for structured documents use XOXO
 Resource Description Framework (RDF) Describing Resources
  • RDF describes everything in triples
    • making a statement about a resource (identified by a URI)
    • describing a certain property of the resource (identified by a URI)
    • specifying a value for that property (a URI or a literal)

  
    Eric Miller
    
    Dr. 
  
]]>
 RDF Graphs RDF is Simple and Complex
  • RDF's abstract model is the idea of descriptive triples
    • the actual RDF model is rooted in description logic
    • RDF itself can only describe individuals (something identified by URI)
  • RDF/XML is an XML syntax for encoding triples
    • the syntax allows a variety of ways to represent the same RDF statements
    • processing RDF/XML with XML tools is likely to fail
    • use RDF parsers to parse all variations of RDF/XML into an abstract RDF graph
  • RDF Schema supports the creation of RDF vocabularies
    • describe the classes of things that can be used in statements
    • describe the properties which can be used for each of these classes
    • describe the allowed values for the supported properties
 RDF Schema Graph RDF in Attributes (RDFa)
  • Microformats can use any kind of markup design
    • this makes it hard to detect microformats when processing a Web page
    • combining microformats can become complicated and ill-designed
  • RDFa defines a syntax for embedding RDF into HTML
    • the vocabulary must be described by some RDF schema language
This document is licensed under a Creative Commons License.
]]>
 More Languages SPARQL
  • RDF graphs can be large and hard to handle
    • querying RDF graphs using XML technologies is hard and slow
    • special data structures need special query languages
    • SPARQL is a query language for querying RDF graphs
  • Using RDF without using SPARQL does not make a lot of sense
    • if the data is simple and restricted, why use RDF?
    • processing unrestricted RDF without a special language is very hard
  • Semantic Web search engines can harvest the Web for RDF
    • the result is a huge graph of RDF describing all semantic Web resources
    • querying into this graph retrieves all formalized semantics on the Web
 Web Ontology Language (OWL)
  • RDF and RDF Schema are rather basic languages
  • OWL adds more sophisticated features to RDF Schema
    • constructions of classes using existing ones
    • characterize relationships (e.g., whether they are transitive, symmetric, functional, etc.)
  • Formal semantics are hard to write and compute
    • no property expressions or datatypes in RDF Schemas
    • not all set operators, restricted cardinality in OWL Lite
    • some restrictions, but a computational guarantee in OWL DL
    • full expressive power in OWL Full (but no computational guarantee)
 Vocabulary Taxonomy Conclusions Some Questions
  • Is the world something that can be objectively formalized and described?
  • If the conceptualization of the world changes, what about the ontology?
  • How can ontology users understand a large ontology?
  • Should users trust ontologies which are based on strict categorization?
  • How much responsibility should we delegate to formalisms?
  • Can computer formalizations fully capture semantics?
 Semantics are Important and Hard
  • Semantics must be captured somewhere
  • Most semantic definitions are using prose and some formalism
  • Completely formal semantics are hard to define and hard to use
  • Semantic Web technologies may share the fate of AI
 Content Management System (CMS) 2009-09-24 Wikipedia (CMS) · Wikipedia (WCMS) Apache · Drupal · MarkLogic A2 assigned (due date: 10/4) The fundamental architecture of the Web only requires a Web server capable of answering HTTP requests on the server side. The question, however, is what that content server is serving when responding to requests. The content served by Web servers may come from files, from some form of managed more or less static content, or from dynamic processes. In this lecture, the idea of a Content Management System (CMS) or, more specifically, a Web Content Management System (WCMS), is introduced in a structured and disciplined way. Abstract

Content on the Web
  • Web technologies describe how to get content into browsers
    • HTML is the universally supported representation for content
    • HTTP allows browsers to GET information from servers
  • Resources are never transmitted or displayed
    • browsers only display resource representations
    • how a representation is produced is entirely up to the server
  • Managing resources and producing representations is a core Web task
    • resource management often is done using some proprietary system
    • mapping resources to representations should be done by rules
    • in many scenarios, resources are content units
  • A Content Management System (CMS) manages any kind of content
    • and it does not necessarily provide Web access
  • A Web Content Management System (WCMS) has Web-specific functionality
    • support for Web representations (HTML & CSS)
    • support for Web patterns (navigation bars)
 Content on the Web Content and Structure
  • Content is what matters most
    • content itself often has some internal structure
    • content may explicitly link to other content
  • Macro-structure often is more representation than content
    • displaying the current context of the content
    • displaying related content
    • displaying some overall structure (site navigation)
  • Content often is reusable across application scenarios
    • publishers have used Content Management Systems for a long time
    • adding a new publication channel to a CMS should be evolutionary
  • Content may require very different support depending on the channel
    • newspapers need fine-tuned layout and good control over content size
    • Web needs good interlinking and navigation
 CMS Evolution
  1. Web servers reading from files
  2. Web servers implementing primitive content management (SSI)
  3. Scripting languages implementing better management
  4. Management code getting hooked up to databases
  5. Better handling of client-specific behavior
  6. Databases getting more diverse (RDB, XML, RDF)
 <q>Content</q> in CMS Serving Content from Files The Rise of the CMS
  • File-based content management works well for small sites
    • simple site structure and small number of files
    • redundant parts can be manually synchronized
    • no software is required other than a Web server
  • Web servers soon developed rudimentary CMS functions (SSI)
    • rudimentary support is better than no support
    • managing a non-trivial setup with SSI still is a challenge
    • SSI allows includes but no backlinks and thus hides dependencies
  • Content management is very similar to code management
    • simple setups require no or little tool support
    • serious projects need tools to manage dependencies and changes
 Serving Content from Files with SSI Files (Opaque Chunks)
  • All major operating systems have file systems
  • Files are typically treated as opaque chunks of data
  • Applications may have special knowledge of file contents
  • Advantages of files:
    • universally supported across major operating systems
    • storage and user management comes for free
    • all that is needed for a Web site is a Web server
  • Disadvantages of files:
    • content access requires file system access
    • setting up parallel servers requires additional effort
    • no support for managing structure, everything handcoded
 File Systems are Databases
  • A file system is a simple hierarchical database
    • it does not know data types and simply stores any content
    • its structure is a tree with a few extra tricks (such as symlinks)
  • Many scenarios have much more structured data models
    • products, people, financial institutions all have complex data models
    • content should be stored and queried based on these models
  • Databases are better optimized for storing structured content
    • better methods for structured storage and retrieval
    • better strategies for managing large datasets
    • sophisticated tools for access control, backup, and versioning
 Tables (Relational Model)
  • Most widely used model for large collections of structured data
  • Very mature products and many skilled people available
  • The biggest advantage is that it is not hierarchical (no structure bias)
  • Advantages of relations:
    • well-understood model and maps well to existing data
    • the non-hierarchical model allows views from different perspectives
    • highly scalable solutions available
  • Disadvantages of Relations:
    • bad for sequences and variable structures (choices, repetitions, …)
    • very bad for structured documents
 ER Model Ordered Trees (XML)
  • XML has a heritage of document processing
  • XML tools can be used standalone and are widely supported
  • XML and HTML have a very similar foundation
  • XML has two built-in directions: hierarchy and ordered children
  • Advantages of XML:
    • maps well to HTML and XHTML
    • well-suited for document-oriented content
  • Disadvantages of XML:
    • not good at representing non-tree data
    • databases not as mature as relational products
 XML Content

The term Mixed content in XML refers to elements which have text content mixed with elements. What these elements do depends on the elements , but the important point is that they are on the same level as the text nodes of the mixed content.

 Directed Graphs (RDF)
  • RDF is the metamodel of the Semantic Web
  • Highly granular, less rigid than tables, less ordered than trees
  • Advantages of RDF:
    • any structure can be mapped to RDF triples
    • support still limited but getting better
  • Disadvantages of RDF:
    • no model for document boundaries and self-contained units
    • bad for sequences
    • very bad for structured documents
 Choose a Matching Metamodel
  • Content has some inherent metamodel properties
    • forcing that into a different metamodel is possible but unwise
  • Using a metamodel which best matches a model is crucial
    • if you have large collections of rigid and highly-structured data: Tables
    • if you have structured documents with rich text: XML
    • if you have fine-granular graph-structured data: RDF
  • Mapping is always possible but has severe limitations
    • things that work effortlessly in one metamodel may be awkward in another
    • there is no such thing as the one metamodel for all needs
    • RDF's claim to be the one metamodel for everything is not backed by facts
 <q>Management</q> in CMS Deconstructing Management
  • What is managed?
  • Who is managing it?
  • What are the management support functions?
  • Are there workflows and processes?
  • Is the management integrated with other processes?
  • Is it likely that processes will be followed?
 Managing Content with Files Integrated Management Functions
  • Separating management and publishing does not work well
    • typical examples for workflows are review and release processes
    • oftentimes publishing-specific roles are required
  • Integrated management takes over all tasks
    • What is managed? Database of structured content.
    • Who is managing it? Registered users based on roles.
    • What are the management support functions? Building a site around the content.
    • Are there workflows and processes? Can be based on users/roles/content.
    • Is the management integrated with other processes? APIs allow extension/integration.
    • Is it likely that processes will be followed? Easier to use than homegrown methods.
 <q>System</q> in CMS System Platform
  • Systems need runtime environments
  • CMS are programs that are installed on some OS
    • integrated Web server or connect with Web server
    • integrated database or connect with database
    • integrated use management or connect with user management
  • Typical steps for setting up a CMS
    1. setting up the runtime environment
    2. installing the CMS software
    3. initializing the CMS installation
    4. migrating existing data into the CMS installation
 Drupal MarkLogic
  • XML-based content management
    • content is stored in XML documents in an XML database
    • non-XML content can be stored as well
    • programming is done in XQuery using extension functions
  • XML-based content management works well for documents
    • XML's ordered trees are well-suited to represent documents
    • MarkMail provides access to well-indexed data of 7,227 mailing lists
  • XML databases are more efficient than XML files
    • XML content is indexed and access is faster
    • building server farms is supported by the database management system
 Conclusions Content vs. Web Pages
  • Managing content is one of the prerequisites for Web publishing
  • Content management can be done generically or Web-specific
  • WCMS provides specific support for Web publishing tasks
  • The most important part of a CMS is its metamodel
 Extensible Markup Language (XML) 2009-09-29 XML 1.0 Press Release · XML Fever · On XML Language Design Spec · Structuring Content with XML · People The Extensible Markup Language (XML) defines a simple way for structuring data. The power and popularity of XML can be explained by its versatility, the platform-independence, the standards and technologies leveraging it, and the number of tools and products supporting it. Understanding XML itself is rather simple, it only depends on a very small set of other technologies. Unicode and URIs are the most important foundations of XML. XML itself specifies two different things: on the one hand the format for structured data, which are called XML documents, and on the other hand a constraint language for XML documents, which is called Document Type Definition (DTD). Abstract

Foundations for XML Identifications
  • Identification of Character Encodings
  • Identification of Languages
    • textual content should be tagged with language information
    • specification based on ISO 639 language tags
    • language identification is about text, not about characters
 Unicode XML's Idea of Content and Names

XML documents can use a wide array of characters. They are defined by Unicode, which currently (Version 5.0) defines more than 100,000 characters (#100,000 added in 2005).

XML and Unicode
  • XML is based on Unicode
    • XML is defined in terms of character structures
    • how these characters are encoded is not part of XML
  • How are XML documents encoded?
    • applications can use any character encoding they like
    • XML processors must support UTF-8 and UTF-16
    • XML processors may support any number of additional encodings
  • How is the encoding encoded?
    • part of the XML document: <?xml version="1.0" encoding="UTF-8"?>
    • bootstrap problem solved heuristically or by out-of-band information
 Uniform Resource Identifier (URI) Identifiers are Essential
  • Uniform Resource Locator (URL) is the old concept
    • introduced to distinguish between locating and naming
    • locating and naming are two ways of identification
    • URLs have been replaced by URIs, technically URLs do not exist anymore
  • URIs identify resources
    • some resources may be retrieved using a protocol: http://dret.net/netdret/
    • not all resource access is retrieval: mailto:dret@berkeley.edu
    • sometimes computers are not required: tel:+1-510-6432253
    • or resources cannot be located: urn:ietf:rfc:2648
    • or location is the only means of identification: geo:27.988056;86.925278
 URIs and REST
  • Representational State Transfer (REST) requires identification
    1. identify all relevant resources
    2. design/use representations for those resources
    3. resources should be linked (via URI) for allowing navigation
  • URIs are the API of RESTful applications
    • URI-identified resources have a uniform interface (HTTP)
    • interaction with those resources is done via HTTP
    • clients can navigate the resource/state space by following links
 XML XML Use Cases
  • XML is a metalanguage supporting application-specific vocabularies
  • RSS (and Atom) are XML vocabularies for newsfeeds
  • OpenDocument (ODF) is a language for office application documents
    • designed for open and interoperable exchange
    • standardized by ISO (which now also standardizes Microsoft's Open XML)
  • Scalable Vector Graphics (SVG) for portable vector graphics
 XML Documents Markup?
  • Structures are encoded using special characters
    • a fundamental difference when comparing to binary formats
    • markup languages can be read and modified using text-based tools
    • programs must treat markup characters in a special way
  • Documents are content interspersed with markup (i.e., structures)
    • XML-aware software interprets the markup
    • XML-unaware software just sees a text file
    • modifications must be made XML-aware (e.g., inserting AT&T as AT&amp;T)
  • You have to pay the
Basic Concepts
  • XML Documents have an XML declaration (optional)
  • There is exactly one document element (a.k.a. root element)
  • Elements may be nested (there is no conceptual limit)
    • elements may be repeated (they can be identified by position)
  • Elements are marked up using tags
    • most elements have content, surrounded by start and end tags
    • empty elements are allowed and may use a special notation
  • Elements may have attributes (zero to any number)
    • attributes can only occur once on an element (i.e., they cannot be repeated)
Tree Syntax
  • Markup is important, but only a notation
  • XML documents are trees with different node types
    • nodes so far: document, element, attribute, text
 Elements
  • Elements can use a wide variety of names
    • Allowed: html, id9832798472, _, :, こんにちは
    • Disallowed: leading numbers, spaces, control characters
  • Element names usually convey some information about the content
    • this is not reliable and highly language-dependent
    • it is very useful when working with a known vocabulary
    • it is potentially harmful when working with an unknown vocabulary
  • Elements are the foundation for XML's versatility
    • they can be nested (<address><city>Berkeley</city><zip>94709</zip>…)
    • they can be repeated (<givenname>Erik</givenname><givenname>Thomas</givenname>)
    • their sequence can convey additional information (given names have a sequence)
 Attributes
  • Additional information pertaining to elements
  • Traditionally, anything that is not considered content
    • SGML is a document markup language
    • XML uses SGML's concepts
    • XML has its roots in the document world
  • Elements: Content (i.e., Data); Attributes: Metadata
  • Documents often distinguish by what is textual content
Attribute Syntax
  • Naming rules are the same as for
  • Attributes always appear within an element's start tag
  • Attributes are name/value-pairs
    • the value is enclosed in single or double quotes
  • Attribute with a single-quote value: elem attr="Single: '"/
  • Attribute with a double-quote value: elem attr='Double :"'/
  • How can attribute values contain both?
 The Price for Markup
  • Markup characters have a special meaning
    • < opens a tag
    • for attribute values, quotes delimit the value
  • The literal use of a markup character requires escaping
    • XML's entities can refer to pieces of content
    • entity syntax is &name; for referring to the entity name
    • XML has 5 predefined entities: &lt;, &gt;, &amp;, &apos;, &quot;
  • Attribute using both kinds of quotes: <elem attr="Single ' and Double &quot;"/>
Attribute using both kinds of quotes: <elem attr="Single ' and Double &quot;"/>]]>
 Mixed Content

The term Mixed content in XML refers to elements which have text content mixed with elements. What these elements do depends on the elements , but the important point is that they are on the same level as the text nodes of the mixed content.

The term Mixed content in XML refers to elements which have text content mixed with elements. What these elements do depends on the elements , but the important point is that they are on the same level as the text nodes of the mixed content.
]]>
 Mixed Content Usage
  • Database people find mixed content irritating
    • cannot be easily mapped to relational structures
    • is more document-like than data-like
    • much harder to optimize for query analysis and query processing
  • Document people find mixed content very intriguing
    • textual content can still be used as simple text
    • markup provides additional information for rich text
    • start with a text-only document and use markup to add structure to it
 Whitespace
  • XML documents often are pretty-printed
  • Whitespace text nodes often are not really content
    • XML whitespace characters are space, tab, newline, and carriage return
    • whitespace text nodes are text nodes containing only whitespace characters
 Significant Whitespace
  • Some whitespace text nodes are relevant
  • Usually text nodes in mixed content elements

Whitespace can be very important!

Whitespace can be very important!
]]>
 Processing XML Observing XML Syntax
  • XML's syntax requires you to use the right characters
  • XML processors (a.k.a. XML parsers) check for these rules
    • if there are problems, the document cannot be interpreted as XML
    • otherwise, the document is said to be well-formed
  • Only well-formed documents can be regarded as a tree
    • other documents are not XML at all, even though they may be close
    • XML processors must report problems to the application (no silent recovery)
 Validity
  • Well-formed documents observe XML rules
    • they observe the XML syntax
    • they observe all well-formedness constraints
  • Applications require the right elements and attributes
  • Validity is a more comprehensive concept
  • Valid documents observe additional rules
 Semantics
  • XML is a language for encoding trees
    • Elements and attributes are labeled node in this tree
    • the labels can be chosen freely by document authors
  • The tree's meaning is nothing XML is concerned with
    • peers must have a mutual understanding of the semantics
    • XML without mutual understanding is almost useless
    • reverse engineering often is possible, but it is risky and brittle
 Conclusions XML Documents
  • XML documents are structured data using markup
  • Elements and Attributes are the main structuring mechanisms
  • Elements and Attributes have names, but have no inherent semantics
  • For using XML successfully, shared semantics are essential
  • Always think about semantics
 Documents, Data, and Databases 2009-10-01 FAQ XML databases often are a good solution for managing document-oriented content, but frequently it is necessary or dictated by existing solutions to use non-XML databases for managing document content. In most cases, these databases will be relational databases. There a two major approaches of how to manage document-oriented content in a relational database. The first approach is to define a mapping between document and relational structures and work with this mapping. The second approach is to use the XML-specific functionality, which is increasingly provided by relational databases, turning them into XML-aware databases. Abstract

Documents are Trees
  • Many documents are trees or almost trees
    • applications may have different internal data models
    • the exchange and processing of documents often is tree-based
  • XML is the most popular format for tree structures
  • Where and how is XML being used?
    • as a transfer syntax (Web Services often are used like this)
    • as artifacts that have a longer lifespan (archiving of business documents)
    • as the application data model (there is nothing but XML)
  • XML usage results in very different requirements for XML tools
    • Web Service programmers often never see the tree
    • archived XML documents need to be searchable
    • XML-centric applications need to store XML efficiently
 Document Content vs. Document Metadata
  • Document content
    • title and abstract
    • chapters and sections
    • tables, lists, figures
    • footnotes, endnotes, sidenotes
    • bibliographic references
    • annotations
    • cross-references
  • Document metadata
    • title
    • creation date
    • author
    • affiliation (document and/or author)
    • version/revision information
    • keywords
    • summary and/or abstract
Metadata Metamodels
  • Metadata is often perceived to be key/value data
    • long history of catalog cards and their management
    • complex metadata is hard to create, maintain, and use
  • Metadata management works well in non-tree models
    • relational metadata works well for regular metadata
    • graph metadata (RDF) works well for any metadata
  • Many document management systems manage metadata and not documents
    • manage all the document metadata in RDBMS (or RDF)
    • treat the documents as BLOBs (PDF or some other opaque format)
    • makes it hard to work with document contents and structure
 Storing XML
  • XML documents are text files
    • they can be stored in file systems (they are self-describing)
    • they can be retrieved by searching through the file system
  • File systems are not designed to store millions of documents
    • standard file system implementation usually slow down dramatically
    • standard procedures (backup/versioning/concurrency) do not work well
  • Problems with File Systems as XML Databases
    • the number of documents is too large
    • there is no structured access
    • there is no access optimization
 Relational Databases Generic XML Storage
  • Relational databases are the state of the art since 1976
    • this is long enough to build highly optimized and robust systems
    • this is long enough to have ER hard-wired into some brains
  • XML is more powerful than ER
    • repetitions of elements do not map well
    • choices do not map well
    • ordered content does not map well
    • mixed content does not map well
  • Storing XML in a relational database is hard
    • it can be done by piggybacking structural information as content
    • using the resulting structures is awkward and very inefficient
 Tree Table
ID Type Name Value Parent Left
1 Root
2 Element a 1
3 Element b 2
4 Element c 2 3
5 Text Text 3
6 Attribute att 42 4
Database Support for XML Why XML and Databases?
  • XML is becoming increasingly popular
    • XML as a document format was first used as wire format
  • What is XML for an application?
    • an (increasingly popular) way to represent the data?
    • the data itself?
    • currently, the representation perspective is more popular
    • as XML is increasingly penetrating applications, this may change
 XML Interchange XML Support in DBMS XML DBMS XML Storage in Databases Model Mapping
  • Relational databases are not good tools for storing XML
    • they might be appropriate if the schema disallows problematic constructs
    • they often are already deployed and applications must live with them
  • If the data model is ER-oriented, relational databases are good tools
    • metadata typically is a good fit for tables
  • If the XML is not visible in the model, it can be structurally inaccessible
    • e.g., a product catalog may contain product descriptions in XHTML rich text snippets
    • for managing the product catalog data, the XHTML is not relevant
  • If the XML is part of the model, it should be accessible structurally
    • if the product catalog XHTML contains links to other products, these links are important
    • if they are hidden in the XHTML, all XHTML snippets have to be parsed
    • ideally, the database should be able to query the XHTML snippet
 XML is Text
  • XML documents can be stored as text
    • databases typically have various datatypes for text storage
    • if the database supports Unicode, any XML document can be stored
  • The XML structure is completely invisible to the database
    • working with the XML requires querying and parsing the XML text
    • this kind of storage does not allow any querying of the XML content
 XML → ∗LOB XML as a Datatype
  • SQL supports a wide variety of datatypes
    • typed values are better than untyped values (they enable type-specific operations)
    • XML can be regarded as just another data type
  • Introducing a datatype lets the database recognize the data
    • XML data can be stored in some format (a persistent DOM)
    • databases can provide functionality avoiding parsing/serialization (DOM-based)
 XML Datatype Mapping XML to Models
  • Model-relevant data must be mapped to the database structures
    • this assumes there is a ER-model which describes the database structure
    • mapping XML is easy by definition because the XML is ER-compliant
  • Is the data accessed as table data?
    • if shredded data is only used to assemble it again, it is just performance overhead
    • if shredded data is accessed relationally, then shredding makes sense
 Shredding (XML → Columns) XML as First-Class Citizen
  • The defines XML as a sub-concept of ER
    • the overall structure of the database is relational
    • attributes may be of type XML, which means storing trees in tables
  • Tables are not the only way to see the world
    • XML trees are an alternative to tables, not a datatype
    • XML-centric applications should not be forced to use tables at all
  • XML can be regarded as replacing the ER-concept altogether
    • the database simply stores XML documents
    • applications can store, query, update, and manage XML documents in the database
 XML DBMS XML in Relational Databases RDBish XML
  • XML schemas can be designed with databases in mind
    • avoid unbounded repetitions of elements
    • avoid choices
    • avoid ordered content
    • avoid mixed content
    • avoid recursion
  • Many XML schemas are designed RDBish for compatibility reasons
    • it was decided that the XML should allow easy mapping to tables
    • the person designing the schema thinks in ER
    • the schema has been generated from a relational database schema
 Problematic XML
  • XML in its full glory is too much for tables
    • XML has been developed as a document format
    • XML is about hierarchy (which intentionally have been left out of ER)
    • XML is about highly irregular structures
  • XML often is said to have two flavors
    • data-oriented XML: regular data which can be easily mapped to tables
    • document-oriented XML: irregular structures which are hard to map to tables
    • real-world XML often is a bit of both (e.g., content and metadata)
  • Hybrid approaches sometimes are a good solution
    • data-oriented can be shredded and stored in tables
    • the document-oriented rest is stored as one object (text or XML)
 SQL/XML SQL/XML:2003
  • SQL/XML provides s
    • it introduces
    • it introduces a number of operations for generating XML from query results
    • it defines mappings to bridge both worlds (SQL and XML)
  • SQL/XML does not change anything about the database model
    • data is still stored in tables only
    • a column of a table may use the XML type
    • queries may return results in XML rather than as SQL result sets
 SQL/XML Example 
SELECT
  e.EmpId,
  e.FirstName,
  e.LastName,
  e.StartDate,
  e.EndDate
FROM Employees e WHERE e.EmpId = 12
SELECT
  XMLELEMENT (NAME "employee",
    XMLATTRIBUTES(e.EmpId as "id"),
    XMLELEMENT(NAME "names",
    XMLELEMENT(NAME "first", e.FirstName),
    XMLELEMENT(NAME "last", e.LastName)),
    XMLELEMENT(NAME "hire-dates",
      XMLATTRIBUTES(e.StartDate as "start", e.EndDate as "end")))
FROM Employees e WHERE e.EmpId = 12
 SQL/XML:2007
  • Adds the concept of XML Tables
  • XML Tables are not tables, they are containers for XML
  • SQL/XML:2007 changes the database's data model
    • it is now possible to have a database with no tables
    • likely use cases are to have both: traditional and XML tables
  • SQL/XML:2007 defines a hybrid database: relational and XML database
 Conclusions Tables and Trees don't Mix
  • Tables and trees are different metamodels
  • Different technologies are used to handle these different models
  • Think before choosing the wrong tool
 Database Technologies do Mix
  • Relational databases are good tools for regular data
  • XML databases are good tools for documents
  • SQL/XML:2007 defines a database that supports both
  • Applications can choose the best mix of tables and trees
 Content Syndication 2009-10-06 Identifying Atom Atom · AtomPub · Validator For many information sources on the Web, it is useful to have some standardized way of subscribing to information updates. Syndication formats such as RSS and Atom can be used by these information sources to publish a feed of updated information items. While RSS and Atom are read-only formats, the Atom Publishing Protocol (AtomPub) build on top of Atom and provides a protocol for submitting new items to feeds. Abstract

Content Feeds
  • Early Web content was static or updated very infrequently
    • there was not yet the requirement to reuse content in different contexts
  • Frequently updated Web content quickly became a very common scenario
    • as commercial interests took over the Web, users should have a reason to re-visit a site
    • presenting a steady stream of new content creates the image of a live Web site
  • There are two major use cases where HTML is not sufficient
    1. users want an efficient way to get the updated content from a site
    2. sites want to aggregate updated content from other sites and re-publish it
  • are designed to support these two use cases
    • container formats for updated items
    • a small amount of metadata about these items for automated processing
 Syndication Formats RSS RSS History
  • The Myth of RSS Compatibility provides a good overview
  • RSS is a schoolbook example for why standards are a good thing
    • was created for the My Netscape portal in March 1999
    • RSS 0.91 (a simplification) was introduced in July 1999 (as an interim solution)
    • the AOL/Netscape merger removed the format from the company's portal
    • RSS was without an owner, and different parties claimed/denied ownership
    • was created by an informal developer group
    • RSS 0.92 (and 0.93 and 0.94) were published without acknowledging RSS 1.0
    • finally, was released as a follow-up to the RSS 0.9x versions
  • Using RSS has become an exercise in managing a menagerie of versions
 RSS 0.9
  • RSS means RDF Site Summary (or Rich Site Summary?)
    • based on an RDF draft and not compatible with the final RDF specification
    • RDF was considered too cumbersome and unstable
    • 0.90 (proto-RDF) was quickly replaced by the non-RDF 0.91 version
  • RSS 0.92+ versions were developed as unilateral specifications
    • starting with RSS 0.91, RSS means Rich Site Summary
    • it is no longer built on RDF, instead it simply uses XML
    • the 0.9x branch eventually was renamed to
RSS 0.91 Example RSS 1.0
  • RSS means RDF Site Summary (this time for real)
    • based on the final RDF specification and thus incompatible with any
    • developed when the Semantic Web and RDF were first heavily marketed (1999)
    • RDF was expected to become the format for metadata on the Web
  • RSS 1.0 makes heavy use of XML Namespaces
  • RSS 1.0 introduces features which were not present in 0.91
    • date information for published items (very relevant for news feeds)
    • individual authors for various items in a feed
  • RSS 1.0 is the latest version of RDF-based RSS
    • the Semantic Web wave is not over yet, but RDF has lost its novelty appeal
    • for a more XML-oriented encoding, provides a better foundation
 RSS 1.0 Example RSS 2.0
  • RSS now means Really Simple Syndication
    • RSS 2.0 is the continuation of the 0.91 branch (which dropped RDF)
    • together with it is the most popular version of RSS
    • migration from 0.91 to 2.0 is easily possible
  • RSS 2.0 tries to avoid the use of XML Namespaces
  • RSS 2.0 is increasingly used with extensions for vendor-specific information
    • the RSS core is minimal, so many applications need extensions
    • many extensions have overlapping functionality
    • most extensions have unclear semantics and unclear versioning policies
 RSS 2.0 Example The Case for Content Management
  • RSS is very rarely produced by hand
    • by definition, RSS contains redundant information for a specific purpose
  • If a is used, RSS can be generated
    • basic metadata can be generated by the CMS (title, author, date)
    • better tagging of content results in better tagging of feeds
    • well-tagged feeds are better foundations for large-scale reuse of feed items
  • Blogging is simply a specialized case of a CMS
    • Web-based interface for controlling everything
    • strictly time-ordered sequenced of published items
    • navigation features primarily based on the time-specific facets of the blog (maybe tags)
    • all blogging tools include feed support
 Consuming RSS
  • RSS feeds often have quality problems
    • surprisingly often feeds do not even deliver well-formed XML
    • the use of embedded markup in RSS is not well-defined
  • Writing an RSS reader from scratch is not a good idea
  • There are three major tasks which RSS readers must do
    1. accept non-XML RSS feeds and fix them to be XML
    2. look at the feed contents and bring them into a unified form
    3. produce a unified view of feeds regardless of the RSS version
 RSS Technical Problems
  • What to put into an item's description
    • the fundamental question is whether a description is text or HTML
    • if there is no well-defined way, then interpretation is client-specific
      • <description>This is a <em>very important</em> blog post …
      <description>This is a &lt;em>very important&lt;/em> blog post …
      <description>This is a blog post about <em> in RSS feeds …
      <description>This is a blog post about &lt;em> in RSS feeds …
      <description>This is a blog post about &amp;lt;em> in RSS feeds …
  • Underspecified and not very robust in various other areas
    • broken RSS is accepted by most readers (but fixing it can change the interpretation)
    • the interpretation of relative URIs is not mentioned in the specifications
    • some minimal semantics (classification) for items would be very useful
 RSS Political Problems
  • Multiple and incompatible are still in widespread use
    • and are incompatible by design (RDF vs. non-RDF)
    • none of the RSS versions is maintained by a universally accepted standards body
  • None of the specifications is being updated or fixed
    • some of the lessons learned by RSS deployment are not used in a new version
    • it is unlikely that a new version will be produced which merges the RSS landscape
  • Invent something new instead of trying to fix RSS
    • started in 2003 (called Echo at first)
    • W3C or IETF would have been promising candidates for a new RSS
    • W3C is more formal, IETF is more developer-centered
    • IETF was chosen over W3C because the of Atom community's preferences
 Atom Atom History
  • RSS's shortcomings were very apparent and could not be fixed
  • In mid-2003, discussions started about an improved format
  • It also became apparent that the format should have a protocol
  • Atom 0.3 was released in December 2003 but had no formal home
  • IETF was chosen as the new home with a working group in June 2004
  • RFC 4287 was published in December 2005
  • AtomPub has been published as RFC 5032 in October 2007
 Atom vs. RSS
  • Standardized by the IETF (well-defined process)
  • Classification of entries (user-defined categories)
  • More XML-like markup design (more nesting)
  • Namespaces are used and supported as standard mechanism
  • Atom feeds must be well-formed XML (there even is a schema)
  • Interpretation of content is well-defined (various content types)
  • Support for xml:lang and xml:base
 Atom Example Atom Content
  • RSS had no safe way of finding out what an entry's content is
    • this led to different implementations being smart about what the RSS author really wanted
    • one of Atom's main goals was to improve this in a well-defined way
    • Atom allows escaped markup (the only way to include non-XML HTML in an XML format)
  • Each content element should have a type (the default is text)
  • Atom's content interpretation algorithm (use first applicable rule):
    1. if type is text, no child elements are allowed (plain text content)
    2. if type is html then RSS's method of escaped markup is used
    3. if type is xhtml then there must be an div containing XHTML markup
    4. if type is an XML media type then the content should be treated as this type
    5. if type starts with text/ then no child elements are allowed
    6. for all other values, the content must be an base64-encoded entity of the specified MIME type
 Atom Content Examples 

  

    One bold foot forward
  

]]>
The "atom:content" element either contains or links to the content of the entry. The content of atom:content is Language-Sensitive.]]>
The <code>atom:content</code> element either contains or links to the content of the entry. The content of <code>atom:content</code> is <a href="http://www.ietf.org/rfc/rfc3066.txt">Language-Sensitive</a>.]]>

iVBORw0KGgoA … TAAAAAElFTkSuQmCC
]]>
]]>
 Atom Categories
  • Atom allows to assign categories to entries
    • each category element must have a term attribute for the category
    • an optional scheme identifies the categorization scheme (ontology, taxonomy, …)
    • an optional label attribute provides a human-readable label for the category
  • AtomPub defines a document format for
  • Three different cases of categorization can be distinguished
    1. use a well-known scheme (such as Dublin Core)
    2. use a private but well-designed scheme (which has a URI and can be reused reliably)
    3. use tags without schemes, which then are little more than content labels
  • Widely-known tags are not easy to handle
    • they are more than just privately assigned tags
    • there is no formal scheme for them, just an emerging consensus
 Switching from RSS to Atom
  • Generate both feeds but serve RSS with an HTTP redirect (301)
    • old subscribers with broken clients can still use the RSS feed
    • old subscribers with correct clients will use the Atom feed
  • Atom exposes more information than RSS (category for tags)
    • the mapping of publishing info to the feed has to be changed/extended
    • for standard metadata use Atom's built-in metadata elements
    • for application-specific metadata consider reusing an existing metadata schema
  • Atom can be used to publish snippets as well as full content
    • content allows any type of content to be used and may contain a complete entry
    • summary allows only text and should provide a condensed version of an entry
    • some Atom sources publish two feeds for summaries and content
  • Generate good Atom and downgrade it to RSS 1.0 & 2.0
 Syndication Aggregation End-User Aggregation
  • Users often have a small number of preferred Web sites
    • news can be tracked by subscribing to their feeds
    • this requires an feed-aware client (Firefox is not a good feed reader)
  • Ajax-based feeds could be implemented as a client-side application
    • retrieve the feed, transform it to HTML, and render the result
    • but the security restriction of XMLHttpRequest gets into the way
    • there is a way around this as used in the Ajax RSS reader
    • Google Reader accesses Google's own copy of the feed ()
  • How can users find a feed?
    • feeds have URIs (they are Web resources) and can be referenced from within HTML
<link rel="alternate" type="application/rdf+xml" title="…" href="…" />
<link rel="alternate" type="application/rss+xml" title="…" href="…" />
<link rel="alternate" type="application/atom+xml" title="…" href="…" />
 Aggregation Intermediaries
  • RSS is a frequently used format between content providers
    • news agencies want to distribute news items as easy as possible
    • by using a single format, producers and consumers can more easily interoperate
  • The caused publishers to look for something better
    • had a head start and still is widely used
    • is a much better format and will eventually replace RSS
  • User-configured portals are the merger between both scenarios
    1. users select a number of feeds as their preferred information sources
    2. the feeds are presented on a single personalized Web page
    3. by sharing and re-publishing items, users are becoming aggregation intermediaries
 FeedBurner Fixing Feeds Load Balancing Statistics/Analytics Atom Publishing Protocol Syndication Format Protocols
  • LiveJournal (very simple text-based protocol)
    • not very good at handling structures (re-inventing for encoding structure)
  • Blogger (now at Google after Google bought Pyra)
    • no support for titles or any other sort of entry metadata
    • protocol from the early days of blogging before tagging became popular
  • MetaWeblog (an attempt to improve Blogger)
    • extends Blogger using a very bad design (RSS XML as XML-RPC structure encoded as XML)
  • Atom Publishing Protocol (AtomPub) is an attempt to provide a clean alternative
    • use the same document structures for feeds and the protocol interacting with them
    • use a REST approach to provide a simple and Web-compatible protocol
    • add and for additional tasks
 RESTified Syndication
  • Atom is a format for retrieving a set of entries as a feed document
    • feeds often are time-based and are refreshed periodically or whenever needed
    • feeds can use any other strategy for deciding what to publish
  • Read-only access to feeds should be complemented by full access
    • full access needs the CUD out of the CRUD set of operations
    • many Web-centric technologies try to build on the Web's REST model of interaction
  • AtomPub builds on Atom and adds a REST-based protocol on top of it
    • POST for creating new entries (sending the request to the collection)
    • PUT for updating existing entries (overwriting the existing entry)
    • DELETE for deleting entries from a collection
 Collections, Members, Entries, Media
  • AtomPub's top-level concept is a collection
    • collections are used for managing and organizing members
    • Atom feed documents are the representation of collections
  • Members of a collection can be entry and media resources
    • entry resources represent metadata and are represented as Atom entries
    • media resources can have any media type and are the data described by entries
    • a media link entry is an entry associated with a member
 Protocol Summary
Resource HTTP Method Representation Description
Introspection GET Atom Service Document Enumerates a set of collections and lists their URIs and other information about the collections
Collection GET Atom Feed A list of member of the collection (this may be a subset of all entries in the collection)
Collection POST Atom Entry Create a new entry in the collection
Member GET Atom Entry Get the Atom Entry
Member PUT Atom Entry Update the Atom Entry
Member DELETE n/a Delete the Atom Entry from the collection
Service Documents
Service Documents represent server-defined groups of Collections, and are used to initialize the process of creating and editing resources.
  • The real top-level construct of AtomPub is the workspace
    • collections on a server are organized into different workspaces
    • workspaces have no AtomPub semantics and no operations can be performed on them
  • Service documents list constraints on the members of collections
    • accept specifies a comma-separated list of media ranges (with entry as special value)
    • categories defines the list of categories that can be applied to members (can be fixed)
    • AtomPub servers are likely to reject operations not satisfying these constraints
 Service Document Example Category Documents
  • Categories are important for creating and reading entries
    • they may contain metadata using any classification scheme
  • contain a list of allowed categories
  • AtomPub defines a document format for standalone category documents
    • a useful interface between AtomPub systems and other systems using classification schemes
 Category Document Example Conclusions Semantic Web Light
  • Syndication creates representations for universal concepts
  • Atom adds some concepts to RSS's model
  • Syndication revolves around the idea of interacting with items
  • Atom-based interaction is one way of implementing REST
  • For more semantics, Atom is only the foundation
 Media Types 2009-10-08 MIME Respect MIME · Registry A3 assigned (due date: 10/18) One of the most important aspect of computer-based communications is the concept of media types, the question what type of information some digital artifact represents, and how it is encoded. The most common standard for this information is the scheme introduced by Multipurpose Internet Mail Extensions (MIME). Media types can be negotiated by peers communicating through HTTP. Some media types allow fragment identifiers, which allow references to a resource to identify a fragment of the complete resource. Abstract

Multipurpose Internet Mail Extensions (MIME)
  • Basic e-mail only supports ASCII text messages
  • MIME was introduced in 1993 to standardize a more powerful message format
    • multiple objects in a single message
    • text having unlimited line length or overall length
    • character sets other than ASCII, allowing non-English language messages
    • binary or application specific files
    • images, audio, video and multi-media messages
  • Resource types are necessary for every automated action with resources
    • Unix started with /etc/mime.types, a list of mappings between extensions and media types
    • the Unix file command uses simple fingerprints (specified in /etc/magic)
    • double-clicking in GUIs needs a file association (based on the file's type) to work
 Unix File Type Handling Windows File Type Handling Media Types and the Web Browsers and Resources
  • Web browsers retrieve resources and render them
    • HTTP can transfer any kind of resource (binary resources must be transfer encoded)
    • resource types cannot (and should not) be inferred from the URI
  • HTTP combines data transfer, transfer management, and metadata
    • basic information about a resource (modification date)
    • information describing the resource's type (media type) and content (language)
    • can be used to request a specific resource variant
  • The resource type received may or may not be supported by the browser
    • built-in support is provided for the core Web resource types (HTML, GIF, JPEG)
    • plug-in support is an add-on to the browser for popular types (PDF, Flash)
    • external applications are standalone applications invoked by the browser
 Firefox Media Type Handling Media Type Control in Browsers
  • In practice, media type control is less than perfect
    • getting control over media types means getting control over your computer
    • applications are rarely written in a perfectly well-designed way
  • XML is handled very poorly by Firefox
    • rendering XML is unbelievably slow (and very primitive)
    • if the XML has errors (real or imagined by Firefox), there is no way to get to it
    • helper applications can be registered, but this is ignored by Firefox
  • Competing application suites hijack as many media types as possible
    • QuickTime and Windows Media Player support the same media types
    • installing or updating these packages often changes many media type settings
 Media Types Content Types
  • MIME splits the world of resource types into Content Types and Subtypes
    • Content types are the main classification of a resource type
    • qualify the format and encoding used for the content
  • Content types classify the world of resource type into 8 areas
    • audio for media types representing exclusively audio signals
    • image for any media type representing two-dimensional images
    • message for resources representing e-mail messages
    • model for complex representations of physical objects (very unpopular)
    • multipart for MIME entities containing multiple individual MIME-tagged resources
    • text for mainly textual material (e.g., HTML is considered to be text)
    • video for media types combining moving pictures with audio
    • application for any resource which cannot by classified anywhere else
    • (example is only used for media type examples, not for real-world resources)
 Subtypes
  • Within each content type, many different data formats are in use
    • content types only allow a broad classification
    • subtypes allow the identification of a specific data format of a resource
  • Subtypes are expected to be registered with the IANA
    • unregistered subtypes can be used but must have a x- prefix
  • Additional qualifiers can be used to be more specific
    • text/plain is the media type for plain text files
    • plain text files have additional properties such as character encoding and language
    • text/plain can be further qualified to text/plain; charset=iso-8859-1
    • not all qualifications are available as media type (e.g., language is not addressed)
 What is XML?
  • XML can be regarded from various point of view
    • XHTML is (almost) identical to HTML and should be regarded as text
    • configuration files are data for some program and should be regarded as application
    • XML schemas can be mixes of these two extreme cases
  • RFC 3023 registers a set of XML-related media types
    • application/xml for generic XML usage (data oriented XML)
    • text/xml for textual XML usage (document oriented XML)
    • application/xml-dtd for DTDs (which are not XML documents)
    • application/xslt+xml (proposed) for XSLT stylesheets (text/css is defined by RFC 2318)
    • image/svg+xml (proposed) for vector graphics using
    • application/xhtml+xml is defined by RFC 3236
    • … and a number of other suggestions for future XML media types
 Media Type Registration
  • Media types need to be registered together with a documentation
    • this makes sense if it is assumed that registered types should be openly accessible
    • this becomes complicated if the types are proprietary and not publicly documented
  • It makes sense to register types even if they are not publicly documented
    • if a Word document is sent by e-mail it should be opened by the Word application
    • IANA registers vnd. prefixed subtypes with less requirements than regular types
    • vendor specific types are often undocumented and may change significantly over time
  • Using well-defined types makes handling resources more stable
    • the IANA registry contains hundreds of types (most of them application types)
    • when designing applications dealing with various content types, use media types as the foundation
 <mime>application/msword</mime> Media Type Text Content Types Plain Text
  • RFC 2046 defines plain text files as a basic media type
    • any text file that does not contain structures which are intended for machine-based processing
    • even files do not count as plain text
  • Guessing of character encoding is hard and unreliable and should be avoided
    • the character encoding can be specified with an additional parameter: text/plain; charset=iso-8859-1
    • if no such parameter is present, ASCII should be assumed as the character encoding
  • For more specific text subtypes, various other subtypes exist
    • calendar for information about calendar entries
    • javascript for JavaScript code (should now be marked as application/javascript)
    • sgml and xml for text with additional markup
 HTML
  • RFC 2854 registers text/html for HTML documents
    • like the character encoding can also be specified as a parameter
    • it is not specific for some version of HTML (version information can be found in the HTML document)
  • are also defined by the media type registration
  • HTML in many cases needs additional resources to be self-contained
    • images which are references by img elements (maybe external image maps)
    • other media referenced by object or applet (or the deprecated embed)
    • stylesheets or scripts which are referenced in the document head (they may reference other files …)
    • generating a truly self-contained HTML is a rather hard task
  • MIME can be used to represent a self-contained MHTML (RFC 2557)
 Comma-Separated Values (CSV)
  • RFC 4180 defines a textual format for spreadsheet data
  • CSV has been used for a long time, but some of the details were solved differently
  • Defining a media type makes it easier for implementations to know what to expect
    • the registration not only registers the type, but also defines it
  • CSV is not overly complex, but some issues have to be solved
    • how to separate lines (CRLF)
    • how to end the file (CRLF is allowed but optional)
    • are there headers allowed (yes, but they are not marked as such)
    • may different lines use different numbers of fields (no)
    • are spaces significant (yes)
    • are quotes significant (no, they are delimiters, so quotes as values must be escaped)
    • how to treat fields with CRLF, commas, or quotes (enclose the value in quotes)
 Application Content Types JSON
  • RFC 4627 registers JSON as a media type
  • The definition of JSON is derived from ECMAScript's object literals
  • JSON is a very limited notation intended for simple structures
    • it allows the four primitive types strings, numbers, booleans, and null
    • it allows the two structured types objects (unordered) and arrays (ordered)
  • The value of the media type in this case is the clean integration into the Web
    • information providers may choose to expose their data in JSON and XML
    • can be used to specify which representation is preferred
 Image Content Types Graphic Interchange Format (GIF)
  • RFC 2046 registers the oldest graphics format on the Web
  • GIF was subject of a long patent debate
    • the compression technique of GIF (LZW) had been patented by Unisys (1983)
    • Unisys wanted to get licensing fees from all commercial online uses of GIF
    • was developed as an effort to develop a copyright-free format
    • in 1999, Unisys changed its tactics and wanted to collect one-time fees ($5000-$7500) from all users
    • all GIF-related LZW expired in 2003/2004, so GIF is freely available now
  • GIF's poor features make PNG the better choice anyway
    • 8 bit color (requires dithering for photographs), binary transparency
    • GIF's animation feature is the only thing that is not available in PNG …
Joint Photographic Experts Group (JPEG)
  • RFC 2046 standardizes the second popular image format for the Web
  • JPEG has been specifically designed for photographs
    • it always is lossy (it cannot preserve the complete information from a random bitmap)
    • it uses perception-based compression (for example, color precision is sacrificed for brightness)
Q = 50, filesize 15,138 bytes Q = 10, filesize 4,787 bytes Q = 1, filesize 1,523 bytes
Portable Network Graphics (PNG)
  • PNG is registered as image/png and is the third major image format
    • PNG was intended to be a royalty- and copyright-free replacement of GIF
    • image formats need to supported by browsers and thus take a long time until they are established
    • IE6 implements PNG in a very rudimentary form, IE7 handles PNG correctly
  • PNG has some advantages over GIF and JPEG
    • lossless, compressed palette, grayscale, or true color images
    • 8 bit alpha channel for gradual opacity (blending into the background)
  • JPEG still is the preferred format for photographic pictures
  • GIF still is the preferred format for animated images
    • MNG and APNG are two available but not widely supported PNG animation formats
 Fragment Identifiers Identification of Resource Fragments
  • URIs identify a resource (based on a scheme and a scheme-specific part)
    • URIs do not necessarily identify a specific representation of a resource
    • any representation-specific operation needs to look at the resource type
  • Fragment identifiers can be used to identify a part of a resource
    • http://dret.net/netdret/publications#wil02h
    • fragments are a client side concept (the HTTP GET requests the complete resource)
    • if the client supports fragment handling, the identifier is interpreted
  • Fragment identifiers and content negotiation have a problematic relationship
    • keeping fragment identifiers stable across resource variants may be hard
    • nothing in the fragment identifier associates it with a specific resource variant
 HTML Fragment Identifiers
  • HTML allows to address named/identified elements in the HTML document
    • the first HTML versions required named <a name="frag-id">incoming anchors</a>
    • newer HTML versions allow <p id="frag-id">every element to have an id</p>
    • browsers support both ways, but the id variant should be preferred
  • Only named/identified fragments can be identified
    • 99.99% of all page authors do not routinely add identifications
    • tools may be smarter and take over that task (e.g., Movable Type identifies all relevant elements)
    • for most pages on the Web this means users cannot link to most elements in them
  • Keeping fragment identifiers stable should be a goal of Web authors
    • identify the key fragments (and maybe provide a better UI than view source)
    • never change identifiers once they have been assigned
 XML Fragment Identifiers
  • XPointer is the language which defines XML fragment identifiers
    • XPointer's history and success is even worse than XLink's
    • the interesting part was never finished, the easier parts have been finalized
  • XPointer supports HTML's approach of identified elements
    • if there is simply a name after the #, it is a shorthand and treated like in HTML
    • … almost, actually, because IDs in XML are a bit more complicated (xml:id or schema-defined)
  • XPointer allows child sequences to navigate the document tree
    • …#element(/1/2) or …#element(intro/3/1)
    • the nice aspect is that elements without an explicit identification can be identified
    • the risky part is that any changes in the document structure may break the XPointer
  • XPointer in all its glory (W3C working draft December 2002 …)
    • an extension of XPath inheriting all the interesting features of XPath
    • adds the concept of range locations between to XPath-identified point locations
 Conclusions Know Your Resources
  • Handling Web resources and technologies requires a common vocabulary
  • Media types are a useful vocabulary for identifying resource types
  • Fragment identifiers add some complexity, in particular for resource variants
 Usability 2009-10-15 Contextual Design (Chapter 3, pp. 41-64) Heuristic Evaluation · useit.com According to the International Organization for Standardization (ISO), usability defines the extent to which a product can be used by specified users to achieve specified goals with effectiveness, efficiency and satisfaction in a specified context of use. We will discuss tradeoffs in the design of Web interfaces to support users goals, and present resources to aid design decisions. Scripting 2009-10-20 DHTML Best Practices · Tutorial · Wikipedia A4 assigned (due date: 11/1) Scripting is used on the majority of today's modern Web sites. Scripting can be used to improve the usability and accessibility of a Web site (for example for validating form data on the client side), it can vastly improve the user experience with new interface design (the smooth scrolling of Google Maps vs. older click to scroll map services), or it can be used to implement behavior that would be impossible without scripting (for example the online applications of Google Docs). Asynchronous JavaScript and XML (Ajax) takes Dynamic HTML (DHTML) to the next level by allowing server access from within scripting code. This is accomplished by using a standardized API for client/server communications, the XMLHttpRequest object. This objects allows using HTTP connections from within scripting code, and thereby allows scripting code to dynamically reload data from a server in response to user interactions. Abstract

Scripting on the Web
  • Web pages were static HTML
    • were the only interactive part of Web pages
    • interaction was only possible by clicking links and visiting new pages
    • CSS introduced limited dynamic behavior (such as mouseOver events)
  • Netscape invented the and LiveScript
    • Java was new and hip, so the language was renamed to
    • pages with scripting (a.k.a. Dynamic HTML or DHTML) allowed richer user interfaces
    • other browsers invented their own versions of DOM/JavaScript
  • Scripting was and is often used to implement missing functionality
    • good scripting supports graceful degradation (leaving the page functional)
    • bad scripting compromises accessibility when the scripting code does not work
  • Any non-trivial scripting has to deal with browser differences
    • help by providing a foundation to build on
 The Joys of Web Design Basic Scripting (DHTML) Basic Scripting (JavaScript) Basic Scripting (CSS) JavaScript Browsers are Platforms
  • Runtime environments are critical for running applications
    • popular computer environments are Windows, MacOS, Linux, and Java
    • popular mobile environments are iPhone, Android, Blackberry, S60, and JavaME/JavaFX
    • popular Web-based environments are JavaScript, Flash, and Java applets
    • popular Web-oriented environments are Silverlight and AIR
  • JavaScript is a scripting language supported by most browsers
    • access to the current document's DOM is the most important part of DHTML
    • JavaScript has code, functions, and interacts with the user through the browser
<p>The current time is <script type="text/javascript">
var currentTime = new Date() ;
document.write(currentTime.getHours() + ":" + currentTime.getMinutes()) ;
</script>.</p>

<p>The current time is .</p>
Compiled vs. Interpreted Languages
  • Programming languages can be compiled or interpreted
  • Compilers can do more than just translate
    • compiled languages are translated before they can be executed
    • check the code for errors that can be determined statically
    • augment the code for performance or analysis purposes
    • compile/optimize the code for different runtime platforms
  • Interpreters provide a less heavyweight environment
    • interpreted languages can be executed directly
    • any change in the code can be tested immediately
    • less tightly coupled software bundles than compiled packages
  • Scripting has become much more popular in the past years
    • server-side languages such as PHP, Ruby/Rails, and Python
 JavaScript and Browsers
  • Scripting used to be too slow for serious applications
    • processors have become much faster
    • language interpreters have become much smarter
  • Implementations now deliver high-performance scripting
  • Modern implementations allow sophisticated applications
    • Google Docs is a set of browser-based desktop applications
    • Processing.js is a JavaScript version of a popular visualization environment
  • Some platforms switch to interpreted languages
    • Palm's webOS uses JavaScript as the native language
    • Google's ChromeOS uses the browser's JavaScript as the native language
 Document Object Model (DOM) From HTML to DOM
  • HTML is a representation for hypermedia documents
    • a representation is required to store and transmit the document
    • HTML uses markup for representing the document structure
  • Browsers must render HTML documents (i.e., apply CSS and execute JavaScript)
    1. GET HTML from server and receive as text/html document
    2. parse document and deal with any errors by fixing them
    3. interpret document as if it had been error-free
    4. GET all additional resources (CSS, images, JavaScript, …)
    5. build internal model (DOM) based on error-free interpretation
    6. apply CSS rules to determine styling of document (e.g., margins and font sizes)
    7. render into visual structure
    8. start executing JavaScript code
    9. listen for events (keyboard, mouse, timer) and execute code
    10. discard everything and start over when user navigates to a different page
 Browser Handling of HTML Document
  • The document (HTML) is the interface language for Web applications
  • Most programming environments have visual interface models
    • almost everything has moved to window-oriented interfaces
    • Windows, MacOS, and Linux provide similar visual metaphors
  • Web applications must use HTML as their model for the interface
    • are a simple way to build an interface
    • forms can be extended with client-side helpers (validation, repeating entries)
 Object
  • Documents are static, programming is dynamic
    • documents and code must be connected
    • objects are a common abstraction in programming languages
  • Objects usually have a type and methods
    • types for HTML-based objects are based on HTML's elements
    • methods define the allowed to interact with objects
    • interactions can be read-only or they can change the document structure
Model
  • Models are idealized/abstract views of something
  • Model interfaces allow to expose that idealized/abstract view
  • DOM introduced a common way of how browsers deal with HTML
    • without a DOM, there can be no interoperable scripting
  • Abstractions are also limitations
    • some vendors add/support extensions to the basic DOM model
    • any code based on these extensions is not interoperable
 Markup to Trees
  • Browsers build a DOM tree from the (fixed) markup
    • scripting code never works on the markup text, it works on the tree
  • DOM trees are an abstraction of the markup
    • trees provide convenient navigation facilities
    • abstractions provide an insulation against irrelevant markup details
  • DOM is available in a large number of programming languages
    • is the language supported in Web browsers
    • DOM is also available for Java, C, C++, Python, Perl, C#, Ruby, …
 DOM History
  • DOM0 was invented by Netscape (backing the LiveScript/JavaScript)
  • DOM1 was the first DOM version produced by the W3C
  • DOM2 is the currently available stable version of DOM
    • major changes to be compliant with XML and the XML Infoset
  • DOM3 is highly modularized and still under development
    • more XML technologies such as the ability to use XPath
 DOM2 Map DOM3 Map Asynchronous JavaScript and XML (Ajax) Ajax = DHTML + HTTP
  • uses JavaScript locally
    • the scripting code reacts to user events and accesses the DOM structure
    • changes are either hardcoded or derived from user events
  • Ajax adds an HTTP request method to JavaScript
    • scripting code can now request additional data from an HTTP server
    • changes can thus be made based on any data received from the server
  • Ajax dramatically reduces the number of page reloads
    • any change of the page can be done without a complete reload
    • based on user interactions, parts of the page can be reloaded
  • Ajax has the same interoperability problems as DHTML
 Ajax and DHTML JavaScript and XML
  • The XMLHttpRequest API has been built for requesting XML via HTTP
    • this is useful because XML is the most popular data format
    • all requested data has to be processed by using XML access methods in JavaScript
  • JavaScript does not have XML as its internal data model
    • the XML received via XMLHttpRequest has to be parsed into a DOM tree
    • DOM access in JavaScript is inconvenient for complex operations
    • alternatively, the XML can be mapped to JavaScript objects (also requires parsing)
  • JavaScript Object Notation (JSON) encodes data as JavaScript objects
    • more efficient for the consumer if the consumer is written in JavaScript
    • this turns the generally usable XML service into a JavaScript-oriented service
    • for large-scale applications, it might make sense to provide XML and JSON
    • using , this can be negotiated on the HTTP protocol level
 JSON Example JSON via Content Negotiation
  • XML or JSON are just different representations
    • they represent the same underlying resource (as identified by the URI)
  • allows to specify preferences
    • clients specify preferences and server respond with the best match
  • HTTP Accept specifies the accepted
    • resources may be available in HTML, XML, or JSON
    • depending on the HTTP request, the server responds with the best representation
    • reduces processing time on the client and can be cached
  • Really smart Ajax frameworks could even hide the content negotiation
    • request JSON or XML, with XML being the lower priority
    • if XML is sent by the server, it has to be parsed into a JavaScript object
    • the end result is always a JavaScript object for the framework user
 JavaScript Frameworks Abstraction and Reality
  • Browsers are not entirely standards-compliant
    • Acid Tests are a way how to test browser compliance
    • compliance depends on what you test for (versions of the standards)
  • Running Acid3 for current browsers is disappointing
    • Chrome and Safari are equal (because they both use WebKit
    • Firefox and Opera are not that bad (but not perfect)
    • IE8 is a disaster
  • In some cases, implementations have to make guesses
    • complex combinations of HTML, CSS, and JavaScript interactions
  • JavaScript frameworks have two major functions
    1. hiding the fact that browsers need a lot of special case handling
    2. providing support for common Web design patterns
 Web Design Patterns
  • Many Web pages use similar ideas/visualizations
  • Factoring them into design patterns enables tool support
  • Providing access to a tree-structured set of resources
  • Displaying image captions based on mouse events
  • Tabs are well-known from desktop applications and popular in Web design
    • Ajax Tabs can even load content dynamically
  • Image-heavy Web sites might need image viewing support
 Popular Frameworks
  • Different needs produce different frameworks
  • There is no such thing as the best JavaScript framework
    • for any given project, decide on the support you need
    • evaluate frameworks for the support they provide
    • evaluate for functional requirements (is there a collapse/expand folder view?)
    • evaluate for non-functional requirements (is the framework openly licensed)
 Important Framework Questions
  • How big is it?
  • How is it licensed?
  • How is it maintained?
  • How well does it support graceful degradation?
  • How well does it mix with other JavaScript code?
  • For professional Web development, don't overuse effects
 Conclusions
  • Scripting has become as essential part of Web-based applications
  • DHTML is local scripting, Ajax is scripting + server access
    • Ajax implements a lightweight client/server model
    • Ajax can range from helper functions to complete local applications
    • Creating better interfaces for Ajax applications is a research issue
  • Deciding between client-side and server-side is hard
  • JavaScript frameworks help developing script-based applications
  • Graceful degradation becomes more important on the mobile Web
 Content vs. Context 2008-10-22 A5 assigned (due date: 11/8) The Web often is regarded as a content delivery platform (Web 1.0) or as an application development platform (Web 2.0). However, as another part of the Web 2.0 model, context also has become much more important, because (a) users can now more easily contextualize content by creating their own content, and (b) mechanisms such as social networking provide additional context that is bound to a user's identity and the social networks in which this user is engaged. In this lecture, we discuss the move from pure content to a more contextualized view of the content on the Web, and we discuss possible developments and their technical and non-technical implications. Abstract

Content Delivery Putting Content into Context
  • Content are decontextualized Web pages
  • Blogging mediates content
    • blogs link to content and put it into context
    • the context of the blog is unknown (invisible readership)
  • Twitter mediates content and context
    • tweets link to content and put it into context
    • twitter's follower concept makes readership explicit
    • retweeting reinforces the validity of context
 Content a la Web 1.0 Content and Blogs Content and Twitter Content and Annotations Context Types Semantics as Context
  • Knowing more about content helps finding it
  • Content can be more explicit about its structure
  • Content can be embedded into contextual meaning
    • ontologies help describing and relating concepts
 Location as Context
  • Location is a central property of real-world objects
  • Knowing location can be critical to utility
  • Location on the Web still is not exposed properly
    • many applications use location internally
    • using location across applications/services is still unexplored
 Social Context
  • Social groups can be critical for information access
  • For the longest time, social groups were essential
    • information is only available as made available by society/culture
    • Web-based information access removes this filter
  • Social context is important in two distinct ways
    1. finding content made available through the social/cultural filter
    2. contextualizing content my using/accessing/annotating it
 Time as Context
  • Content often has an implicit aspect of time
  • Finding something should depend on (at least) two time dimensions
    • the time when the content was created
    • the time when the content was searched/accessed
    • optionally, updates and annotations can be used as well
 Conclusions
  • Content is more valuable with context
  • The Web becomes increasingly context-aware
  • More ubiquity means more opportunities for contextualization
  • Context often is partly implicit and hard(er) to control
  • Context technologies develop much faster than content technologies
 Picture Formats 2009-10-27 GIF · JPEG · PNG Pictures are the only multimedia content on the Web that is widely supported by standardized formats. The most important picture formats are the Graphics Interchange Format, the Joint Photographic Experts Group (JPEG) format, and the Portable Network Graphics (PNG) format. These picture formats target different application areas and depending on the picture material, choosing one format over the other can make a big difference. Audio and video in many cases are not handled by the browser itself, but are included in this overview of multimedia on the Web. Abstract

Images vs. Graphics
  • Pictures can be encoded in a wide variety of ways
  • Images are bitmaps of pixels that represent a picture
    • it takes some kind of scanning process to produce images
    • images have a certain resolution based on the quality of the scanning process
    • scanning can take place in a scanner, in a fax machine, or in a camera's CCD
  • Graphics are composed out of graphic primitives
    • graphics can be searchable, stylable, and scalable
    • the graphics format can have very different capabilities (2D vs. 3D)
  • Graphics preserve model-level information
    • this only makes sense if there is a model
    • rendering and styling can be an expensive process (e.g., video games)
    • images can be a snapshot of some specific view of graphics
  • Today's Web supports images, but not graphics
 Image Formats Graphics Interchange Format (GIF) Graphic Interchange Format (GIF)
  • RFC 2046 registers the oldest graphics format on the Web
  • GIF was subject of a long patent debate
    • the compression technique of GIF (LZW) had been patented by Unisys (1983)
    • Unisys wanted to get licensing fees from all commercial online uses of GIF
    • was developed as an effort to develop a copyright-free format
    • in 1999, Unisys changed its tactics and wanted to collect one-time fees ($5000-$7500) from all users
    • all GIF-related LZW expired in 2003/2004, so GIF is freely available now
  • GIF's poor features make PNG the better choice anyway
    • 8 bit color (requires dithering for photographs), binary transparency
    • GIF's animation feature is the only thing that is not available in PNG …
Joint Photographic Experts Group (JPEG) Joint Photographic Experts Group (JPEG)
  • RFC 2046 standardizes the second popular image format for the Web
    • ISO 10918 is the standard for the actual image format
  • JPEG has been specifically designed for photographs
    • it always is lossy (it cannot preserve the complete information from a random bitmap)
    • it uses perception-based compression (for example, color precision is sacrificed for brightness)
Q = 50, filesize 15,138 bytes Q = 10, filesize 4,787 bytes Q = 1, filesize 1,523 bytes
JPEG 2000
  • JPEG has some problems (for example, it is never lossless)
  • JPEG 2000 is a completely new standard
    • it uses wavelets instead of DCT as the compression algorithm
    • it includes support for lossless encoding (JPEG is always lossy)
    • it even comes with its own transmission protocol (JPIP)
    • image/jp2 (JP2) and image/jpx (JPX) are the two JPEG 2000 MIME types
  • Support for JPEG 2000 is good but not universal
 Portable Network Graphics (PNG) Portable Network Graphics (PNG)
  • PNG is registered as image/png and is the third major image format
    • PNG was intended to be a royalty- and copyright-free replacement of GIF
    • image formats need to supported by browsers and thus take a long time until they are established
    • IE6 implements PNG in a very rudimentary form, IE7 handles PNG correctly
  • PNG has some advantages over GIF and JPEG
    • lossless, compressed palette, grayscale, or true color images
    • 8 bit alpha channel for gradual opacity (blending into the background)
  • JPEG still is the preferred format for photographic pictures
  • GIF still is the preferred format for animated images
    • MNG and APNG are two available but not widely supported PNG animation formats
 Alpha Channel Effects
Other Image Formats Tagged Image File Format (TIFF)
  • Standard file format format for scanned images
    • none of the limitations of
    • ability to represent any kind of bitmapped image information
    • compression is supported but always lossless (not as effective as JPEG)
  • Popular for scanned images and similar applications
    • native support in browsers is the exception (only Safari)
 Icons
  • Original favicon (a.k.a. page icon or urlicon) format
    • the image format used by Windows for its icons (which is identical to the cursor format)
  • Important on the Web only for Web site icons
    • most browsers will display the icons in the address bar
    • most browsers will remember the icon for bookmarks and shortcuts
  • Icons appear as meta information of a Web page
    • standards are not really tight and based on best practices and browsers
    
]]>
 Graphics Formats Image Concepts
  • Pictures of the real world are always lossy
    • there is no (accessible) underlying model of the real world
    • the real world is always pictured as well as technically possible
  • Graphics start from some model and turn that into an image
    • an image of a circle will always appear jagged on close inspection
    • a graphical picture of a circle is always perfectly round
  • Graphics can be customized based on presentation needs
    • line and text color can be coordinated with the rest of Web page design
    • any information can be used as input to the graphics → image conversion
  • Graphics can be searched for concepts and contents
 Scalable Vector Graphics (SVG) Graphics for the Web
  • Identified as important media type a long time ago
    • W3C work on SVG for a long time
    • complex format with little support and little success on the Web
  • Embedding SVG in HTML is more art than science
    • using SVG requires browser-specific HTML code
    • backwards compatibility must support bitmapped version of the SVG pictures
  • Support for SVG is provided by many newer tools
    • the SVG code produced not always is a very good version of SVG
    • SVG has not yet been adopted as something that people really care about
 Image vs. Graphics

Source: carto.net

 SVG is XML Video and Audio Download vs. Streaming
  • Web resources usually are downloaded
    • browsers may choose to implement incremental rendering (e.g., HTML or images)
    • the resource is completely downloaded and stored
  • Streaming means that there is no complete download
    • TV and phone calls are classical examples of streaming
    • any navigable media type can use streaming (iPaper is streamed PDF)
    • some data sources cannot be downloaded (e.g., a security camera)
  • Streaming often is also used because of security issues
    • downloads make it easy to get content and redistribute it
    • streaming makes redistribution much harder (content must be destreamed)
    • the data formats for streaming are often undisclosed
 Streamed Paper Video and Audio on the Web
  • Internet Protocols only provide best-effort connections
    • Quality of Service (QoS) requires end-to-end QoS provisioning
    • QoS was never implemented on the Internet for economic reasons
  • Data types and expectations co-evolve with the infrastructure
    • faster processors and graphics chips can handle high-resolution video
    • faster networks and better compression make high-resolution feasible
  • Almost all data traffic will eventually move to an Internet
    • TV and telephony are two very popular examples
    • almost all telephony is handled on a (not the) Internet today anyway
  • The public Internet and an Internet are not the same thing
    • companies and the military often have separate networks
    • using Internet technologies for building a network is cost-efficient
    • security and economics decide how Internets are connected
 Content Delivery Networks (CDN)
  • High-volume traffic is better not routed from one place
    • Google and YouTube only look like a a site
    • sophisticated routing and load balancing helps handling traffic
  • Content Delivery Networks (CDN) are designed for high-volume low-latency delivery
    • clients in different parts of the world will be served by different servers
    • the internal data distribution and management is handled by the CDN
  • CDNs are required when sites start handling large traffic volumes
    • CDN services can be bought by site/service owners
    • Akamai and Limelight are two popular services
  • CDN are usually hidden by other technologies
    • DNS responses for CDN hostnames are returned based on the request
    • prepackaged video codecs for Flash/Silverlight have built-in CDN support
 CDN Request Routing Audio on the Web
  • Audio is not very popular on the Web
    • the Web is mostly visually oriented
    • audio content without playback controls is not user-friendly
    • most sites using multimedia use video instead of audio
  • Internet radios such as Pandora often use Flash
    • they are standalone applications running in a browser
    • content is often delivered via HTTP to circumvent firewalls
  • Audio formats exist in many different variations
 Video on the Web
  • Video formats have been evolving quickly for a while now
    • video signals have a lot of redundancy that is hard to compute
  • Depending on the application, algorithms ideally behave differently
    • for playback of recorded content, encoding can be very expensive
    • symmetric scenarios (such as video conferencing) better use symmetric codecs
  • Handling video in effectively implements dynamic codecs
    1. YouTube started serving better quality a while ago
    2. the servers and the Flash plug-in have to be updated
    3. browsers reload the Flash code every time they load a YouTube page
  • Video encoding combines time-enabled and audio
    • both signals must be carefully synchronized
    • sophisticated encodings use variable bitrates and even vary between video/audio rates
 Conclusions Multimedia on the Web
  • Images are the only supported media types on the Web
  • Graphics and video and audio are not universally supported
  • Image formats serve different purposes on the Web
  • PNG for graphics and JPEG for photographic images
  • GIF should be avoided (still required for animated images)
 Internet Architecture 2009-10-29 TCP/IP Internet Architecture · TCP/IP Overview · Timeline The Internet is the technical infrastructure on top of which the Web is built. Some of the services provided by the Internet are essential for the Web, most importantly the naming service and the data transfer service. The Domain Name System (DNS) provides the human-readable names for computers, which can then be used in the addresses of Web servers and ultimately Web pages. The Transmission Control Protocol (TCP) provides the reliable data transfer service between Web Servers and Web Browsers, building on the very robust Internet Protocol (IP). Abstract

Computer Networks Network History
  • First regarded as a convenient workaround for floppy disks
    • real computer scientists write compilers
    • the value of computer networks depends on their size
  • Early networking solutions were vendor-specific islands
    • DECnet for Digital Equipment Corporation (DEC) customers
    • XNS for Xerox customers
    • SNA for IBM customers
    • transmitting data between these networks was very cumbersome
  • Bridging networks transparently became increasingly important
    • more computers and networks increase the benefit of interconnections
    • layering being used for internetworks, not only for networks
 Networks vs. Internetworks
  • Specific networks use specific abstractions
    • how to address nodes (computers, phones, PDAs, RFID tags)
    • how to address applications on these nodes
    • how to transmit data to these applications
  • Internetworks provide a network-independent abstraction
    • nodes are addressed uniformly (IP addresses)
    • applications are identified uniformly (ports)
    • data transmission uses one set of protocols (TCP/UDP)
 Internet
  • Very early start and a lot of experience
    • pragmatic and evolutionary approach
    • if it's not broken, don't fix it
  • Standardization by independent technical experts
    • avoids the designed by committee effect of consortiums
    • conservative and concentrating on stability
    • implementations are required to prove technical feasibility
    • simplicity whenever possible
 Internet Principles
Be liberal in what you accept, and conservative in what you send.

Jon Postel, RFC 1122

Whenever possible, communications protocol operations should be defined to occur at the end-points of a communications system, or as close as possible to the resource being controlled.

J. Saltzer, D. Reed, D. Clark, End-to-end Arguments in System Design

 Internet Protocols Network Convergence Internet Protocol (IP) IP Features
  • End-to-end data transfer (IP addresses)
  • Hiding lower-level heterogeneity
  • Connection-less (each packet routed individually)
  • Unreliable (packets may be lost or duplicated)
 IP Address
  • IP identifies nodes by an IP address
  • IP addresses are globally unique (and can be geocoded)
  • IP uses 4 bytes for addresses (e.g., 128.32.226.29)
    • maximum number of addresses: 232 = 4 billion
    • IPv6 extends the address format to 16 bytes (2128 addresses) = ~ 1038
  • IP address shortage led to the some trickery using IP addresses
    • Dynamic Host Configuration Protocol (DHCP) is used to assign addresses on-demand
    • Network Address Translation (NAT) uses one IP address for more than one device
  • IP addresses are well-organized
    • important for routing (i.e., sending packets to the target host)
    • not ideally suited for mobile or ad-hoc networks
 Transmission Control Protocol (TCP) TCP Features
  • Flow-controlled (avoiding congestion)
  • Reliable (no data lost or duplicated)
  • Connection-oriented
  • Application addressing
 Reliable Connections
  • IP may drop or duplicate packets
    • TCP adds serial numbers in data packets
    • if problems are detected, TCP recovers automatically
  • TCP avoids network congestion and system overload
    • slow start avoid flooding receivers with data they cannot process
    • fast retransmit for avoiding timeouts when losing data
    • a sliding window for controlling the amount of outstanding packets
 TCP Window Domain Name System (DNS) Naming vs. Addressing
  • IP addresses depend on network topology and organization
    • reorganizing a network may change all IP addresses
    • identifying important hosts should not be address-based
  • Names are supposed to be more stable than addresses
    • a name is an abstract identification of something
    • names can be used to obtain more information
  • Network services should use names instead of addresses
    • before using the service, a mapping has to be performed
    • the Domain Name System (DNS) is providing this service
 DNS Properties
  • DNS has a bootstrap problem
    • DNS provides a service and should thus be identified by a name
    • for resolving names into addresses, the DNS service is required
  • DNS configuration is part of basic Internet configuration
    • Dynamic Host Configuration Protocol (DHCP) provides , netmask, gateway, and DNS server address
  • DNS names are hierarchically structured
    • ischool.berkeley.edu, edu is the Top-Level Domain (TLD)
    • TLDs are either generic (gTLD) or country code (ccTLD)
    • subdomains are federated (e.g., edu, us, uk, tv)
 Names Matter
  • Names are not unique and namespaces are finite
    • name disputes arise which were irrelevant before the Web
    • cybersquatting as a popular way to make money
  • Names can be worth a lot of money
    • business.com was sold for $7.5M in 1999 and again for $345M in 2007
  • Name inflation can be used to generate money
  • Names can have political significance
    • ccTLDs are assigned based on the UNO's idea of what a country is
  • Names can have symbolic significance
    • Catalonia managed to get a domain of its own (cat)
 Domain Name Space DNS Namespace Organization
  • Domain owners can organize the assignment of subdomains
    • berkeley.edu is an U.S. educational institution
    • ethz.ch is a Swiss university
    • imperial.ac.uk is a British university
    • uts.edu.au is an Australian university
  • Special rules may apply (Germany does not assign car license plate names)
  • Organizations may be countries or companies
    • countries have national organizations for assigning names
    • companies may create an internal multi-level namespace (www.ischool.berkeley.edu)
 Using DNS
  • DNS is used by virtually all Internet applications
    • names are more stable than addresses
  • E-mail has some dedicated features built into DNS
    • special entries (MX records) identify the e-mail server for a domain
    • fallback entries help dealing with failing e-mail servers
  • most URIs are based on DNS names
    • http://ischool.berkeley.edu/ identifies the access protocol and the host
    • the browser first performs a DNS lookup
    • a TCP connection is then established to the address returned by the DNS
 Conclusions
  • The Internet is a network of networks
  • The Internet provides basic data transfer services
  • IP is used to send datagrams between end-points
  • TCP is used for reliable communications (building on IP)
  • DNS provides human-readable names for Internet hosts
 Web Foundations (URI & HTTP) 2009-11-03 HTTP · Cool URIs Live HTTP Headers · HTTP and CGI · URI Spec · HTTP Spec The Web's architecture has very simple principles revolving around the ideas of placing a heavy emphasis on a consistent and global identification mechanism for resources, a standardized way of how resource representations can be retrieved, and a standardized way of how resource representations should be usable by using standardized media types. Based on the Internet, the Web's transport protocol transmits representations of resources identified by a Uniform Resource Identifier (URI) between Web servers and clients. The most important protocols for data transfer on the Web is the Hypertext Transfer Protocol (HTTP). Abstract

Web Server Service
  • Web servers do more than just deliver files
  • They receive a request for acting on a resource
    • this may be a simple file retrieval
    • additional information is available from the request's header fields
    • the request URI may contain additional query information
    • the request may transmit complex data (such as a form submission)
  • Processing can mean anything, it is transparent for the client
    • the result of processing yields a resource representation
    • in many cases, a Web server is just part of an application
    • the application server is the application-specific logic
 Uniform Resource Identifier (URI) Resource Identification
  • The Web is centered around resources
    • HTTP has been designed to manipulate resources
    • HTTP provides methods for getting, putting, updating, and even deleting resources
  • Resources are useful abstractions for interfaces
    • instead of an API, interaction is built around manipulating resources
    • APIs change and bind closely, documents can better withstand change and bind loosely
    • the whole Web is built around resources, not APIs
 URI Schemes 
URI = scheme ":" hier-part [ "?" query ] [ "#" fragment ]
http://dret.net/lectures/web-fall09/foundations#uri-schemes
  • URIs in their general case are very simple
    • the scheme identifies how resources are identified
    • the identification may be hierarchical or non-hierarchical
  • Many URI schemes are hierarchical
    • it is then possible to use relative URIs such as in a href="../"
    • the slash character is not just a character, in URIs it has semantics
  • Query components specify additional information
    • it is non-hierarchical information further identifying the resource
    • in most cases, it can be regarded as input to the resource
 Hypertext Transfer Protocol (HTTP) DNS & HTTP

The two basic protocols which every Web browser must implement are DNS access and HTTP. However, most operating systems provide an API for DNS access, so the browser can use this service locally and only has to implement HTTP. TCP (which is required as the foundation for HTTP) is usually provided by the operating system.

 The Web's Protocol

provided by CacheLogic Inc.

 HTTP Basics HTTP Messages
  • HTTP needs a reliable connection
    • the foundation for HTTP is the
    • DNS resolution yields an IP address
    • open TCP connection to port 80 or port specified in URI (http://rosetta.ischool.berkeley.edu:8085/)
  • HTTP is a text-based protocol
    • the connection is used to transmit text messages
    • all HTTP messages are human-readable (not all entities, though)
    • basic HTTP operations can be carried out by hand
start-line
message-header *

message-body ?
 HTTP Header Fields
  • Header fields contain information about the message
    • general header: Date as the message origination date
    • request header: Accept-Language indicates language preferences
    • response header: Server contains system information
    • entity header: Content-Type specifies the media type of the entity
  • HTTP defines a number of header fields
    • unknown fields must be ignored (extensibility)
    • unstandardized fields should use a X- prefix
  • HTTP is about acting on these fields
    • HTTP defines what HTTP implementations must or should do
 HTTP Content Negotiation
  • have interaction semantics
    • depending on the header type they convey different information
  • HTTP Content Negotiation allows representation negotiation
    • the client specifies a number of preferred content properties
    • the server responds with the representation that fits best
  • Server-driven negotiation can use a number of request header fields
    • Accept uses a list of
    • Accept-Charset uses a list of character sets
    • Accept-Encoding uses a list of content encodings
    • Accept-Language uses a list of language codes
    • User-Agent specifies the client's identification
  • Client-driven negotiation lets the server send a list of URIs
    • two steps are required to get the best alternate representation
 HTTP Requests
  • After opening a connection, the client sends a request
    • the method indicates the action to be performed on the resource
    • HTTP's most interesting methods are: GET, HEAD, POST
    • other interesting methods are: PUT, DELETE
  • The URI identifies the resource to which the request should be applied
    • absolute URIs are required when contacting proxies
    • absolute paths are required when contacting a server directly
    • the URI may contain query information
  • The Host header field must be included in every request
Method Request-URI HTTP/Major.Minor
[Header]*

[Entity]?
 HTTP GET
  • Retrieval action based on the URI
    • maybe implemented by reading a file
    • maybe implemented by processing a file (PHP)
    • maybe implemented by invoking a process
  • Semantics may change based on header fields
    • If-*: only reply with the entity if necessary
    • Range: only reply with the requested part of the entity
  • Cacheability depends on header fields of the response
GET / HTTP/1.1
Host: ischool.berkeley.edu
 HTTP Responses
  • The server's response to interpreting a request
    • the status code is given numerically and as text
    • 2** for variations of ok
    • 3** for redirections
    • 4** are different client side problems (404: not found)
    • 5** are different server side problems
  • Header fields specify additional information
    • information about the server
    • information about the entity (media type, encoding, language)
HTTP/Major.Minor Status-Code Text
[Header]*

[Entity]?
 HTTP Performance
  • HTTP/1.0 allowed one transaction per connection
    • TCP connection setup and teardown are expensive
    • TCP's slow start slows down the initial phase of data transfer
    • typical Web pages use between 10-20 resources (HTML + images + CSS + scripts)
    • typically, these resources are stored on the same server
  • HTTP/1.1 introduces persistent connections
    • the TCP connection stays open for some time (10 sec is a popular choice)
    • additional requests to the same server use the same TCP connection
  • HTTP/1.1 introduces pipelined connections
    • instead of waiting for a response, requests can be queued
    • the server responds as fast as possible
    • the order may not be changed (there is no sequence number)
 HTTP Connection Handling HTTP Authentication HTTP Access Control
  • HTTP servers can deny access through access control
    • 401 Unauthorized means the resource is access controlled
    • 403 Forbidden means the resource is inaccessible
    • 405 Method Not Allowed signals a request using the wrong request method
  • Two different approaches to unauthorized access are possible
    • repeat the HTTP request with the proper authentication credentials
    • redirect to a and establish an authenticated
HTTP Authentication Basic HTTP Authentication
  • Authentication is based on authentication realms
    • a set of resources for which the authentication is required
    • an opaque name which is used to signal which login is required
    • username/password often is specific for a given realm
  • Users supply username and password through the client
  • Authorization is handled on the server side
    • HTTP/1.0 401 Unauthorized
	WWW-Authenticate: Basic realm="SokEvo"
    GET /private/index.html HTTP/1.0
	Authorization: Basic QWxhZGRpbjpvcGVuIHNlc2FtZQ==
 Repeated Access
  • Clients typically access more than one protected resource
    • a perfectly stateless client would always request authentication from the user
    • using the realm clients can identify repeated accesses
  • Web interactions by default are perfectly stateless
    • each request is completely independent from other requests
    • stateless interactions make the Web loosely coupled and scalable
    • concepts like the realm or introduce state
  • Clients remember the authentication and replay it automatically
    • browsers provide little control over this feature
    • logging out of HTTP authenticated sessions is hard
 Login Page
  • works with browser controls (including the window)
    • no possibility to log out without using browser-specific controls
    • client side security depends on browser security measures
  • Using gives more freedom in session management
    • and are completely application-based
    • if there were secure personal browsers this would not work very well
 Conclusions
  • HTTP is much more than file transfer
    • it is a protocol for the concept of resource manipulation
    • it is a distinct step away from the API approach to building distributed systems
  • HTTP servers can be configured to deliver good or bad service
    • this is a question of how well they are configured on the HTTP level
    • it is also a question of how good the Web design is
    • both issues together are required to set up a good Web server
 Security & Privacy 2009-11-05 Security · Privacy · Browser Security Browser Options · HTTPS · HTTPS Spec TCP and thus HTTP are clear-text protocols, which make no attempt to hide the data being transmitted. For secure data transfers, it thus is necessary to use additional technologies for providing secure data transfers. For the Web, the most interesting security feature are secure HTTP interactions, which are provided by HTTP over SSL (HTTPS), a protocol that layers an encryption layer (SSL or TLS) between TCP and HTTP. For any task involving personalization and/or trust, it is not only necessary to have a concept for providing privacy, but also to have concepts for identity and how to prove identity, which needs authentication. Abstract

Security Concepts Identification
  • Identity is required for any non-anonymous communications
    • groups can have an identity (facebook members see more than non-members)
    • pseudonyms are hidden identities (the real identity is not visible)
    • personal identity should be tied to a person itself
  • Proof of Identity is important for any privileged operation
    • signatures and seals are traditional ways
    • traditional ways are mostly protected by law (but not really safe)
    • more modern ways often include technical methods for
  • Client identity on the Web can be bound in three ways:
    1. Computer (most of the time identified by an )
    2. Browser (in the form of a stored cookie)
    3. User (identified through some authentication method)
 Authentication
  • Authentication is the process of verifying an identity
    • the weakest form of authentication is simply trust
    • legal consequences can make it more risky to falsify authentication
    • technical measures should make it hard to impossible to falsify authentication
  • Authentication on the Web comes in many different flavors
    • implicitly by accessing a server from some range
    • presenting a cookie from a previous formal authentication
    • presenting a password as a proof of identity
    • proving that you are owning additional authentication hardware (often PIN-enabled)
  • Risk and potential damage should justify authentication methods
 Authorization
  • Authorization is the question of allowing operations
    • is necessary to identify the initiator
    • is necessary to verify the initiator's identity
    • if the initiator is authorized, the operation can be performed
  • Web pages often are public or restricted access
    • public web pages do not require any identification (and thus authentication)
    • restricted access Web pages can be group pages (internal company pages)
    • personal access is another popular scenario (email, facebook, online banking)
  • Web servers have well-defined ways of performing authentication
 Browser Security & Privacy Trust and Security on the Web
  • Web-based applications introduce many risks
    • do you trust your browser? (it may not safeguard your information)
    • do you trust your computer? (it may have a virus)
    • do you trust your network? (it may be monitored on various levels)
    • do you trust the server? (it may be a fake phishing server)
  • Most of these risks are amplified by the Web's scale
    • phishing and spamming only work because the Web makes fraud more effective
  • Controlling Web access is important for safe browsing
    • trusting shared browsers is risky (they may store logins and cache pages)
    • trusting the network can be risky (more and more networks are wire-tapped)
    • trusting the server is risky (phishing and poor server security)
 Privacy Options Security Options Encryption Options Security 101 Cryptography
  • Cryptography is structured into different layers
    • layering is a well-established principle for separation of concerns
  • Cryptographic primitives implement very basic functionality
    • changes and advancements in this area are limited to very specialized researchers
    • it is easy to make fatal mistakes which then challenge everything built on top if it
  • Cryptographic protocols assemble primitives into application-level solutions
    • primitives solve very basic security problems (fingerprints, encryption, …)
    • protocols combine these into applications (digital signatures, secure communications, …)
 One-Way Function
  • Hashes (or message digests) are well-known in computer science
  • One-way functions are cryptographically safe hashes
    • very hard to find an input producing a given output
    • very hard to find two inputs producing the same output (collision)
 Secret-Key Cryptography Plausible Encryption
  • Secret-Key is was most people think of when thinking of encryption
    • symmetric cryptography is another popular term
  • One key for encryption and decryption
  • Revealing the key makes encrypted data openly readable
  • Good for long-term relationships with few partners
    • exchange secret keys as part of the initial setup of a relationships
    • adding partners requires a secure channel for key exchange
    • changing keys requires a secure channel for key exchange
  • Almost impractical in an environment with many ad-hoc partners
 Notice the Arrow Public-Key Cryptography Implausible Encryption
  • Public-Key intuitively is hard to accept as a concept
    • asymmetric cryptography is another popular term
  • Key pairs of one public and one secret key
    • key generation is the process of generating these key pairs
  • The public key can be made available to the public
    • only the secret key can do the inverse operation of the public key
  • Good for short-term relationships with many partners
    • publish your public key so that it can be used worldwide
    • everybody can encrypt data using the public key
    • only the owner of the secret can can decrypt the message and read it
  • Computationally expensive and not good for a large amounts of data
 No Arrow Here … Cryptographic Protocols Building Secure Applications
  • Cryptographic primitives in most cases are not sufficient
    • they provide basic functionality for fundamental tasks
    • they must by combined to provide solutions for real-world problems
  • Typical problem #1: How to ensure key authenticity
    • with insecure keys, the majority of cryptographic methods is worthless
  • Typical problem #2: How to communicate securely without shared keys
    • many interesting scenarios are based on ad-hoc interactions
    • secret-key does not work, public-key needs to verify the peer
  • Typical problem #3: How to check authenticity and integrity of data
    • integrity can be done with checksums, but these could be forged
    • authenticity needs a cryptographically secure way of combining identity and data
 Certificate
  • Certificates are digital signatures issued by a trusted party
    • most digital signatures are created with certified public keys
    • this means the digital signature is created based on a digitally signed key
  • Who can you trust on the Web?
    • trust can only start to grow based on initial trust in something
    • many systems come with pre-installed trust (root certificates)
    • certificates from other issuers will cause browsers to complain
  • Certificates (like domain names) are a very easy way to make money
    • in theory there are different levels of certificates with different levels of identity checking
    • in practice most sites choose the cheapest one that does not give an error message
 HTTP over SSL (HTTPS) Secure Communications
  • Public-Key cryptography is computationally expensive
    • it is possible to encrypt all traffic using asymmetric key pairs
    • this generates considerably more load on the server side
  • Combining public-key and secret-key cryptography
    1. check the public key for authenticity (using a )
    2. generate a key for a secret-key encryption scheme
    3. use the public key to securely transmit the secret key
    4. use the secret key for securely transmitting the payload
  • Combines the advantages of both methods
    • the lower complexity of secret-key algorithms
    • the ability of public-key algorithms to work without a secure channel
 HTTP and Security
  • HTTP sends clear-text messages
  • Making HTTP secure requires additional mechanisms
  • Encryption is done by a layer on top of TCP
    • Secure Sockets Layer (SSL) is the protocol layer invented by Netscape
    • Transport Layer Security (TLS) is the standardized Internet version
    • TLS adds more encryption schemes and more flexibility
  • Lower-level methods may also provide encryption
    • Virtual Private Networks (VPN) provide IP-based encryption
    • WEP and WPA provide network interface encryption
 HTTP and SSL Conclusions Internet Security
  • Security is hard
  • Certificates are used to guarantee a party's authenticity
  • Certificates are digital signatures issued by trusted parties
  • One authenticated, public keys can be used to securely communicate
  • Encryption on the Web is based on HTTPS
 State Management 2009-11-10 Wikipedia State · Cookies Spec HTTP is a stateless protocol, where each request/response interaction is a separate interaction and there is no protocol support for longer sessions (such as a user logging in and working on a Web site as an identified user). State management refers to mechanisms which provide support for this kind of scenario, the most popular choice for state management are cookies. Another possibility is URI-based state management. This lecture is a first glimpse into the world of Representational State Transfer (REST), the Web's fundamental model of handling interaction with resources. Abstract

Session HTTP and Sessions
  • HTTP has no session concept
    • interactions are HTTP request/response pairs and not site visits
    • HTTP/1.1 does not change this, it is only a performance optimization
    • servers can not reliably identify users interacting with a Web site
  • Sessions should not be used to track resource state
    • the semantics of resource interactions should not depend on client state
    • application behavior can depend on client state
  • HTTP's concept of stateless interaction is important
    • the Web's idea is to use loose coupling between clients and servers/resources
    • retrofitting the Web with tight coupling through server state is bad design
 Client-Side State
  • Sessions should be maintained on the client
    • the client has all relevant information about a session
    • when the server restarts, no information will be lost
    • if something has to be persistent, it should be a resource
  • Small and short-term solutions may work well with server state
    • scaling these solutions typically introduces many problems
    • debugging can be hard because the state is transient
    • integration with other clients can become a difficult problem
  • Three ways of client-side state are possible
    1. sending back and forth state as part of the interaction
    2. store state in the server and refer to it from the client (not recommended)
    3. store state at a URI and use the URI to refer to that state
 State in HTML or HTTP State in the Server Application State as a Resource Stateless Shopping
  • Typical session scenarios can be mapped to resources
    • Client: Show me your products
    • Server: Here's a list of all the products
    • Client: I'd like to buy 1 of http://ex.org/product/X, I am "John"/"Password"
    • Server: I've added 1 of http://ex.org/product/X to http://ex.org/users/john/basket
    • Client: I'd like to buy 1 of http://ex.org/product/Y, I am "John"/"Password"
    • Server: I've added 1 of http://ex.org/product/Y to http://ex.org/users/john/basket
    • Client: I don't want http://ex.org/product/X, remove it, I am "John"/"Password"
    • Server: I've removed http://ex.org/product/X to http://ex.org/users/john/basket
    • Client: Okay I'm done, username/password is "John"/"Password"
    • Server: Here is the total cost of the items in http://ex.org/users/john/basket
  • This is more than just renaming session to resource
    • all relevant data is stored persistently on the server
    • the shopping cart's URI can be used by other services for working with its contents
    • instead of hiding the cart in the session, it is exposed as a resource
 Reusing Resources Cookies Tracking Sessions
  • Invented as a way to compensate for HTTP's lack of state
    • application state is being sent to the client (Set-Cookie2)
    • the client transmits application state in requests (Cookie)
  • Cookies do not contain code that is executed
    • some data that represents application state (by value or by reference)
    • this data is stored by the client and returned to the server
    • the client is not supposed to interpret the data in any way
  • Cookies can be used in many different ways
    • when used for tracking application state they are unproblematic
    • when used for tracking resource state they introduce problems
  • Cookies tightly bind clients to opaque concepts on the server
 Cookies for State Management Third-Party Cookie Advertising & Privacy
  • Big ad servers are digital hubs in the commercial Web
    • consumers switch content providers but get the same ad provider
    • tracking consumers across content providers is very valuable
  • Cookies set by ad providers are sent very frequently
    • each site that uses the ad provider triggers the cookies to be sent
    • detailed profiling can be employed for creating consumer profiles
  • Content and ad providers can cooperate for better profiling
    • consumers log in to content providers are are reliably identified
    • their personal profile can be matched with the ad provider's profile
    • ad provider consolidation makes this scenario realistic
 Browsers Assemble Web Pages

Typical Web resources (HTML pages) are assembled from a number of resources retrieved by HTTP. Any resource not originating on the server that is hosting the HTML page is considered a third-party resource. If the HTTP response for such a resource contains a cookie, it is a third-party cookie.

 Cookie-Less State Tracking Cookie Support
  • Authentication can be tracked with
    • this is possible because authentication is built into HTTP
  • Other session concepts are not supported by HTTP
    • cookies have become the generic solution for all session tracking
  • Cookies are increasingly limited by browsers
    • cookies have gained some notoriety as privacy invaders
    • browsers have more restrictive default settings
    • an increasing number of users restricts cookie support
  • Session-oriented Web sites often depend on cookies
 URI Rewriting
  • are a piece of information stored on the client
    • they are sent by the server as a result of a request
    • they are returned by the browser in a response to the same site
  • The same information can also be encoded in the URI
    • normally a response contains a cookie and an HTML page
    • the same effect is achieved when all links include the cookie value
    • this method often results in very long URIs
  • Some Web application frameworks switch automatically
    • J2EE checks for cookie support and switches to URI rewriting if required
  • Problems with bookmarks and caches
 Hidden Form Fields
  • transmit session information via HTTP
  • encodes session information in URIs
  • are a way to send data to a server
    • in most cases this is data that is entered by the user
  • Hidden form fields can be used to send data that is part of the HTML
    • hidden form fields are never displayed to the user
    • their predefined values are sent as part of the form submission
  • Hidden form fields are essentially the same as
    • they can only be used if the interaction is based on forms
    • they also require the Web page to be dynamically generated for each request
    • the values end up as URI query string or request entity
 Conclusions Session for Application State
  • Sessions should only be used for application state
  • Cookies are the best way to track sessions
    • cookies should be self-contained rather than referential
  • Alternative methods are URI rewriting and hidden form fields
    • more robust than cookies but unpleasant side-effects
 Representational State Transfer (REST) 2009-11-17 A6 assigned (due date: 11/22) REST vs. SOAP · What is REST? · REST Interfaces RESTwiki Representational State Transfer (REST) is an architectural style for building distributed systems. The Web is an example for such a system. REST-style applications can be built using a wide variety of technologies. REST's main principles are those of resource-oriented states and functionalities, the idea of a unique way of identifying resources, and the idea of how operations on these resources are defined in terms of a single protocol for interacting with resources. REST-oriented system design leads to systems which are open, scalable, extensible, and easy to understand. Abstract

The Web as a System
  • The Web is one distributed hypermedia system
    • the main architectural components are URIs, HTTP, and HTML
    • all other Web technologies are built on that foundation
    • if they are not, they are very likely not well-designed Web technologies
  • The Web is amazingly open, scalable, extensible, and easy to understand
    • openness allows new technologies to be introduced
    • scalability ensures that the system does not contain bottlenecks
    • extensibility allows the Web to evolve without any redesign of existing parts
    • simplicity makes sure that the system survives and evolves
  • No other information system gets even close to the Web
    • but not all information system designs can accept the Web's limitations
    • REST should be seen as a guideline how to build a true Web application
    • other applications will continue to be built using other approaches
 Web System Design
There are two ways of constructing a software design: One way is to make it so simple that there are obviously no deficiencies, and the other way is to make it so complicated that there are no obvious deficiencies. The first method is far more difficult.

C. A. R. Hoare, The Emperor's Old Clothes, 1980 Turing Award Lecture

 Technologies and Implementations Object-Orientation
  • Object-Orientation is a Software Engineering Style
    • it can be applied to any programming language
    • depending on the language, this is more or less easy
    • OO languages support or even enforce certain design patterns
    • spaghetti code can be written in every programming language
  • Implementations can always be bad or good
    • the quality of the implementation depends on the programmer
    • programmers can be better supported and controlled with an OO language
    • implementation quality metrics must be based on the product, not the language
  • Good programmers always produce good code
  • Bad programmers always produce bad code
  • Average programmers need good tools to produce good code
 Technologies are Tools
  • Technologies help solving problems
    • they are built with certain goals in mind
    • they specialize in solving a problem in a specific way
  • Technology choices are very important
    • the technology (i.e., the tool) shapes the way a problem is solved
    • working against the tool is possible, but hard and rarely done
  • Technologies sometimes cloud the more important issues
    1. the problem must be well-defined and fully understood
    2. the general approach to solve a problem must be identified
    3. finally, a technology supporting this approach must be chosen
 Implementations are Products
  • Implementations are built on (and shaped by) technologies
  • Implementation = Concepts + Technologies
  • Implementation quality depends on both factors
    • the right choice of technologies is very important
    • the responsible use of that foundation is equally important
  • Good REST is as hard to grasp as good OO
    • products may claim that they are REST/OO
    • they may even use technologies which support REST/OO
    • only careful inspection reveals the truth of this claim
    • in most cases, this only happens when the product needs to be changed
 REST Principles Definition
  • Resources are defined by URIs
  • Resources are manipulated through their representations
  • Messages are self-descriptive and stateless
  • There can be multiple representations for a resource
  • Application state is driven by resource manipulations
 Resources
  • Resources are defined by URIs
    • resources can never be accessed or manipulated directly
    • REST works with resource representations
  • Resources are all the things we want to work with
    • if you cannot name something, you cannot do anything with it
    • a popular resource type on the Web are documents
    • documents usually are a structured collection of information
  • Documents are abstract concepts of descriptive resources
    • they may be used in different contexts (e.g., formats)
    • different applications may be interested in different representations
    • the underlying resource is always the same
 State
  • State is represented as part of the content being transferred
    • server interruptions do not create problems for the client
    • it is possible to switch between servers for different interactions
    • clients can simply store the representation to save the state
  • State transfer makes the system scalable
    • data transfer is not state-specific (no stateful connection handling)
    • state is transferred between client and server
 Establishing a Common Model
  • Distributed systems must be based on a shared model
    • traditional systems must agree on a common API
    • REST systems structure agreement into three areas
  • REST is built around the idea of simplifying agreement
    • nouns are required to name the resources that can be talked about
    • verbs are the operations that can be applied to named resources
    • content types define which information representations are available
 Nouns
  • Nouns are the names of resources
    • in most designs, these names will be URIs
    • URI design is a very important part of a REST-based system design
  • Everything of interest should be named
    • by supporting well-designed names, applications can talk about named things
    • new operations and representations can be introduced
  • Separating nouns from verbs and representations improves extensibility
    • applications might still work with resources without being able to process them
    • introducing new operations on the Web does not break the Web
    • introducing new content types on the Web does not break the Web
 Verbs
  • Operations which can be applied to resources
  • The core idea of REST is to use universal verbs only
    • universal verbs can be applied to all nouns
  • For most applications, HTTP's basic methods are sufficient
    • GET: Fetching a resource (there must be no side-effects)
    • PUT: Transfers a resource to a server (overwriting if there already is one)
    • POST: Adds to an existing resource on the server
    • DELETE: Discards a resource (its name cannot be used anymore)
  • Corresponding to the most popular basic database operations
    • CRUD: Create, Read, Update, Delete
 <http>POST</http>ing
  • POST adds instead of an overwriting update
  • POST can have different effects
    • by POSTing, state is changed and a new resource is created
    • by POSTing, only the existing resource is changed
    • the server signals the difference using HTTP responses (200 OK or 201 Created)
  • This is a design choice
    • if the added information needs to be accessible individually, create a new resource
    • for changes of an existing resource, no new resource has to be created
  • Make sure that resources are navigable using URIs
    • if appropriate, a relationship can be represented in the resource format
 Content Types
  • Representations should be machine-processable
    • they don't have to, they may be opaque to applications
    • in many cases, machine-processable representations are advantageous
  • Resources are abstractions, REST passes representations around
    • resources can have various representations (i.e., content types)
    • clients can request content types they are interested in
  • Adding or changing content types does not change the system architecture
    • different clients and servers support different content types
    • allows content types to be negotiated dynamically
 REST vs. <q>Web Services</q>
  • REST is a description of the Web's design principles
    • it is not something new, it is simply a systematic view of the Web
    • REST's claim is to be able to learn from the Web's success
  • Web Services (the SOAP flavor) do not build on REST
    • they use HTTP as a transport protocol
    • they re-create Web functionality through additional specifications (WS-*)
    • they have been built by programmers using a top-down approach
  • REST and Web Services have different design approaches
    • REST starts at the resources and takes everything from there
    • Web Services focus on messages, which in most cases are operations
 REST Implementation REST Technologies
  • REST is not tied to a particular set of technologies
    • are the most common choice for nouns
    • methods are the most common choice for verbs
    • is the most common choice for content types
  • Choosing other technologies should have a very good reason
    • building a REST system should make it open and accessible
    • technology choices are as important as architectural choices
 URIs
  • REST requires a lot of URI design
    • instead of being generated as a side-effect, they are the core of the system
  • Designing URIs and starting from them is a new way of thinking
    • URIs are much more powerful than just being an address of a Web page
  • URIs are names for concepts
    • concepts are never transmitted, only their representation
    • having to focus on concepts rather than representations is helpful
 HTTP
  • HTTP is the most successful RESTful protocol
    • HTTP's author Roy Fielding coined the term REST in his Ph.D. thesis
  • HTTP should be regarded as an application-level protocol
    • Web Service technologies use HTTP as a transport protocol
    • HTTP has much more to offer than a firewall-penetrating pipe
  • Web infrastructure is built around proper HTTP usage
    • caching is built into HTTP and caches optimize the Web transparently
    • authentication can be done using HTTP's authentication methods
    • secure data transfer can be done using
XML
  • URI-identified resources are abstract concepts
    • for machine-based processing, XML is a good representation
    • for human-oriented interactions, HTML probably is a better choice
  • Connections to other resources must be done by URI
    • XML does not make built-in assumptions about identifiers
    • but it does support URIs, for example with XInclude and XML Base
    • RESTful applications are about navigating a Web of URI-identified resources
 Conclusions Better Services
  • REST is an architectural style
    • URI/HTTP/HTML/XML may be replaced
    • the general principle of resource-based interaction remains valid
  • RESTful system designs can create better systems
    • a little bit more design effort in the beginning
    • a lot less headaches later
  • SOA often are not really RESTful
    • SOA often focuses on operations
    • REST focuses on resources
  • RESTful design is a good starting point for OO implementations
 Semantic Web 2009-11-19 Microformats · Which Semantic Web? RDFa · FAQ · RDF · OWL The Semantic Web can either be understood as a prepackaged set of languages and technologies for representing semantics and working with them, or as a more general idea of Web Semantics, which instead of predefining certain languages and technologies just looks at the various options of how more semantics can be represented on the Web. Taking the latter approach, this lecture looks at the various ways in which semantics can be introduced on the Web, and what is required in these scenarios in terms of technology and information sharing. Abstract

Information About the Web Web Resources
  • Every Web resource is identified by URI
  • Resources can be self-contained or represent real-world objects/concepts
    • A Web page with a commentary on it is the commentary
    • A home page may represent a person and not just an abstract home page
    • many philosophical questions about identity, equality, and ontology
  • Practically speaking, many Web resources have implicit links
    • technical limitations (finding inbound links is hard)
    • linking between pages about the same resource
    • linking between similar resources
 Resources on the Web Describing Resources with Microformats Metadata as Markup Overlay Surfacing Concepts
  • are embedded into HTML Web content
    • there is no need for alternative representations
    • machines use the same resources as humans (and extract microformat data)
  • Many larger Web sites are using structured back-ends
    • for pure HTML publishing, semantics are translated into HTML/CSS
    • more semantics can be represented by also producing microformats
  • Microformats can also help in decentralized service scenarios
    • centralized: data has to be submitted to a hub
    • decentralized: crawlers search for microformats and use what they find
    • advantage of decentralization: loose coupling and independent control
 Publishing Resources as XML Different Representations XML as Custom Markup
  • XML can represent arbitrary (textual) data
    • works very well for tree-structured and document-style data
    • works not so well for graph-like data with no inherent order
  • XML only defines a syntax for representing ordered trees
    • what the elements and attributes mean requires agreement on the vocabulary
    • XML is a good tool for data exchange, but only for the syntax part
  • Agreement can be based on three different approaches
    1. everybody always uses a universally useful vocabulary (XHTML)
    2. user (groups) agree on vocabularies based on mutual interest
    3. user (groups) agree on modules and build vocabularies with these modules (UBL)
 Transforming Resources into RDF Translation into a Universal Metamodel The Semantic Web Vision
  • RDF is a more general model for structured data
    • it is very fine-granular and can represent (almost) everything
    • its granularity becomes problematic in scenario with coarse granularity
  • Transforming resources can be done in two basic ways
    1. using well-defined mappings between HTML and RDF (GRDDL)
    2. extracting information by analysis such as NLP
  • Transforming resources does not require cooperation of the service
    • any service can be crawled and the resources can be transformed
  • Transforming resources does not allow access to all data
    • crawling can have limits and thus the RDF data is limited as well
    • with everything as RDF the complete RDF graph could be queried
 Transforming Services into RDF Bypassing Web Publishing Trillions of Triples
  • Turn the structured data from the back-end into RDF
    • the complete dataset can be transformed in one processing step
    • relationships between resources may be better preserved than by crawling
    • keeping the snapshot current can become a significant problem
  • With everything in one database (a triple store) queries become possible
    • SPARQL queries can query the graph of complete service data
    • large datasets easily translate into several billion triples
    • triple store implementation moves to native RDF databases
  • Large triple stores can process complex SPARQL queries
    • performance is a problem with large datasets
    • RDF's simplicity bites back in the form of reification
 Conclusions Exposing Structured Data
  • Exposing semantics makes data more valuable
  • Various technologies and approaches can be chosen
  • What are the economics behind allowing/supporting reuse?
  • Different implementations fit different scenarios
 Architecture of the World Wide Web 2009-11-24 A7 assigned (due date: 12/6) Architecture? · Architecture Summary Architecture The Web's architecture has very simple principles revolving around the ideas of placing a heavy emphasis on a consistent and global identification mechanism for resources, a standardized way of how resource representations can be retrieved, and a standardized way of how resource representations should be usable by using standardized media types. This lecture presents an overview of these architectural principles and illustrates them with using blogs as an example of Web-based applications. Abstract

Today's Reading
Summary of Ian Jacobs, Norman Walsh, Architecture of the World Wide Web, Volume One, World Wide Web Consortium, Recommendation REC-webarch-20041215, December 2004
  • Examples (or counter-examples) for the following principles, practices, and constraints:
    • URIs identify a single resource (versioning, aliases)
    • URI opacity (assuming a specific resource representation)
    • Available representations (XML namespaces as really bad example)
    • Hypertext links (resource representations should be good Web citizens)
    • Orthogonality (identification, interaction, and representation are orthogonal)
 Parsimony Keep It Simple
  • Loose coupling vs. tight coupling
    • fewer requirements for cooperation mean fewer potential sources of problems
    • taking independent developments into consideration (graceful degradation)
  • Parsimony may conflict with optimization
    • a fully backlinked Web would be a very different hypermedia system
    • modifying resources would be expensive and require considerable efforts
    • an uncontrolled Web allows failure and innovative development
  • Programming Languages vs. Frameworks
    • programming languages are very simple and very powerful
    • frameworks are more complex and have some choices built into them
    • both can be used to build good systems
    • framework applications are more likely to not do really innovative things
 Web Design as System Design
There are two ways of constructing a software design: One way is to make it so simple that there are obviously no deficiencies, and the other way is to make it so complicated that there are no obvious deficiencies. The first method is far more difficult.

C. A. R. Hoare, The Emperor's Old Clothes, 1980 Turing Award Lecture

 Technology Blinders
  • Web architecture is an additional set of constraints
    • it is not a very complicated set of constraints
    • but it still makes life more complicated than in an unconstrained world
    • it may require a major redesign of an application
  • Technology providers sometimes ignore Web architecture
    • multimedia presentation concepts are often disconnected from the Web
    • hypermedia researchers often regard the Web as inferior (or not as hypermedia at all)
    • questions of client capabilities are often ignored (or brushed aside using statistics)
  • Integration vs. Transport
    • integrating into the Web requires applications to conform to Web architecture
    • sitting on top of the Web just requires to use HTTP for data transfer
    • many Web Technologies are not integrated into the Web
    • many Web Applications are not integrated into the Web
 Principles Identification
  • Everything should be identified in a uniform way
  • Identification and access methods evolve over time
    • sms: and callto: did not exist when the Web was created
  • Identification and access support evolve over time
    • tel: now can be supported by an increasing number of clients
  • The Web is one huge proof for the power of network effects
    • it also is a lesson for many who did not take it seriously and failed
 Interaction
  • Many URI schemes are named after protocols
    • http: can be accessed using the Hypertext Transfer Protocol (HTTP)
    • ftp: can be accessed using the File Transfer Protocol (FTP)
    • mailto: sends electronic mail using the Simple Mail Transfer Protocol (SMTP)
  • Some URI schemes do not really imply a protocol
  • Some URI schemes have no protocol for dereferencing resources
    • urn: URIs are abstract names from some namespace
    • urn:ietf:rfc:2648 identifies an IETF standard and not some specific copy
    • geo:27.988056,86.925278 identifies a physical resource (accessing it is really hard)
 Data Formats
  • Agreement on the interpretation of resource representations
  • HTML was the first standardized data format on the Web
    • CSS and XML have become successful formats as well
  • Some data formats are de-facto standards as Web formats
    • GIF and JPEG for images and PNG as the successor of GIF
  • Some formats are less integrated but still widely used
    • PDF for paginated documents
  • Some formats have become replacements for missing standards
    • Flash for audio and video because no single format was sufficiently successful
  • Some formats were intended to become standards but failed
    • SVG for vector graphics
    • SMIL for multimedia presentations
 Identifier, Resource, and Representation Constraints and Good Practices Constraints
  • Some things on the Web can be inconsistent
    • guaranteeing consistency by design can lead to tight coupling
    • well-defined ways of handling inconsistencies are better scalable
  • Some things on the Web are not perfect
    • technologies being used in ways not anticipated (XML, XML Namespaces)
    • company goals vs. the greater good (browser war)
  • The ideal Web vs. the real Web
    • dealing with a given landscape can introduce additional constraints
    • handling these constraints should not violate the general principles
 Good Practices
  • Design for openness and extensibility is a key factor
    • design for and support evolution and extension and reuse
    • try to be a good Web citizen by embracing integration
  • Design with the Web in mind
    • use Web standards where appropriate (URIs for identification)
    • even intranet applications typically evolve and should be designed for the Web
  • Make content visible, accessible, usable, reusable
    • URI design guidelines should be defined and followed
    • think about aggregation and granularity and access to resources
    • use well-defined and well-documented XML for B2B scenarios
    • reuse existing vocabularies or vocabulary parts whenever possible
 Blogs as Web Applications Blog in XML Support URI Guessing (Year Index) Support URI Guessing (Month Index) Support URI Guessing (Day Index) Support Spontaneous Navigation Publishing as Atom Feed Blog as Atom Feed Conclusions Web Architecture Essentials
  • Principles (violating these causes architectural problems)
  • Constraints (disregarding these causes technical problems)
  • Good Practices (ignoring these causes user problems)
 Internationalization (I18N) & Localization (L10N) 2009-12-01 I18N & L10N Markup Unicode · History · Content · Tag Set Many publishing environments need to support multiple languages. In many cases, the requirement to support multiple languages surfaces in later stages of a product development or publishing solution, which can cause major design changes, driving up costs. Internationalization (I18N) is the approach to design systems which can adapt to different locales. Localization (L10N) is the activity to identify, define, and encode locales, based on internationalized software. For languages using different alphabets, Unicode is the most popular character set today and provides a variety of encoding schemes, each of them being a Unicode Transformation Format (UTF). Abstract

Characters Characters and Computers
  • American Standard Code for Information Interchange (ASCII)
    • for the first time a basic set of characters had a universally accepted encoding
    • many Internet protocols (such as HTTP) encode their information in ASCII commands
  • ASCII is a very limited repertoire of characters
    • basic ASCII contains 128 characters (7 bit) with a number of control chars
    • no variants of characters (german umlauts, french accents) are supported
    • various code pages extending ASCII to 8 bit exist and are hard to distinguish
  • Character is not a trivial concept when regarded globally
    • european languages all have writing systems based on a small number of atoms
    • other languages and writing systems have vastly different ideas of language atoms
 Characters
Character. (1) The smallest component of written language that has semantic value; refers to the abstract meaning and/or shape […]

The Unicode Standard, Version 4.0, Addison-Wesley, 2003

  • The alphabetic approach is only one of several possibilities
    • A character in Japanese hiragana and katakana scripts corresponds to a syllable (usually a combination of consonant plus vowel)
    • Korean Hangul combines symbols for individual sounds of the language into square blocks, each of which represents a syllable; depending on the user and the application, either the individual symbols or the syllabic clusters can be considered to be characters
    • In Indic scripts each consonant letter carries an inherent vowel that is eliminated or replaced using semi-regular or irregular ways to combine consonants and vowels into clusters; depending on the user and the application, either individual consonants or vowels, or the consonant or consonant-vowel clusters can be perceived as characters
    • Arabic and Hebrew vowel sounds are typically not written at all; when they are written they are indicated by the use of combining marks placed above and below the consonantal letters
 Glyphs
[A Glyph is] a recognizable abstract graphic symbol which is independent of a specific design.

ISO/IEC 9541:1991, Information Technology – Font Information Interchange

  • Visual rendering introduces the notion of a glyph.
  • There is not a one-to-one correspondence between characters and glyphs
    • A single character can be represented by multiple glyphs (each glyph is then part of the representation of that character); these glyphs may be physically separated from one another
    • A single glyph may represent a sequence of characters (this is the case with ligatures, among others)
    • A character may be rendered with very different glyphs depending on the context
    • A single glyph may represent different characters (e.g. capital Latin A, capital Greek A and capital Cyrillic A)
 Character Sets History of Character Sets
  • Text documents need ways to represent characters
    • computers handle bits, not characters
    • to handle characters, computers need a mapping from characters to bits
  • For a long time, computers were doing their work in a very isolated way
    • I think there is a world market for maybe five computers. (¬ T. J. Watson)
  • With more computers being used, more data is exchanged between computers
  • Data rot happens on all levels (media, formats, applications)
  • Standardization of character sets started in the 60's
    • ASCII was the first generally accepted character set
    • EBCDIC was invented and marketed by IBM (and a terribly designed character encoding)
    • ISO 8859 was the first attempt to better support character sets beyond ASCII
    • asian scripts were always a problem because of the number of characters they need
 ASCII 1963 ASCII 1965 ASCII 1967 Beyond ASCII
  • ASCII is called ASCII for a reason
    • it works well for english-speaking countries
    • the majority of other languages cannot be represented
  • Character sets and the 8 bit computer start to collide
    • ASCII is very convenient because characters and bytes correspond 1:1
    • every character set expanding ASCII will make this more complicated
    • complications can occur within and/or outside of the character set
  • Introducing a character set beyond 8 bit is a fundamental change
    • dealing with and counting bytes is a seductively simple idea
  • Introducing several 8 bit character sets saves the 8 bit world
    • by introducing several character sets, each of them can remain 8 bit
    • the complexity has now been shifted to the handling of various character sets
 ISO 8859
  • A family of character sets rather that a single character set
    • each ISO 8859 family member is an 8 bit character set (256 characters)
    • the lower half (128 characters) are always the same (ASCII)
    • the upper half is supporting different user groups and changes between versions
  • ISO 8859 files cannot be identified by inspection
    • ASCII characters can always be safely interpreted (identical on all ISO 8859 code pages)
    • the upper half can only be interpreted if the code page is well-known
  • ISO 8859 environments must carefully track the code pages being used
    • failure to do so results in misinterpretation of characters
 ISO 8859-1 (Latin-1) & ISO 8859-2 (Latin-2)

Latin-1 (Western European)

Latin-2 (Central European)
ISO 8859-7 (Greek) & ISO 8859-15 (Latin-9)

Greek

Latin-9
Unicode Basics ISO 8859 Problems
  • One document can only contain characters from one character set
    • mixing characters from different sets is impossible
  • An increasing number of character sets does not make life easier
    • in particular, if they sometime differ only slightly (e.g., Latin-1 vs. Latin-9)
  • For bigger character sets, the 8 bit approach is not working at all
    • the ISO 8859 approach allows only 128 special characters (the lower half is ASCII)
  • ISO 8859 is as good as it gets with 8 bit
    • to improve this approach, the 8 bit philosophy must be abandoned
 Unicode
  • Generalization takes characters beyond one and even two bytes
    • Unicode has been designed to cover all characters of the world
    • Unicode recently added its 100,000th character
    • for handling this character set, a more structured approach is required
  • Unicode cleanly separates various conceptual steps
    • characters are collected and are then part of the character repertoire
    • characters are then identified by a unique code point (written as U+0041)
    • a Character Encoding Scheme (CES) then maps the Coded Character Set (CCS) based on a Character Encoding Form (CEF)
  • XML is ASCII for the 21st century
    • purists sometimes consider Unicode too big or dangerous
    • Unicode is well-established and is necessary in a globalized economy
 Unicode Character Count
  • Unicode has the ability to encode 17 × 216 = 1'114'112 characters
    • this means that currently ~10% of the available space is used
  • Characters are organized into 17 planes of 216 = 65'536 characters
  • Planes are numbered from 0 to 16
  • Plane 0 is the Basic Multilingual Plane (BMP)
    • all characters which in practical use today are part of the BMP (well, almost …)
  • Planes beyond the BMP contain rare and historic characters
 Unicode Encodings
Aא好
Code pointU+0041U+05D0U+597DU+233B4
UTF-841D7 90E5 A5 BDF0 A3 8E B4
UTF-1600 4105 D059 7DD8 4C DF B4
UTF-3200 00 00 4100 00 05 D000 00 59 7D00 02 33 B4
 UTF-8
  • UTF-8 is one of the two standardized encodings for XML
    • every ASCII document by definition is a UTF-8 document
    • UTF-8 must be supported by every XML implementation
  • UTF-8 is not trivial, but it is widely supported and easy to implement
    • there is no 1:1 correspondence between bytes and characters
    • each Unicode character is encoded by 1-6 bytes
    • UTF-8 is good for europeans (1 byte per ASCII character)
  • Non-Unicode documents must be transcoded to UTF-8
    • keeping track of resource character encodings is a good idea
 Other UTFs
  • UTF-16 stores every character as 2 or more bytes
    • BMP characters are stored as 2 bytes
    • astral plane characters are stored as 4 bytes
    • UTF-16 is the other encoding (in addition to UTF-8) required by XML
  • UTF-32 stores every character as 4 bytes
    • very simple and very inefficient (requires four times more space than ASCII)
  • Multi-byte formats introduce the problem of byte order
    • UTF-16/32BE and UTF-16/32LE are stored with guaranteed endian
    • UTF-16/32 may use a Byte Order Mark (BOM) (U+FEFF) to detect the endian
 Character Set Identification
  • HTTP, XML, and HTML support character set identification
    • HTTP supports the Content-Type header field
      • Content-Type: text/html; charset=utf-8
    • XML encodes the character set in the XML declaration
      • <?xml version="1.0" encoding="utf-8"?>
    • HTML supports the meta element in the document's head
      • <meta http-equiv="Content-Type" content="text/html;charset=utf-8">
  • Different identifications serve different purposes
  • Conflicting identifications are a sign of management problems
 Unicode is Complex
  • Many languages have characters which are composed
    • german umlauts are vowels with a double dot
    • french accents and the cedilla are also added to regular characters
  • Unicode contains composed as well as composing characters
    • for most languages, composed characters are considered to be regular characters
    • in some circumstances, it might be required to compose a character out of a base and a diacritical mark
    • as a result, the question arises how to define the equality of these variants
  • Unicode defines normal forms which prescribe one variant
    • a complex field with different concepts of equivalence (canonical and compatibility)
    • based on the equivalence forms, there are four normalization forms
Transcoding
  • Handling Unicode in an unconstrained environment is not easy
    • Unicode's size and variability make character processing harder then it used to be
    • for some applications with limited character need, this might be too much
  • In all text-based environments, well-defined rules must be defined for:
    1. the encoding of documents (maybe if variants, such as Unicode normalization forms)
    2. accepted incoming encodings and how they are mapped to the internal encoding
    3. available outgoing encodings and how they can be requested and will be generated
  • Transcoding is the activity of changing the encoding of data
    • ideally, transcoding should be lossless and round-trip proof
    • in any scenario with non-trivial encodings, this is not an easy goal
    • even staying with one encoding can be a problem (if there are variations allowed)
  • Transcoding is essential for maintaining data quality
 Internationalization (I18N) What is Language?
  • Language is more than the encoding of individual words
    • languages and culture are deeply intertwined and inseparable
    • ideally, systems/solutions should adapt to culture, not only to language
    • on a superficial level, adapting to language is useful a first step
  • Languages have properties which are beyond character sequences
    • for right-to-left languages, screen layout should be done right-to-left
    • if languages are mixed, one language has to be the preferred language
  • Language identification and selection are basic I18N tasks
    • the one language fits all assumption is becoming increasingly inappropriate
    • the just switch the labels strategy also may be too little for true L10N
 Beyond Language
  • Directionality is the concept how written language is organized
    • early languages had no inherent directionality (even zigzag writing was possible)
    • the majority of today's languages are left-to-right languages
    • Arabic and Hebrew are two popular right-to-left languages
    • Chinese can be written top-to-bottom and right-to-left
    • Mongolian (using the Uighur alphabet) is written top-to-bottom and left-to-right
  • Icons can be very culture-specific and often need to be localized as well
    • icons are pictorial metaphors (rooted in language and/or culture)
    • language-specific metaphors may not work across languages (e.g., OK gesture)
    • culture-specific metaphors may not work across cultures (e.g., owls)
 Definition
Internationalization is the design and development of a product, application or document content that enables easy localization for target audiences that vary in culture, region, or language.
  • I18N starts at the design phase and influences the complete process
  • The upfront costs of I18N are considerable (increased complexity)
  • The costs of retroactive I18N are much higher than that
 I18N Tasks
  1. UI elements (windows, menus) must be modified to accept translated text
  2. Static text must be made configurable
  3. Icons and graphics must be changed to be more culturally appropriate
  4. Sound files that contain spoken language must be re-recorded
  5. Online help must be translated
  6. Dynamic text (dates, times) must be formatted using the locale
  7. Text handling code must calculate word breaks using the locale
  8. Tabular data must be sortable using the locale
 Localization (L10N) Definition
Localization refers to the adaptation of a product, application or document content to meet the language, cultural and other requirements of a specific target market (a locale).
  • Without proper I18N, L10N is a very expensive and risky process
  • L10N based on internationalized products should be straight-forward
  • during L10N for specific locales, new areas for I18N might be identified
    • some non-internationalized part of the product is identified as inappropriate
  • L10N should be regarded as an input for improved I18N
 L10N Tasks
  1. Create translations for all interface elements
  2. Translate all static texts
  3. If necessary, create localized icons and graphics
  4. Any spoken text must be recorded in the target language
  5. Make sure that the localized product uses the localized online help
  6. Formatting of data types must be treated locale-specific
  7. If necessary, dictionaries and other language tools must be integrated
  8. Sorting functions in the code must respect the locale
 Language Identification in Resources Language Codes
  • Language identification is required in many contexts
    • successful identification needs a standardized set of language tags
  • RFC 4646 defines Tags for Identifying Languages
    • two letter codes such as en are interpreted according to ISO 639-1
    • three letter codes such as eng are interpreted according to ISO 639-2
    • x- indicates a value which is not standardized (requires mutual agreement)
    • subtags such as en-US specify additional properties (regions, dialects, scripts, …)
  • RFC 4647 defines the Matching of Language Tags
    • matchmaking between requested and available languages
    • how to proceed if en-US or de are requested but available variants are en and de
 ISO 639-2 Code List IANA Language Subtag Registry URIs for Multilingual Resources Naming Language Variants
  • URIs are intended to be names for resources (not for representations)
    • when does a representation become an individual resource?
    • the distinction is a gradual one (and it also applies to versioning over time)
    • if I send you a bookmark of a german page, should you get an english variant?
  • Advantages when language variants are different resources
    • language variants can be identified (e.g., bookmarked) reliably
  • Advantages when language variants are the same resource
    • when accessing the resource, a more appropriate variant can be negotiated dynamically
  • Ideally, language variants should use a mix of both approaches
    • have their own URI so that they can be bookmarked and handled individually
    • remain identifiable as a variant of the more generic language-independent resource
 Variant Naming Variations
  • URIs can be used in a number of different ways
    • any Web-based publishing should have a strategy for using URIs
    • being consistent is almost as important as doing it right
  • For variations of URIs, various parts of the Web architecture can be used
 DNS Domains 
http://en.example.com/some/page
  • Defines DNS subdomains for all supported languages
  • Advantages
    • offers easy load-balancing to different servers (but code must kept in sync)
    • bookmarks identify the language variant
  • Disadvantages
    • no easy way to get from one variant to another in terms of URI navigation
    • language management requires DNS updates
    • DNS names have not been designed for this kind of usage
 Constructed Paths 
http://example.com/en/some/page
  • Encodes the language as the first path segment of the URI path
  • Advantages
    • bookmarks identify the language variant
  • Disadvantages
    • logically, the URI path does not represent the hierarchy of resources
    • no easy way to get from one variant to another in terms of URI navigation
    • hard to maintain if this is used as the actual layout of documents in directories
 Query Strings 
http://example.com/some/page?lang=en
  • Use a query string to get the language in the required language variant
  • Advantages
    • bookmarks identify the language variant
    • http://example.com/some/page is usable for the abstract resource
  • Disadvantages
    • does not allow proper caching of pages
    • search engines ignore query strings and thus cannot link to the variants
    • against the idea of query strings which are meant for dynamic content
    • hard to combine with other query string information (if required)
 DNS TLDs 
http://example.us/some/page
  • Use DNS TLDs to identify the supported languages
  • Advantages
    • bookmarks identify the language variant
  • Disadvantages
    • requires registration of many TLDs (effort, costs, domain squatting)
    • countries do not identify languages (requires additional mechanism)
    • because of the country/language complexity, cross-language links are hard to maintain
    • DNS names have not been designed for this kind of usage
 Cookies 
http://example.com/some/page
  • Store the cookie with the language preference and use the cookie setting
  • Advantages
    • language is not part of the URI and thus the URI identifies the resource
    • bookmarks (and any intra-page links) will automatically yield the preferred language variant
  • Disadvantages
    • does not work if are disabled or intercepted
    • it is not possible to create a bookmark for the language variant
 Content Negotiation 
http://example.com/some/page
  • Use browser settings and to get the variant
  • Advantages
    • language is not part of the URI and thus the URI identifies the resource
    • bookmarks (and any intra-page links) will automatically yield the preferred language variant
  • Disadvantages
    • does not work if the user agent does not support
    • does not work if the user has not configured the browser correctly
    • switching between languages requires configuration of the browser
    • it is not possible to create a bookmark for the language variant
 Path Segment Name 
http://example.com/some/page.en
  • Use . which uses the resource's extension
  • Advantages
    • bookmarks identify the language variant
    • maps easily to extensions in file systems
  • Disadvantages
    • does not mix well if additional properties (such as the format) are required
    • no easy way to get from one variant to another in terms of URI navigation
    • not officially recognized as URI structure (just simplifies parsing and recognition)
 URI Sub-Delimiter Comma 
http://example.com/some/page,en
  • Use , for specifying a parameter to a URI path segment
  • Advantages
    • bookmarks identify the language variant
    • makes the language identifiable as a special part of the URI
    • http://example.com/some/page is usable for the abstract resource
  • Disadvantages
    • does not mix well with parameters for other resource dimensions
    • not officially recognized as URI structure (just simplifies parsing and recognition)
 URI Sub-Delimiter Semicolon 
http://example.com/some/page;lang=en
  • Use ; for specifying a parameter to a URI path segment
  • Advantages
    • bookmarks identify the language variant
    • makes the language identifiable as a special part of the URI
    • can be combined with parameters for other resource dimensions
    • http://example.com/some/page is usable for the abstract resource
  • Disadvantages
    • not officially recognized as URI structure (just simplifies parsing and recognition)
 Now What?
  • There is no perfect solution for naming language variants
    • the Web architecture does not provide support for variant preference in a URI
    • there is a strong separation between static URIs and dynamic content negotiation
  • The name-based solutions (using ., ;, and ,) are very similar
    • any difference between them is how they are used in practice
    • from the URI point of view, path segments containing ., ;, and , are not treated in any special way
    • the only officially recognized special path segments are . and ..
  • Theoretically, HTTP could define scheme-specific semantics
    • for http URIs, , and/or ; could interact with content negotiation
    • while this is an interesting idea with many useful applications, it is very unlikely to happen
  • Use , or ; for language variants, but do not expect magic to happen
 Conclusions Babelification
  • I18N is a non-trivial task and requires some planning
  • L10N involves much more than just translating text
  • Producing fully localized products is a complex issue
  • The locale is based on language and culture and customs
  • Successful L10N requires the input of local people
 Web Trends 2009-12-03 Web architecture in many cases simply lays the groundwork for developing application areas. In this final lecture we briefly look at some of the current trends on the Web, and how they connect to Web architecture. While the drivers of the trends often are not exclusively technical, they often have a substantial background in technology as an enabler of applications. Not all of the technological issues are within the realm of Web architecture, but increasingly the Web ties together a lot of formerly disconnected application areas, and serves as an integration and unification platform. Abstract

Web Versioning Past Developments
  • Web 2.0
  • Blogging and micro-blogging
  • Social networks
  • Web services
 The Location-Aware Web Location-Based Services Location and the Web
  • The current Web is unaware of location as a concept
    • it supports other concepts such as media types and languages
    • many applications use location and expose location in the UI
    • mobile access and location-oriented services become more widespread
  • How to express location as a Web-level concept?
    • should there be a URI scheme for locations?
    • should there be location support in HTTP?
    • should there be location metadata in HTML?
    • should there be location access in Web Apps? (there is)
    • should there be location concepts in RDF?
  • Thinking beyond location-oriented applications
    • applications typical are closed verticals
    • mashability and repurposing requires exposed application models
 The Mobile Web Mobile Phones Native vs. Web
  • iPhone switched from Web-based to native
    • iPhone OS 2.0 introduced native apps and the App Store
    • the native app model have proven to be the killer feature of the iPhone
    • other mobile platforms try to replicate the model
  • In-App Purchases allow apps to use micropayments
  • Phones will have increasingly powerful browsers
    • phones might also have only browsers (ChromeOS)
    • might be a better long-term strategy
  • Currently too many players on the mobile OS market
    • consolidation is inevitable
    • the Web is moving to , why not the mobile Web as well?
 Web Apps HTML5 for Advanced Web Apps The CSS3 Effect?
  • HTML5 itself defines the language and the DOM
  • Web Apps need additional functionality in terms of APIs
  • Turning the browser into an OS
    • the Java vision without the Java language
    • where do you stop? (3D rendering, device access, introspection)
 Web Semantics Understanding the Web Understanding Resources
  • Finding needles in a haystack
    • search engines are amazing but limited in their understanding
    • more understanding of resources allows better resource access
  • Semantics are always embedded into perspectives/models
    • who has the right to understand/describe resources?
    • how do I know how the resource was interpreted and described?
    • how can I find alternative descriptions?
    • where is the line between data and information/knowledge?
  • The tough challenges are not on the technical level
    • kid-friendly search engines
    • browser-level or OS-level content filtering
    • Internet-level or Web-level filtering
 Web of Things Connecting Real-World Objects Internet vs. Web
  • Internet of Things is the current buzzphrase
    • a useful and necessary first step (establish connectivity)
    • not enough to enable ubiquitous accessibility and usability
  • Apply Web architecture to the Internet of Things
    • expose Things via RESTful Web services
    • from the Web's perspective, things are just resources
  • Some new twists and challenges have to be solved
    • how to find things (you cannot search for them with Google)
    • how to find the interesting things (there are too many things)
    • how to interact with them (is HTTP sufficient as the uniform interface?)
    • how to support real-time interactions (push instead of pull)
 The Real-Time Web Tracking in Real-Time Pull vs. Push
  • Pulling is the underlying principle of the current Web
    • interactions are initiated by the client
    • servers don't have to keep track of their clients
  • Translating between the Pull and Push patterns
    • translating push into pull is trivial: store the push, wait for the pull
    • translating pull into push requires some optimization
  • Pushing is faster but more expensive
    • scalable pushing infrastructures can be built (iPhone push notifications)
    • building it in a distributed way makes it more flexible and scalable
  • PubSubHubbub builds a push model around feeds
    • feeds declare that the content is also available through a hub
    • hubs can be configured into a multi-level hierarchy
    • clients connect to hubs and are notified by their hub
 Conclusions
  • Web Architecture as Foundation
    • The Web has become an essential infrastructure
    • Understanding the Web helps understanding services
    • Web architecture and the Web will keep evolving
    • Certain principles are surprisingly stable over time
    • Don't spend too much time analyzing buzzphrases