Abstract

The Extensible Markup Language (XML) has been introduced in 1998 to enable content providers to publish their content on the Web in an application-specific format. HTML was considered as conveying not enough semantics, since its only purpose was (and is) the preparation of content for Web-based publishing. XML was the first step towards machine-readable data formats for the Web, a trend that since its invention has been taken to higher levels with the idea of the Semantic Web. XML appeared when the Web was in the steepest part of its success curve, and since then has taken over as the globally accepted format for the exchange of machine-readable structured data.

XML Executive Summary

• More and more value switches from goods to information
• Information sharing needs well-defined structures
• Business agility and flexibility are critical success factors
• Standardized formats prevent lock-in and incompatibilities
• XML is the most successful format for structured data
• XML technologies are widely used and universally available
• XML for B2B enables better workflow engineering
• XML for B2C is a good interface between B2B and Web interfaces
• XML is a mission-critical success factor for optimizing ROI and minimizing interoperability risks in today's fast-moving globalized fragmented business landscape …

What's the Plan?

• XML Basics and how to apply them
• Describing classes of XML documents
• Combining different vocabularies of XML documents
• Selecting parts of an XML document
• Transforming XML into something else (or XML again)
• A more complicated way to describe classes of XML documents
• Even more ways of describing classes of XML documents
• How does all of this relate to databases?
• What to expect as future developments

What are we doing?

• Assignments
  • blogs as the common theme (the perfect XML application example)
  • how to create an XML document representing a blog
  • how to write a schema describing this document's structure
  • how to select parts of the blog (posts, titles, comments, …)
  • how to transform blogs (into HTML, RSS, Atom, …)
  • how to extract blog information from an XML database
• Tools
  • XML editor such as Altova XML Spy (XSLT and XQuery included)
  • XSLT Processor such as Saxon
  • XQuery Processor such as Saxon
  • XML database such as MarkLogic or eXist

About Me

• Computer Science at Technical University of Berlin (TUB) (88-91)
• Ph.D. at ETH Zürich (92-97)
• Post-Doc at ICSI, Berkeley (97/98)
• Various activities back in Switzerland (98-06)
  • teaching at ETH Zürich and FHNW
  • working as independent consultant (training, courses, consulting)
  • research in various XML-related areas
• Visiting Assistant Professor at the School of Information (since fall 2006)
  • technical director of the Information and Service Design (ISD) program

About this Course

• Course Web page: http://dret.net/lectures/xml-fall07/
• Course mailing list: subscribe at majordomo@ischool.berkeley.edu
  • no subject (leave blank)
  • body of message: subscribe i242
• Letter grade based on final exam (30' oral)

About these Slides

• Generated from XSLidy XML
  • all Slidy presentations are generated from this source
  • 242.xml for importing the syllabus into SylViA
  • toc.html for displaying the summary on the course's Web page
• Designed for online presentation and use (lots of links!)
  • for printing, use a (all slides), and then s (smaller font) a couple of times
• A good real-world example for XML applications
  • XSLidy is useful, but there is no interface (XML editing only)
  • SylViA is useful, but there is no interface (XML editing or XSLidy export)

Additional Resources

• My Online Glossary at http://dret.net/glossary/
  • suggestions, updates, corrections are very welcome (really!)
  • XML-based and XSLT-generated HTML pages
• My bibliography at http://dret.net/biblio/
  • suggestions, updates, corrections are very welcome (really!)
• The World Wide Web Consortium (W3C)
  • the organization which invented XML
  • as well as (almost) all other technologies covered in this course

XML Yin & Yang

• XML is …
  • … great for exchanging trees (if this is what you want to do)
  • … platform-independent (even your mobile phone processes XML)
  • … a foundation for other technologies (some of which we will look at)

• XML is not …
  • … a programming language (ever programmed comma-separated values?)
  • … capturing semantics (without higher-layer consensus, XML is worthless)
  • … ensuring interoperability (we both use bits! we can interoperate!)

Why Use XML?

• Because you want to share data
  • share it in a format which is widely used and easy to use
  • enable others to use it on various platforms with existing tools
• Because you want to share data cheaply
  • it is easier to use XML than to invent something new
  • it is even easier to use an existing XML schema than to invent a new one
• Because you want to share data openly
  • if you invent new formats, people must process them
  • avoid applying the security through obscurity principle inadvertently
  • application-specific processing should be deferred to higher layers

Is XML Self-Describing?

• XML is often said to be self-describing
  • many people think this is the same as self-explanatory
  • the catch is what exactly it is you refer to by describing
• Database data cannot live without a database
  • database data is simply content, the structure is provided by a DBMS
  • XML documents have their structure encoded within them
  • compared to database data, XML in fact is self-describing
• What is the gap between self-describing and self-explanatory?
  • it is impossible to find out how the document could be modified
  • there are no semantics associated with neither structure nor content
  • so self-describing means, you can guess a lot, but you maybe wrong

XML is Character-Based

• XML is not a binary format, it is based on Unicode
  • binary structures cannot (or rather should not) be described using XML
• Multimedia formats often are binary
  • image formats such as GIF, JPEG, and PNG
  • audio formats such as MP3 and AAC
  • video formats such as MPEG4 and H.264
• But: multimedia also uses many XML formats
  • vector graphics formats such as Scalable Vector Graphics (SVG)
  • Synchronized Multimedia Integration Language (SMIL) for describing presentations

XML is a Syntax for Trees

• Not all data is easily represented by trees
  • overlapping markup (multiple views of the same content)
  • graph-like structures which are less constrained than trees
• What is it that you have in your tree?
  • XML encodes a structure purely on the syntactic level
  • what the structures mean is in no way described by XML
  • XML structures must be accompanied by semantic descriptions

XML Usages

• XML can be used in different ways
  • people should be able to use your XML directly using standard tools
  • if they absolutely need a set of special tools, something is wrong
• XML is hip, so everybody wants to use it
  • many things have been created ad-hoc and without much planning
  • if you start something which is XML-based, use XML responsibly
  • if you have to use some bad XML, complain about it
• Finding the balance can be hard
  • XML is great for prototyping and experiments
  • once you decide to redesign your XML, it may be too late
  • XML documents may be short-lived, XML schemas are definitely not

Web Technologies

• Early Web: URI+HTTP+HTML
  • URIs identify resources (in a human-readable way)
  • HTTP retrieves resources (using a simple protocol)
  • HTML is the resource format (using a simple data format)
• The early Web was a distributed hypermedia system
  • not designed by hypermedia researchers or companies
  • simple enough to be adopted very fast
• The Web today uses many different technologies
  • URI+HTTP+HTML for basic Web publishing
  • CSS & JavaScript (maybe even Ajax) for advanced publishing
• JavaScript & XML (a.k.a. Ajax)
  • scripts dynamically loading data from a server
  • machine-to-machine interaction: the server and the script

From Humans to Machines

• The Web was designed for humans
  • HTML is a language for describing page layout and links
  • machines were only used for implementing it
• Search engines were the first machine users on the Web
  • they made the Web's success possible
  • they demonstrated how hard it is to understand HTML pages
  • search engines are still a very active field of research
• A bigger Web needs more automation

HTML is for Humans

• HTML is a format for dead ends
  • HTML is good for rendering Web pages
  • HTML is bad for understanding Web pages
  • the browser is a dead end (from a machine's point of view)
• Web growth in the late 90's was enormous
  • everybody was putting information online
  • but this information was inaccessible for machines
• How can this information be made accessible to machines?
  • HTML is not the right format (slightly better than fax machines)
  • there was no other widely accepted format for structured data

A Machine-Friendly Web

• Information should be published in a machine-understandable format
  • HTML is good for rendering Web pages
  • HTML is bad for understanding Web pages
  • understanding is the key term here: application semantics!
• Information should be published in application-specific formats
  • HTML is one application: Rendering documents for humans
  • machines need other structures to process Web content
• 1996: W3C Working Group SGML on the Web
  • HTML is just one document type defined with SGML
  • SGML is a very complex and expensive technology
  • how can SGML be made easily and widely usable?

SGML, HTML, and XML

• Standard Generalized Markup Language (SGML)
  • a language for designing document types
  • a very complex standard with many expensive and non-interoperable implementations
• Hypertext Markup Language (HTML)
  • implements a simple SGML document type
  • its syntax is SGML syntax, it is not defined by HTML itself
  • uses very few SGML features, dedicated processors are rather easy to build
• Extensible Markup Language (XML)
  • a language for designing document types (i.e., classes of documents)
  • a greatly simplified version of SGML, omitting many obscure features
  • a specification with no optional parts!

XML Documents on the Web

• XML's idea was that content should be published as XML
  • stylesheets could then be used to render human-readable views
  • machines could simply use the underlying XML
• There are (almost) no XML documents on the Web
  • stylesheet support depends on browsers (software has a long life!)
  • many content providers do not want to publish machine-readable data
• There are many XML documents behind HTML documents
  • content does not have to be made public in a machine-readable way
  • browser-independent HTML can be produced from XML
  • XML technologies can be leveraged on the server-side

XML Documents Elsewhere

• XML is not used as intended, but it is very successful
  • as a server-side foundation for Web publishing
  • as a B2B-focused format with no Web publishing in mind
• XML has been successful because of different reasons
  • being there at the right time (Internet bubble)
  • politically correct (the W3C is OS-agnostic)
  • technically sound (simple and no optional parts)
  • human-readable based on a well-known syntax
  • great for rapid prototyping and experiments

Used Everywhere

• Very small: Messages from sensors
  • e.g., building automation or car electronics
  • mostly implemented in hardware or firmware
• Very large: Genome sequences
  • encoding the results of genome analyses
  • yields very large XML documents (several gigabytes)
• Very different processing requirements
  • very fast processing (time critical applications)
  • memory-conserving processing (very large documents)
  • incremental processing (streaming)
  • random access (only small parts required)

This Course and XML

• XML is the ASCII for the 21^st century
  • information professionals should know and use XML
  • you will see it in many projects
  • you will hopefully use it in many projects
  • you will be able to build and test prototypes very rapidly
• What do you need for using XML?
  • XML and some kind of schema language
  • XSLT for processing it

Sharing Concepts

• XML is a syntax for trees
  • trees are just structured data
  • for doing something useful, you must understand the trees
• Schema-based sharing of concepts is possible
  • HTML works great because everybody is using it
  • Anything beyond HTML's capabilities needs a new schema
• General sharing of concepts is hard
  • the AI community tried for decades and failed
  • micro-formats are a more humble approach to reusable shared concepts
  • agreement in communities gets exponentially harder with their size

The Semantic Web

• Technologies for describing concepts
  • the foundation of successful interaction is mutual understanding
  • describe your XML using Semantic Web technologies
• XML core technologies do not convey any meaning
  • XML is a language for exchanging trees
  • XML schema languages describe what trees may be exchanged
  • XML schema languages are for markup design
• Semantic Web technologies have received a lot of attention
  • and a lot of research funding
  • success for the most general approaches is questionable
  • proven failure as demonstrated by AI's failure
  • modest approaches are much more promising and likely to succeed