Abstract

In most scenarios today, structured data is exchanged using XML. For non-trivial data structures, it is important to have a schema for the data structures, so that users know what to expect and/or what to generate. XML Schema is the most popular language for XML schemas today. XML Schema has many powerful and complex features, which means that any problem can be solved in many different ways in XML Schema. This lecture describes some guidelines for writing well-designed, open, extensible, and well-documented XML schemas.

Common Models

• Cooperation is impossible without a common model
  • syntax (like XML) is not an application-level model, it is a representation for model-based data
  • standard syntax is good, but it is not sufficient for achieving interoperability
• Trivial data models can be understood without a schema
  • appropriate for micro-formats and comparable vocabularies
  • larger vocabularies need better definitions and descriptions
• Designing for change
  • more complex data models are more likely to evolve
  • loose coupling should allow individual versioning of components
  • Openness and Extensibility should be part of the data design

What is a Type?

• A type is a set of values
  • the values can be enumerated (phone number types home, mobile, office)
  • the values can be described by extension (intervals, regular expressions)
• DTD have (almost) no types
  • element content is always #PCDATA (any number of any characters)
  • attributes most often are CDATA (any number of any characters)
  • attributes may have enumerated types (but no extensional types)
  • attributes may use ID/IDREF

XML Schema vs. DTD

Document/Data Perspectives

• XML as documents is text interspersed with structure
  • XML captures text structures that support document processing
  • without these structures, the text remains usable (as unstructured text)
  • structure is good, but not indispensable
• XML as data is structure filled with data
  • programmers think about classes and objects, so they need types
  • without structure, data-centric XML is completely useless
  • programmers often view XML as wire format and types as the portal to their objects

What are Simple Types?

• Simple types describe values not structured by XML markup
  • they describe attribute values (date="2006-10-03")
  • they describe element content (<phone>+1-510-6432253</phone>)
• Simple types can be used for elements or attributes
  • XML Schema treats contents in elements and attributes equally
  • simple type libraries can be designed independent of their eventual use
• Simple types are available in three flavors
  • atomic types: one value of one type (one number in some range)
  • union types: one value of a union of types (a number or the string undefined)
  • list types: a whitespace-separated list of values (phone type="home office")

Named vs. Anonymous

• Types can be named or anonymous
  • named types have a name and can be referenced (and thus be reused)
  • anonymous types have no name and can only be used where they are defined

 <xs:element name="home" type="phoneType"/>
 <xs:element name="office" type="phoneType"/>
 <xs:simpleType name="phoneType">
  <xs:restriction base="xs:string">
   <xs:maxLength value="30"/>
  </xs:restriction>
 </xs:simpleType>

 <xs:element name="business">
  <xs:simpleType>
   <xs:restriction base="xs:string">
    <xs:maxLength value="30"/>
   </xs:restriction>
  </xs:simpleType>
 </xs:element>

Type Definitions

• Simple types are sets of values
  • named simple types are sets of values with a name (and thus reusable)
  • anonymous simple types are sets of values defined where they are needed
• Simple types are defined to represent model-level information
  • in most cases, they will have restrictions associated with them
  • they may also simply be tags for semantics (fax and phone numbers share the same value space)
• XML Schema has a library of built-in datatypes
  • ur-types are the conceptual grounding of all types
  • primitive types are the types that are there by definition
  • derived types are based on primitive types
  • users can derive their own types using simple type restriction

Type Hierarchy

Built-In Types

<?xml version="1.0" encoding="UTF-8"?>
<xs:schema xmlns:xs="http://www.w3.org/2001/XMLSchema">
 <xs:simpleType name="integer">
  <xs:restriction base="xs:decimal">
   <xs:fractionDigits value="0" fixed="true"/>
  </xs:restriction>
 </xs:simpleType>
 <xs:simpleType name="nonNegativeInteger">
  <xs:restriction base="integer">
   <xs:minInclusive value="0"/>
  </xs:restriction>
 </xs:simpleType>
 <xs:simpleType name="positiveInteger">
  <xs:restriction base="nonNegativeInteger">
   <xs:minInclusive value="1"/>
  </xs:restriction>
 </xs:simpleType>
</xs:schema>

How to Restrict

• Simple types can be derived by restriction
  • the base type must be a simple type
  • the derived type will be a simple type
  • all simple types form a tree, rooted ad the anySimpleType
• Restriction are based on facets
  • each restriction can use 0-n facets
  • facets can be refined in further simple type restrictions
  • XML Schema designers should try to restrict types as much as possible

Facets

• Facets define a certain way of restricting a simple type
  • facets are independent, but they may interact (minLength and maxLength)
  • XML Schema defines 12 constraining facets which may be used for restrictions
  • length, minLength, maxLength, pattern, enumeration, whiteSpace, maxInclusive, maxExclusive, minExclusive, minInclusive, totalDigits, fractionDigits
• Facets may be repeated in different levels of the type hierarchy
  • they may only further restrict the facet (e.g., reducing the maxLength)
  • facets apply to all directly or indirectly derived subtypes
  • facets may be fixed (no further restriction is allowed)
• Not all facets are applicable to all types
  • the applicability depends on the primitive type being used

Facet Applicability

Patterns

• Patterns restrict the lexical space of simple types
  • most other facets restrict the value space (e.g., intervals of numbers)
  • in many cases, patterns are useful additions to value-oriented facets
• Patterns are regular expressions
  • they support many common regex constructs and Unicode
  • the language pattern allows de, de-CH, and other tags
  • the pattern checks for lexical correctness, not against a code list

([a-zA-Z]{2}|[iI]-[a-zA-Z]+|[xX]-[a-zA-Z]{1,8})(-[a-zA-Z]{1,8})*

Simple Type Examples

<?xml version="1.0" encoding="UTF-8"?>
<xs:schema xmlns:xs="http://www.w3.org/2001/XMLSchema">
 <xs:simpleType name="myIntegerType">
  <xs:restriction base="xs:integer">
   <xs:minInclusive value="10000"/>
   <xs:maxInclusive value="99999"/>
  </xs:restriction>
 </xs:simpleType>
 <xs:simpleType name="stockKeepingUnitType">
  <xs:restriction base="xs:string">
   <xs:pattern value="\d{3}-[A-Z]{2}"/>
  </xs:restriction>
 </xs:simpleType>
 <xs:simpleType name="USStateType">
  <xs:restriction base="xs:string">
   <xs:enumeration value="AK"/>
   <xs:enumeration value="AL"/>
   <xs:enumeration value="AR"/>
   <!-- and so on ... -->
  </xs:restriction>
 </xs:simpleType>
</xs:schema>

Facet Limitations

• Facets limit one dimension of a type's value space
  • using pattern, the lexical space can also be restricted
  • restrictions should be made as specific as possible
  • no limitations are possible beyond the predefined facets
• There is no connection to the context within the document
  • facets cannot make references to other values (e.g., neighboring attributes)
• Additional constraints should be documented
  • documentation enables applications to implement constraint checking
  • other schema languages (like Schematron) may be used to express these constraints

What is a Complex Type?

• Complex types describe the allowed element content
  • they describe what the element may contain (the element's content model)
  • they describe the attributes that an element may have (the element's attribute list)
• Complex types do not define the element name
  • the complex type defines which content is allowed for the element
  • the element definition uses the complex type to define the allowed element content
• Complex types have similar properties to simple types
  • they can be named or anonymous
  • Complex Type Derivation can be used to construct a type hierarchy

Complex Type Example

<?xml version="1.0" encoding="UTF-8"?>
<xs:schema xmlns:xs="http://www.w3.org/2001/XMLSchema">
 <xs:element name="billingAddress" type="addressType"/>
 <xs:element name="shippingAddress" type="addressType"/>
 <xs:complexType name="addressType">
  <xs:sequence>
   <xs:element name="name" type="xs:string"/>
   <xs:element name="street" type="xs:string"/>
   <xs:element name="city" type="xs:string"/>
   <xs:element name="state" type="xs:string" minOccurs="0"/>
   <xs:element name="zip" type="xs:decimal"/>
  </xs:sequence>
  <xs:attribute name="country" type="xs:NMTOKEN"/>
 </xs:complexType>
</xs:schema>

Complex Types & Content Types

• Complex types can have different kinds of content
  • simple content refers to simple type content using additional attributes
  • complex content is anything else (anything beyond simple type content)
• Complex Type Derivation heavily depends on this classification

DTD Content Models

• Defining Elements in DTDs uses a compact syntax
  • XML Schema supports the same facilities with a more verbose syntax
  • XML Schemas adds features which DTDs do not support
• DTDs allow elements to be mandatory, optional, repeatable, or optional and repeatable
  • XML Schema allows the cardinality to be specified
• DTDs allow sequences (,) and alternatives (|)
  • XML Schema introduces a (very limited) operator for all groups
• Apart from the syntax, XML Schema content models are not very different

Mixed Content

• DTDs define mixed content by mixing #PCDATA into the content model
  • DTDs always require mixed content to use the form ( #PCDATA | a | b )*
  • the occurrence of elements in mixed content cannot be controlled
• XML Schema defines mixed content outside of the content model
  • the content model is defined like an element-only content model
  • the mixed attribute on the type marks the type as being mixed
• XML Schema mixed content can use all model groups
  • it is possible to constrain element occurrences in the same way as in element-only content
  • in practice, this feature is rarely used (mixed content often is very loosely defined)

 <xs:element name="p" type="mixedType"/>
 <xs:complexType name="mixedType" mixed="true">
  <xs:choice maxOccurs="unbounded" minOccurs="0">
   <xs:element ref="b"/>
   <xs:element name="i" type="xs:string"/>
   <xs:element name="u" type="xs:string"/>
  </xs:choice>
  <xs:attribute ref="class"/>
 </xs:complexType>

Empty Content

• DTDs have a special keyword for empty elements
  • instead of the content model, the keyword EMPTY is used
  • empty elements may still have attribute lists associated with them
• XML Schema empty types are defined implicitly
  • there is no explicit keyword for defining an empty type
  • if a type has no model group inside it, it is empty (it still may have attributes)

Schema Use and Evolution

• Schemas encode the current state of the data model
  • some part of the data model has been fixed
  • other parts may be open to specific user needs
  • at certain points, extension points allow evolution
• Development becomes harder
  • Openness: Not all data structures are known in advance
  • Extensibility: Future developments may allow additional structures to appear
  • both aspects must be accounted for in the code (or it will break)

Wildcards

• XML Schema allows wildcards for specifying open content models
  • any allows any element to appear (namespaces can be specified)
  • anyAttribute allows any attribute to appear (namespaces can be specified)
• Wildcards can specify the processing requirements
  • skip: do not validate any content
  • lax: validate if a schema is available
  • strict: validate always (error if no schema is available)
• Wildcards can specify the permissible namespace(s)
  • ##any: contents from any namespace are allowed
  • ##other: contents from all namespaces other than the targetNamespace are allowed
  • ##targetNamespace: contents from the targetNamespace are allowed
  • ##local: any unqualified content (not in a namespace)
• Programming must take wildcards into account

Wildcard Example

<?xml version="1.0" encoding="UTF-8"?>
<xs:schema xmlns:xs="http://www.w3.org/2001/XMLSchema">
 <element name="purchaseReport">
  <complexType>
   <sequence>
    <element name="regions" type="r:RegionsType"/>
    <element name="parts" type="r:PartsType"/>
    <element name="htmlExample">
     <complexType>
      <sequence>
       <any namespace="http://www.w3.org/1999/xhtml" minOccurs="0" maxOccurs="unbounded" processContents="skip"/>
      </sequence>
     </complexType>
    </element>
   </sequence>
   <attribute name="period" type="duration"/>
   <attribute name="periodEnding" type="date"/>
  </complexType>
 </element>
</xs:schema>

Openness

• Open schemas allow users to add structures they care about
  • openness can be used to allow random content
  • openness can be used for modularization
• Schemas should only define the core components for the service
  • optional components bind them into the core schema
  • having modularized schemas better supports evolution
• Openness based on namespaces has its disadvantages
  • developers must be disciplined enough to respect the schema
  • many people dislike namespaces
  • naïve processing becomes very hard because of the variability

Extensibility

• Openness talk about a schema being open to other content
  • extensibility is the ability to create a new version of the schema
  • if the schema is open, new instances can validate against the old schema
  • if the schema is well-designed, old programs may process new instances
• Extensible schemas have well-designed type hierarchies
  • instead of adding totally new content, existing types are extended
  • if the processing code is type-based, type casting can be used
  • code reuse allows the processing of new types with existing code
  • Complex Type Extension is a kind of open content
• Extensibility has its natural limits
  • an extension policy should tell developers what to expect
  • old code should break if the semantics of content are changed

Schema/Namespace Policy

• Can a namespace change over time?
  • may schema documentation become outdated? extended? replaced?
  • this should be clearly documented in the namespace description
• The XML XML Namespace was widely believed to be defined by XML
  • xml:lang and xml:space defined by XML
  • xml:base was added by XML Base
  • xml:id was added by xml:id
• When defining namespaces, plan ahead and publish everything
  • dependencies, change management, and versioning issues are important
  • there still is no accepted standard for namespace descriptions
  • useful ideas are examples, test suites, sample code, dependencies

Namespace Descriptions

Erik Wilde, Structuring Namespace Descriptions, 15th International World Wide Web Conference (WWW2006), Edinburgh, UK, May 2006.

Plan Ahead

• Design for Change is very important for data modeling
• In a REST design, data models are nicely decoupled
• Open schemas allow users to use additional information
• Extensible schemas allows schema evolution
• Developers must know what to expect
• Development becomes harder, code becomes more robust and long-lived