Abstract

One of the most important aspect of computer-based communications is the concept of media types, the question what type of information some digital artifact represents, and how it is encoded. The most common standard for this information is the scheme introduced by Multipurpose Internet Mail Extensions (MIME). Media types can be negotiated by peers communicating through HTTP. Some media types allow fragment identifiers, which allow references to a resource to identify a fragment of the complete resource.

Multipurpose Internet Mail Extensions (MIME)

• Basic e-mail only supports ASCII text messages
• MIME was introduced in 1993 to standardize a more powerful message format
  • multiple objects in a single message
  • text having unlimited line length or overall length
  • character sets other than ASCII, allowing non-English language messages
  • binary or application specific files
  • images, audio, video and multi-media messages
• Resource types are necessary for every automated action with resources
  • Unix started with /etc/mime.types, a list of mappings between extensions and media types
  • the Unix file command uses simple fingerprints (specified in /etc/magic)
  • double-clicking in GUIs needs a file association (based on the file's type) to work

Unix File Type Handling

image/bmp   bmp
image/cgm
image/g3fax
image/gif   gif
image/ief   ief
image/jpeg   jpeg jpg jpe
image/naplps
image/png   png
image/prs.btif
image/prs.pti
image/tiff   tiff tif

0 string  P1  image/x-portable-bitmap
0 string  P2  image/x-portable-graymap
0 string  P3  image/x-portable-pixmap
0 string  P4  image/x-portable-bitmap
0 string  P5  image/x-portable-graymap
0 string  P6  image/x-portable-pixmap
0 string  IIN1  image/tiff
0 string  MM\x00\x2a image/tiff
0 string  II\x2a\x00 image/tiff
0 string  \x89PNG  image/x-png
1 string  PNG  image/x-png
0 string  GIF8  image/gif

Windows File Type Handling

Browsers and Resources

• Web browsers retrieve resources and render them
  • HTTP can transfer any kind of resource (binary resources must be transfer encoded)
  • resource types cannot (and should not) be inferred from the URI
• HTTP combines data transfer, transfer management, and metadata
  • basic information about a resource (modification date)
  • information describing the resource's type (media type) and content (language)
  • Content Negotiation can be used to request a specific resource variant
• The resource type received may or may not be supported by the browser
  • built-in support is provided for the core Web resource types (HTML, GIF, JPEG)
  • plug-in support is an add-on to the browser for popular types (PDF, Flash)
  • external applications are standalone applications invoked by the browser

Firefox Media Type Handling

Media Type Control in Browsers

• In practice, media type control is less than perfect
  • getting control over media types means getting control over your computer
  • applications are rarely written in a perfectly well-designed way
• XML is handled very poorly by Firefox
  • rendering XML is unbelievably slow (and very primitive)
  • if the XML has errors (real or imagined by Firefox), there is no way to get to it
  • helper applications can be registered, but this is ignored by Firefox
• Competing application suites hijack as many media types as possible
  • QuickTime and Windows Media Player support the same media types
  • installing or updating these packages often changes many media type settings

Content Types

• MIME splits the word of resource types into Content Types and Subtypes
  • Content types are the main classification of a resource type
  • Subtypes qualify the format and encoding used for the content
• Content types classify the world of resource type into 8 areas
  • audio for media types representing exclusively audio signals
  • image for any media type representing two-dimensional images
  • message for resources representing e-mail messages
  • model for complex representations of physical objects (very unpopular)
  • multipart for MIME entities containing multiple individual MIME-tagged resources
  • text for mainly textual material (e.g., HTML is considered to be text)
  • video for media types combining moving pictures with audio
  • application for any resource which cannot by classified anywhere else
  • (example is only used for media type examples, not for real-world resources)

Subtypes

• Within each content type, many different data formats are in use
  • content types only allow a broad classification
  • subtypes allow the identification of a specific data format of a resource
• Subtypes are expected to be registered with the IANA
  • unregistered subtypes can be used but must have a x- prefix
• Additional qualifiers can be used to be more specific
  • text/plain is the media type for plain text files
  • plain text files have additional properties such as character encoding and language
  • text/plain can be further qualified to text/plain; charset=iso-8859-1
  • not all qualifications are available as media type (e.g., language is not addressed)

What is XML?

• XML can be regarded from various point of view
  • XHTML is (almost) identical to HTML and should be regarded as text
  • configuration files are data for some program and should be regarded as application
  • XML schemas can be mixes of these two extreme cases
• RFC 3023 registers a set of XML-related media types
  • application/xml for generic XML usage (data oriented XML)
  • text/xml for textual XML usage (document oriented XML)
  • application/xml-dtd for DTDs (which are not XML documents)
  • application/xslt+xml (proposed) for XSLT stylesheets (text/css is defined by RFC 2318)
  • image/svg+xml (proposed) for vector graphics using Scalable Vector Graphics (SVG)
  • application/xhtml+xml is defined by RFC 3236
  • … and a number of other suggestions for future XML media types

Media Type Registration

• Media types need to be registered together with a documentation
  • this makes sense if it is assumed that registered types should be openly accessible
  • this becomes complicated if the types are proprietary and not publicly documented
• It makes sense to register types even if they are not publicly documented
  • if a Word document is sent by e-mail it should be opened by the Word application
  • IANA registers vnd. prefixed subtypes with less requirements than regular types
  • vendor specific types are often undocumented and may change significantly over time
• Using well-defined types makes handling resources more stable
  • the IANA registry contains hundreds of types (most of them application types)
  • when designing applications dealing with various content types, use media types as the foundation

application/msword Media Type

SECURITY CONSIDERATIONS:
None known.


PUBLISHED SPECIFICATION:

Specification by example:

   From any microsoft word application select "Save As..." from the
   "File" menu.  Enter a filename, make sure that "Normal" is specified
   for the file type, and click "Save".

Company Contact:

   Microsoft Inc.

   16011 NE 36th Way
   Box 97017
   Redmond WA, 98073-9717

Plain Text

• RFC 2046 defines plain text files as a basic media type
  • any text file that does not contains structures which are intended for machine-based processing
  • even Comma-Separated Values (CSV) does not count as plain text
• Guessing of character encoding is hard and unreliable and should be avoided
  • the character encoding can be specified with an additional parameter: text/plain; charset=iso-8859-1
  • if no such parameter is present, ASCII should abe assumed as the character encoding
• For more specific text subtypes, various other subtypes exist
  • calendar for information about calendar entries
  • javascript for JavaScript code (should now be marked as application/javascript)
  • sgml and xml for text with additional markup

HTML

• RFC 2854 registers text/html for HTML documents
  • like Plain Text the character encoding can also be specified as a parameter
  • it is not specific for some version of HTML (version information can be found in the HTML document)
• HTML Fragment Identifiers are also defined by the media type registration
• HTML in many cases needs additional resources to be self-contained
  • images which are references by img elements (maybe external image maps)
  • other media referenced by object or applet (or the deprecated embed)
  • stylesheets or scripts which are referenced in the document head (they may reference other files …)
  • generating a truly self-contained HTML is a rather hard task
• MIME can be used to represent a self-contained MHTML (RFC 2557)

Comma-Separated Values (CSV)

• RFC 4180 defines a textual format for spreadsheet data
• CSV has been used for a long time, but some of the details were solved differently
• Defining a media type makes it easier for implementations to know what to expect
  • the registration not only registers the type, but also defines it
• CSV is not overly complex, but some issues have to be solved
  • how to separate lines (CRLF)
  • how to end the file (CRLF is allowed but optional)
  • are there headers allowed (yes, but they are not marked as such)
  • may different lines use different numbers of fields (no)
  • are spaces significant (yes)
  • are quotes significant (no, they are delimiters, so quotes as values must be escaped)
  • how to treat fields with CRLF, commas, or quotes (enclose the value in quotes)

JSON

• RFC 4627 registers JSON as a media type
• The definition of JSON is derived from ECMAScript's object literals
• JSON is a very limited notation intended for simple structures
  • it allows the four primitive types strings, numbers, booleans, and null
  • it allows the two structured types objects (unordered) and arrays (ordered)
• The value of the media type in this case is the clean integration into the Web
  • information providers may choose to expose their data in JSON and XML
  • Content Negotiation can be used to specify which representation is preferred

Graphic Interchange Format (GIF)

• RFC 2046 registers the oldest graphics format on the Web
• GIF was subject of a long patent debate
  • the compression technique of GIF (LZW) had been patented by Unisys (1983)
  • Unisys wanted to get licensing fees from all commercial online uses of GIF
  • Portable Network Graphics (PNG) was developed as an effort to develop a copyright-free format
  • in 1999, Unisys changed its tactics and wanted to collect one-time fees ($5000-$7500) from all users
  • all GIF-related LZW expired in 2003/2004, so GIF is freely available now
• GIF's poor features make PNG the better choice anyway
  • 8 bit color (requires dithering for photographs), binary transparency
  • GIF's animation feature is the only thing that is not available in PNG …

Joint Photographic Experts Group (JPEG)

• RFC 2046 standardizes the second popular image format for the Web
• JPEG has been specifically designed for photographs
  • it always is lossy (it cannot preserve the complete information from a random bitmap)
  • it uses perception-based compression (for example, color precision is sacrificed for brightness)

Q = 50, filesize 15,138 bytes	Q = 10, filesize 4,787 bytes	Q = 1, filesize 1,523 bytes

Portable Network Graphics (PNG)

• PNG is registered as image/png and is the third major image format
  • PNG was intended to be a royalty- and copyright-free replacement of GIF
  • image formats need to supported by browsers and thus take a long time until they are established
  • IE6 implements PNG in a very rudimentary form, IE7 handles PNG correctly
• PNG has some advantages over GIF and JPEG
  • lossless, compressed palette, grayscale, or true color images
  • 8 bit alpha channel for gradual opacity (blending into the background)
• JPEG still is the preferred format for photographic pictures
• GIF still is the preferred format for animated images
  • MNG and APNG are two available but not widely supported PNG animation formats

Identification of Resource Fragments

• URIs identify a resource (based on a scheme and a scheme-specific part)
  • URIs do not necessarily identify a specific representation of a resource
  • any representation-specific operation needs to look at the resource type
• Fragment identifiers can be used to identify a part of a resource
  • http://dret.net/netdret/publications#wil02h
  • fragments are a client side concept (the HTTP GET requests the complete resource)
  • if the client supports fragment handling, the identifier is interpreted
• Fragment identifiers and content negotiation have a problematic relationship
  • keeping fragment identifiers stable across resource variants may be hard
  • nothing in the fragment identifier associates it with a specific resource variant

HTML Fragment Identifiers

• HTML allows to address named/identified elements in the HTML document
  • the first HTML versions required named <a name="frag-id">incoming anchors</a>
  • newer HTML versions allow <p id="frag-id">every element to have an id</p>
  • browsers support both ways, but the id variant should be preferred
• Only named/identified fragments can be identified
  • 99.99% of all page authors do not routinely add identifications
  • tools may be smarter and take over that task (e.g., Movable Type identifies all relevant elements)
  • for most pages on the Web this means users cannot link to most elements in them
• Keeping fragment identifiers stable should be a goal of Web authors
  • identify the key fragments (and maybe provide a better UI than view source)
  • never change identifiers once they have been assigned

XML Fragment Identifiers

• XPointer is the language which defines XML fragment identifiers
  • XPointer's history and success is even worse than XLink's
  • the interesting part was never finished, the easier parts have been finalized
• XPointer supports HTML's approach of identified elements
  • if there is simply a name after the #, it is a shorthand and treated like in HTML
  • … almost, actually, because IDs in XML are a bit more complicated (xml:id or schema-defined)
• XPointer allows child sequences to navigate the document tree
  • …#element(/1/2) or …#element(intro/3/1)
  • the nice aspect is that elements without an explicit identification can be identified
  • the risky part is that any changes in the document structure may break the XPointer
• XPointer in all its glory (W3C working draft December 2002 …)
  • an extension of XPath inheriting all the interesting features of XPath
  • adds the concept of range locations between to XPath-identified point locations

Know Your Resources

• Handling Web resources and technologies requires a common vocabulary
• Media types are a useful vocabulary for identifying resource types
• Fragment identifiers add some complexity, in particular for resource variants