(4) Server Logs

(5) Connecting to the Server

(6) Log File Display

• Navigating to the log file directory

  cd ~/mobapps-logs

• Apache writes two log files
  • mobapps_access_log is the access log of the server
  • mobapps_error_log is the error log of the server
  • the structure of the entries is defined in the server configuration [http://httpd.apache.org/docs/2.2/logs.html]
• Displaying the lines added to the access log

  tail -f mobapps_access_log

• Displaying and filtering the lines added to the access log

  tail -f mobapps_access_log | grep "/~dret"

(7) MobApps Server Logs

(8) Server Log Analysis

(10) Resource Orientation

• REST [Representational State Transfer (REST)]'s focus on resources is good for caching
  • resource representations are sent in HTTP messages and can be cached
  • HTTP methods allow caching to be used only when appropriate
• HTTP is supposed to supply useful metadata in HTTP messages
  • some HTTP headers are about the client or server agent
  • some HTTP headers are about the entity contained in the message
  • some HTTP headers serve general purposes
• Most applications require a mix of static and dynamic resources
  • some resources never change (code and chrome)
  • some resource classes are constantly growing (your Flickr images)
  • some resources are ephemeral (any representation based on time)

(11) Cache Locations

• HTTP allows caches to exist in various locations
  • reverse proxies are used for removing load from application servers
  • proxy caches are used for making the network more effective
  • client caches are used for making the client work faster
• Caching in many cases is not (easily) predictable
  • caching depends on the scenario but always needs HTTP header information
  • being a well-behaving HTTP citizen is a good idea
  • caches should not change application behavior (it is just optimization)
• Client Caches can be provided by platforms or applications
  • many desktop browsers have their own HTTP implementations and caches
  • mobile (and modern) platforms should move HTTP (and caching) into the platform

(12) Caching in the Platform

(13) HTTP Metadata

GET /4026/4519445803_06922b9171_b.jpg HTTP/1.1
Host: farm5.static.flickr.com
User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.2.3) Gecko/20100401 Firefox/3.6.3 (.NET CLR 3.5.30729)
Accept: image/png,image/*;q=0.8,*/*;q=0.5
Accept-Language: en-us,en;q=0.7,de-de;q=0.3
Accept-Encoding: gzip,deflate
Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7
Keep-Alive: 115
Connection: keep-alive
Referer: http://www.flickr.com/photos/dret/4519445803/sizes/l/

HTTP/1.1 200 OK
Date: Fri, 16 Apr 2010 17:02:20 GMT
Content-Type: image/jpeg
Connection: keep-alive
Server: Apache/2.0.52 (Red Hat)
Cache-Control: max-age=315360000
Expires: Mon, 28 Jul 2014 23:30:00 GMT
Last-Modified: Wed, 14 Apr 2010 04:57:32 GMT
Accept-Ranges: bytes
Content-Length: 361084
X-Cache: MISS from photocache510.flickr.gq1.yahoo.com
X-Cache-Lookup: MISS from photocache510.flickr.gq1.yahoo.com:81
Via: 1.1 photocache510.flickr.gq1.yahoo.com:81 (squid/2.7.STABLE6)

(14) Caching in the Application

(15) Caching in the Cloud

(16) Cache-Friendly Services

• Do not use aliases (different URIs for the same resource)
• Reuse content whenever possible (no copies of the same resource)
  • reuse has to be Web-friendly (symlinks save disk space but are not cache-friendly)
• Serve static content with long expiration times
• Serve dynamic content with synchronized expiration times
  • even dynamic content often does not expire immediately
• Avoid touching all files on your hard disk
  • this will update all timestamps and make cached copies invalid
• Only use HTTPS when necessary
  • HTTPS is end-to-end encrypted and bypasses some caches

(18) Caching Conformance

• Caching is optional and HTTP implementations may not cache at all
  • caching eliminates some requests entirely
  • caching allows responses to be more effective (304 Not Modified)
• Implementations that support caching must follow the rules
  • any cached local resource must always use the latest version
  • if HTTP metadata expires that version it should not be used
  • servers can mark resources to disallow the use of stale versions
• HTTP allows explicit and heuristic expiration
  • not all resources are explicitly marked with expiration dates
  • caches are allowed to guess expiration dates (e.g., using Last-Modified)

(19) Mobile Safari Caching

• Mobile Safari does a bad job at caching [http://www.phpied.com/iphone-caching/]
  • caching behavior is not documented and cannot be changed
  • the mobile browsing experience is substantially degraded
• It seems that caching is based on the platform's minimal RAM [Constraints of Mobile Applications]
  • nothing bigger than 15KB is cached (some sources say the limit is 25KB)
  • total cache size is 1.5MB (very likely caused by RAM-only caching)
  • there seems to be no persistent cache (no storage outside of RAM)
• AppCache [Offline-Capable Applications; Caching Application Code (1)] can be used to improve caching
  • risky strategy because it may have unintended side-effects
  • inconvenient because everything must be packaged as an application
  • incomplete because this only works for HTML5 implementations (no proxy support)
  • clients may start treating AppCache differently if it becomes (too) widely-used

(20) Native Application Caching

• Caching is important for almost any network activity
  • Web-based applications by definition primarily use HTTP communications
  • native applications in most cases also use HTTP communications
  • instead of implementing HTTP they typically use HTTP libraries
• NSURLRequest [http://developer.apple.com/iphone/library/documentation/Cocoa/Reference/Foundation/Classes/NSURLRequest_Class/Reference/Reference.html] is the iPhone OS API for requesting URIs
  • NSURLRequestCachePolicy [http://developer.apple.com/iphone/library/documentation/Cocoa/Reference/Foundation/Classes/NSURLRequest_Class/Reference/Reference.html#//apple_ref/doc/uid/20001695-DontLinkElementID_1] allows to set the cache policy for a request
  • HTTP headers are set based on the selected request policy
• How is the caching architecture of the iPhone library?
  • caching is only done in RAM (i.e., there is no persistent cache)
  • caching is done on a per-process base (i.e., there is no shared cache)
• Mobile platforms seem to not support HTTP very well
  • caching has to be implemented in the application (Flickr app [Caching in the Application (1)])
  • it could be made reusable (Facebook app and three20 [http://github.com/facebook/three20/])

(22) More Metadata is Better

• HTTP supports a comprehensive set of metadata fields
  • more metadata can be used to support better caching
• Tuning caching is important but non-trivial and application-specific
  • requires knowledge about what can/will change and when
  • HTTP servers should automate/support as much as possible
• HTTP servers support more advanced interactions with clients
  • clients can advertise compression support with Accept-Encoding

(23) Freshness of Responses

• Expires tells a cache how long data is fresh
  • defined by HTTP/1.0 and easy to use and support
  • caches will typically simply reuse data before the expiration date
  • caches have to check for changes after the expiration date
• Cache-Control supports a more nuanced caching model
  • introduced in HTTP/1.1 to address Expires limitations
  • expiration dates can now be made relative
  • support for caching and control of authenticated content
• Caching support should set the right headers
  • Apache supports caching and uses both headers
  • caching only works when validators are present

(24) Configuration Steps

• Configure the server to support setting expiration dates

  LoadModule expires_module modules/mod_expires.so

  • httpd.conf (line 97)
• Configure the setting of dates for a specific directory

  ExpiresActive On

  • ./src/caching/.htaccess (line 1-1)
• Configure how dates should be set for certain types

  # expire GIF images after a week in the client's cache
  ExpiresByType image/gif "now plus 1 week"
  
  # expire JPEG images after they are a week old
  ExpiresByType image/jpeg "modification plus 1 week"
  
  # expire PNG images after they are a month old
  ExpiresByType image/png "modification plus 1 month"

  • ./src/caching/.htaccess (line 3-10)
• Content will now be served with the appropriate headers [src/caching/]
• Apache's mod_expires [http://httpd.apache.org/docs/2.2/mod/mod_expires.html] supports a wide variety of settings

(25) Entity Tags (Etags)

• Timestamps are weak identifiers for caching
  • timestamps may change even if nothing changed
• ETag is a server-assigned identifier for a response entity
  • caches store ETags alongside the actual representation
  • when validating content clients/caches include the ETag
• What is the relationship between URIs and ETags?
  • URIs identify abstract resources that can be accessed via the Web
  • ETags identify a concrete version of such a resource
  • ETags change when something relevant happens to the resource

(26) The Problem with ETags

• ETags are meant to be unique tags for one version
  • CMS-based servers can use IDs generated from the CMS
  • file-based servers have to generate unique identifiers
• File-based content often uses rule-based ETags
  • Apache uses inode-size-timestamp as the value of the ETag
  • such an ETag depends on the actual file system (not just the file itself)

    FileETag INode MTime Size

• Server farms often use load balancers and replicated files
  • file replication will produce different inode values
  • ETag-based caching in such a scenario is server-specific
  • the files are equal but not identical
• ETags should be configured as correctly as possible

  FileETag MTime Size