Reading Notes: Principles and Patterns of Organizing Systems Spring 2011

The Glushko chapter 1 of the IFIOIR book says that IO and IR are now so intertwined as the conceptual and practical foundations of our 21st century culture and economy that it makes little sense to us to force an artificial boundary between them. Do you agree with this claim? Are there better arguments to be made for it?

The core argument of the Glushko chapter is in section 1.1.4, From Collection Categories to Organizing Systems. The proposal is to replace or complement the traditional mutually exclusive category view of organizing systems with a dimensional view that describes collection types as configurations of choices in a complex multidimensional space of design decisions. What are the arguments for this new perspective? Are there better arguments?

The Chapter introduces many of these dimensions in 5 coarse groups: what, why, when, how much, and by whom or what process the organizing system is defined. But the dimensions don't fit cleanly into these 5 groups and no claims are made that these are the right or only dimensions. Are there better ways to arrange or prioritize them?

Pautasso and Wilde's analysis of the architectural concept of loose coupling has the goal of replacing a one-dimensional and vaguely defined distinction with a much richer framework of 12 facets. How is this goal analogous to the goals of the Glushko paper? P & W explicitly reject the use of dimensions in their framework to emphasize that the facets are not completely independent. Glushko rejected the use of facets in Chapter 1 because that word is overloaded in information science in contexts like faceted classification. What are dimensions and facets anyway? Are there better terms we should use to avoid confusion? (aspects, issues, ?)

Regardless of what we call them, can we relate the design dimensions/facets of organizing systems with those for the architecture of coupling? Are any of them equivalent or raising related concerns?

Can we distinguish technology or domain-independent dimensions/facets from those that depend on the technology or domain?

The Bray blog post proposes three facets for comparing communication methods. We assigned this reading mostly to show you another example of a visualization for comparing faceted descriptions, but are there any other useful parallels in that blog post? Are any of Bray's facets important in descriptions of organizing systems?

We are starting with domains — libraries, museums, archives — that are the traditional ones,because presumably that will make it easier to come to consensus about the right dimensions/facets/design issues that best distinguish organizing systems. Here is some guidance and some of my thoughts about the readings:

Emerging Convergence

This paper was in the Fall 2010 202 syllabus, so some of you might have read it already, but even if you did please revisit it because you'll surely get more out of this time. Focus on the first few pages that compare and contrast different types of OSs, and on the issues raised in effective digitization because I think the relationship between originals and digital surrogates might be a key dimension for us. You don't need to go into the stuff after page 376 about training professionals.

Playing Catch up in the Digital Library Race

Think about the scale of the collections and whether you can find a common technical infrastructure that yields interoperability for the scholar, the casual inquirer or the K-12 student. What does this somewhat mangled phrase mean anyway?

Kindle Lending

Make sure you understand what Amazon designed this feature to do so you can understand how the Kindle Lending Club has built upon it. I included some of the comments on the story in the assigned reading because it seems pretty controversial and seems to be a little Napsterization w.r.t. Amazon. I don't have a Kindle — if one of you does, perhaps you can try this to see how it works and how long it takes to get a bite on your offer to lend a book.

Tool Lending

Think about what does and what doesn't fit into the category of tools for the various libraries mentioned in the article. What other kinds of tool libraries might you personally use? This library won't be replaced by a digital library anytime soon because thingness as opposed to digitalness is a pretty important design dimension (We will revisit this issue later when we look at the distinction of information resources vs. non-information resources on the Web). The story says tool lending a great fit for the Bay Area — but I don't understand why.

Seed Library

A very straightforward article that I mentioned in Chapter 1 of IFIOIR, but more nuance than you might at first realize. Returning something other than what you borrowed violates a defining principle of a library, but make you detect the interesting curatorial implication there.

Software Component Library

A little technical and nerdy, but there's a good discussion of different approaches for managing reusable software components and the limitations of simple keyword models of IR. Note how the creation of metadata for each component is intrinsically part of creating the component and how the library functions are tightly coupled into the development environment.

Abu Dhabi Museums

A fascinating discussion about a universal museum that raises questions of cultural parity, imperialism, and authenticity about museums and their role in national identity. Can you recognize what's going on in Abu Dhabi in the description of museum in the Trant Emerging Convergence article?

Crowdsourced Science

Are the efforts here ploys to stimulate public interest rather than advance scientific knowledge or are they clever ways to leverage Shirky's cognitive surplus of online brainpower? How is the traditional role of curators changed? Think about what does and what doesn't fit into the category of science for the various projects mentioned in the article. How do we distinguish them from other organizing systems that use social classification like Flickr and Yelp?

We are now analyzing a more diverse set of organizing system domains with the goal of determining whether the dimensions that characterize the traditional domains still apply.

Flea Markets

Before you read the article, try to come up with a 2-dimensional analysis of the retail sector in which flea markets typify one of the quadrants. Then, given that this article comes from a journal of Marxist thought and analysis, not surprisingly there is an analysis of the different types of sellers in flea markets that highlights some economic contradictions. Can we see economic dimensions when we contrast public libraries, Google books, and the Kindle? If you have a smart phone, think about the organizing system used in the App Market and consider how that market compares to flea markets. Do the same for eBay.

At the Zoo

The title does an excellent job summarizing the paper, which argues that zoos have a distinctive niche as an organizing system domain because they have multiple purposes and goals. But their intended users seem unable or unwilling to appreciate this institutional complexity. Can you clearly differentiate zoos from museums, theme parks, circuses, and wildlife conservation centers?

Museum Content and Community in Second Life

First, if you don't know what the Exploratorium is (an interactive science museum and a lot more) visit it on the web at exploratorium.edu (and note edu here) or better yet visit the actual Exploratorium in San Francisco. The paper describes various efforts to do interactive museum exhibits in Second Life, a multi-user 3D virtual world platform. The paper's main goal is to explain the sorts of things you can do in Second Life that you can't do in a physical museum. It also describes a range of possible relationships between virtual worlds and physical worlds. See if you can characterize or categorize different regions on this augmented reality continuum.

Wiki Genres

We have all used Wikipedia and most of us have used wikis in group projects, but we probably haven't analyzed them as instances of organizing systems or classify them as Poole and Grudin do. The paper discusses several factors that shape the extent and use of wikis, many of which concern the who does the work and who gets the benefit trade-off that we are familiar with from 202. But there are several other issues of governance and provenance that were new to me so make sure you spot them.

Interoperable Data and Articles

When publishers first adopted digital submission of articles, CD-ROM and online access and storage, many aspects of scholarly research and communicated were greatly improved, but the nature of the scientific article and journal per se changed relatively little. But more recently Web-native publication models have emerged that embrace more modular and heavily hyperlinked architectures, creating networks of articles that blur the traditional notions of articles and journals and that interconnect scientific findings with their underlying research data or to discipline-specific data repositories. There are several different kinds of linking between articles and data going on in this article… make sure you understand them all.

The essence of Service Oriented Architecture is that a business model is a loosely-coupled composition or choreography of services. These business components can be a mix of core, internal ones that a business does itself and outsourced ones provided by other businesses or service providers. Some services are exposed as APIs or data feeds, which by themselves might be too granular to be thought of as business componentsbut in combination they can create business value. APIs and data feeds impose organization but because they also offer other organizing systems the chance to use them in new ways they can be thought of as implementing latent organization. Is this just the at the last minuteend point on the early vs. late bindingdimension in the Pautasso and Wilde paper?

Document Engineering

This short reading describes the Drop Shipment design pattern, which is the most common business model on the Internet. Only the very largest and very smallest online stores actually own the inventory that they sell. But if all drop-shipment retailers are implementing this design pattern for their organizing system, how do they differentiate themselves? There are some important sub-patterns or variants of drop shipment in different industries. If industries where products are highly configurable with components that suffer from rapid obsolescence, rather than ship finished goods from inventory a retailer might want to postpone final assembly until the customer places the order. How would you adapt drop shipment in this case? What industry sectors do this?

Flexible Value Structures in Banking

Another short reading that describes how the financial services industry is adopting service architectures to enable efficient specialization of firms. The two figures nicely illustrate how you can think of a firm as a collection of modular services that can be organized in different combinations to implement the different business models. Make sure you understand these rather cryptic sentences: Coordination between modules is primarily via non-hierarchical mechanisms of coordination (p. 34); …value networks run at lower cost by providing flexibility to connecting applications and reducing interface-negotiation efforts between collaborating parties (p. 35); Using and exchanging such a [capability or module] map facilitates collaborating parties gaining a common understanding about banking modules (p. 36).

Japanese farms look to the cloud

Read this carefully, because there is an amazing amount of rich detail crammed into a short article. Imagine what this farm looks like - it has 72 workers and 60 different crops spread over 100 hectares, an area slightly larger than the Berkeley central campus. Imagine you are Mr. Shinpuku sitting in his office looking at his web browser. What are the data sources that are combined in the cloud and mashed up in the Fujitsu farming application? How will this application help bring the concepts of lean manufacturing and continual improvement to farming? Why is it essential that Japanese farms adopt sophisticated information systems like this one? Does it seem like a good investment for Shinpuku?

Data Integration In Mashups

This paper was a nice find for us because it very tightly introduces the idea that both data and services can be ingredients that are mixed-and-matched to create new applications. The first paper contrasts data level, process level, and presentation level mashup architectures and the most important technical challenges posed at each level. The heart of the paper introduces a multi-dimensional framework for comparing tools for creating mashups, and here you will see some of the issues raised in the Pautasso and Wilde paper we read a few weeks ago but with the new focus on the capability of the mashup tools for dealing with those issues. The data refresh dimension, push vs. pull, frames some design choices that I don't think we've seen in previous readings. Before I read this paper I thought that having an API might be a useful dimension to contrast organizing systems but that is probably too coarse a characterization because of all the additional design choices for APIs that this paper discusses.

Digital Libraries and Data-Intensive Computing

This paper discusses the important domain of scientific data collections that is barely mentioned in 202. Section II describes the characteristic properties of such collections, the most important of which is probably their huge scale. Section III discusses the capabilities of data intensive computing environments that this scale and typical uses of scientific data mandate. A lot of us have experience with business and commercial computing, and it is worth thinking about how the typical patterns of information retrieval and processing in those domains differ from the patterns in scientific computing.

Data Curation + Process Curation

If the title makes sense to you please explain it to me because I don't quite get it. But the paper makes important points about the need to manage and organize processes (and services) as well as data. It reinforces the analysis in the Data Integration in Mashups paper that APIs need to be carefully designed and documented to be useful. The most important section of this paper is What Metadata to Curate, and please take a close look at Figure 2, the Curation Continuum that nicely captures the classic trade-off between effort and value.

From the user perspective, things on the Web just work, and do so across a surprisingly wide array of applications and scenarios. This is the first time a truly decentralized hypermedia system has ever succeeded, regardless of the efforts of decades of hypermedia research and enterprise IT integration efforts. In this first set of readings, we explore some of the facets that make this possible. This is the set of introductory readings leading up to the more advanced REST-oriented readings of next week.

Seven Web Ways

I have to admit that I am torn about this. It puts some things in nicely digestible ways, but gets other things plain wrong. For example, Pass by reference rather than by value is of course a good idea when sharing published data, but as we will see next week, the ST in REST stands for state transfer and means the exact opposite: if you need to make changes to things, pass the changed things. So as much as I like this as a popularized attempt to explain how the Web works, it misses some of the finer details we will have to get right to understand Web Architecture as a useful pattern for building organizing systems. To make the most out of this reading, find at least one example where some supposedly Web-oriented application or service got one of the seven Web ways wrong, and remember how you noticed and then try to think of how this could be improved.

Links in HTML

HTML is a language that is quite a bit richer than most people think. In this blog post, I simply went through the exercise of identifying all ways in which HTML can link to related resources. Many of these declarative links are not used very often; find at least five that you think you have never encountered on the Web, and try to explain why. Many declarative links in HTML are now replaced by programmatic links embedded in scripting. HTML5 almost exclusively focuses on enhancing the scripting capabilities of browsers, and mostly ignores the declarative approach of HTML. Can you think of some of the things nowadays done with scripting (or in the near future with HTML5) that could be done declaratively? How would HTML need to change for that? Would there be an advantage over scripting, and if so, what would it be?

Web Linking

Linking is what makes the Web the Web. Some media types are good at linking (HTML of course comes to mind), some are not. Many multimedia formats are bad at linking or simply have no linking at all (think GIF, PNG, JPEG, MP3). With RFC 5988, anything can be linked, because the links can now be carried outside of the actual media type, in an HTTP header. Starting from the observation of some currently link-poor media types, come up with three ideas where you thought additional context (links = context) would have provided a more useful Web experience. And how would you expect the UI to look like?

Media and Resource Types

Without being too REST-centric, just use this as a starting point to think about media types (such as HTML) and resource types (such as my Flickr pages) in the context of your behavior as a human Web user. How much of your Web consumption is (also) based on expecting certain types of resources and not just HTML, often meaning that you're getting upset when a Web page is improved and you have to re-learn how it works. For humans, this is just annoying and hopefully relearning only means to look at how the layout changed, and try to adjust to the new structure. For machine clients, this often breaks everything and the clients have to be changed manually (there is very rich research literature on how clients can be made a bit more robust, which of course is particularly relevant for crawlers). Try to think of how much of your Web consumption could be automated just because of HTML, and how much of it depends on a more nuanced understanding of patterns in the HTML. Have you encountered Web automators? How are they working, and how well are they working?

The Role of Hypermedia

This is by far the most difficult reading for this week, and the main goal is to give it a first read and try to extract a little bit of the emphasis it puts on hypermedia as one of the core principles of REST. We will re-read this paper later, when we have looked at REST in greater detail. Try to look at it from a real-world perspective: When you enter a store, you often don't know what you will encounter, and there is no upfront directory of all the services you might need. Instead, you discover those as you go along, with the information desk linking (pointing) you to the right floor, the salesperson linking you to the closet cash register, and the cashier linking you to the gift wrapping station. This service system can fluidly reconfigure without the need for any centralized registry to be updated, and as long as you can go back, you can even deal yourself with local failures (this cash register is actually closed, where is the next one, please?). Try to come up with an example where this linking analogy fails, and where a centralized registry (or upfront knowledge of all service components) would be required to make things work.

Today's reading give a simple and high-level understanding of what REST is. In both readings, the focus in on the fundamental simplicity of the way how REST models systems: universal interactions and self-describing representations of the things that matter. We will get into more detail in the three follow-up lectures about the main REST constraints, so this serves just as a light introduction to the topic.

How I Explained REST to My Wife

I like how this explanation starts from many observations of human language. I am still convinced that it would be a very interesting exercise to compare human language(s) and REST: Many verbs in human language are universal, and it seems to me that the human mind even has a tendency to universally apply verbs (I have heard people using googling as a general verb for searching). My hypothesis would be that the richness and versatility and extensibility of human language to a large extend is rooted in relatively few verbs that can be applied to a much bigger set of nouns, and that additions to the set of verbs in fact happen much less often than additions to the set of nouns, simply because the set of things you can do grows slower than the set of things you can do something with. I have no idea whether there is any credibility to any of this, but I do like the idea that there may be.

Roots of the REST/SOAP Debate

This one is much more technical in its discussion and probably better suited for those with a technical (and maybe even middleware-oriented) background. The discussions around SOAP vs. REST (which is a very bad way of framing it, but that's the main label people use) have been going for a long time now, and this is a very early contribution to this debate. It has become clear now that (a) REST is a different and valuable approach of designing organizing systems, and that (b) REST comes with a set of constraints and trade-offs that are not a good fit for every organizing system. But that's what's a core of every design problem: understanding the constraints and requirements of the problem, and then using a design that is a good (ideally, the optimal) solution for that problem.

These two papers highlight very different schools of thought when it comes to URIs. The first one simply looks at the practical utility of having stable identifiers for whatever you want to talk about, whereas the second one is based on a lot of assumptions what the things are you want to talk about.

Cool URIs don't change

This classical paper very simply makes the point that URIs should be stable, that it would be friendly to make them legible (but that is not really something you have to do), and to maybe even smart enough and avoid coupling between the URI, and the served media type. There's not really anything in there that is a surprise, but in the light of recent discussions such as the (still ongoing) hashbang URI discussion, it still is interesting reading.

Cool URIs for the Semantic Web

One of the central premises of the Semantic Web is that anything can be identified by an HTTP URI (not just any URI, but an HTTP URI, which is an important difference), and that by GETting that URI, you should be able to retrieve an RDF description of what has been identified. This design transfers the (identified by various schemes, accessible through a uniform interface identified by the URI scheme, using self-describing representations through the uniform interface) design space left open by REST to a narrow interpretation of (always identified by HTTP URI, the uniform interface is get a description , use RDF for everything). One interesting consequence of this specific choice in the design spectrum left open by REST is that it becomes hard to actually talk about things (as opposed to descriptions of them), because every identified thing is supposed to be an RDF description. To solve that problem, it has been decided that there could be either URI patterns or HTTP status codes signifying that a URI is either identifying a thing, or a description.

Representation and hyperlinking are deeply intertwined: Representations (often) have internal structure, and hyperlinks are the external structure between resources. One of the tough design challenges in REST is the choice of granularity: What to make internal structure of the representation (in which case the structural relationship is not accessible for RESTful interactions), and what to turn into links (in which case aggregating fine-granular resources can become cumbersome and inefficient). This weeks readings look at both of these problems.

Still Building the Memex

Don't be tempted to follow the 40 links to the research projects and products that are listed in this paper. This paper is a great overview of how differently the general Memex problem of organizing information and making it accessible can be approached. Try to spot the many projects and products that have hardcoded centralized components in them. Sometimes it's simply a design flaw (or maybe an accepted limitation), but sometimes there is an organizational reason for that, such as a centralized authority or similar issues where decentralization would introduce a whole new layer of complexity by making it necessary to handle provenance, trust, authority, and scenarios where adversarial parties try to game the system in their interest (a.k.a. spam).

Hypermedia Types

Only read up to (and including) section 1.4 of this paper. This paper tried to make the leap from hypertext just as a narrative structure to present human-readable information, to a more generalized view of hypertext as a data type that allows applications to purposefully interact with exposed data. This is mostly an analytical exercise in the sense that there is a difference in the kind of interactions that links are used for, and there is a difference in how existing media types make use of these interaction types. Think about your interactions from A3: where do they fit in? Does the framework presented here even apply, or if not, why do you think it is not adequate?

REST and RDF Granularity

This is a short piece about a very fundamental difference between two popular Web-oriented methodologies. REST incorporates interaction at the very core of its model and fragments the universe into resources that are interlinked. Links can be used by interacting with the linked resource according to the link semantics, and the identification scheme. RDF, on the other hand, abstracts from that and attempts to build something that sometimes also is referred to as the Giant Global Graph. In this model, all data is just there, and then applications can freely navigate this graph. It is not a coincidence that RDF and Semantic Web approaches flourish mostly where there are few big centralized players, such as in government data.

The topic of that day is the Semantic Web and more specifically Linked Data as an alternative approach for how to come up with an architecture and design patterns for organizing systems. The readings assume that you have some basic familiarity with Semantic Web technologies (RDF, schema languages and ontologies such as RDFS and OWL) from 202; if you cannot remember anything of that, you may first want to go back to the 202 readings before looking at today's PPOS readings.

Linked Data

The idea of the Semantic Web has been around for a while and is based on the idea that all data should use the same metamodel (RDF), and that all identified concepts and things are identified by URIs. The basic approach of the Semantic Web has been discussed in 202 already. In the last years, the Semantic Web term has been increasingly replaced by the Linked Data term, which in a nutshell refers to a set of best practices layered on top of the original Semantic Web idea. Because pure RDF uses URIs exclusively for identification, it is necessary to come up with additional rules (such as the ones described in this note) to ensure that the URIs are not just identifiers, but also allow interaction. The main motivation behind this design pattern is to make it easy to use RDF to find more RDF, so it's essentially a design pattern that should allow the Semantic Web to better support the vision of the Giant Global Graph (all RDF on the Semantic Web seamlessly connected so that it can be traversed with RDF-only technologies).

Describing Linked Datasets with the VoID Vocabulary

In reality, the current landscape of Linked Data converts individual datasets of interesting size (such as Wikipedia) into RDF by implementing customized mappings. The resulting RDF then is stored, and may have linkages to RDF data that is stored in another one of those major Linked Data hubs. VoID is an attempt to make the dependencies between those datasets explicit. One could imagine that an RDF-supporting toolset could become VoiD-aware and would start retrieving RDF from the various hubs, thus implementing a system that would have a data management layer underlying the actual data layer. VoiD is still young, but the growing set of available Linked Data datasets has made it clear that there has to be a way of making dependencies between datasets explicit, so that the Giant Global Graph view can be maintained layered on top of the interlinked datasets.

Linked Data and Service Orientation

This paper contrasts the approaches of REST and the Semantic Web (or Linked Data). The trade-offs in that space are interesting and at the core of what we are constantly discussing in our seminar: Almost any organizing system architecture can be used to implement almost organizing system; the question often is how to create a design that is effective, efficient, and has additional desirable properties such as openness, flexibility, decentralization, and openness. When reading this paper, keep your scenario from the assignments in mind and try to analyze the different system you would come up with if you used REST or Semantic Web technologies. Is one implementation clearly more desirable than the other? And if so, what is the most important aspect that made you think that there is a big difference in how well the architecture works for your scenario?

REST and Linked Data: A Match Made for Domain Driven Development?

REST and Linked Data have a number of similarities, for example in the areas of identification and linking. However, the communities working in these two areas are surprisingly disjoint, and the question is why this is the case, and what could be improved if the communities were more working together. This paper argues that both sides can learn from each other, with REST having the richer model to think of APIs and interaction with data, and with Linked Data having an established and semantically rich data model that can be used to satisfy the self-describing message constraint in REST. Combining these two areas indeed may be the best path to a Web of Data that is rich both in interaction and in representation, but it remains to be seen how the communities develop.

The readings this week are at times tough to read because they assume a lot of domain knowledge about libraries, but skip through that if it slows you down.

The Strongest Link: Libraries and Linked Data

The first few pages are pretty straightforward about the benefits of semantic encoding of bibliographic data and other traditional library information sources. There is a growing awareness that the web and search engines have radically changed how people use and think about libraries and that libraries need to do a better job integrating themselves into the virtual work spaces and activities of their users. This is challenging for many reasons. The primary technical barriers relate to the vast amount of legacy bibliographic information in card-oriented formats created according to cataloguing rules designed when users needed rich description to decide whether it was worth paying the high transaction costs of obtaining the resource described. Rich description is far less necessary when you can follow the link from the description to the thing and quickly decide its relevance, but converting the rich human-oriented data to lean computer-oriented data isn't easy. But equally challenging barriers come from the bias that libraries have always had toward resources that they could completely control, making them averse to interconnecting their bibliographic resources to the open and uncontrolled web. Graze through the Appendix and follow some of the links to tools or projects.

Resource Description and Access (RDA): Cataloging Rules for the 20^th Century

When I ran across this article at first I thought it would be a puff piece about the benefits of the proposed RDA cataloging rules that will support the use of semantic web (RDF) encoding of the information in traditional library records to make that information more technology-independent. But closer reading (and realizing that it was written in 2007 – look at the title) revealed that it is highly critical of RDA because it is supposedly too tied to the legacy data formats and cataloguing rules. I like this paper because it has a nice discussion of changes in the information resources managed by libraries, changes in library catalog technology, and changes in user activities and expectations that together argue for more radical changes in metadata design and publishing practices.

Principles of E-Government Architecture

This 2003 paper might seem a little dated, but we've included it is very easy to read and because its focus is on architectures that need to last a very long time. Government IT is likely to be extremely heterogeneous with lots of legacy software and data and this paper proposes a strategy for the government services organizing system that works in that challenging context. It is (thankfully) a little thin on technical details about the web services implementation of this organizing system (because they would be out of date) so focus on the principles. The paper is making an argument for a publish and subscribe or event driven architecture that has some aspects of centralized control while allowing a lot of flexibility in how services are delivered at the endpoints (incremental automation and different user interfaces for existing services, easy new service composition etc).

Restful Web Services vs. Big Web Services: Making the Right Architectural Decision

This is a very dense paper but we've talked about REST a lot so I think you can get through it. It tries to take a fair and balanced look at the debate between REST and big web services. Section 2 SOAP and the WS-* Stack describe the concepts and technology implied in the Principles of E-Government Architecture paper, so go back to that paper and make sure you can figure out where they go in that proposal. The technology comparison in Section 7 is the heart of the paper, and after reading this and reflecting on the Section 9 Conclusion section you should understand that REST works best for simple and ad hoc integration scenarios and WS-* is needed if you need to satisfy the quality of service requirements typical of enterprise or inter-enterprise scale service systems.