API Strategy and Design for Geospatial Data and Services

GEOINT 2017

Erik Wilde , API Academy, CA Technologies
June 5, 2017

Creative Commons License [http://creativecommons.org/licenses/by/3.0/]

This work is licensed under a CC
Attribution 3.0 Unported License
[http://creativecommons.org/licenses/by/3.0/]

Contents Erik Wilde : API Strategy and Design for Geospatial Data and Services

Contents

Erik Wilde : API Strategy and Design for Geospatial Data and Services

(2) Summary

APIs play a central role when it comes to collecting, managing, providing, and using geospatial information. In this training, we look at general aspects of API strategy, design, and management. For each of those aspects, we specifically look at the additional challenges that geospatial data introduces, and how they can be addressed by using established design patterns and technologies. APIs play a central role when it comes to using geospatial data and services for sense-making and intelligence. The reason for this is that the complexity and openness of geospatial information means that any framework for collection, management, and analytics has to be open by design. This openness must span both the ability to add new producers and consumers on demand, and it must also cover the ability to introduce new data and services without the need to re-engineer or re-factor existing data and services. This aspect of an open and extensible ecosystem of geospatial services will also be covered in this training.



Introduction

Outline (Introduction)

  1. Introduction [3]
  2. Introduction to APIs [6]
    1. APIs from 50,000 feet [3]
    2. Three Drivers of the API Economy [3]
  3. API Design [13]
    1. Messages vs. Models [3]
    2. API Scope [3]
    3. Developer Experience (DX) [4]
    4. Diversity, Decentralization, and Loose Coupling [3]
  4. Geospatial APIs [14]
    1. Geospatial API Issues [5]
    2. What is GeoJSON? [5]
    3. IETF GeoJSON Working Group [4]
  5. Conclusions [1]
Introduction Erik Wilde : API Strategy and Design for Geospatial Data and Services

(4) About Me



Introduction Erik Wilde : API Strategy and Design for Geospatial Data and Services

(5) API Academy [http://www.apiacademy.co/]

API Academy

Introduction Erik Wilde : API Strategy and Design for Geospatial Data and Services

(6) Microservice Architecture

Microservice Architecture: Aligning Principles, Practices, and Culture

Introduction to APIs

Outline (Introduction to APIs)

  1. Introduction [3]
  2. Introduction to APIs [6]
    1. APIs from 50,000 feet [3]
    2. Three Drivers of the API Economy [3]
  3. API Design [13]
    1. Messages vs. Models [3]
    2. API Scope [3]
    3. Developer Experience (DX) [4]
    4. Diversity, Decentralization, and Loose Coupling [3]
  4. Geospatial APIs [14]
    1. Geospatial API Issues [5]
    2. What is GeoJSON? [5]
    3. IETF GeoJSON Working Group [4]
  5. Conclusions [1]

APIs from 50,000 feet

APIs from 50,000 feet Erik Wilde : API Strategy and Design for Geospatial Data and Services

(9) APIs are Languages

  • APIs in their current sense evolved from the original API
    • Local APIs originated as a way to build systems (i.e., one big thing)
    • Network APIs were invented decades ago but nobody found the silver bullet
    • Web APIs are a loose and evolving collection of principles and patterns
  • Languages become necessary when coordination is required
    • Human language is a result of improving ecological fitness
    • Human written language is a result of new organizational challenges
  • Same problems/approaches/answers like other language problems
    • Domain-only: Domains figure out their problems and their languages
    • Universal: Find the universal grammar and use it everywhere
    • Hybrid: Use principles and patterns but avoid überformats


APIs from 50,000 feet Erik Wilde : API Strategy and Design for Geospatial Data and Services

(10) Two Views of API Scope/Focus

  • Internal: Using APIs to break up monolithic systems
    • Focus on systems/software engineering and better modularization
    • Turn a system into an ecosystem.
  • External: Using APIs to build up ecosystems
    • Focus on distributed/decentralized systems and better connectivity
    • Enable an ecosystem by loosely coupling systems.
  • Both views are complementary and mostly a difference in perspective


APIs from 50,000 feet Erik Wilde : API Strategy and Design for Geospatial Data and Services

(11) One Perception of API Value

  • System/ecosystem boundaries are blurry
    • System/ecosystem perspective is a question of perspective
  • APIs provide a foundation for communications in an ecosystem
    • Internal: Value is in system changeability
    • External: Value is in system agility
  • API discussions should always be aware of both perspectives
    • Separate internal/external API strategies are counter-productive


Three Drivers of the API Economy

Three Drivers of the API Economy Erik Wilde : API Strategy and Design for Geospatial Data and Services

(13) Driver #1: Push

  • Many environments change at increasing velocities
    • Economics change
    • Incentives change
    • Technologies change
  • Organizations are pushed to better handle change
    • They have to become better at reacting to external changes
    • They have to become better at executing internal changes
    • They have to become better at embracing changing


Three Drivers of the API Economy Erik Wilde : API Strategy and Design for Geospatial Data and Services

(14) Driver #2: Pull

  • External: Participating in the API economy becomes increasingly beneficial
    • Interfacing with customers/partners becomes easier
    • The number of API-ready customers/partners keeps increasing
    • The network effect [https://en.wikipedia.org/wiki/Metcalfe%27s_law] guarantees for the trend to continue
  • Internal: Being able to plan and execute changing quickly improves fitness
    • Traditionally, IT is perceived as sluggish and heavyweight
    • Increasingly, IT becomes the lube for the organizational machinery
    • Additionally, IT can allow completely new classes of business models


Three Drivers of the API Economy Erik Wilde : API Strategy and Design for Geospatial Data and Services

(15) Driver #3: Follow

  • Having an API strategy becomes increasingly important
    • Organizations have executed on this for a long time (Amazon API memo: 2002)
    • Organizations share experiences and tooling around API strategies
  • Using APIs becomes less of an if and more of a how
    • Business pushes IT to make sure strategy can be executed quickly
    • IT can leverage others as justification and leaders


API Design

Outline (API Design)

  1. Introduction [3]
  2. Introduction to APIs [6]
    1. APIs from 50,000 feet [3]
    2. Three Drivers of the API Economy [3]
  3. API Design [13]
    1. Messages vs. Models [3]
    2. API Scope [3]
    3. Developer Experience (DX) [4]
    4. Diversity, Decentralization, and Loose Coupling [3]
  4. Geospatial APIs [14]
    1. Geospatial API Issues [5]
    2. What is GeoJSON? [5]
    3. IETF GeoJSON Working Group [4]
  5. Conclusions [1]

Messages vs. Models

Messages vs. Models Erik Wilde : API Strategy and Design for Geospatial Data and Services

(18) APIs are Languages

  • Interfaces should hide domain complexity (encapsulation)
    • Expose as little as possible of the domain complexity
    • Resist the temptation to expose your domain model
  • Interfaces are languages for communication
    • They establish a language between API provider and consumer
    • Keeping the language simple keeps the API usable
  • Model Creep is an obstacle for API adoption
    • There is no one true model for any domain
    • Keeping the language simple minimizes conflict potential


Messages vs. Models Erik Wilde : API Strategy and Design for Geospatial Data and Services

(19) APIs and Messages

  • Messages are the sentences in API communications
    • They transport meaning between collaborating peers
    • They are independent of how peers implement their activities
  • Messages can follow patterns that make APIs easier to use
    • API styles are focusing on specific communication styles
    • Metamodels help to provide structure (plain text, binary, XML, JSON, RDF, …)
    • API models add message models (SOAP, HAL, GraphQL, …)
    • Vocabularies help to describe specific APIs
  • The usual balancing act of familiarity and avoiding Maslow's hammer [https://en.wikipedia.org/wiki/Law_of_the_instrument]


Messages vs. Models Erik Wilde : API Strategy and Design for Geospatial Data and Services

(20) Make Messaging the Start

  • Make sure to identify relevant resources so that you can refer to them
    • Resource refers to anything that is worth exchanging messages about
    • Unique and stable identity helps when assigning resource identifiers
  • Any service resource must have a representation in messages
    • Your data model is not your object model
    • Your object model is not your resource model
    • Your resource model is not your message model
  • Design interaction forms to be as succinct as possible


API Scope

API Scope Erik Wilde : API Strategy and Design for Geospatial Data and Services

(22) APIs as Tools

  • APIs exist to solve specific problems
    • Think about the problem and not the solution
    • Design a capability that hides solution details
  • Collaborate with consumers and gather their feedback
    • API first is all about driving solutions from the API
    • Use approaches that make incremental design easy


API Scope Erik Wilde : API Strategy and Design for Geospatial Data and Services

(23) Tools and Problems

Hammer and Screw

API Scope Erik Wilde : API Strategy and Design for Geospatial Data and Services

(24) KISS API Design

  • The only thing that matters is the service surface
    • Make the surface as shallow as possible to reduce coupling
    • Resist the temptation to expose your model just because you can
  • Concepts must be well-defined but not necessarily deeply modeled
  • There is no single correct path to well-designed APIs
    • What is the capability that the API is exposing?
    • What are the assumptions about API consumers?


Developer Experience (DX)

Developer Experience (DX) Erik Wilde : API Strategy and Design for Geospatial Data and Services

(26) Design Thinking

  • Design is driven by creating products/solutions
    • How well does the product address a given problem?
    • How easy is the designed product to use?
  • APIs often are part of a bigger picture
    • Developers have to use APIs to design their applications
    • How easy is it for them to use APIs as part of their solution?
    • How easy is it to create new APIs when the existing ones do not work?


Developer Experience (DX) Erik Wilde : API Strategy and Design for Geospatial Data and Services

(27) Ad-Hoc API Integration

Client-Side API Integration

Developer Experience (DX) Erik Wilde : API Strategy and Design for Geospatial Data and Services

(28) Pre-Packaged API Integration

Server-Side API Integration

Developer Experience (DX) Erik Wilde : API Strategy and Design for Geospatial Data and Services

(29) Design Coherence

  • Establish a culture of API design
    • This is how APIs are usually looking
    • Here are some tools to help with design and documentation
    • Here is a breakdown of what the API landscape looks like
  • Observing reality helps to document and establish culture
    • One way to think of establishing coherence is by setting guidelines
    • An alternative path is to document and disseminate existing practices


Diversity, Decentralization, and Loose Coupling

Outline (Diversity, Decentralization, and Loose Coupling)

  1. Introduction [3]
  2. Introduction to APIs [6]
    1. APIs from 50,000 feet [3]
    2. Three Drivers of the API Economy [3]
  3. API Design [13]
    1. Messages vs. Models [3]
    2. API Scope [3]
    3. Developer Experience (DX) [4]
    4. Diversity, Decentralization, and Loose Coupling [3]
  4. Geospatial APIs [14]
    1. Geospatial API Issues [5]
    2. What is GeoJSON? [5]
    3. IETF GeoJSON Working Group [4]
  5. Conclusions [1]
Diversity, Decentralization, and Loose Coupling Erik Wilde : API Strategy and Design for Geospatial Data and Services

(31) Diversity

  • API designs should focus on the API Quality
    • Standardizing on technologies is no guarantee for quality
  • Teams are incentivized to create good APIs
    • Feedback from API consumers comes early and often
    • Being a good citizen improves the chances of success
    • Monitoring quickly reveals patterns of usage and popularity
  • Standardize on good practices instead of mandated patterns
  • Consumers should have a seamless experience across APIs


Diversity, Decentralization, and Loose Coupling Erik Wilde : API Strategy and Design for Geospatial Data and Services

(32) Decentralization

  • Centralization has become an IT anti-pattern
  • APIs should be easily composable by consumers
    • Robust identity allows resource discovery without out-of-band knowledge
    • Interlinking exposes control flows in a self-describing way
  • Individual services own and document their service APIs
  • Cross-service examples and use cases need to respect this autonomy


Diversity, Decentralization, and Loose Coupling Erik Wilde : API Strategy and Design for Geospatial Data and Services

(33) Loose Coupling

  • Apply well-known Patterns for Robust Extensibility [http://dret.typepad.com/dretblog/2016/04/robust-extensibility.html]
  • Modern service architecture encourages service changes and replacements
    • Design for service changes: Non-breaking changes as the goal
    • Design for landscape changes: Self-describing interactions allow changing workflows
  • Hypermedia is a well-known pattern for service and landscape changes


Geospatial APIs

Outline (Geospatial APIs)

  1. Introduction [3]
  2. Introduction to APIs [6]
    1. APIs from 50,000 feet [3]
    2. Three Drivers of the API Economy [3]
  3. API Design [13]
    1. Messages vs. Models [3]
    2. API Scope [3]
    3. Developer Experience (DX) [4]
    4. Diversity, Decentralization, and Loose Coupling [3]
  4. Geospatial APIs [14]
    1. Geospatial API Issues [5]
    2. What is GeoJSON? [5]
    3. IETF GeoJSON Working Group [4]
  5. Conclusions [1]

Geospatial API Issues

Geospatial API Issues Erik Wilde : API Strategy and Design for Geospatial Data and Services

(36) API Complexity

  • Geospatial models tend to have intrinsic complexity
    • Well-known in the geospatial community
    • Not as well-known outside of that particular community
  • Geospatial information sharing can have simple scenarios
    • Extracting information based on simple query models
    • Submitting information based on simple insertion models


Geospatial API Issues Erik Wilde : API Strategy and Design for Geospatial Data and Services

(37) API Audience

  • Design APIs with an audience in mind
    • Designing into the void rarely leads to good UX/DX
    • All you have to do is know these 28 OGC standards.
  • Using simple standards means accommodating bigger audiences


Geospatial API Issues Erik Wilde : API Strategy and Design for Geospatial Data and Services

(38) Geo API Landscape Scale

  • Different environments have different ideas of landscape scale
    • Small projects may have a handful of services
    • Large adopters may have hundreds or thousands of APIs
  • Using any form of discovery may not be necessary from the start
    • Consider the possibility when starting an API effort
    • Having a plan helps teams to be aware of the need later on


Geospatial API Issues Erik Wilde : API Strategy and Design for Geospatial Data and Services

(39) GEO API Landscape Diversity

  • Not all services use the same technical foundations
    • Different communications mechanisms
    • Different metamodels (XML/JSON/RDF) for data models
    • Different models for representing application concepts
    • In a Web context, these are concepts [webconcepts.info/concepts/] such as HTTP headers [http://webconcepts.info/concepts/http-header/], media types [http://webconcepts.info/concepts/media-type/], and link relations [webconcepts.info/concepts/link-relation/]
  • Handling diversity can be challenging because of different design universes
    • Depending on choice, similar concepts/models may be represented differently
    • Introducing a generalized abstraction layer is unlikely to be helpful
  • Discovering documentation provides a better overview of the ecosystem
    • A reality-based overview of the technology landscape
    • A possible starting point or guideline for getting teams started


Geospatial API Issues Erik Wilde : API Strategy and Design for Geospatial Data and Services

(40) Geo API Landscape Decentralization

  • Decentralization means no single point of control
  • Decentralization means no automatic top-down visibility
    • API gateways are one way of improving visibility (and control)
    • Instrumenting the connective fabric is another way out of this
  • Complete and uncontrolled decentralization is an ideal
    • Observe how the landscape develops
    • Identify pain points from the design and operational point of view
    • Identify popular patterns so that teams have a starting point
    • Convincing teams instead of mandatory guidelines


What is GeoJSON?

What is GeoJSON? Erik Wilde : API Strategy and Design for Geospatial Data and Services

(42) It's for Riding Animals!

fish.jpggoose.jpg

What is GeoJSON? Erik Wilde : API Strategy and Design for Geospatial Data and Services

(43) GeoJSON Fish

  • Transformed from Strava [http://www.strava.com/]-originated activity [http://www.strava.com/activities/523719323] GPX [src/fish.gpx]
  • Github [http://github.com/] understands and renders the resulting GeoJSON [src/fish.geojson]
{
    "type": "Feature",
    "properties": {
        "name": "World Water Day Fish",
        "time": "2016-03-22T11:01:42.000Z"
    },
    "geometry": {
        "type": "LineString",
        "coordinates": [
            [
                8.548238184303045,
                47.40594316273928,
                459.6000061035156
            ],
            [


What is GeoJSON? Erik Wilde : API Strategy and Design for Geospatial Data and Services

(44) GeoJSON Goose

  • Transformed from Strava [http://www.strava.com/]-originated activity [http://www.strava.com/activities/527214108] GPX [src/goose.gpx]
  • Github [http://github.com/] understands and renders the resulting GeoJSON [src/goose.geojson]
{
    "type": "Feature",
    "properties": {
        "name": "Easter Goose",
        "time": "2016-03-26T11:03:27.000Z"
    },
    "geometry": {
        "type": "LineString",
        "coordinates": [
            [
                8.551550712436438,
                47.40528577007353,
                401
            ],
            [


What is GeoJSON? Erik Wilde : API Strategy and Design for Geospatial Data and Services

(45) Representing Geometries or Features

  • GeoJSON has object types for single and multiple geometries and features
    • Geometry [http://tools.ietf.org/html/rfc7946#section-3.1] represents 6 geometry types (see below) and one collection
    • Feature [http://tools.ietf.org/html/rfc7946#section-3.2] represents single or multiple spatially bound things
  • GeoJSON's fundamental geometry construct is the position [https://tools.ietf.org/html/rfc7946#section-3.1.1]
    • Point [https://tools.ietf.org/html/rfc7946#section-3.1.2] has one position
    • LineString [https://tools.ietf.org/html/rfc7946#section-3.1.4]/MultiPoint [https://tools.ietf.org/html/rfc7946#section-3.1.3] have an array of positions
    • Polygon [https://tools.ietf.org/html/rfc7946#section-3.1.6]/MultiLineString [https://tools.ietf.org/html/rfc7946#section-3.1.5] have an array of arrays of positions
    • MultiPolygon [https://tools.ietf.org/html/rfc7946#section-3.1.7] has an array of polygons
  • Features have properties and an associated geometry
  • Feature and Geometry also come as FeatureCollection [https://tools.ietf.org/html/rfc7946#section-3.3]/GeometryCollection [https://tools.ietf.org/html/rfc7946#section-3.1.8]
  • GeoJSON is not meant to represent the full complexity/fidelity of GIS models


What is GeoJSON? Erik Wilde : API Strategy and Design for Geospatial Data and Services

(46) Using GeoJSON

  • Meant as a simple and lightweight representation
    • Use it in services/APIs where geometries/features need to be represented in a simple way
    • Do not expect full fidelity of possibly complex underlying models
  • Not meant as a synchronization format for full-fidelity GIS access/interchange
    • Trying to GISify GeoJSON very likely makes it non-interoperable
  • Geo-tools such as Leaflet [http://leafletjs.com/] often support GeoJSON out of the box [http://leafletjs.com/examples/geojson.html]


IETF GeoJSON Working Group

IETF GeoJSON Working Group Erik Wilde : API Strategy and Design for Geospatial Data and Services

(48) GeoJSON Problems

  • Despite being popular, the GeoJSON specification [http://geojson.org/geojson-spec.html] was not created/managed by any SDO
    • Lingering doubts about the stability/stewardship of the specification
    • Some policies require that only SDO-published specifications can be used
  • Usage around CRS [http://geojson.org/geojson-spec.html#coordinate-reference-system-objects] had caused some interoperability problems
    • Default rule (WGS84) plus inheritance meant implementations often did not implement the specification
    • Naming of specific CRS was left open, resulting in namespace fragmentation
  • Extensibility and openness were not specified clearly enough
    • Lack of guidance made it too easy for implementers to do the wrong thing
    • Experience showed which areas were the most problematic ones


IETF GeoJSON Working Group Erik Wilde : API Strategy and Design for Geospatial Data and Services

(49) IETF GeoJSON WG

  • The IETF GeoJSON Working Group [https://datatracker.ietf.org/wg/geojson/] started work in October 2015
    • The main item on the charter was to revise and publish GeoJSON as an RFC
    • An additional item is to publish a specification for GeoJSON Text Sequences
    • Additional work items are currently under consideration
  • IETF working groups are open to all interested participants
  • IETF RFCs are often accepted as standards that are stable and safe to use


IETF GeoJSON Working Group Erik Wilde : API Strategy and Design for Geospatial Data and Services

(50) IETF GeoJSON WG Work

  • Adopted a draft starting from the GeoJSON Specification [http://geojson.org/geojson-spec.html]
    • List of changes available in Appendix B [https://tools.ietf.org/html/rfc7946#appendix-B]
  • Started draft about GeoJSON Text Sequences [http://datatracker.ietf.org/drafts/draft-ietf-geojson-text-sequence]
    • Applies JSON Text Sequences (RFC 7464) [http://tools.ietf.org/html/rfc7464] to GeoJSON
    • How to represent sequences of GeoJSON objects in one streamable representation
    • Useful in scenarios where sequences/streams of GeoJSON objects are exchanged
  • Two more work items currently considered
    • Providing guidance and a registry for GeoJSON extensions
    • Adding time information to GeoJSON as an initial extension


IETF GeoJSON Working Group Erik Wilde : API Strategy and Design for Geospatial Data and Services

(51) GeoJSON Changes

  • Discouraging the use of non-WGS84 CRS
  • Clarifying the extensibility of position arrays
  • Clarifying lines as being Cartesian (instead of being geodesic)
  • Clarifying the definition of a bounding box
  • Making it clear that extensions cannot change the base semantics
  • Defining a new media type application/geo+json


Conclusions

Outline (Conclusions)

  1. Introduction [3]
  2. Introduction to APIs [6]
    1. APIs from 50,000 feet [3]
    2. Three Drivers of the API Economy [3]
  3. API Design [13]
    1. Messages vs. Models [3]
    2. API Scope [3]
    3. Developer Experience (DX) [4]
    4. Diversity, Decentralization, and Loose Coupling [3]
  4. Geospatial APIs [14]
    1. Geospatial API Issues [5]
    2. What is GeoJSON? [5]
    3. IETF GeoJSON Working Group [4]
  5. Conclusions [1]
Conclusions Erik Wilde : API Strategy and Design for Geospatial Data and Services

(53) Thanks! Q&A

What's Next?

June 5, 2017 GEOINT 2017