Accelerating API Availability with Microservices

GEOINT 2018

Erik Wilde
API Academy, CA Technologies

April 23, 2018

Creative Commons License [http://creativecommons.org/licenses/by/3.0/]

This work is licensed under a CC
Attribution 3.0 Unported License
[http://creativecommons.org/licenses/by/3.0/]

Contents Erik Wilde : Accelerating API Availability with Microservices

Contents

Erik Wilde : Accelerating API Availability with Microservices

(2) Summary

Microservices are a relatively recent trend in the enterprise IT architecture space. The main idea of microservice architecture (MSA) is to strictly decouple service implementation, which means that the development and change of MSA services can be quicker than in architectures where any development and change is limited by complex interdependencies and resulting slower implementation cycles. In order to make the pattern of many independently developed services economically and operationally viable, MSA requires high degrees of automation and tooling for development, deployment, and operations. One possible way to look at MSA is to consider it as the application of DevOps ideas within the SOA space.



Introduction

Outline (Introduction)

  1. Introduction [4]
  2. Digital Transformation [14]
    1. APIs from 50,000 feet [2]
    2. APIs: 3 Drivers, 2 Perspectives, 1 Lifecycle [5]
    3. Digital Transformation: Why? [2]
    4. Digital Transformation: What? [3]
    5. Digital Transformation: How? [2]
  3. Microservices [14]
    1. Microservices as a Pattern [8]
    2. The MSA Journey [3]
    3. Microservices and APIs [3]
  4. Accelerating API Availability [7]
    1. From Integration to APIs [4]
    2. GraphQL [3]
  5. Conclusions [2]
Introduction Erik Wilde : Accelerating API Availability with Microservices

(4) @dret on Twitter/GitHub

dret

Introduction Erik Wilde : Accelerating API Availability with Microservices

(5) API Academy [http://www.apiacademy.co/]

API Academy

Introduction Erik Wilde : Accelerating API Availability with Microservices

(6) Microservice Architecture

Microservice Architecture: Aligning Principles, Practices, and Culture

Introduction Erik Wilde : Accelerating API Availability with Microservices

(7) About Me



Digital Transformation

Outline (Digital Transformation)

  1. Introduction [4]
  2. Digital Transformation [14]
    1. APIs from 50,000 feet [2]
    2. APIs: 3 Drivers, 2 Perspectives, 1 Lifecycle [5]
    3. Digital Transformation: Why? [2]
    4. Digital Transformation: What? [3]
    5. Digital Transformation: How? [2]
  3. Microservices [14]
    1. Microservices as a Pattern [8]
    2. The MSA Journey [3]
    3. Microservices and APIs [3]
  4. Accelerating API Availability [7]
    1. From Integration to APIs [4]
    2. GraphQL [3]
  5. Conclusions [2]

APIs from 50,000 feet

APIs from 50,000 feet Erik Wilde : Accelerating API Availability with Microservices

(10) Digital Transformation

Digital Transformation: Because being more agile than the competition just may be a good idea...

APIs from 50,000 feet Erik Wilde : Accelerating API Availability with Microservices

(11) APIs are a Pattern

Uniform interface Self-describing representations and stateless interactions


APIs: 3 Drivers, 2 Perspectives, 1 Lifecycle

APIs: 3 Drivers, 2 Perspectives, 1 Lifecycle Erik Wilde : Accelerating API Availability with Microservices

(13) API Driver #1: Push

  • Many environments change at increasing velocities
    • Economics change
    • Incentives change
    • Technologies change
  • Organizations are pushed to better handle change
    • They have to become better at reacting to external changes
    • They have to become better at executing internal changes
    • They have to become better at embracing changing


APIs: 3 Drivers, 2 Perspectives, 1 Lifecycle Erik Wilde : Accelerating API Availability with Microservices

(14) API Driver #2: Pull

  • External: Participating in the API economy becomes increasingly beneficial
    • Interfacing with customers/partners becomes easier
    • The number of API-ready customers/partners keeps increasing
    • The network effect [https://en.wikipedia.org/wiki/Metcalfe%27s_law] guarantees for the trend to continue
  • Internal: Being able to plan and execute changing quickly improves fitness
    • Traditionally, IT is perceived as sluggish and heavyweight
    • Increasingly, IT becomes the lube for the organizational machinery
    • Additionally, IT can allow completely new classes of business models


APIs: 3 Drivers, 2 Perspectives, 1 Lifecycle Erik Wilde : Accelerating API Availability with Microservices

(15) API Driver #3: Follow

  • Having an API strategy becomes increasingly important
    • Organizations have executed on this for a long time (Amazon API memo: 2002)
    • Organizations share experiences and tooling around API strategies
  • Using APIs becomes less of an if and more of a how
    • Business pushes IT to make sure strategy can be executed quickly
    • IT can leverage others as justification and leaders


APIs: 3 Drivers, 2 Perspectives, 1 Lifecycle Erik Wilde : Accelerating API Availability with Microservices

(16) Two Views of API Scope/Focus

  • Internal: Using APIs to break up monolithic systems
    • Focus on systems/software engineering and better modularization
    • Turn a system into an ecosystem.
    • Use Private APIs as a way to reduce tight coupling
  • External: Using APIs to build up ecosystems
    • Focus on distributed/decentralized systems and better connectivity
    • Enable an ecosystem by loosely coupling systems.
    • Use Public APIs as a way to (better) interface with partners/public
  • Both views are complementary and mostly a difference in perspective


APIs: 3 Drivers, 2 Perspectives, 1 Lifecycle Erik Wilde : Accelerating API Availability with Microservices

(17) One Management Lifecycle

API Lifecycle

Digital Transformation: Why?

Digital Transformation: Why? Erik Wilde : Accelerating API Availability with Microservices

(19) Traditional IT

The Perfect Machine

Digital Transformation: Why? Erik Wilde : Accelerating API Availability with Microservices

(20) Business Agility

Farmers' Market Sacramento

Digital Transformation: What?

Digital Transformation: What? Erik Wilde : Accelerating API Availability with Microservices

(22) The Web Works

  • MSA is about service autonomy in service-oriented landscapes
    • Developer autonomy is the freedom to choose the best tools for the job
    • Operations autonomy means that every service is operated standalone
  • MSA is also about effectiveness in autonomy-focused service-oriented landscapes
    • Rapid provisioning means that getting started is an easy task
    • Basic monitoring establishes visibility into service operations
    • Rapid deployment increases the velocity of service landscape evolution


Digital Transformation: What? Erik Wilde : Accelerating API Availability with Microservices

(23) The Web, Industrialized

  • MSA applies Web Architecture's loose coupling [http://dret.net/netdret/publications#pau09a] to the API space
    • Build services with no hidden dependencies
    • Choose the best tool for the job
    • Decouple service provisioning from delivery architecture
    • Monitor service state and usage
    • React quickly when customers or market demand change
    • Experiment ruthlessly and keep experimentation cost low
  • MSA is all about being webby while minimizing the cost


Digital Transformation: What? Erik Wilde : Accelerating API Availability with Microservices

(24) Digitalization Dividends [http://queue.acm.org/detail.cfm?id=2956643]

  1. Permissionless Innovation
  2. Enable Failure
  3. Disrupt Trust (replaced by autonomy and accountability)
  4. You Build It, You Own It
  5. Accelerate Deprecations
  6. End Centralized Metadata
  7. Concentrate the Pain
  8. Test Differently


Digital Transformation: How?

Digital Transformation: How? Erik Wilde : Accelerating API Availability with Microservices

(26) The Path to Microservices [http://martinfowler.com/articles/microservices.html]

  1. Componentization via services
  2. Organized around business capabilities
  3. Products not projects
  4. Smart endpoints and dumb pipes
  5. Decentralized governance
  6. Decentralized data management
  7. Infrastructure automation
  8. Design for failure
  9. Evolutionary design


Digital Transformation: How? Erik Wilde : Accelerating API Availability with Microservices

(27) Microservices for the Masses?

  • Microservices are an investment and not a default choice
    • The cost is the need for advanced infrastructure and teams
    • The return is increased flexibility and agility
    • Cost/benefit for microservices is targeted at long term capabilities
  • What are the prerequisites for considering microservices? [http://martinfowler.com/bliki/MicroservicePrerequisites.html]
    1. Rapid Provisioning: Automated process to quickly create server infrastructure
    2. Monitoring: Multi-level monitoring from service health to business-level logic
    3. Rapid Deployment: Automated ways to quickly move from code to running services
  • Moving to MSA is a transition process and not one Big Bang
    • Just like the Web is never complete/final, neither is a microservices landscape


Microservices

Outline (Microservices)

  1. Introduction [4]
  2. Digital Transformation [14]
    1. APIs from 50,000 feet [2]
    2. APIs: 3 Drivers, 2 Perspectives, 1 Lifecycle [5]
    3. Digital Transformation: Why? [2]
    4. Digital Transformation: What? [3]
    5. Digital Transformation: How? [2]
  3. Microservices [14]
    1. Microservices as a Pattern [8]
    2. The MSA Journey [3]
    3. Microservices and APIs [3]
  4. Accelerating API Availability [7]
    1. From Integration to APIs [4]
    2. GraphQL [3]
  5. Conclusions [2]

Microservices as a Pattern

Microservices as a Pattern Erik Wilde : Accelerating API Availability with Microservices

(30) Bottom-Up Motivation: Service Scalability

  • Build modularized components that can be scaled individually
    • Monitoring allows insights into system status and load
    • Rapid Provisioning allows to react quickly when new capacity is required
    • Rapid Deployment means simple ways to develop/test/start/scale services
  • Microservice implementations are more flexible than integrated approaches
    • Scaling can be done per service instead of scaling bigger deployment units
    • Code can be retired more easily by removing services from deployment
    • Fewer unintended side-effects since all interactions are through APIs


Microservices as a Pattern Erik Wilde : Accelerating API Availability with Microservices

(31) Top-Down Motivation: Digital Transformation

  • IT capabilities and management often are not aligned
    • Capabilities are growing and business demands are growing as well
    • Management becomes complex and impacts execution speed
  • There is a difference in separating interfaces and implementations
    • Interfaces are useful for accessing capabilities
    • Implementations must support easy modifications and additions
  • The first wave of API focus (SOA) focused on interfaces only
    • Focusing on APIs as business-aligned capabilities is a necessary first step
    • Agility suffers when implementations are not business-aligned
    • Conway's Law [https://en.wikipedia.org/wiki/Conway%27s_law] means that changing implementation structure can be hard


Microservices as a Pattern Erik Wilde : Accelerating API Availability with Microservices

(32) The SOA Crisis

  • SOAP-style SOA are traditional IT monoliths exposing some capabilities
  • MSA #1 goal is to create a loosely coupled service landscape
    • Shared data storage creates coupling
    • Shared infrastructure creates coupling
    • Shared libraries create coupling
    • Loose coupling allows to pick the right tool for the job
    • Loose coupling allows to refactor/retire as required
    • Loose coupling blurs the public/partner/private lines
  • Using APIs is one of the few overlaps between SOAP-style SOA and MSA


Microservices as a Pattern Erik Wilde : Accelerating API Availability with Microservices

(33) Microservices are SOA!

Microservices are SOA!

Microservices as a Pattern Erik Wilde : Accelerating API Availability with Microservices

(34) Microservices are SOA done Right!

  • Decouple implementations and not just interfaces
  • Velocity is determined by how hard it is to make changes
  • The biggest problem is not access to capabilities but capability agility
    • Providing access to capabilities is better than doing nothing
    • Agile management of the capability landscape goes further than that
  • Microservices apply Web architecture to Enterprise IT
    • Implementation details are completely invisible to service users
    • Service consumption only uses well-defined and uniform interfaces


Microservices as a Pattern Erik Wilde : Accelerating API Availability with Microservices

(35) The Dreaded Monolith

Where you come from…

Microservices as a Pattern Erik Wilde : Accelerating API Availability with Microservices

(36) Microservices: The Starting Point

Where you think you're going…

Microservices as a Pattern Erik Wilde : Accelerating API Availability with Microservices

(37) Microservices Over Time

By Lachlan Donald from Melbourne, Australia (Sharpest tool in the shed) [CC BY 2.0 (http://creativecommons.org/licenses/by/2.0)], via Wikimedia Commons

The MSA Journey

The MSA Journey Erik Wilde : Accelerating API Availability with Microservices

(39) Transformation: Changing vs. Changeable

  • Changing implies a one-time activity
  • Changeable implies an intrinsic property

    Don't focus on what to change into;
    Focus on embracing change as your mode of existence.



The MSA Journey Erik Wilde : Accelerating API Availability with Microservices

(40) Strangling the Monolith

Strangler Vine

The MSA Journey Erik Wilde : Accelerating API Availability with Microservices

(41) How to Strangle Well

  • Manage access to old/new capabilities through a router or gateway
    • Transitioning is a management process old/new should be unaware of
  • Data sync/update is a separately managed process
    • Sophistication and cost of sync/update depends on consistency requirements
    • When one of the services disappears sync/update can be removed
  • Manage/monitor transition of service replacement
    • At some point in time decommissioning may be the most economical option
    • Until then the strangled monolith is the best case scenario


Microservices and APIs

Microservices and APIs Erik Wilde : Accelerating API Availability with Microservices

(43) There is no Microservices API

  • Microservices are all about how to implement services
  • APIs are all about how to interact with services
  • Microservices are an implementation detail from the API perspective
  • Microservice-implemented APIs might exhibit certain properties
    • Changing more often because of faster implementation cycles
    • More diverse API landscapes because of more autonomy
    • Smaller scope because of solutions targeted at specific problems


Microservices and APIs Erik Wilde : Accelerating API Availability with Microservices

(44) Designing Microservice APIs

  • API consumers just use an API and not a Microservice API
    • Microservice API design is no different from API design in general
  • Balancing autonomy/diversity and coherence/usability is necessary
    • Microservices should have freedom to design their APIs
    • API design should be driven by design culture and patterns
    • Microservices strategy needs to distill/document/distribute design patterns


Microservices and APIs Erik Wilde : Accelerating API Availability with Microservices

(45) Focus Areas for Microservice APIs

  • Be explicit about openness and extensibility
  • Provide glue that makes combining services easy
    • At the minimal level, always use global, unique, and persistent identifiers
    • Consider using hypermedia as a way to support service orchestration
  • Be explicit about how services signal end-of-life


Accelerating API Availability

Outline (Accelerating API Availability)

  1. Introduction [4]
  2. Digital Transformation [14]
    1. APIs from 50,000 feet [2]
    2. APIs: 3 Drivers, 2 Perspectives, 1 Lifecycle [5]
    3. Digital Transformation: Why? [2]
    4. Digital Transformation: What? [3]
    5. Digital Transformation: How? [2]
  3. Microservices [14]
    1. Microservices as a Pattern [8]
    2. The MSA Journey [3]
    3. Microservices and APIs [3]
  4. Accelerating API Availability [7]
    1. From Integration to APIs [4]
    2. GraphQL [3]
  5. Conclusions [2]

From Integration to APIs

From Integration to APIs Erik Wilde : Accelerating API Availability with Microservices

(48) Integration Platforms and Applications

api-integration-platforms.png

From Integration to APIs Erik Wilde : Accelerating API Availability with Microservices

(49) Moving to API Management

api-integration-api-management.png

From Integration to APIs Erik Wilde : Accelerating API Availability with Microservices

(50) Faster API Implementation through Microservices

api-integration-microservices.png

From Integration to APIs Erik Wilde : Accelerating API Availability with Microservices

(51) Manage Your Services/APIs

API Lifecycle

GraphQL

GraphQL Erik Wilde : Accelerating API Availability with Microservices

(53) GraphQL History

graphql-logo.png
  • Facebook wanted a way for clients to more specifically query content
    • Facebook's dataset is a very large interconnected graph
    • Clients want specific parts of it depending on various context factors
  • GraphQL is not an API but a language for building APIs
    • GraphQL defines a type system and a query language that produces JSON results
    • The query language can be integrated into an API in a variety of ways [http://graphql.org/learn/serving-over-http/]
  • GraphQL also has mutations [http://graphql.org/learn/queries/#mutations] as a way to update data
    • Input types [http://graphql.org/learn/schema/#input-types] look like types but only accept input data


Conclusions

Outline (Conclusions)

  1. Introduction [4]
  2. Digital Transformation [14]
    1. APIs from 50,000 feet [2]
    2. APIs: 3 Drivers, 2 Perspectives, 1 Lifecycle [5]
    3. Digital Transformation: Why? [2]
    4. Digital Transformation: What? [3]
    5. Digital Transformation: How? [2]
  3. Microservices [14]
    1. Microservices as a Pattern [8]
    2. The MSA Journey [3]
    3. Microservices and APIs [3]
  4. Accelerating API Availability [7]
    1. From Integration to APIs [4]
    2. GraphQL [3]
  5. Conclusions [2]
Conclusions Erik Wilde : Accelerating API Availability with Microservices

(57) Loosely Coupled Implementations



Conclusions Erik Wilde : Accelerating API Availability with Microservices

(58) Thanks! Q&A

What's Next?

April 23, 2018 GEOINT 2018