A Conversation about REST, Cloud Computing and Microservices
Recent architectural trends in software engineering advocate a fine-grained decomposition of applications into loosely-coupled microservices that can be independently developed, continuously integrated and deployed in the Cloud. In this talk we look at how to recompose reusable microservices back into integrated systems using a variety of software connectors. To do so we will introduce the concept of RESTful conversation and propose RESTalk: a domain-specific, developer-friendly visual notation to model all possible correct interaction sequences (or patterns) that can be performed interacting with RESTful Web service APIs.
Abstractions
Microservices
Will this component
always terminate?
function f() {
...
return 42;
}
Development
Will this service
run forever?
while (true) {
on f {
return f()
};
}
Operations
Will this microservice continuously change?
while (true) {
on f {
- return f()
+ //return f()
+ return f2()
};
}
DevOps
Microservices
The microservice architectural style is an approach to developing a single application as a suite of small services, each running in its own container and communicating with lightweight mechanisms, often an HTTP resource API. These services are built around business capabilities and independently deployable by fully automated deployment machinery. There is a bare minimum of centralized management of these services, which may be written in different programming languages and use different data storage technologies.
Martin Fowler and James Lewis
Microservices
The microservice architectural style is an approach to developing a single application as a suite of small services, each running in its own container and communicating with lightweight mechanisms, often an HTTP resource API. These services are built around business capabilities and independently deployable by fully automated deployment machinery. There is a bare minimum of centralized management of these services, which may be written in different programming languages and use different data storage technologies.
Martin Fowler and James Lewis
How small is a Microservice?
Monolith
Macro
Micro
Nano
How small is a Microservice?
One team has full control of the entire devops (code, build, test, release, deploy and operate) cycle
Iterate fast: Many small frequent releases better than few large releases
Loosely Coupled Microservices
Avoid dependencies: If you have to hold a release until some other team is ready you do not have two separate microservices
Avoid cascading failures: A failed microservice should not bring down the whole system
Decomposition
Recomposition
How to connect two microservices?
The Web as a Software Connector
SOAP over HTTP
grpc+proto over HTTP/2
The Web as a Tunnel
HTTP-*
AsyncSync
Coupling (Time)
1:N1:1
Cardinality
Shared Web Resource
Multicast
REST/WWWMEST/ESB
ResourceQueue
Shared MemoryMessage Passing
One more difference
Conversations
Multiple interactions (message exchanges) between two or more participants to achieve a goal
Gregor Hohpe
RESTful Conversations
Client/Server: Clients always begin every request-response interaction
Requests are addressed to URI
A request message is always followed by a response message, unless the server is not responding (connection refused) or not responsive (client timeout)
There may be different possible responses to the
same request message. The choice is entirely up to the server.
Statelessness: every request is self-contained and therefore independent of the previous ones
Uniform Interface:
there is a fixed set of request methods a resource can support. It
is possible to distinguish safe (GET) or idempotent (PUT, DELETE) requests from unsafe ones (POST)
Hypermedia: responses embed related URIs, which may
be used to address subsequent requests
Link: </payment?booking=42>; rel="payment"
Servers control the conversation by embedding links into representations/responses
Clients drive the conversation by deciding which link to follow
Servers only reveal the immediate next steps
Clients need to understand the link relation semantics to choose which link to follow
How to Describe RESTful Conversations?
RESTalk
HTTP interaction
GET / HTTP/1.1
Host: restalk-patterns.org
HTTP/1.1 200 OK
Link: </patterns>; rel="list"
Sequential Conversation
GET / HTTP/1.1
HTTP/1.1 302 Found
Location: /blog
GET /blog HTTP/1.1
HTTP/1.1 200 OK
Link: </blog/add>; rel="add"
POST /blog/add HTTP/1.1
Slug: my post
HTTP/1.1 201 Created
Location: /blog/my-post/
Hyperlink Flow
Describe how URIs are discovered during the conversation
URI found in responses flow into URIs used in following requests
Branches and Loops
POST /job HTTP/1.1
HTTP/1.1 202 Accepted
Location: /job/42
GET /job/42 HTTP/1.1
HTTP/1.1 200 OK
GET /job/42 HTTP/1.1
HTTP/1.1 303 See Other
Location: /job/42/output
GET /job/42/output HTTP/1.1
HTTP/1.1 200 OK
RESTalk
RESTful Conversation Patterns
One more thing
Microservices
Microservices prefer letting each service manage its own database, either different instances of the same database technology, or entirely different database systems - an approach called Polyglot Persistence.
Which RESTful conversation pattern would you like to discuss?
POST New Collection ItemPOST Once ExactlyPOST-PUT CreationLong Running Operation with PollingServer-side Redirection with Status CodesClient-side Navigation following HyperlinksIncremental Collection Traversal(Partial) Resource EditingConditional Update for Large ResourcesBasic AuthenticationCookie-based Authentication
How to discover how to create a resource within a collection?
POST New Collection Item
Ask the server
POST New Collection Item
POST New Collection Item
Client discovers form with new resource attributes
Server determines new resource URI
Client must understand media type schema
What if the POST fails?
How to avoid creating duplicate resources?
POST-PUT Creation
POST Once Exactly
POST-PUT Creation
True or False?
The client can send POST requests multiple timesThe client knows the link to the created resource before the start of the conversation
POST-PUT Creation
POST Once Exactly
What if the client times out because the server is taking too long to reply?
Long Running Operation with Polling
Long Running Operation with Polling
Long Running Operation with Polling
Clients can share the URI with the results
Clients can cancel the long running operation
Polling: how frequently?
Garbage collection
True or False?
The job output resource gets automatically deleted once the client has read it onceThe client can read the job output multiple timesThe client must delete the job output after reading it
Long Running Operation with Polling
How to decouple clients from evolving resource URIs?
Server-side Redirection with Status Codes
Server-side Redirection with Status Codes
Server-side Redirection with Status Codes
Avoid breaking clients
Refer to server "nearest" to the client
Redirect can be temporary or permanent
Can your client be redirected any time?
Latency may increase
How to avoid hard-coding resource URIs into clients?
Client-side Navigation following Hyperlinks
Client-side Navigation following Hyperlinks
Client-side Navigation following Hyperlinks
Loose Coupling
Standardize Link Relation Semantics
Latency may increase
How to let a client retrieve some items of large collections?
Incremental Collection Traversal
Hypermedia
Incremental Collection Traversal
Incremental Collection Traversal
Save bandwidth
Cannot guarantee consistency
How to discover how to edit a resource?
(Partial) Resource Editing
(Partial) Resource Editing
(Partial) Resource Editing
Client discovers form with editable resource attributes
Idempotent full updates with PUT
Client must understand media type schema
PATCH is not idempotent
How to efficiently check if updates are possible?
Conditional Update for Large Resources
Declare client expectations
Conditional Update for Large Resources
Conditional Update for Large Resources
Save bandwidth, avoid bad requests
HTTP/1.1 Support Required (otherwise 417 Expectation Failed)
Microservices emphasize their small size while minimizing coupling among them is what really matters to enable continuous change and independent evolution
After decomposing the monolith, choose the appropriate software connector to recompose the microservices together
REST
REpresentational State Transfer (Web)
REmote Synchronous Transaction (RPC)
REmote aSynchronous noTification (Bus)
Conclusion
RESTful conversations are enabled and driven by hyperlinks embedded in resource representations
RESTalk visualizes conversations: modeling all possible correct request-response interactions and the hyperlink flow linking them
RESTful Conversation Patterns simplify the design of loosely-coupled RESTful APIs
The BAC Theorem
When Backing up a whole microservice architecture, it is not possible to have both Consistency and Availability
Corollaries
Microservice architectures eventually become inconsistent after disaster strikes when recovering from independent backups
Achieving consistent backups can be attempted by limiting the full availability of the system and synchronizing the backups
Outlook
RESTalk with Multi-party Conversations
RESTful Conversation Mining
More RESTful Conversation Patterns
Dealing with the consequences of the BAC theorem
References
Guy Pardon, Cesare Pautasso, Olaf Zimmermann, Consistent Disaster Recovery for Microservices: the BAC Theorem, IEEE Cloud Computing, DOI: 10.1109/MCC.2017.455154634, In Press
Ana Ivanchikj, Cesare Pautasso, and Silvia Schreier, Visual modeling of RESTful conversations with RESTalk, Journal of Software & Systems Modeling, pp. 1-21, May, 2016. (Journal-First Best Paper Award at MODELS 2016)
Jim Webber, Savas Parastatidis, Ian Robinson, REST in Practice: Hypermedia and Systems Architecture, O'Reilly, 2010
Cesare Pautasso, Erik Wilde, Push-Enabling RESTful Business Processes, 9th International Conference on Service Oriented Computing (ICSOC 2011), Paphos, Cyprus, December 2011
Cesare Pautasso, BPMN for REST, In Proceedings of the 3rd International Workshop on BPMN (BPMN 2011), Lucerne, Switzerland, November 2011
Frank Leymann, Olaf Zimmermann, Florian Haupt, Silvia Schreier, Ana Ivanchikj, Guy Pardon, Mathias Weske, Adriatik Nikaj, Sankalita Mandal, Hagen Overdick, Jesus Bellido, Rosa Alarcón, Alessio Gambi, Daniele Bonetta, Achille Peternier, Erik Wilde, Mike Amundsen, Stefan Tilkov, James Lewis