Your browser doesn't support the features required by impress.js, so you are presented with a simplified version of this presentation.

For the best experience please use the latest Chrome, Safari or Firefox browser.

The Stream Software Connector Design Space

Frameworks and Languages for Distributed Stream Processing

Masiar Babazadeh, Cesare Pautasso
http://www.pautasso.info
c.pautasso@ieee.org
@pautasso

Software Connectors

Streaming Connector

Basic Concepts

Design Space

Survey

Design Space Overview

Linear Pipeline

Parallel Flows

Directed Acyclic Graph

Arbitrary Graph

Declarative

Select Sum(O.cost)
From Orders O, Fulfillments F [Range 1 Day]
Where O.orderID = F.orderID And F.clerk = "Sue"
    And O.customer = "Joe"
Continuous Query Language (CQL)

Imperative

void->void pipeline Minimal {
    add IntSource;
    add IntPrinter;
}
void->int filter IntSource {
    int x;
    init { x = 0; }
    work push 1 {
        push(x++); }
    }
int->void filter IntPrinter {
    work pop 1 {
        print(pop());
    }
}
StreamIt

Cluster

Cloud

Pervasive

Web Browser

Static

General topology changes (e.g. operator addition) require to stop and restart the stream

Predefined dynamic behavior can be expressed with optional operators and branches

Dynamic: Operator

Operators can be reconfigured at run-time (e.g. degree of parallelism, resource allocation)

Failed operators can be migrated without altering the logical topology

Dynamic: Topology

Operators can be added (or removed) from the topology without stopping the stream

Load Shedding

Dynamic Initialization

Dynamic Adaptive Controller

Replication

Reconfiguration

Summary

Method

Conclusion

Pipeline Topology

Pipeline Representation

Deployment

Pipeline Flexibility

Load Balancing

Fault Tolerance

Use a spacebar or arrow keys to navigate