Your browser doesn't support the features required by impress.js, so you are presented with a simplified version of this presentation.

For the best experience please use the latest Chrome, Safari or Firefox browser.

Streaming Across the Web of Things

Cesare Pautasso
http://www.pautasso.info
cesare.pautasso@usi.ch
@pautasso

Goals

  1. Provide a stream processing framework for JavaScript
  2. Deploy streaming pipelines across the Web
    (on Web-enabled devices, Web servers and Web browsers)

Target Platform

Why JavaScript?

  1. Low-level streaming protocols (WebSockets, WebRTC)
  2. Dynamically typed streams (send/receive JSON)
  3. Actor-like parallelism (HTML5 WebWorkers, Node.js cluster)
  4. Shared-memory parallelism (TigerQuoll)
  5. Runs on almost everything

Stream Programming Model

Operators

var k = require('WLS.js');
k.createOperator(function(msg) { 
  //operator function
  var chunks = msg.data.split(’ ’);
  chunks.map(function(v) {
    //output to the stream
    k.send({ word : v });
  });
}).start();

Producer

var k = require('WLS.js');

function gps(position) {
    k.send(
     { lat : position.coords.latitude,
       lon : position.coords.longitude });
}

setInterval(function(){
//HTML5 API in the Web Browser
  navigator.geolocation.getCurrentPosition(gps);
}, 1000);

Stateful Filter

var k = require('WLS.js');

//initialize operator state
var sum = 0;
var samples = 0;

k.createOperator(function(msg) {
  //handle next stream message

  sum += msg.temperature;
  samples ++;
  
  //check average temperature
  if (sum/samples > 25) {
    k.send({'weather' : 'warm'});
  } else {
    k.send({'weather' : 'cold'});
  }

}).start();

UI Consumer

var k = require('WLS.js');

k.createOperator(function(msg) {

  //handle weather condition message
  var w = msg.weather;
  if (w) {
    k.setDOM('#weather', 'Temperature is: ' + w)
  }
  
  //handle gps location message
  var lat = msg.lat;
  var lon = msg.lon;
  if (lat && lon) {
    k.setDOM('#position', 
         'Position is: ' + lat + "," + lon);
  }
  
}).start();

Stream Topologies

Many ways to configure topologies (operator graphs):

Interactive Console

run op.js

Run an operator somewhere (returns the operator ID)

bind op1 op2

Establish a stream connection between two operators

stop op1

Disconnect and stop the operator

unbind op1 op2

Disconnect the operators without stopping them

update op1 script op.js

Update the code of the running operator

update op1 worker N

Use N parallel worker threads to run the operator

run op.js peer

Start the operator on the given peer (server or browser)

deploy op1 peer

Make it possible to run op1 on the given peer

undeploy op1 peer

No longer use peer to run op1

migrate op1 peer

Run op1 only on the given peer

exec pipeline.k
exec pipeline.json

Start the given operator topology

Topology Example

Topology Example (JSON)


{
  "topology" : {
    "id" : "example-topology",
    
    "operators" : [
{ "id" : "producer", "script" : "webcam.js", "deploy" : { "peer" : "IP:port", "path" : "/webcam", "sensors" : ["webcam"] } }, { "id" : "filter", "script" : "filter.js" }, { "id" : "consumer", "script" : "consumer.js", "deploy" : { "browser" : true, "path" : "/consumer" } }
], "bindings" : [
{ "from" : "producer", "to" : "filter", "type" : "roundrobin" }, { "from" : "filter", "to" : "consumer", "type" : "broadcast" }
] } }

Deployment Constraints

peer : [ "IP:port" ]

The operator can be deployed on these peers

sensors : [ "webcam", "gps" ]
actuators : [ "light-switch" ]

The operator requires access to some type of sensor/actuator

browser : undefined

The operator can run both on a browser or a server

browser : false

The operator cannot run on a browser

browser : true

The operator can only run on a browser

path : "/webcam"

The browser gets the operator from this URI path

Broadcast Bindings

Round Robin Bindings

Runtime Architecture

Architecture

Architecture (Browser)

Initial Results

Web Streaming Protocols

Web Streaming Protocols

Parallel Filter

Parallel Filter

Next Steps

Conclusion

Acknowledgement

Liquid Web Streams project (SMARTWORLD)

Credits

Masiar Babazadeh

Daniele Bonetta

Andrea Gallidabino

Mattia Candeloro

References

Related Work

Use a spacebar or arrow keys to navigate

Timer