Prof. Dr. Cesare Pautasso

Performance Comparison Between BPMN 2.0 Workflow Management Systems Versions

Vincenzo Ferme, Marigianna Skouradaki, Ana Ivanchikj, Cesare Pautasso, Frank Leymann

18th Working Conference on Business Process Modeling, Development, and Support (BPMDS 2017), Essen, Germany

June 2017

Abstract

Software has become a rapidly evolving artifact and Workflow Management Systems (WfMSs) are not an exception. WfMSs' changes may impact key performance indicators or resource consumption levels may change among different versions. Thus, users considering a WfMS upgrade need to evaluate the extent of such changes for frequently issued workload. Deriving such information requires running performance experiments with appropriate workloads. In this paper, we propose a novel method for deriving a structurally representative workload from a given business process collection, which we later use to evaluate the performance and resource consumption over four versions of two open-source WfMSs, for different numbers of simulated users. In our case study scenario the results reveal relevant variations in the WfMSs' performance and resource consumption, indicating a decrease in performance for newer versions.

Download

DOI: 10.1007/978-3-319-59466-8_7

PDF: ▼benchflow-bpmds2017.pdf (492KB)

Citation

Bibtex

@inproceedings{2017:benchflow:bpmds,
	author = {Vincenzo Ferme and Marigianna Skouradaki and Ana Ivanchikj and Cesare Pautasso and Frank Leymann},
	title = {Performance Comparison Between BPMN 2.0 Workflow Management Systems Versions},
	booktitle = {18th Working Conference on Business Process Modeling, Development, and Support (BPMDS 2017)},
	year = {2017},
	month = {June},
	publisher = {Springer},
	address = {Essen, Germany},
	doi = {10.1007/978-3-319-59466-8_7},
	abstract = {Software has become a rapidly evolving artifact and Workflow Management Systems (WfMSs) are not an exception. WfMSs' changes may impact key performance indicators or resource consumption levels may change among different versions. Thus, users considering a WfMS upgrade need to evaluate the extent of such changes for frequently issued workload.  Deriving such information requires running performance experiments with appropriate workloads. 
In this paper, we propose a novel method for deriving a structurally representative workload from a given business process collection, which we later use to evaluate the performance and resource consumption over four versions of two open-source WfMSs, for different numbers of simulated users. In our case study scenario the results reveal relevant variations in the WfMSs' performance and resource consumption, indicating a decrease in performance for newer versions.},
	keywords = {BenchFlow, BPMN, Performance Regression, Performance Testing, workflow engine, Workflow Management Systems}
}