Write a Blog >>
Wed 21 Jun 2017 10:20 - 10:45 at Auditorium, Vertex Building - Systems and Performance Chair(s): Dan Grossman

We present the scalable, elastic operator scheduler in IBM Streams 4.2. Streams is a distributed stream processing system used in production at many companies in a wide range of industries. The programming language for Streams, SPL, presents operators, tuples and streams as the primary abstractions. A fundamental SPL optimization is operator fusion, where multiple operators execute together in the same process. Streams 4.2 automatically performs fusion at submission time, because we discovered that in practice, customers did not have the expertise to do so. However, this presented a new problem: potentially thousands of operators would execute together in the same process, with no user guidance for thread placement. We needed a way to automatically figure out how many threads to use, with arbitrarily sized applications on a wide variety of hardware, and without any input from programmers. Our solution has two components. The first is a scalable operator scheduler that minimizes synchronization, locks and global data, while allowing threads to execute any operator and dynamically come and go. The second are elastic algorithms to dynamically adjust the number of threads to optimize performance, using the principles of trust and establishing trends. We demonstrate our scheduler’s ability to scale to over a hundreds threads, and our elasticity algorithm’s ability to adapt to diferent workloads on an Intel Xeon system with 176 logical cores, and an IBM Power8 system with 184 logical cores.

Wed 21 Jun

Displayed time zone: Amsterdam, Berlin, Bern, Rome, Stockholm, Vienna change

10:20 - 12:00
Systems and PerformancePLDI Research Papers at Auditorium, Vertex Building
Chair(s): Dan Grossman University of Washington
10:20
25m
Talk
Low-Synchronization, Mostly Lock-Free, Elastic Scheduling for Streaming Runtimes
PLDI Research Papers
Scott Schneider IBM Research, Kun-Lung Wu IBM Research
Media Attached
10:45
25m
Talk
Practical Partial Evaluation for High-Performance Dynamic Language Runtimes
PLDI Research Papers
Thomas Wuerthinger Oracle Labs, Christian Wimmer , Christian Humer Oracle Labs, Switzerland, Andreas Woess Oracle Labs, Lukas Stadler Oracle Labs, Austria, Chris Seaton Oracle Labs, Gilles Duboscq Oracle Labs, Doug Simon Oracle Labs, Matthias Grimmer Oracle Labs, Austria
Media Attached
11:10
25m
Talk
Responsive Parallel Computation: Bridging Competitive and Cooperative Threading
PLDI Research Papers
Stefan K. Muller , Umut A. Acar Carnegie Mellon University, Robert Harper CWI
Media Attached
11:35
25m
Talk
StreamQRE: Modular Specification and Efficient Evaluation of Quantitative Queries over Streaming Data
PLDI Research Papers
Konstantinos Mamouras University of Pennsylvania, Mukund Raghothaman University of Pennsylvania, Rajeev Alur University of Pennsylvania, Zachary G. Ives University of Pennsylvania, Sanjeev Khanna University of Pennsylvania
Media Attached