8 October 2019
Beam in NYC Kickoff. Food, Beer, Soda, sponsored by Oden Technologies.
Oden Technologies will talk about their cloud-edge-hybrid deployment of Beam for real-time manufacturing analytics. Tyler Akidau will share his insights on streaming sql on Apache Beam and its management.
More Details Forthcoming. I'll continue to announce updates to membership as available.
One SQL to Rule Them All: An Efficient and Syntactically Idiomatic Approach to Management of Streams and Tables
What does it mean to execute streaming queries in SQL? What is the relationship of streaming queries to classic relational queries? Are streams and tables the same thing? What sorts of extensions are necessary to support robust stream processing SQL?
Over the last three years, members of the Apache Beam, Calcite, and Flink communities have held numerous discussions on the topic of streaming SQL semantics and syntax. Earlier this year, we formalized these ideas in our SIGMOD 2019 publication, “One SQL to Rule Them All: An Efficient and Syntactically Idiomatic Approach to Management of Streams and Tables”, in which we presented a three-part proposal for integrating robust streaming into SQL:
Time-varying relations as a foundation for both classical tables and streaming data.
Event-time semantics to ensure correctness despite out-of-order data.
Minimal syntax extensions for controlling materialization of time-varying results.
In this talk, I’ll give an overview of the contributions in the paper, discuss current implementations in Beam, Calcite, and Flink, and talk through ideas for future work.
Tyler Akidau is a senior staff software engineer at Google, where he is the technical lead for the Data Processing Languages & Systems group, responsible for Google's Apache Beam efforts, Google Cloud Dataflow, and internal data processing tools like Google Flume, MapReduce, and MillWheel. His also a founding member of the Apache Beam PMC.
Though deeply passionate and vocal about the capabilities and importance of stream processing, he is also a firm believer in batch and streaming as two sides of the same coin, with the real endgame for data processing systems the seamless merging between the two. He is the author of the `Streaming Systems` book and `Streaming 101` / `Streaming 102` articles from O’Reilly.
Title: Spanning the Edge and Cloud with Apache Beam
Speaker Bio: Devon Peticolas is a Senior Data engineer who works for Oden Technologies on the Data Science team.
Abstract: Oden Technologies provides a real-time mission-critical analytics platform to manufacturers. Mission-critical means real-time metrics and insights need to be computed in a way that’s resilient to the network connectivity issues that surround our customer’s factories, may the be in South Africa or a small midwestern town with only one ISP.
In this talk we’ll go over how Oden is using Apache Beam’s portability to do duel-deployments of our streaming infrastructure both on Dataflow in Google Cloud Platform and on Apache Flink running on on-premise hardware
and Talk #3 (order may change)
Speaker: James Maidment is Engineering Director at Oden Technologies