Apache Beam Meetup




Apache Beam meetup

1 May 2019


Added 02-May-2019

18:30 - Registrations, pizza and drinks.

18:45 - kick-off

19:00 - 1st talk: Beam at Lyft.

19:30 - 2nd talk: Building a data lake using Beam at LKQ by Datatonic.

19:50 - 3rd talk: Beam in-depth: schema support in Beam by Google


1st talk
For the first talk, we welcome Thomas Weise (https://www.linkedin.com/in/thomas-weise-0b57a63). Thomas is Software Engineer, Streaming Platform at Lyft. He is also a PMC member for Apache Beam and Apache Flink and contributor to several more of the ASF ecosystem projects. His talk will be about dynamic pricing at Lyft with a combination of various data sources, machine learning models, and streaming infrastructure for low latency, reliability and scalability. Dynamic pricing allows Lyft to quickly adapt to real world changes and be fair to drivers (by say raising rates when there's a lot of demand) and fair to passengers (by let’s say offering to return 10 mins later for a cheaper rate). The streaming platform powers this use case by bringing together the best of two worlds using Apache Beam; ML algorithms in Python and Apache Flink as the streaming engine. Learn how the pricing legacy infrastructure was migrated to become the first production ready deployment on the new portable Flink runner and how Beam's portability framework enables the execution of Python code on Flink.

2nd talk
Joe Cullen (https://www.linkedin.com/in/joseph-cullen-97a8727a), a Data Engineer at Datatonic, a team of problem solvers working on cutting-edge Machine Learning solutions in the fields of Media, Telecom, and Retail will talk about their journey with LKQ, building reusable Beam pipelines to ingest CSV data onto Google Cloud Platform with the goal to build a data lake to facilitate machine learning and analytics.

3rd talk
Reuven Lax (https://www.linkedin.com/in/reuven-lax-a82818/), senior software engineer at Google, will present one of the features he has been working on extensively: schema support and how it can be useful for you!

Who should attend
Everyone interested in Data Engineering, Data Science and Machine Learning, who wants to learn about one of the newer and exciting Apache projects focused on batch & stream processing of data. We try to cover both business value as well as digging deeper technically.