28 February 2019
6:00 - Doors & Food
6:30 - Talk 1
7:15 - Talk 2
7:45 - Wrap & Chat
Talk 1: Pros and cons of containerizing data workflows (and how to have the best of both worlds)
Speaker: Tian Xie, Data Engineer @ Enigma
At Enigma, we run over one hundred workflows to ingest public data into our system. Running so many workflows also means managing dependencies and deployment for each of those workflows. Over time, we have iterated over several solutions to this problem and this is our story. Spoiler: docker is involved, but (*plot twist*) it only leads to another set of problems in the 2nd act.
Tian Xie has been working in the NYC tech start-up scene for the last eight years on consumer video rendering, on-demand shipping, and now data engineering at Enigma Technologies.
Talk 2: TBD