How To Use Airflow For ML Research And Development




How to use Airflow for ML research and development

4 June 2019

New York

Added 01-Jan-1970

Think Airflow isn't useful for machine learning R&D? Think again! Please join us for this talk on how one Data Scientist is leveraging Airflow to make his model research and development experience easier and more productive.

Meet-up topic details:
So far we have mostly talked about airflow in a production setting, covering how data engineers use airflow for data on-boarding and running production models. In this talk, we will be covering how data scientists can use airflow for model research and development and how data engineers can work with data scientists to assist them in the R&D process.

Unlike a production setting, the R&D workflow is less well defined, requires trial and error and frequent resetting of tasks and output as data, models and parameters change. It's less about recovering from failures and making sure everything is running smoothly and more about generating insights and assessing predictive power with different models and parameters.