Data Swindon: Azure, IoT, Real Time Data, Analytics And ETL




Data Swindon: Azure, IoT, Real Time Data, Analytics and ETL

4 September 2019


Added 01-Jan-1970

18.15 – 18:30 Meet & Greet

18:30 - 19:20
TALK #1 - Paul Andrew "ETL in Azure Made Easy with Data Factory Data Flows" (Level 2)
What happens when you combine a cloud orchestration service with a Spark cluster?! The answer is a feature rich, graphical, scalable data flow environment to rival any ETL tech we’ve previously had available in Azure. In this session we’ll look at Azure Data Factory and how it integrates with Azure Databricks to produce a powerful abstraction over the Apache Spark analytics ecosystem. Now we can transform data in Azure using Databricks, but without the need to write a single line of Scala or Python! If you haven’t used either service yet, don’t worry, you’ll get a quick introduction to both before we go deeper into the Data Factory Data Flow feature.

19.20 – 19:40 Break & Pizza

19:40 - 20:30
TALK #2 - Paul Andrew "Beyond IoT Real-time Data Ingestion with Azure Stream Analytics" (Level 3)
The desire and expectation to use real-time data is constantly growing, businesses need to react to market trends instantly. In this new data driven age a daily ETL load/processing window isn’t enough. We need a constant stream of information and analytics achieved in real-time. In this session will look at how that can be achieved using Azure Stream Analytics. Building streaming jobs that can blend and aggregate data as it arrives to drive live Power BI dashboards. Plus, we’ll explore how a complete lambda architecture can be created when combining stream and batch data together.