When: Aug. 8, 2019, 6 p.m.
Where: Active Campaign
1 North Dearborn St Suite 500, Chicago IL, 60602
This event is no longer accepting registrations.
189 going so far
Katie Simmons, a data engineer at ActiveCampaign, will speak about the challenges and benefits of using Airflow for ETL at a rapidly growing company. ActiveCampaign has many thousands of databases - some including tables with up to a trillion rows - several APIs and new source requests coming in every week. This lightning talk will be an overview of using Airflow to extract, load and transform that data into our data lake so that it can be used for Business Intelligence and Data Science.
I first encountered SQLAlchemy several years ago. I didn't get it. It seemed every line I attempted to write would drop me into 50 tabs of labarynthine documentation. Why do we have the ORM *and* Core? Should I build my tables as `Table` instances or should I be extending `Base`? How is `Base` more declarative than a function that returns `Table`s?? Can I please just write SQL??? :sob: I'm still hesitant to peek too far behind the curtain, but I do think I've finally wrapped my head around the philosophical underpinnings of the library and the different problems SQLAlchemy allows us to solve. After all, who among us works with databases that aren't problems in and of themselves?
One of the most common languages used by Python developers is some shell script (sh, bash, cmd.exe, or PowerShell), but why torture yourself with poor design decisions from the 70s when you know Python?