Ray is a framework for distribution and scaling of clustered, high-performance, Python applications. It is used in several ML/AI systems and production deployments. This talk explains the problems that Ray solves, including rapid execution of “tasks” and management of distributed state, such as model parameters during training. I’ll use several example applications to illustrate. You'll learn when and how to use Ray in your projects.
I first encountered SQLAlchemy several years ago. I didn't get it. It seemed every line I attempted to write would drop me into 50 tabs of labarynthine documentation. Why do we have the ORM *and* Core? Should I build my tables as `Table` instances or should I be extending `Base`? How is `Base` more declarative than a function that returns `Table`s?? Can I please just write SQL??? :sob: I'm still hesitant to peek too far behind the curtain, but I do think I've finally wrapped my head around the philosophical underpinnings of the library and the different problems SQLAlchemy allows us to solve. After all, who among us works with databases that aren't problems in and of themselves?
Katie Simmons, a data engineer at ActiveCampaign, will speak about the challenges and benefits of using Airflow for ETL at a rapidly growing company. ActiveCampaign has many thousands of databases - some including tables with up to a trillion rows - several APIs and new source requests coming in every week. This lightning talk will be an overview of using Airflow to extract, load and transform that data into our data lake so that it can be used for Business Intelligence and Data Science.
Not all data is easily accessible. Taking info from a website that requires authentication, interaction, or even just to load a fancy script. This talk will discuss using Selenium to level up your web scraping skills, with examples and suggested practices.
Failure can be scary. There are real costs to a company and its users when software crashes, models are inaccurate, or when systems go down. The emotional stakes feel high-- no one wants to be responsible for a failure. We can lower the stakes by creating spaces to learn from failures, and minimize their impact. This talk introduces two ways to address failure: blameless post-mortems, to learn from an incident; and pre-mortems, to identify modes of failure upfront.