Ray is a framework for distribution and scaling of clustered, high-performance, Python applications. It is used in several ML/AI systems and production deployments. This talk explains the problems that Ray solves, including rapid execution of “tasks” and management of distributed state, such as model parameters during training. I’ll use several example applications to illustrate. You'll learn when and how to use Ray in your projects.
Most computer languages offer "int"s and "reals" and maybe some support for "complex" or fixed point decimal. Python goes further. This talk will discuss built-in numeric types (such as Rational and Decimal), numeric types from Numpy, and the Abstract Base Classes that make it possible to add your own specialized numeric type and have it appear as part of the language.
Have you ever wondered how your computers knows what programs are running? What about what happens behind the scenes when you start a program? This talk will cover the basics of how processes work, and how your operating system keeps track of what's running. By the end of it you will know enough to write your own basic versions of 'ps' or 'top'.
The rapid growth of Python is, in part due, to it's exceptional toolkit for Data Analysts, Scientists, and Engineers. Packages like Pandas, Scikit-Learn, PySpark, and Dask have become staples for teams looking to process data. However, when processing large amounts of data there are times when Python might not be the right solution for your task. In this conversation, we'll learn about Cloud based Data Warehouses, such as Google's BigQuery, Amazon's Redshift, and Snowflake. You'll learn about the advantages of these platforms compared to in-memory processing in Python. We'll also show examples of how you can use Apache Airflow to automate recurring tasks, turning your Data Warehouse into the cornerstone of your Data Science infrastructure.
Join us as we describe our migration from a limiting cloud deployment on long-running VMs with shared infrastructure to a streamlined immutable infrastructure built on top of Docker and K8s. We'll also discuss techniques to support local development during this transition. Many teams wish they could reap the widely known benefits of Kubernetes (K8s), but most struggle to migrate to a new infrastructure while simultaneously supporting two deployment models and avoiding impacts to the velocity of software development. In this talk, we describe the particular challenges we faced during our incremental migration from multiple long-running singleton EC2 instances to a containerized solution. We'll highlight: - What challenges motivated us to transition to K8s? - Approaching an infrastructure migration incrementally to minimize impacts to local development and production deployments - Developing a solution to provide the same abstraction for local development that exists in production - Concurrently supporting multiple deployment models to reduce risk and simplify migration - Strategy variations for synchronous and asynchronous services - Networking challenges with Vagrant and Docker - Integrating K8s with a CI/CD pipeline - Tuning the environment
If you have attended a few ChiPy events, chances are you have used the chipy.org website. The ChiPy Web Guild is a group of volunteers that help maintain the site. In this talk, I will give a brief description of how the Web Guild works and touch on some aspects of the ChiPy.org site. We will then go through an example of how team members were able to address a flaw in the ChiPy.org code enhancing user experience. Finally, I will share some thoughts on what I learned and what the group might work on next.
Ten mentees will present the projects that they have been working on with their mentoors for the past 3 months.
We will talk to you about Nielsen's Connect Platform, our global, unified, open data ecosystem powered by Microsoft Azure and how we're building platform components using Python. Specifically, we'll deep dive into object-oriented data flows. As more and more data scientists write software beyond statistical models, object thinking from the field of programming can help them write test-able, maintainable and reusable components.
Machine Learning is something you'll see referenced very frequently now in everything from marketing materials to sales pitches, and job postings. With so much hype it can be hard to distinguish what people mean when they say Machine Learning. In this talk we will demystify Machine Learning by understanding its core concepts and applying that knowledge to real world examples. We'll explain basic concepts like linear algebra and loss functions, figure out when to use machine learning and build an ML model that we'll be able to use in real world apps. Here’s an in-depth list of what we'll cover: * What Machine Learning is and where it’s being used * How to recognize when machine learning is necessary * Math 101 * Linear Regression * Live Coding Session Salary Estimator * Q & A
I made a package, pyplot-themes, that helps make it easier to: 1. have decent looking matplotlib/pandas plots 2. have some decent color palettes 3. create your plot themes https://pypi.org/project/pyplot-themes/