ChiPy __main__ Meeting May 2020
When: May 14, 2020, 6 p.m.
Where: Remote Meeting
This event is no longer accepting registrations.
19 going so far
From Spreadsheets to DataFrames: Escaping Excel Hell with Python
By: Ryan McCoy
Experience Level: Novice
A spreadsheet is a wonderful invention and an excellent tool for certain jobs. All too often, however, spreadsheets are called upon to perform tasks that are beyond their capabilities. It is like the old saying, “If the only tool you have is a hammer, every problem looks like a nail.” But some problems are better addressed with a screwdriver, with glue, or with a swiss army knife.
Python is often called the Swiss army knife of the programming world, due to its versatility and flexibility in use. That is why it has become increasingly popular over time. Companies can adopt Python to perform some uniquely complex processes over the long-term.
During this talk, Ryan will discuss his firsthand account of Excel Hell and how he managed to escape it using Python. He will also discuss of the relevant libraries he uses for web scraping, data processing, analysis, and visualization, including Requests, Pandas, Flask, and Airflow, as well as few strategies he uses when approaching problems with data.
Ryan S. McCoy is a Data Engineer at gotem, LLC, where he is responsible for helping modernize the systems, data infrastructures, and analytics of companies primarily in the Financial Services industry, including Investment Managers, Hedge Funds, Venture Capital funds, and data vendors. Previously he spent a decade at several institutional investment funds located in St. Louis.
Github Repo -> https://github.com/ryansmccoy/spreadsheets-to-dataframes
Intro to Python kubernetes (k8s) client
By: Nate Rock
Experience Level: Intermediate
Length: 30 Minutes
Python has a great library for interacting with kubernetes (k8s) clusters. This talk will discuss two quick tools to get your feet wet when it comes to interacting with k8s using python and show you some of the things to look out for, as well as the basics of local vs intra-cluster security.
"the phone calls are coming from inside the house!"
The first service is a simple flask based application that will be running as a pod inside the cluster exposing the endpoint using a Service and Ingress resources. When you call the "/pod/versions" endpoint, it will return the versions of any applications running in the cluster as JSON. There are some security constraints built into k8s that you should be aware of when trying to access the k8s API internally. We will walk you through how to allow this service to access this API even with Role Based Access Control (RBAC) enabled using a ServiceAccount. This method will only grant this specific service inside a particular namespace read-only access to pod information for the cluster.
The second application will make use of this flask endpoint and be run from your local command. k8s config file to get access. We will then use it to compare a secondary application running in a different namespace. This is a smaller version of some real world tooling we use at Rally Health as we migrate from mesos to k8s and need to compare state between these two environments as well as between clusters in different environments. These techniques are just the tip of the iceberg, but ideally they should give you some idea as to what the kubernetes python client is capable of handling.
By: Erik Johnson
Experience Level: Novice
pudb is a feature-rich terminal-based debugger that is a great alternative to Python's built-in debugger (pdb). This demo will demonstrate how to launch into the debugger, as well as how to use its remote functionality to connect to and troubleshot multi-process apps which do not run in the foreground.