Pandas

Pandas is a Python library for data analysis. Started by Wes McKinney in 2008 out of a need for a powerful and flexible quantitative analysis tool, pandas has grown into one of the most popular Python libraries. It has an extremely active community of contributors.

Pandas is built on top of two core Python libraries—matplotlib for data visualization and NumPy for mathematical operations. Pandas acts as a wrapper over these libraries, allowing you to access many of matplotlib's and NumPy's methods with less code. For instance, pandas' .plot() combines multiple matplotlib methods into a single method, enabling you to plot a chart in a few lines.

Before pandas, most analysts used Python for data munging and preparation, and then switched to a more domain specific language like R for the rest of their workflow. Pandas introduced two new types of objects for storing data that make analytical tasks easier and eliminate the need to switch tools: Series, which have a list-like structure, and DataFrames, which have a tabular structure.

Pandas tutorials

Here are some analysis-focused pandas tutorials that aren't riddled with technical jargon.

• Pandas cookbook (Julia Evans) - This tutorial uses real-world data and presents a problem to solve or question to answer in every example. Great for putting pandas' capabilities in context of the actual analytical workflow.
• Practical Data Analysis with Python (Anita Raichand) - Provides code examples for four specific analytical tasks: data munging, aggregation, visualization, and time series analysis.
• [VIDEO SERIES] Easier data analysis in Python with pandas (Data School) - A series of video tutorials for pandas newbies who know some Python. Each video answers a student-posed question using real-world data.
• An Introduction to Pandas (Michael Hansen) - This tutorial covers the basics of pandas with a complete analysis of weather data—from reading in data to creating charts.
• Modern Pandas (Tom Augspurger) - An intermediate tutorial for experienced Python users looking to stay sharp on pandas.

Pandas data structures

Series

You can think of a series as a single column of data. Each value in the series has a label, and these labels are collectively referred to as an index. This is demonstrated in the output below. 0-4 is the index and the column of numbers to the right contain the values.

0    22
1    27
2    31
3    33
4    34

DataFrames

While series are useful, most analysts work with the majority of their data in DataFrames. DataFrames store data in the familiar table format of rows and columns, much like a spreadsheet or database. DataFrames makes a lot of analytical tasks easier, such as finding the averages per column in a dataset.

You can also think of DataFrames as a collection of series—just as multiple columns combined make up a table, multiple series make up a DataFrame.

       home_page_visits  like_messages  messages  searches                
0                784            492       292       102   
1                793            500       287       106   
2                253            172       110        40   
3                134             95        55        33   
4                501            331       182       119

Note: In Mode, the results of your SQL queries are automatically converted into DataFrames and made available in the list variable "datasets." To describe or transform the results of Query 1, use datasets[0], for the results of Query 2, use datasets[1] and so on.

For more on manipulating pandas data structures, check out Greg Reda's three-part tutorial, which approaches the topic from a SQL perspective.

Pandas features

Time series analysis

• Time Series / Date functionality (Official Pandas Documentation)
• Times series analysis with pandas (EarthPy)
• Timeseries with pandas (Jupyter)
• Complete guide to create a Time Series Forecast (with Codes in Python) (Analytics Vidhya)

split-apply-combine

Split-apply-combine is a common strategy used during analysis to summarize data—you split data into logical subgroups, apply some function to each subgroup, and stick the results back together again. In pandas, this is accomplished using the groupby() function and whatever functions you want to apply to the subgroups.

• Group By: split-apply-combine (Official Pandas Documentation)
• Summarizing Data in Python with Pandas (Brian Connelly)
• Using Pandas: Split-Apply-Combine (Duke University)

Data visualization

• Visualization (Official Pandas Documentation)
• Simple Graphing with IPython and Pandas (Chris Moffitt)
• Beautiful Plots With Pandas and Matplotlib (The Data Science Lab)

Pivot tables

• Reshaping and Pivot Tables (Official Pandas Documentation)
• Pandas Pivot Table Explained (Chris Moffitt)
• Pivot Tables in Python (O'Reilly)

Working with missing data

• Working with missing data (Official Pandas Documentation)
• Handling missing data (O'Reilly)