Pandas · Lesson 1 of 4

DataFrames and Series

Load data, inspect it and select rows and columns.

Beginner
14 min read
3 objectives

What you will learn

Create and load DataFrames
Inspect with head/info/describe
Select with loc and iloc

pandas is Python's standard library for working with tables of data: spreadsheets, CSV files, database results. Its two core types are the Series (one column of labeled values) and the DataFrame (a table of Series sharing the same row index). If you can picture a spreadsheet, you can picture a DataFrame.

pip install pandas

Creating a DataFrame

import pandas as pd

df = pd.DataFrame({
    "name": ["Ada", "Linus", "Grace", "Alan"],
    "country": ["UK", "FI", "US", "UK"],
    "age": [36, 28, 45, 41],
    "salary": [120, 95, 150, 110],
})
print(df)

Output

    name country  age  salary
0    Ada      UK   36     120
1  Linus      FI   28      95
2  Grace      US   45     150
3   Alan      UK   41     110

Loading real data

df = pd.read_csv("employees.csv")
df = pd.read_excel("report.xlsx")       # needs openpyxl
df = pd.read_json("data.json")
# from a database: pd.read_sql("SELECT * FROM users", connection)

First look

df.head()          # first 5 rows
df.shape           # (rows, columns)
df.info()          # column names, types, non-null counts
df.describe()      # count, mean, std, min, quartiles, max for numbers
df["country"].value_counts()   # frequency of each value

Always do this before anything else. Surprises such as unexpected types, empty columns or odd ranges show up immediately.

Selecting columns and rows

df["name"]                      # one column -> Series
df[["name", "salary"]]            # several -> DataFrame
df.loc[0]                         # a row by label
df.loc[0:1, "name":"age"]         # label slices are inclusive
df.iloc[0:2, 0:2]                 # by integer position, end exclusive

Filtering with boolean masks

uk = df[df["country"] == "UK"]
rich_uk = df[(df["country"] == "UK") & (df["salary"] > 115)]   # & | ~, with parentheses!
names = df[df["age"] > 30]["name"]
df.query("age > 30 and country == 'UK'")

New columns, vectorized

Operate on whole columns at once instead of looping; it is far faster and shorter.

df["salary_k"] = df["salary"] * 1000
df["senior"] = df["age"] >= 40
df["name_upper"] = df["name"].str.upper()
df = df.sort_values("salary", ascending=False)

# Write your solution here

Up next · Lesson 2Cleaning DataMissing values, duplicates, types and text cleanup.