Selecting and Filtering: loc, iloc and query

Master pandas selection with loc, iloc, boolean masks, isin, between and query, and learn how to update rows safely without chained assignment.

• Beginner
• 16 min read
• 4 objectives

Before this lessonLesson 2: Reading and Writing Data: CSV, Excel, JSON, SQL, Parquet

The introduction lesson showed the basics: df["col"] for a column and a boolean mask for filtering. That gets you surprisingly far, but as soon as you want specific rows and specific columns at once, or you want to change values in only some rows, you need the two workhorses of pandas selection: .loc and .iloc.

Getting selection right matters for more than tidiness. Most confusing pandas bugs (values that do not update, warnings about copies, off-by-one slices) come from mixing up labels and positions. Once the mental model clicks, the rest of the library feels much more predictable.

Labels vs positions

Every DataFrame has an index (row labels) and columns (column labels). .loc selects by those labels. .iloc selects by integer position, like a Python list, no matter what the labels are. The syntax for both is [rows, columns].

import pandas as pd

df = pd.DataFrame(
    {"team": ["core", "web", "core", "data"],
     "salary": [120, 95, 130, 110],
     "years": [5, 2, 8, 4]},
    index=["ada", "grace", "linus", "margaret"],
)

print(df.loc["grace", "salary"])            # one value by label
print(df.loc[["ada", "linus"], ["team", "years"]])
print(df.iloc[0, 1])                        # first row, second column
print(df.iloc[-1])                          # last row as a Series

95
       team  years
ada    core      5
linus  core      8
120
team      data
salary     110
years        4
Name: margaret, dtype: object

Slicing: the end is included with loc

Here is the classic trap. Label slices with .loc include the end label, because labels are not necessarily ordered numbers and "up to and including" is what people mean. Position slices with .iloc follow Python rules and exclude the end.

import pandas as pd

df = pd.DataFrame(
    {"salary": [120, 95, 130, 110], "years": [5, 2, 8, 4]},
    index=["ada", "grace", "linus", "margaret"],
)

print(df.loc["grace":"linus"])   # includes linus
print(df.iloc[1:3])              # rows 1 and 2, not 3
print(df.loc[:, "salary":"years"].shape)

       salary  years
grace      95      2
linus     130      8
       salary  years
grace      95      2
linus     130      8
(4, 2)

Boolean filters, combined

A condition like df["salary"] > 100 produces a Series of True and False. Put it inside .loc to keep matching rows, and add a column list to pick columns at the same time. Combine conditions with & (and), | (or) and ~ (not), and wrap each condition in parentheses because those operators bind tighter than comparisons.

import pandas as pd

df = pd.DataFrame({
    "name": ["Ada", "Grace", "Linus", "Margaret", "Ken"],
    "team": ["core", "web", "core", "data", "web"],
    "salary": [120, 95, 130, 110, 88],
    "years": [5, 2, 8, 4, 1],
})

senior_core = df.loc[(df["team"] == "core") & (df["years"] >= 5), ["name", "salary"]]
print(senior_core)

print(df.loc[df["team"].isin(["web", "data"]), "name"].tolist())
print(df.loc[df["salary"].between(90, 115), "name"].tolist())
print(df.loc[~(df["team"] == "web"), "name"].tolist())

    name  salary
0    Ada     120
2  Linus     130
['Grace', 'Margaret', 'Ken']
['Grace', 'Margaret']
['Ada', 'Linus', 'Margaret']

isin replaces a long chain of | conditions, and between is an inclusive range check. Both make filters shorter and harder to get wrong.

query: filters that read like English

When conditions pile up, the brackets get noisy. df.query() takes the condition as a string and lets you use column names directly, plus and, or and not. Reference a Python variable with @.

import pandas as pd

df = pd.DataFrame({
    "name": ["Ada", "Grace", "Linus", "Margaret", "Ken"],
    "team": ["core", "web", "core", "data", "web"],
    "salary": [120, 95, 130, 110, 88],
})

min_salary = 100
print(df.query("salary >= @min_salary and team != 'web'"))
print(df.query("team in ['web', 'data']").shape)

       name  team  salary
0       Ada  core     120
2     Linus  core     130
3  Margaret  data     110
(3, 3)

query is great for exploration and readable pipelines. Column names with spaces need backticks, for example df.query("`unit price` > 5").

Updating rows safely

To change values in some rows, do the selection and the assignment in one .loc call. A common beginner pattern is chained indexing like df[df["team"] == "web"]["salary"] = 100. The first bracket returns a new object, and the assignment modifies that temporary copy, not df. Since pandas 3.0 turned on Copy-on-Write by default, this pattern never updates the original.

import pandas as pd

df = pd.DataFrame({
    "name": ["Ada", "Grace", "Linus", "Ken"],
    "team": ["core", "web", "core", "web"],
    "salary": [120, 95, 130, 88],
})

# Right: one .loc call with a mask and a column
df.loc[df["team"] == "web", "salary"] += 10
df.loc[df["name"] == "Ada", ["team", "salary"]] = ["lead", 140]
print(df)

    name  team  salary
0    Ada  lead     140
1  Grace   web     105
2  Linus  core     130
3    Ken   web      98

The same rule applies when you take a subset to work on later. If you filter a DataFrame and then add columns to the result, call .copy() to make it clear you want an independent table: web = df[df["team"] == "web"].copy().

Fast single values: at and iat

For reading or writing exactly one cell inside a loop, .at[row, col] (labels) and .iat[i, j] (positions) skip some of the overhead of .loc/.iloc. You will rarely need them if you stick to vectorized code, but they are handy in small scripts.

import pandas as pd

df = pd.DataFrame({"salary": [120, 95]}, index=["ada", "grace"])
df.at["grace", "salary"] = 99
print(df.iat[1, 0])

99

Recap

• .loc uses labels and includes the end of a slice; .iloc uses positions and excludes it.
• Both take [rows, columns], so you can filter rows and pick columns in one step.
• Combine conditions with &, |, ~ and parentheses, or use isin, between and query.
• Update with a single df.loc[mask, col] = value; chained indexing never writes back.
• Use .copy() when you keep a filtered subset to modify later.

# Write your solution here