Python Pandas

Learning Objectives

Pandas is designed for working with structured and tabular data. It is widely used by data analysts, data scientists and business intelligence professionals.

What Is Pandas?

Pandas is a Python library designed for data manipulation and analysis.

It provides powerful structures for working with structured data, especially tables containing rows and columns.

Raw Data
    ↓
Pandas DataFrame
    ↓
Clean
    ↓
Transform
    ↓
Analyze
    ↓
Business Insights

Why Pandas Matters

Real-world data rarely arrives perfectly prepared. Businesses commonly store information in spreadsheets, CSV files, databases and APIs.

Pandas gives you tools to inspect, clean, transform and analyze this information efficiently.

Customer Data
Sales Data
Inventory Data
Marketing Data
       ↓
     Pandas
       ↓
Clean + Transform + Analyze
       ↓
Business Decisions

Installing Pandas

Install Pandas with pip.

python -m pip install pandas

If you are using a virtual environment, activate it before installing Pandas.

python -m pip show pandas

Importing Pandas

The standard convention is to import Pandas using the alias pd.

import pandas as pd

print(pd.__version__)

Pandas Series

A Series is a one-dimensional labeled data structure.

import pandas as pd

sales = pd.Series([
    1200,
    1500,
    900,
    1800
])

print(sales)

0    1200
1    1500
2     900
3    1800
dtype: int64

Notice that Pandas automatically provides an index.

You can also provide custom labels.

sales = pd.Series(
    [1200, 1500, 900],
    index=[
        "Monday",
        "Tuesday",
        "Wednesday"
    ]
)

print(sales)

DataFrames

A DataFrame is a two-dimensional labeled data structure consisting of rows and columns.

import pandas as pd

data = {
    "Product": [
        "Laptop",
        "Phone",
        "Tablet"
    ],
    "Price": [
        1200,
        800,
        500
    ],
    "Quantity": [
        2,
        5,
        3
    ]
}

df = pd.DataFrame(data)

print(df)

  Product  Price  Quantity
0  Laptop   1200         2
1   Phone    800         5
2  Tablet    500         3

A DataFrame is one of the most important structures you will use as a Python data analyst.

Working with Columns

Select a column using its name.

print(df["Product"])

You can select multiple columns by providing a list.

selected = df[
    ["Product", "Price"]
]

print(selected)

A calculated column can also be created.

df["Total"] = (
    df["Price"] *
    df["Quantity"]
)

print(df)

  Product  Price  Quantity  Total
0  Laptop   1200         2   2400
1   Phone    800         5   4000
2  Tablet    500         3   1500

Working with Rows

Pandas provides several ways to access rows.

print(df.iloc[0])
print(df.iloc[1])

You can also select multiple rows.

print(df.iloc[0:2])

Later we will use loc[] and iloc[] to select rows and columns more precisely.

Using loc[]

loc[] selects data using labels.

import pandas as pd

df = pd.DataFrame({
    "Name": [
        "Ali",
        "Sara",
        "Ahmed"
    ],
    "Marks": [
        85,
        92,
        78
    ]
})

print(df.loc[0])

print(
    df.loc[0, "Name"]
)

Name     Ali
Marks     85
Name: 0, dtype: object

Ali

loc[] is especially useful when working with meaningful row labels.

Using iloc[]

iloc[] selects data using integer positions.

print(df.iloc[0])
print(df.iloc[0, 1])

head() and tail()

When working with a large dataset, you do not want to print every row immediately.

print(df.head())

head() displays the first five rows by default.

print(df.tail())

tail() displays the last five rows by default.

You can specify the number of rows.

print(df.head(10))
print(df.tail(3))

info() and describe()

Two extremely useful functions for quickly understanding a dataset are info() and describe().

df.info()

info() shows information such as column names, non-null values and data types.

df.describe()

describe() provides statistical summaries for numerical columns.

Understanding Data Types

Use dtypes to inspect the data types of DataFrame columns.

print(df.dtypes)

You can change a column's type with methods such as astype().

df["Marks"] = df[
    "Marks"
].astype(float)

print(df.dtypes)

Filtering Data

Filtering is one of the most important Pandas skills for data analysis.

import pandas as pd

df = pd.DataFrame({
    "Product": [
        "Laptop",
        "Phone",
        "Tablet",
        "Monitor"
    ],
    "Price": [
        1200,
        800,
        500,
        300
    ]
})

expensive = df[
    df["Price"] > 700
]

print(expensive)

  Product  Price
0  Laptop   1200
1   Phone    800

Multiple conditions can be combined using & and |.

result = df[
    (df["Price"] > 500) &
    (df["Price"] < 1300)
]

print(result)

Sorting Data

Use sort_values() to sort rows according to a column.

sorted_df = df.sort_values(
    "Price"
)

print(sorted_df)

To sort from highest to lowest:

sorted_df = df.sort_values(
    "Price",
    ascending=False
)

print(sorted_df)

You can sort by multiple columns as well.

df.sort_values(
    ["Price", "Product"]
)

Handling Missing Data

Real-world datasets often contain missing values. Pandas represents missing data using values such as NaN or NA, depending on the data and operation.

print(df.isna())

Count missing values in each column:

print(
    df.isna().sum()
)

You can remove rows containing missing values:

cleaned = df.dropna()

Or replace missing values:

df["Price"] = df[
    "Price"
].fillna(0)

Finding Duplicate Data

Duplicate records can distort analytical results.

duplicates = df.duplicated()

print(duplicates)

Remove duplicate rows with:

df = df.drop_duplicates()

You can also check duplicates based on selected columns.

df.duplicated(
    subset=["Product"]
)

Creating New Columns

Data analysis often requires calculated columns.

df["Revenue"] = (
    df["Price"] *
    df["Quantity"]
)

You can also create columns using conditions.

df["Category"] = "Regular"

df.loc[
    df["Price"] >= 1000,
    "Category"
] = "Premium"

print(df)

This is a useful pattern when creating business categories.

Using groupby()

groupby() is one of the most powerful Pandas features for business analysis.

import pandas as pd

sales = pd.DataFrame({
    "Category": [
        "Electronics",
        "Electronics",
        "Clothing",
        "Clothing"
    ],
    "Sales": [
        1200,
        1800,
        700,
        900
    ]
})

result = sales.groupby(
    "Category"
)["Sales"].sum()

print(result)

Category
Clothing       1600
Electronics    3000
Name: Sales, dtype: int64

This allows you to answer questions such as:

• Which category generated the most sales?
• What is the average revenue per region?
• How many orders came from each channel?

Aggregation

Pandas supports multiple aggregation functions.

summary = sales.groupby(
    "Category"
)["Sales"].agg([
    "sum",
    "mean",
    "min",
    "max"
])

print(summary)

You can also perform multiple aggregations across different columns.

summary = df.groupby(
    "Category"
).agg({
    "Sales": ["sum", "mean"],
    "Quantity": "sum"
})

Combining DataFrames

Real-world projects often contain multiple datasets. Pandas provides several tools for combining them.

concat()

combined = pd.concat([
    df1,
    df2
])

merge()

result = pd.merge(
    customers,
    orders,
    on="CustomerID"
)

merge() is conceptually similar to a database JOIN.

Reading CSV Files

CSV files are extremely common in business and data-analysis work.

import pandas as pd

df = pd.read_csv(
    "sales.csv"
)

print(df.head())

You can inspect the imported dataset using:

print(df.info())
print(df.describe())

CSV
 ↓
read_csv()
 ↓
Inspect
 ↓
Clean
 ↓
Transform
 ↓
Analyze

Reading Excel Files

Pandas can also read Excel workbooks.

import pandas as pd

df = pd.read_excel(
    "sales.xlsx"
)

print(df.head())

Depending on your environment and workbook format, an appropriate Excel engine may need to be installed.

python -m pip install openpyxl

You can select a specific worksheet when necessary.

df = pd.read_excel(
    "sales.xlsx",
    sheet_name="Sales"
)

Exporting Data

After cleaning or analyzing a dataset, you may want to save the result.

Export to CSV

df.to_csv(
    "clean_sales.csv",
    index=False
)

Export to Excel

df.to_excel(
    "clean_sales.xlsx",
    index=False
)

Practical Example — Ecommerce Sales Analysis

Let's build a small ecommerce dataset and analyze it.

import pandas as pd

sales = pd.DataFrame({
    "Product": [
        "Laptop",
        "Phone",
        "Laptop",
        "Tablet",
        "Phone",
        "Monitor"
    ],

    "Category": [
        "Electronics",
        "Electronics",
        "Electronics",
        "Electronics",
        "Electronics",
        "Electronics"
    ],

    "Region": [
        "Punjab",
        "Sindh",
        "Punjab",
        "KPK",
        "Sindh",
        "Punjab"
    ],

    "Sales": [
        2400,
        1600,
        1800,
        1200,
        2000,
        900
    ]
})

print(sales)

Total Sales

total_sales = sales["Sales"].sum()

print(
    "Total sales:",
    total_sales
)

Total sales: 9900

Average Sales

average_sales = sales[
    "Sales"
].mean()

print(
    "Average:",
    average_sales
)

Average: 1650.0

High-Value Orders

high_value = sales[
    sales["Sales"] > 1500
]

print(high_value)

Sales by Region

regional_sales = sales.groupby(
    "Region"
)["Sales"].sum()

print(regional_sales)

This small example demonstrates the type of workflow used in real business analysis.

Dataset
   ↓
Inspect
   ↓
Filter
   ↓
Calculate
   ↓
Group
   ↓
Analyze
   ↓
Insight

Common Pandas Mistakes

df = pd.DataFrame(data)

Practice Exercises

Mini Challenge — Ecommerce Sales Dashboard Dataset

Imagine you have been given an ecommerce sales dataset. Your job is to prepare it for a management dashboard.

Starter Code

import pandas as pd

sales = pd.DataFrame({
    "Product": [
        "Laptop",
        "Phone",
        "Tablet",
        "Monitor",
        "Keyboard",
        "Phone"
    ],

    "Region": [
        "Punjab",
        "Sindh",
        "Punjab",
        "KPK",
        "Balochistan",
        "Sindh"
    ],

    "Quantity": [
        2,
        5,
        3,
        4,
        10,
        2
    ],

    "Sales": [
        2400,
        4000,
        1500,
        1200,
        1000,
        1600
    ]
})

print(sales.head())

sales["Revenue"] = (
    sales["Sales"] *
    sales["Quantity"]
)

high_value = sales[
    sales["Revenue"] > 5000
]

regional = sales.groupby(
    "Region"
)["Revenue"].sum()

print("Total revenue:")
print(sales["Revenue"].sum())

print("Average revenue:")
print(sales["Revenue"].mean())

print("High-value sales:")
print(high_value)

print("Regional revenue:")
print(regional)

sales.to_csv(
    "clean_sales.csv",
    index=False
)

Knowledge Check

Lesson Summary

Pandas is one of the most important tools for practical Python data analysis. It allows you to work with structured datasets using powerful and readable operations.

You now have the foundation for working with real-world datasets in Python. In the next lesson, we will focus specifically on one of the most important skills for a data analyst: Data Cleaning.