Python Generators

Learning Objectives

By the end of this lesson, you will understand how generators work and why they are extremely useful when processing data efficiently.

What Is a Generator?

A generator is a special type of iterator that produces values one at a time instead of creating all values at once.

Generators are created using functions containing the yield keyword.

def numbers():

    yield 1
    yield 2
    yield 3


for number in numbers():
    print(number)

1
2
3

Unlike a normal function, the generator does not execute completely when it is called. It produces each value when requested.

The yield Keyword

The yield keyword is what makes a function a generator function.

def generate_numbers():

    yield 10
    yield 20
    yield 30

Each yield produces one value.

When Python reaches a yield, execution is temporarily paused. When the generator is requested again, execution continues from that point.

Generator Functions

A function containing yield becomes a generator function.

def countdown():

    yield 3
    yield 2
    yield 1


generator = countdown()

print(generator)

Calling the function does not immediately execute all of its code.

Instead, Python creates a generator object.

Using next()

You can manually retrieve values from a generator using the built-in next() function.

def numbers():

    yield 10
    yield 20
    yield 30


generator = numbers()

print(next(generator))
print(next(generator))
print(next(generator))

10
20
30

After the final value has been produced, another next() call raises StopIteration.

Generators with for Loops

In normal Python programs, generators are often consumed with for loops.

def even_numbers():

    for number in range(2, 11, 2):
        yield number


for number in even_numbers():
    print(number)

2
4
6
8
10

The for loop automatically requests values from the generator until the generator is exhausted.

yield vs return

Understanding the difference between these two keywords is essential.

def get_number():

    return 10

    # Code after return
    # will not execute

def get_numbers():

    yield 10
    yield 20
    yield 30

Generator State

One of the most powerful features of generators is that they remember their execution state.

def demo():

    print("Step 1")
    yield 100

    print("Step 2")
    yield 200

    print("Step 3")
    yield 300


generator = demo()

print(next(generator))
print(next(generator))
print(next(generator))

Step 1
100
Step 2
200
Step 3
300

Notice that the generator does not restart from the beginning after every next() call.

Memory Efficiency

One of the biggest advantages of generators is their ability to produce values lazily rather than storing an entire sequence in memory.

Consider generating one million numbers.

numbers = [number for number in range(1000000)]

The list stores all one million values.

numbers = (number for number in range(1000000))

The generator produces values when they are requested.

Processing Large Data

Imagine processing millions of records from a data source.

Loading everything into a list may require a large amount of memory.

A generator can instead produce records one at a time.

def generate_orders():

    for order_id in range(1, 1000001):
        yield {
            "order_id": order_id
        }


for order in generate_orders():

    if order["order_id"] == 5:
        print(order)

The generator does not need to create a million dictionaries before processing begins.

Generator Expressions

Python provides a shorter syntax for creating simple generators called a generator expression.

The syntax looks similar to a list comprehension, but uses parentheses instead of square brackets.

numbers = (number * 2 for number in range(5))

for number in numbers:
    print(number)

0
2
4
6
8

The expression produces values lazily.

List vs Generator

Lazy Evaluation

Generators use a concept called lazy evaluation.

Lazy evaluation means that a value is calculated only when it is needed.

def numbers():

    print("Generating 1")
    yield 1

    print("Generating 2")
    yield 2

    print("Generating 3")
    yield 3


generator = numbers()

print("Generator created")

print(next(generator))
print(next(generator))

Generator created
Generating 1
1
Generating 2
2

Notice that the third value is never generated because it was never requested.

send()

Advanced generators can receive values from the caller using send().

def receiver():

    while True:

        value = yield

        print("Received:", value)


generator = receiver()

next(generator)

generator.send("Hello")
generator.send("Python")

Received: Hello
Received: Python

close()

The close() method tells a generator to stop execution.

def numbers():

    try:

        yield 1
        yield 2
        yield 3

    finally:

        print("Generator closed")


generator = numbers()

print(next(generator))

generator.close()

1
Generator closed

The finally block is useful for cleanup when a generator is closed.

throw()

The throw() method allows an exception to be injected into a generator at its current pause point.

def demo():

    try:

        yield "Waiting..."

    except ValueError:

        yield "ValueError received"


generator = demo()

print(next(generator))

print(generator.throw(ValueError))

Waiting...
ValueError received

This is an advanced feature and is less common in everyday beginner-level Python code.

Practical Generator Examples

Example 1 — Even Numbers

def even_numbers(limit):

    for number in range(2, limit + 1, 2):
        yield number


for number in even_numbers(10):
    print(number)

Example 2 — Squares

def squares(limit):

    for number in range(1, limit + 1):
        yield number ** 2


for square in squares(5):
    print(square)

1
4
9
16
25

Example 3 — Reading Lines

Files are naturally iterable, and generators can be used to transform or filter file data.

def valid_lines(lines):

    for line in lines:

        line = line.strip()

        if line:
            yield line


data = [
    "Ali",
    "",
    "Sara",
    "Ahmed"
]

for line in valid_lines(data):
    print(line)

Example 4 — Data Pipeline

def numbers():

    for number in range(1, 11):
        yield number


def even_numbers(values):

    for number in values:

        if number % 2 == 0:
            yield number


def squares(values):

    for number in values:
        yield number ** 2


pipeline = squares(
    even_numbers(
        numbers()
    )
)


for value in pipeline:
    print(value)

4
16
36
64
100

This demonstrates how generators can be chained together to create efficient data-processing pipelines.

Common Mistakes

Practice Exercises

Mini Challenge — Sales Data Generator

Build a generator that processes sales records one at a time.

sales = [
    {"product": "Laptop", "amount": 1200},
    {"product": "Phone", "amount": 800},
    {"product": "Tablet", "amount": 500},
    {"product": "Monitor", "amount": 300},
    {"product": "Keyboard", "amount": 100}
]


def high_value_sales(records, minimum):

    for sale in records:

        if sale["amount"] >= minimum:
            yield sale


for sale in high_value_sales(sales, 500):
    print(sale)

{'product': 'Laptop', 'amount': 1200}
{'product': 'Phone', 'amount': 800}
{'product': 'Tablet', 'amount': 500}

Your Tasks

1. Create a generator that yields only sales above a specified amount.
2. Create another generator that calculates a discount for each sale.
3. Create a generator that yields only products with an amount greater than 700.
4. Chain two generators together.
5. Convert the final generator to a list when you actually need all results at once.

Knowledge Check

Test your understanding of Python generators.

Lesson Summary