Generators and Iterators

Generators and iterators allow Python to produce values one at a time (lazy evaluation) instead of creating entire collections in memory.

This is critical when:

• Data is large (files, streams, APIs)
• Values are infinite or unknown
• You only need partial results

---

Iterables vs. Iterators

Iterable

An iterable is any object that can be looped over.

Examples:

• list
• tuple
• string
• set
• dictionary
• file

nums = [1, 2, 3]
for n in nums:
    print(n)

Note

Internally, Python converts this into an iterator.

---

Iterator

An iterator is an object that:

• Keeps track of position
• Produces next value on demand

---

Iterator Protocol

An object is an iterator if it implements:

• __iter__() → returns iterator object
• __next__() → returns next value
• Raises StopIteration when done

---

graph TD A[Iterable] --> B[iter] B --> C[Iterator] C --> D[next] D --> E[Value] C --> F[StopIteration]

---

Manual Iteration Example

nums = [10, 20, 30]

it = iter(nums)

print(next(it))  # 10
print(next(it))  # 20
print(next(it))  # 30
# next(it) -> StopIteration

---

Custom Iterator

class CountDown:
    def __init__(self, start):
        self.current = start

    def __iter__(self):
        return self

    def __next__(self):
        if self.current <= 0:
            raise StopIteration
        value = self.current
        self.current -= 1
        return value


for num in CountDown(3):
    print(num)

---

Generators

Generators are a simpler way to create iterators.

Instead of writing a class, you use yield.

---

Key Idea

• return → ends function
• yield → pauses function and saves state

---

Generator Function Example

def generate_numbers(n):
    i = 1
    while i <= n:
        yield i
        i += 1


gen = generate_numbers(3)

for num in gen:
    print(num)

---

Execution Flow

graph TD Call --> GeneratorObject GeneratorObject --> StartExecution StartExecution --> Yield Yield --> Pause Pause --> Resume Resume --> Yield Yield --> StopIteration

---

Important Behavior

• Function does NOT execute immediately
• Runs only when next() is called
• State is preserved between calls

---

Generators vs. Lists

# List (memory heavy)
nums = [x*x for x in range(1000000)]

# Generator (memory efficient)
nums = (x*x for x in range(1000000))

Generator:

• Computes values only when needed
• Uses constant memory

---

Generator Expressions

Compact syntax similar to list comprehensions.

gen = (x * 2 for x in range(5))

for val in gen:
    print(val)

---

Infinite Generators

Generators can produce infinite sequences.

Caution

Must be controlled manually.

---

Example: Infinite Counter

def infinite_counter():
    num = 1
    while True:
        yield num
        num += 1


counter = infinite_counter()

for _ in range(5):
    print(next(counter))

---

Output

1
2
3
4
5

---

Mental Model

graph TD Start --> Yield1 Yield1 --> Resume Resume --> Yield2 Resume --> Yield3 Resume --> InfiniteLoop

---

Sending Values into a Generator

Generators can receive input using send().

---

Example

def accumulator():
    total = 0
    while True:
        value = yield total
        total += value


acc = accumulator()

print(next(acc))     # start generator
print(acc.send(10))  # 10
print(acc.send(5))   # 15

---

Key Rule

Note

Always call next() first (called priming)

---

Closing a Generator

When close() is called:

• GeneratorExit is raised inside generator
• You can clean up resources

Why we need to close generators?

Caution

In normal loops, resources are automatically released when the loop ends. But with generators store their state between yields, they can hold onto resources (like file handles, network connections, or memory) for a long time. If you don’t close them properly, you might end up with resource leaks.

---

Example

def resource_manager():
    print("Resource opened")
    count = 0

    try:
        while True:
            yield count
            count += 1
    except GeneratorExit:
        print(f"Cleaning up... used {count} times")


gen = resource_manager()

print(next(gen))
print(next(gen))

gen.close()

---

Output

Resource opened
0
1
Cleaning up... used 2 times

---

Flow

graph TD Start --> Yield Yield --> Resume Resume --> Yield Close --> GeneratorExit GeneratorExit --> Cleanup Cleanup --> Stop

---

Interactive Generator Example

def order_processor():
    total_orders = 0
    print("System Ready")

    try:
        order = yield

        while True:
            print(f"Processing: {order}")
            total_orders += 1
            order = yield

    except GeneratorExit:
        print(f"System Shutdown. Total orders: {total_orders}")


processor = order_processor()

next(processor)  # start

processor.send("Pizza")
processor.send("Burger")

processor.close()

---

yield from Delegation

Used to simplify nested generators

---

Without yield from

def chain():
    for x in [1, 2]:
        yield x
    for y in [3, 4]:
        yield y

---

With yield from

def chain():
    yield from [1, 2]
    yield from [3, 4]