Codectionary / Developer documentation / Python

Generators (yield)

A generator is a special kind of function that produces a sequence of values lazily, one at a time, using the yield keyword instead of return. Each call to yield pauses the function, remembering its state, and resumes exactly where it left off when the next value is requested. This makes generators highly memory-efficient for large or infinite sequences, since values are produced on demand rather than all at once.

Syntax

def gen():\n    yield value

Examples

Basic Generator Function

A function that yields multiple values over several calls instead of returning once.

def count_up_to(n):
    i = 1
    while i <= n:
        yield i
        i += 1

for number in count_up_to(5):
    print(number)  # 1, 2, 3, 4, 5

Generator Expression

A compact, lazy alternative to a list comprehension, using parentheses instead of square brackets.

squares = (n ** 2 for n in range(1, 6))
print(squares)          # <generator object ...>

for square in squares:
    print(square)  # 1, 4, 9, 16, 25 - computed lazily, one at a time

Using next() Manually

Manually pulling values from a generator one at a time with next().

def simple_gen():
    yield "first"
    yield "second"
    yield "third"

gen = simple_gen()
print(next(gen))  # first
print(next(gen))  # second
print(next(gen))  # third
# print(next(gen))  # would raise StopIteration - no more values

Memory-Efficient Large Sequences

A key advantage of generators: processing huge sequences without holding them all in memory at once.

def read_large_file_lines(filepath):
    with open(filepath) as f:
        for line in f:
            yield line.strip()

# Only one line is held in memory at a time, no matter how large the file is
# for line in read_large_file_lines("huge_log.txt"):
#     process(line)

Best practices

  • Use generators instead of building a full list in memory when working with large datasets or infinite sequences
  • Remember a generator can only be iterated once - once exhausted, you need to call the generator function again to start over
  • Use generator expressions (parentheses) instead of list comprehensions when you only need to loop through the values once
  • Combine generators with functions like sum(), max(), or any() for memory-efficient aggregation over large sequences

Runtime & versions

Python interpreter. Feature availability depends on the language and runtime version used by your project; consult the official reference below.

Python language reference

Specifications & further reading