Codectionary / Developer documentation / Python

Generators (yield)

A generator is a special kind of function that produces a sequence of values lazily, one at a time, using the yield keyword instead of return. Each call to yield pauses the function, remembering its state, and resumes exactly where it left off when the next value is requested. This makes generators highly memory-efficient for large or infinite sequences, since values are produced on demand rather than all at once.

Syntax

def gen():\n    yield value

Examples

Basic Generator Function

A function that yields multiple values over several calls instead of returning once.

def count_up_to(n):
    i = 1
    while i <= n:
        yield i
        i += 1

for number in count_up_to(5):
    print(number)  # 1, 2, 3, 4, 5

Generator Expression

A compact, lazy alternative to a list comprehension, using parentheses instead of square brackets.

squares = (n ** 2 for n in range(1, 6))
print(squares)          # <generator object ...>

for square in squares:
    print(square)  # 1, 4, 9, 16, 25 - computed lazily, one at a time

Using next() Manually

Manually pulling values from a generator one at a time with next().

def simple_gen():
    yield "first"
    yield "second"
    yield "third"

gen = simple_gen()
print(next(gen))  # first
print(next(gen))  # second
print(next(gen))  # third
# print(next(gen))  # would raise StopIteration - no more values

Memory-Efficient Large Sequences

A key advantage of generators: processing huge sequences without holding them all in memory at once.

def read_large_file_lines(filepath):
    with open(filepath) as f:
        for line in f:
            yield line.strip()

# Only one line is held in memory at a time, no matter how large the file is
# for line in read_large_file_lines("huge_log.txt"):
#     process(line)

Best practices

  • Use generators instead of building a full list in memory when working with large datasets or infinite sequences
  • Remember a generator can only be iterated once - once exhausted, you need to call the generator function again to start over
  • Use generator expressions (parentheses) instead of list comprehensions when you only need to loop through the values once
  • Combine generators with functions like sum(), max(), or any() for memory-efficient aggregation over large sequences

At a glance

Purpose
Scripting and general-purpose applications
File extension
.py
Runs in
Python interpreter
Usually used with
Python standard library and packages

Specifications & further reading

Related Python documentation

def
The def keyword in Python is used to define functions - reusable blocks of code that perform specific tasks. Functions are fundamental to organizing code, avoiding repetition, and making programs more maintainable. A function can accept inputs (parameters), perform operations, and return outputs. Functions are one of the core building blocks of clean, modular Python code.lambda
A lambda is a small, anonymous, single-expression function, created without the def keyword or a name. Lambdas are restricted to a single expression whose result is automatically returned - they can't contain multiple statements or assignments. They're most useful as short, throwaway functions passed as arguments to other functions like sorted(), map(), and filter().return
The return statement exits a function immediately and optionally sends a value back to the caller. A function can return any type - including multiple values as a tuple - or nothing at all, in which case it implicitly returns None. Once return executes, no further code in the function runs, making it useful for early exits as well as producing a final result.*args and **kwargs
*args and **kwargs let a function accept a variable number of arguments. *args collects any extra positional arguments into a tuple, while **kwargs collects extra keyword arguments into a dictionary. The names 'args' and 'kwargs' are just convention - what matters is the * and ** prefixes. This pattern is common in wrapper functions, decorators, and APIs that need to stay flexible about their inputs.