BugForge
All lessons
Python · 03/10 6 min

Iterators, generators and comprehensions

One-shot generators, zip truncation, sorting keys, and changing a collection while looping.

Generators run once

lines = (l.strip() for l in open("app.log"))
errors = [l for l in lines if "ERROR" in l]
count = sum(1 for _ in lines)    # 0: the generator is already exhausted

map, filter, zip, file objects and generator expressions are all one-shot. Wrap in list(...) if you need two passes.

zip stops at the shortest

list(zip([1, 2, 3], ["a", "b"]))           # [(1, 'a'), (2, 'b')]: 3 silently dropped
list(zip([1, 2, 3], ["a", "b"], strict=True))  # ValueError (3.10+)

Indexes and pairs

for i, line in enumerate(lines, start=1):   # human line numbers
for a, b in zip(xs, xs[1:]):                # consecutive pairs
for k, v in d.items():                      # not `for k, v in d`

Sorting

sorted(users, key=lambda u: u.age)                 # new list
users.sort(key=lambda u: (-u.score, u.name))       # in place, returns None!
result = users.sort()                              # ❌ result is None

sorted is stable: equal keys keep their original order, so you can sort by a secondary key first and the primary key second.

Don't change what you're looping over

for x in items:
    if x < 0: items.remove(x)        # ❌ skips the element after each removal
items = [x for x in items if x >= 0] # ✅ build a new list

for k in d:
    if d[k] == 0: del d[k]           # ❌ RuntimeError: dict changed size
for k in list(d): ...                # ✅ iterate over a snapshot

Small but deadly

  • any([]) is False, all([]) is True.
  • range(1, n) stops at n - 1.
  • dict keeps insertion order (3.7+), set doesn't: sort before comparing output.
  • A comprehension with a side effect ([print(x) for x in xs]) is a smell; use a loop.
Practise it now
Next in Python
Exceptions and context managers