All About Iteration

# All About Iteration

## Menno Finlay-Smits

```
hello@menno.io
GitHub: mjs
Mastodon: @menn0@mastodon.nzoss.nz
Twitter: @mjs0
```

---

# Iteration?

Wikipedia says:

- Iteration is the repetition of a process in order to generate a (possibly unbounded) sequence of outcomes. 
- Each repetition of the process is a single iteration.
- The outcome of each iteration is then the starting point of the next iteration.

---

# While loops (for completeness)

Do something while some condition is true.

```python
i = 0
while i < 10:
    print(i)
    i += 1
```

Infinite loop:

```python
while True:
    print("Hello")
```

This is the last time `while` will be mentioned.

---

# for

- Iterating "over" sequences and other objects.
- Often (but not necessarily) collections of things

```python
for item in ["one", "two", "three"]:
    print(item)
```

---
class: middle, inverse

# Built-in types that can be iterated over

---

.left-column[
  ## Lists
]
.right-column[
  ```python
  for item in ["one", "two", "three"]:
     ...
  ```

```
  "one"
  "two",
  "three"
  ```
]

---

.left-column[
  ## Lists
  ## Tuples
]
.right-column[
  ```python
  for item in ("one", "two", "three"):
     ...
  ```

```
  "one"
  "two",
  "three"
  ```
]

---

.left-column[
  ## Lists
  ## Tuples
  ## Strings
]
.right-column[
  ```python
  for char in "hello":` 
     ...
  ```

```
  "h"
  "e",
  "l"
  "l"
  "o"
  ```
]

---

.left-column[
  ## Lists
  ## Tuples
  ## Strings
  ## Files
]
.right-column[
  ```python
  for line in open("file.txt"):
     ...
  ```

```
  "first line\n"
  "second line\n"
  ...
  ```

This works for text files only.
]

---

.left-column[
  ## Lists
  ## Tuples
  ## Strings
  ## Files
  ## Bytes
]
.right-column[
  ```python
  for b in b"hello":
     ...
  ```

Iterating over bytes gives the integer byte values:

```
  104
  101
  108
  108
  111
  ```
]

---

.left-column[
  ## Lists
  ## Tuples
  ## Strings
  ## Files
  ## Bytes
  ## Dictionaries
]
.right-column[
  ```python
    d = {1: "one", 2: "two", 3: "three"}
    for key in d:
       ...
    for key in d.keys():  # same as previous
       ...
    for key in d.values():
       ...
    for key in d.items():
       ...
    ```
]

---

.left-column[
  ## Lists
  ## Tuples
  ## Strings
  ## Files
  ## Bytes
  ## Dictionaries
  ## Sets
]
.right-column[
  ```python
    s = {1, 2, 3}
    for v in s:
       print(v)
  ```

```
  1
  2
  3
  ```

]

---

# Types that can't be iterated over

- Non-collection types like bools, ints and floats

```python
for x in 123:
   ...

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: 'int' object is not iterable
```

- Arbitrary objects

```python
class Foo: pass

for x in Foo():
    ...

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: 'Foo' object is not iterable

```

---

# Iterables and iterators

---

# Iterator

- An object that returns successive values 
- Implements the `__next__` magic method
- Raises `StopIteration` to tell Python there's no more
- This is the "Iterator Protocol"

---

# Iterable

- An object that can return an iterator
- Implements the `__iter__` magic method
- This is the "Iterable Protocol"

The Python runtime calls `__iter__` and then `__next__` on the returned
iterator when asked to iterate over something.

It's common for an object to implement both `__iter__` and `__next__`.

---

# Implementing your own iterable

```python
class Doubler:
    def __init__(self, start, end):
        self.i = start
        self.end = end

def __iter__(self):
        return self  # "just use me as an iterator"

def __next__(self):
        if self.i > self.end:
            raise StopIteration
        out = self.i
        self.i *= 2
        return out

for n in Doubler(2, 60):
    print(n)

# prints: 2 4 8 16 32
```

---

# Iterating Without a For Loop

```python
>>> x = [1, 2, 3]
>>> i = iter(x)    # Get an iterator from x
>>> i.__next__
<method-wrapper '__next__' of list_iterator object at 0x7f2927e16d10>
>>> next(i)
1
>>> next(i)
2
>>> i.__next__()   # Not recommended, next() handles some 
                   # edge cases
3

>>> next(x)        # Can't iterate over an iterable directly
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: 'list' object is not an iterator
```

---

# Generator functions

An easier way to implement many types of iterators.

```python
def doubler(start, end):
    i = start
    while i < end:
        yield i    # Return the next value, execution goes 
                   # back to the caller
        i *= 2

for n in doubler(2, 60):
    print(n)

# prints: 2 4 8 16 32

print(doubler(2, 60))
# prints: <generator object doubler at 0x7f3735972b20>
```

---

# List comprehensions

A convenient way of expressing iterations where the output in a list.

```python
print([x.upper() for x in ["foo", "bar", "qaz"]])
# ["FOO", "BAR", "QAZ"]
```

Is equivalent to:

```python
y = []
for x in ["foo", "bar", "qaz"]:
    y.append(x.upper())
print(y)
```

---

# List comprehension performance

- List comprehensions are usually *faster* 
- Iteration happens inside the Python runtime - implemented in C 
- Not slower execution of multiple Python bytecode instructions
- Around 33% faster in some cases

---

# List comprehensions conditions

A conditional expression may be include to filter out certain values.

```python
print([
    x.upper() for x in ["foo", "foobar", "qaz"] 
    if len(x) == 3
])
# "FOO", "QAZ"]
```

Is equivalent to:

```python
y = []
for x in ["foo", "bar", "qaz"]])
    if len(x) == 3:
        y.append(x.upper())
print(y)
```

---

# Nested list comprehensions

- There can be multiple `for` parts in a single list expression - nested loops!
- Can be hard to read

```python
>>> [(x, y) for x in range(2) for y in range(3)]
[(0, 0), (0, 1), (0, 2), (1, 0), (1, 1), (1, 2)]
```

Note: The `range` built-in function returns a iterator which returns sequences
of numbers.

---

# Generator expressions

- Combines list comprehensions & generator functions
- Results in a generator object (which is an iterator)
- Good for large results which would otherwise eat a lot of memory as a list comprehension
- Surround in parens instead of square brackets

---

# Generator expressions example

```python
>>> g = (x*x for x in range(1, 10))
>>> g
<generator object <genexpr> at 0x7f27b5d0e180>
>>> next(g)
1
>>> next(g)
4
>>> next(g)
9
>>> for y in g:
...     print(y)
16
25
36
49
64
81
```

---

# Infinite iteration

- Iterators don't necessarily have to stop
- This is often fine - something else will stop the iteration

```python
>>> def squares():
        i = 0
        while True:
            yield i*i
            i += 1
>>> for i, v in zip(range(6), squares()):
...     print(i, v)
0 0
1 1
2 4
3 9
4 16
5 25
```

---

# itertools

Standard library package filled with useful goodies.

- `cycle([iterable])` - repeat the given iterable forever
- `repeat(value[, n])` - repeat the value `n` times (or forever)
- `chain(p, q, ...)` - join iterables together
- `dropwhile(pred, iterable)` - start returning values when the function `pred` (predicate) returns false

Many more...

---

# Questions?