Chapter 3: Working with Data Structures

This chapter covers Python’s built-in data structures: lists, tuples, dictionaries, and sets, plus bytes and bytearray for raw binary data. You’ll need them whether you’re storing a list of favorite pizza toppings or building a complex data model.

In this chapter you will learn:

• How to create, change, sort, and copy lists
• How to use tuples for fixed data, unpack them, and return several values from a function
• How to store and look up values by key with dictionaries
• How to remove duplicates and compare groups of items with sets and frozen sets
• How to work with raw binary data using bytes and bytearray
• How to choose the right structure for the job

1. Lists

A list is an ordered, mutable collection. It can hold elements of any data type, and one list can mix several types.

1.1. Defining and initializing lists

Write a list as items separated by commas inside square brackets []:

# A simple list of integers
my_list = [1, 2, 3, 4]

# A mixed list (it can even hold another list)
mixed_list = [42, "hello", 3.14, True, [1, 2, 3]]

# An empty list, ready to be filled later
empty_list = []

print(my_list)  # Output: [1, 2, 3, 4]
print(mixed_list)  # Output: [42, 'hello', 3.14, True, [1, 2, 3]]
print(len(my_list))  # Output: 4
print(len(empty_list))  # Output: 0

1.2. Accessing and slicing

List indexes start at 0, so the first item is at index 0. Negative indexes count from the end, so -1 is the last item. To get several items at once, take a slice with my_list[start:stop:step]. The item at the stop index is not included.

my_list = [1, 2, 3, 4]

# Access the third element (index 2)
print(my_list[2])  # Output: 3

# Access the last element
print(my_list[-1])  # Output: 4

# Get a sublist from index 1 up to (but not including) index 3
print(my_list[1:3])  # Output: [2, 3]

# Reverse a list using slicing
print(my_list[::-1])  # Output: [4, 3, 2, 1]

1.3. Modifying lists

Lists are mutable, so you can change an element by assigning to its index. To add elements, use append(), insert(), or extend(). To remove them, use remove(), pop(), or del.

my_list = [1, 2, 3, 4]

# Change an element
my_list[3] = 5
print(my_list)  # Output: [1, 2, 3, 5]

# append() adds one item to the end
my_list.append(6)
print(my_list)  # Output: [1, 2, 3, 5, 6]

# insert() puts an item at a specific index (here, 10 goes in at index 1)
my_list.insert(1, 10)
print(my_list)  # Output: [1, 10, 2, 3, 5, 6]

# extend() adds every item from another list
my_list.extend([7, 8])
print(my_list)  # Output: [1, 10, 2, 3, 5, 6, 7, 8]

Watch the difference between append() and extend(): my_list.append([7, 8]) would add the whole list [7, 8] as a single item at the end.

The next example continues with the same list and takes items back out:

# remove() deletes the first item with a given value
my_list.remove(10)
print(my_list)  # Output: [1, 2, 3, 5, 6, 7, 8]

# pop() removes an item and returns it (the last one by default)
last_item = my_list.pop()
print(last_item)  # Output: 8
print(my_list)  # Output: [1, 2, 3, 5, 6, 7]

# del removes the item at an index without returning it
del my_list[0]
print(my_list)  # Output: [2, 3, 5, 6, 7]

You can also give pop() an index, as in my_list.pop(0), to remove and return the item at that position.

1.4. Sorting lists

There are two ways to sort a list, and they behave differently:

• sorted(my_list) returns a new sorted list and leaves the original alone.
• my_list.sort() sorts the list in place (it changes the original) and returns None.

scores = [42, 7, 19, 88, 3]

# sorted() gives you a new list
print(sorted(scores))  # Output: [3, 7, 19, 42, 88]
print(scores)  # Output: [42, 7, 19, 88, 3]

# sort() changes the list itself
scores.sort()
print(scores)  # Output: [3, 7, 19, 42, 88]

# Both accept reverse=True for largest-first order
scores.sort(reverse=True)
print(scores)  # Output: [88, 42, 19, 7, 3]

# A common mistake: sort() returns None, so don't save its result
result = scores.sort()
print(result)  # Output: None

Strings sort alphabetically. To sort by something else, pass a function as key. Here, key=len sorts words by their length:

words = ["banana", "kiwi", "apple"]

print(sorted(words))  # Output: ['apple', 'banana', 'kiwi']
print(sorted(words, key=len))  # Output: ['kiwi', 'apple', 'banana']

1.5. Looping and list comprehensions

A for loop goes through the items one at a time. A list comprehension is a one-line shortcut for a loop that builds a new list. You can add an if to keep only some items.

my_list = [1, 2, 3, 4]

# Looping through a list
for number in my_list:
    print(number)

# List comprehension: square each number
squared = [x ** 2 for x in my_list]
print(squared)

# List comprehension with a condition: keep only even numbers
evens = [x for x in my_list if x % 2 == 0]
print(evens)

# Output:
# 1
# 2
# 3
# 4
# [1, 4, 9, 16]
# [2, 4]

1.6. Common pitfalls & performance tips

Trying to access an index that doesn’t exist raises an IndexError:

my_list = [1, 2, 3]

try:
    print(my_list[3])
except IndexError as error:
    print(error)  # Output: list index out of range

Copying a list also catches people out. Assigning a list to a new name (same = original) doesn’t copy anything: both names point to the same list. copy() makes a shallow copy, which is a new outer list that still shares any lists nested inside it. To copy the nested lists too, make a deep copy with copy.deepcopy():

import copy

original = [[1, 2], [3, 4]]

same = original                 # not a copy: one list, two names
shallow = original.copy()       # new outer list, same inner lists
deep = copy.deepcopy(original)  # new outer list and new inner lists

original[0].append(99)

print(same)  # Output: [[1, 2, 99], [3, 4]]
print(shallow)  # Output: [[1, 2, 99], [3, 4]]
print(deep)  # Output: [[1, 2], [3, 4]]

As for speed, reading or replacing an item by index takes the same short time no matter how long the list is. Programmers write this as O(1). Adding or removing at the end with append() and pop() is also O(1) on average. But insert(0, x), pop(0), and remove() have to shift every item after the change, so they get slower as the list grows (O(n), where n is the length of the list). If you need to add and remove items at the front, as in a queue, use collections.deque, which is fast at both ends:

from collections import deque

queue = deque(["Alice", "Bob"])
queue.append("Charlie")  # join at the back
first = queue.popleft()  # leave from the front, quickly

print(first)  # Output: Alice
print(queue)  # Output: deque(['Bob', 'Charlie'])

2. Tuples

Tuples are like lists, except that they are immutable: once you create a tuple, it can’t change. That makes tuples a good fit for data that should stay the same.

2.1. Creating tuples

Write a tuple with parentheses (), or leave them out when the meaning is clear. A tuple with a single item needs a trailing comma. Without it, the parentheses are just grouping, like in math.

# Defining a tuple
my_tuple = (1, 2, 3)

# Parentheses are optional when the context is unambiguous
another_tuple = 4, 5, 6

# The comma makes the tuple, not the parentheses
single = (7,)
not_a_tuple = (7)

print(my_tuple)  # Output: (1, 2, 3)
print(another_tuple)  # Output: (4, 5, 6)
print(type(single))  # Output: <class 'tuple'>
print(type(not_a_tuple))  # Output: <class 'int'>

2.2. Accessing tuple elements

Indexing and slicing work just as they do for lists. So do len(), in, + (join), and * (repeat), although + and * build new tuples instead of changing the old ones.

my_tuple = (1, 2, 3, 4, 5)

print(my_tuple[1])  # Output: 2
print(my_tuple[1:4])  # Output: (2, 3, 4)
print(len(my_tuple))  # Output: 5
print(3 in my_tuple)  # Output: True

print((1, 2) + (3, 4))  # Output: (1, 2, 3, 4)
print((1, 2) * 3)  # Output: (1, 2, 1, 2, 1, 2)

2.3. Immutability

Any attempt to modify a tuple raises a TypeError:

my_tuple = (1, 2, 3)

try:
    my_tuple[1] = 10
except TypeError as error:
    print(error)  # Output: 'tuple' object does not support item assignment

If you need a different tuple, build a new one, for example my_tuple + (4,).

2.4. Unpacking tuples

Tuples are often used to keep related pieces of information together, such as a person’s name and age. Unpacking pulls the items out into separate variables in one step. You need exactly one variable per item, or Python raises a ValueError.

person = ("Alice", 25)

# One variable per item, in order
name, age = person
print(name)  # Output: Alice
print(age)  # Output: 25

# A handy trick: swap two variables without a temporary one
a, b = 1, 2
a, b = b, a
print(a, b)  # Output: 2 1

Unpacking is also how a function can return more than one value. When you write return quotient, remainder, Python packs the two values into a tuple, and the caller can unpack them:

def divide(x, y):
    quotient = x // y
    remainder = x % y
    return quotient, remainder

result = divide(10, 3)
print(result)  # Output: (3, 1)

# Unpack straight into two variables
quotient, remainder = divide(10, 3)
print(f"10 / 3 = {quotient} remainder {remainder}")  # Output: 10 / 3 = 3 remainder 1

2.5. When to use tuples

Tuples suit fixed collections such as coordinates, RGB values, or days of the week.

They can also be dictionary keys, which lists can’t. Dictionary keys (and set members) must be hashable. Python computes a number, called a hash, from the value and uses it to find things quickly. That only works if the value can’t change, so numbers, strings, and tuples are hashable, but lists are not. (A tuple is hashable only if everything inside it is hashable too.)

# Tuples work as dictionary keys
grid = {(0, 0): "start", (2, 3): "treasure"}
print(grid[(2, 3)])  # Output: treasure

# Lists don't, because they can change
try:
    grid[[0, 0]] = "oops"
except TypeError:
    print("A list can't be a dictionary key")  # Output: A list can't be a dictionary key

3. Dictionaries

Use a dictionary when you need a lookup table. It stores data as key-value pairs: you find a value quickly through its key, and the keys describe what each value is. Each key appears only once, and since Python 3.7, dictionaries remember the order in which you added the keys.

3.1. Creating dictionaries

Create a dictionary with curly braces {} or the dict() function. All three of these build the same dictionary:

# Using curly braces
my_dict = {'name': 'John', 'age': 30, 'city': 'New York'}
print(my_dict)  # Output: {'name': 'John', 'age': 30, 'city': 'New York'}

# Using dict() with keyword arguments
my_dict = dict(name='John', age=30, city='New York')
print(my_dict)  # Output: {'name': 'John', 'age': 30, 'city': 'New York'}

# Using dict() with a list of (key, value) tuples
my_dict = dict([('name', 'John'), ('age', 30), ('city', 'New York')])
print(my_dict)  # Output: {'name': 'John', 'age': 30, 'city': 'New York'}

If your keys and values are in separate lists, zip() pairs them up by position, and dict() turns the pairs into a dictionary:

keys = ['name', 'age', 'city']
values = ['John', 30, 'New York']

my_dict = dict(zip(keys, values))
print(my_dict)  # Output: {'name': 'John', 'age': 30, 'city': 'New York'}

3.2. Accessing and modifying data

Square brackets look up a value by its key, and the in operator checks whether a key exists:

my_dict = {'name': 'John', 'age': 30, 'city': 'New York'}

name = my_dict['name']
print(name)  # Output: John
print('name' in my_dict)  # Output: True

# A missing key raises a KeyError
try:
    my_dict['country']
except KeyError as error:
    print(f"KeyError: {error}")  # Output: KeyError: 'country'

For a key that might be missing, use the get() method. It returns a default value instead of raising an error, or None if you don’t give a default:

country = my_dict.get('country', 'United States')
print(country)  # Output: United States
print(my_dict.get('country'))  # Output: None

Assigning to an existing key updates its value, and assigning to a new key adds a new pair:

# Update a value
my_dict['age'] = 31

# Add a new key-value pair
my_dict['occupation'] = 'Engineer'

print(my_dict)  # Output: {'name': 'John', 'age': 31, 'city': 'New York', 'occupation': 'Engineer'}

You can also delete pairs:

# Using del
del my_dict['city']

# Using pop (also returns the value)
age = my_dict.pop('age')

print(age)  # Output: 31
print(my_dict)  # Output: {'name': 'John', 'occupation': 'Engineer'}

3.3. Looping through dictionaries

Looping over a dictionary gives you its keys. Loop over items() instead to get each key and value together. (There are also keys() and values() if you only need one side.)

my_dict = {'name': 'John', 'age': 30, 'city': 'New York'}

for key in my_dict:
    print(key, my_dict[key])

for key, value in my_dict.items():
    print(f"{key}: {value}")

# Output:
# name John
# age 30
# city New York
# name: John
# age: 30
# city: New York

3.4. Advanced methods

To merge two dictionaries, use the | operator (Python 3.9+), which builds a new dictionary, or update(), which merges into an existing one. The older {**dict1, **dict2} syntax also builds a new dictionary. When both dictionaries have the same key, the value from the second one wins:

defaults = {'theme': 'light', 'font_size': 12}
user_settings = {'font_size': 16}

settings = defaults | user_settings
print(settings)  # Output: {'theme': 'light', 'font_size': 16}

defaults.update(user_settings)
print(defaults)  # Output: {'theme': 'light', 'font_size': 16}

A dictionary comprehension builds a dictionary in a single expression:

# Create a dictionary with numbers and their squares
squares = {x: x**2 for x in range(5)}
print(squares)  # Output: {0: 0, 1: 1, 2: 4, 3: 9, 4: 16}

4. Sets

A set is an unordered collection of unique elements. Sets are good for membership testing, removing duplicates, and mathematical operations such as unions and intersections. Because they have no order, you can’t index a set (my_set[0] is an error).

4.1. Creating sets

Create a set with curly braces {} or the set() constructor. Duplicates are dropped automatically:

# Define a set
my_set = {1, 2, 3}
print(my_set)  # Output: {1, 2, 3}

# Duplicates disappear
numbers = set([1, 2, 2, 3, 3, 3])
print(numbers)  # Output: {1, 2, 3}

# An empty set must be created with set(), not {} (which creates an empty dict)
empty_set = set()
print(type(empty_set))  # Output: <class 'set'>
print(type({}))  # Output: <class 'dict'>

Turning a list into a set is the quickest way to remove duplicates. The set doesn’t keep the original order, so use sorted() if you want a predictable list back:

names = ["bob", "alice", "bob", "carol", "alice"]

unique_names = sorted(set(names))
print(unique_names)  # Output: ['alice', 'bob', 'carol']

4.2. Set operations

add() adds an item. remove() deletes one and raises a KeyError if it isn’t there, while discard() does nothing in that case:

my_set = {1, 2, 3}

# Add an item
my_set.add(4)
print(my_set)  # Output: {1, 2, 3, 4}

# Remove an item
my_set.remove(2)
print(my_set)  # Output: {1, 3, 4}

# Discard an item (no error if the item isn't present)
my_set.discard(10)
print(my_set)  # Output: {1, 3, 4}

# remove() complains about a missing item
try:
    my_set.remove(10)
except KeyError as error:
    print(f"KeyError: {error}")  # Output: KeyError: 10

The mathematical set operations are written as operators. Each operator also has a method version, such as set_a.union(set_b) or set_a.intersection(set_b):

set_a = {1, 2, 3}
set_b = {3, 4, 5}

# Union: All unique items from both sets
print(set_a | set_b)  # Output: {1, 2, 3, 4, 5}

# Intersection: Items common to both sets
print(set_a & set_b)  # Output: {3}

# Difference: Items in set_a but not in set_b
print(set_a - set_b)  # Output: {1, 2}

# Symmetric difference: Items in either set, but not in both
print(set_a ^ set_b)  # Output: {1, 2, 4, 5}

4.3. Membership testing

Sets are optimized for fast membership testing: checking in on a set stays fast however big the set gets. That makes a set a good choice for something like a guest list:

guest_list = {"Alice", "Bob", "Charlie"}

print("Alice" in guest_list)  # Output: True
print("Mallory" in guest_list)  # Output: False

4.4. Frozen sets

A frozen set is a set that can’t be changed after you create it. You make one with frozenset(). Everything that reads a set works on it (in, len(), union, intersection, and so on), but methods that change a set, such as add() and remove(), don’t exist.

frozen = frozenset([1, 2, 3, 4])
print(frozen)  # Output: frozenset({1, 2, 3, 4})

# Reading works just like a set
print(3 in frozen)  # Output: True
print(frozen | {5})  # Output: frozenset({1, 2, 3, 4, 5})

# Changing it doesn't
try:
    frozen.add(5)
except AttributeError as error:
    print(error)  # Output: 'frozenset' object has no attribute 'add'

Because a frozen set can’t change, it is hashable (see Section 2.5), so it can be a dictionary key or a member of another set. A regular set can be neither. Frozen sets are handy when the key is a group of things whose order doesn’t matter:

# Each key is a pair of friends; {"Alice", "Bob"} is the same pair as {"Bob", "Alice"}
friends_since = {
    frozenset({"Alice", "Bob"}): 2019,
    frozenset({"Bob", "Carol"}): 2022,
}
print(friends_since[frozenset({"Bob", "Alice"})])  # Output: 2019

# A regular set doesn't work as a key
try:
    friends_since[{"Carol", "Dave"}] = 2024
except TypeError:
    print("A set can't be a dictionary key")  # Output: A set can't be a dictionary key

5. Bytes and bytearrays: raw binary data

Everything a computer stores (text, images, sound) is made of bytes. A byte is a number from 0 to 255. Python has two types for working with bytes directly, which you’ll meet when reading files in binary mode or sending data over a network:

• bytes: an immutable sequence of integers from 0 to 255
• bytearray: the same thing, but mutable

5.1. Bytes

Create a bytes object from a list of numbers, or write a bytes literal with a b in front of the quotes. When Python displays bytes, it shows each byte that matches a printable ASCII character as that character, and the rest as escape codes like \x00. That is only a display convenience: each item is still a number.

data = bytes([72, 105, 33])
print(data)  # Output: b'Hi!'

# Indexing gives you a number, slicing gives you bytes
print(data[0])  # Output: 72
print(data[0:2])  # Output: b'Hi'
print(list(data))  # Output: [72, 105, 33]

# A bytes literal builds the same object
print(data == b"Hi!")  # Output: True

# Every value must fit in one byte
try:
    bytes([256])
except ValueError as error:
    print(error)  # Output: bytes must be in range(0, 256)

# And bytes can't be changed
try:
    data[0] = 74
except TypeError as error:
    print(error)  # Output: 'bytes' object does not support item assignment

5.2. Converting between text and bytes

A string holds characters; bytes hold numbers. To turn text into bytes, you encode it, and to turn bytes back into text, you decode them. UTF-8 is the most common encoding. Some characters take more than one byte:

text = "café"

encoded = text.encode("utf-8")
print(encoded)  # Output: b'caf\xc3\xa9'
print(len(text))  # Output: 4
print(len(encoded))  # Output: 5

decoded = encoded.decode("utf-8")
print(decoded)  # Output: café

The é became two bytes (\xc3 and \xa9), which is why the bytes version is one longer than the string.

5.3. Bytearray

A bytearray works like bytes, but you can change it in place, much like a list of small numbers. That makes it useful when you build up binary data piece by piece.

buffer = bytearray(b"hello")

# Replace the first byte (74 is the code for "J")
buffer[0] = 74
print(buffer)  # Output: bytearray(b'Jello')

# Add a byte to the end (33 is "!")
buffer.append(33)
print(buffer)  # Output: bytearray(b'Jello!')

# bytearray(n) makes n zero bytes
print(bytearray(3))  # Output: bytearray(b'\x00\x00\x00')

# Convert to bytes when you're done changing it
print(bytes(buffer))  # Output: b'Jello!'

6. Comparative overview & use cases

Use a list for data that changes over time and where order matters, and a tuple for fixed data, including dictionary keys. If you need to look values up quickly by unique keys (like a mini-database), use a dictionary. When order doesn’t matter and you want to avoid duplicates or do set operations, use a set, or a frozen set when the set itself needs to be a key. Use str for text and bytes for raw binary data, and convert between them with encode() and decode().

7. Best practices and performance tips

A tuple uses a little less memory than a list with the same items, because a list needs extra bookkeeping (and often some spare room) so it can grow. The difference is small, so choose between them based on whether the data should change. Sets and dictionaries use noticeably more memory than a list of the same size. They trade that extra space for fast lookups.

Accessing elements by index in lists and tuples is O(1), and lookups in dictionaries and sets are also O(1) on average. Checking x in my_list, on the other hand, looks through the list item by item (O(n)), so if you test membership often on a large collection, use a set. Adding or removing at the front of a list is O(n); use collections.deque for queues.

A few habits prevent common errors. Be careful with indexes in lists and tuples, use the get() method for dictionaries to avoid KeyError, and use discard() for sets if you’re not sure whether an element exists.

8. Advanced tips and tricks

You can combine these structures to represent more complex data. For instance, a list of dictionaries can represent a table of data:

students = [
    {'name': 'Alice', 'age': 24, 'major': 'Physics'},
    {'name': 'Bob', 'age': 22, 'major': 'Mathematics'},
    {'name': 'Charlie', 'age': 23, 'major': 'Computer Science'}
]

for student in students:
    print(f"{student['name']} studies {student['major']}")

# Output:
# Alice studies Physics
# Bob studies Mathematics
# Charlie studies Computer Science

When you pass data to a function as a tuple, the function can’t change it by accident.

List, dictionary, and set comprehensions are a compact way to create new collections. They make code shorter and, once you get used to them, easier to read. Notice how the same expression gives three different results:

numbers = [1, 2, 2, 3, 3, 3]

print([n * n for n in numbers])  # Output: [1, 4, 4, 9, 9, 9]
print({n * n for n in numbers})  # Output: {1, 4, 9}
print({n: n * n for n in numbers})  # Output: {1: 1, 2: 4, 3: 9}

To inspect a data structure while debugging, use built-in functions like len() and type(), or plain print statements. When your data isn’t behaving as expected, Python’s debugger (pdb) can save you a lot of time.

Key takeaways

• Lists are ordered and changeable. append() and pop() at the end are fast; sorted() returns a new list, while sort() changes the list and returns None.
• Assigning a list to another name doesn’t copy it. copy() makes a shallow copy; use copy.deepcopy() for nested lists.
• Tuples can’t change. Unpack them into variables, and use them to return several values from a function.
• Dictionaries map unique keys to values, remember insertion order, and find values quickly. Use get() when a key might be missing.
• Sets hold unique items and make membership tests fast. Frozen sets are the unchangeable, hashable version, so they can be dictionary keys or set members.
• bytes and bytearray hold integers from 0 to 255; encode() and decode() convert between text and bytes.

Practice

Exercise 1: Tidy Up a Shopping List

You have this shopping list, with some items written down twice:

shopping = ["milk", "eggs", "bread", "milk", "apples", "eggs"]

Build a sorted list of the unique items, print it, and then print how many different items there are.

shopping = ["milk", "eggs", "bread", "milk", "apples", "eggs"]

unique_items = sorted(set(shopping))
print(unique_items)
print(f"{len(unique_items)} different items")

# Output:
# ['apples', 'bread', 'eggs', 'milk']
# 4 different items

Exercise 2: Count the Words

Count how many times each word appears in the sentence "the cat sat on the mat with the hat". Store the counts in a dictionary, then print each word and its count in the order the words first appear. Hint: sentence.split() turns the sentence into a list of words, and counts.get(word, 0) gives you the current count, or 0 for a new word.

sentence = "the cat sat on the mat with the hat"

counts = {}
for word in sentence.split():
    counts[word] = counts.get(word, 0) + 1

for word, count in counts.items():
    print(f"{word}: {count}")

# Output:
# the: 3
# cat: 1
# sat: 1
# on: 1
# mat: 1
# with: 1
# hat: 1

Exercise 3: Summarize Test Scores

Write a function summarize(scores) that returns three values: the lowest score, the highest score, and the average. Call it with [72, 95, 88, 64, 81], unpack the result into three variables, and print each one. You can use the built-in functions min(), max(), sum(), and len().

def summarize(scores):
    lowest = min(scores)
    highest = max(scores)
    average = sum(scores) / len(scores)
    return lowest, highest, average

low, high, avg = summarize([72, 95, 88, 64, 81])
print(f"Lowest: {low}")
print(f"Highest: {high}")
print(f"Average: {avg}")

# Output:
# Lowest: 64
# Highest: 95
# Average: 80.0