Different ways to merge dictionaries in Python

In this article, we have explored Different ways to merge dictionaries in Python.

Table of contents:

1. Introduction
2. A summary of 6 common methods (Sorted by Python versions)
   3. Union Operators "|" (Python 3.9+ Only)
   4. Unpacking Method: "**" (Python 3.5+)
   5. update() Method (All Python Versions)
   6. Transferring items to new dictionaries (All Python versions)
3. Applying these methods to complex scenarios

Introduction

In Python, dictionaries can store data with multiple pairs of keys and values. The keys are unique in every dictionary while values can be duplicated. In real working scenarios, we often need to merge two or more dictionaries.

This article focuses on:

• By listing 6 approaches and alternative ways of writing, one can quickly understand other people's codes in prior Python versions, especially working in a team and needed to update based on the existed codes;
• By setting some advanced application scenarios, one can decide on the most suitable approach for different application scenarios when writing codes independently.

A summary of 6 common methods (Sorted by Python versions)

Before we start, here's the basic background:
We have two dictionaries: d1 and d2, without overlaps in keys or values. We need to merge them into a new dictionary.

# The basic dictionaries
d1 = {'name': "Claudia", 'country': "China", 'age': 23}
d2 = {'gender': "non-binary", 'Major': "CS"}

1: Union Operators "|" (Python 3.9+ Only)

d_temp1 = d1 | d2
print(d_temp1)

Output: {'name': 'Claudia', 'country': 'China', 'age': 23, 'gender': 'non-binary', 'Major': 'CS'}

• Compared to other methods, using Union Operators is the simplest and most readable one. However, we can only write in this way in Python 3.9+.
• If we want to use "|" in Python 3.x versions, we need to break down dictionaries into a list of key-value pairs with items() method and collect them into a new dictionary using dict() method, shown as below:

dict(d1.items() | d2.items())

2: Unpacking Method: "**" (Python 3.5+)

print({**d1,**d2})

Output: {'name': 'Claudia', 'country': 'China', 'age': 23, 'gender': 'non-binary', 'Major': 'CS'} (Same as using "|")

• "**" can be used to unpack many data types. Here it means to unpack the dictionaries: d1 and d2, and obtain all the key-value pairs in them. Then we use {} to rearrange it as a new dictionary.
• This method is also readable, but still, limited to Python 3.5+ versions.

3: update() Method (All Python Versions)

# 3.1 Using copy() and update()
d_temp2 = d1.copy() 
d_temp2.update(d2)
print(d_temp2)

• .update() is a built-in Python method to "update" d1 with adding d2 to it. It's quite easy to understand the logic at the first sight.
• Disadvantage: It would modify the original d1 if used directly. So we need to use copy() function to copy d1 and update on the copied one d_temp2.
• There is also an alternative method using deepcopy() with the same coding logic.

# 3.2 Using deepcopy() and update()
from copy import deepcopy
d_temp3 = deepcopy(d1)
d_temp3.update(d2)
print(d_temp3)

4: Transferring items to new dictionaries (All Python versions)

4.1 Basic method, though too many lines

d_temp4 = dict()
for k, v in d1.items():
    d_temp4[k] = v
for k, v in d2.items():
    d_temp4[k] = v
print(d_temp4)

• When adding new elements to an existing dictionary, we would use the assignment operator "=" with the syntax: dict_name[key] = value.
• Based on this logic, without being limited to Python versions issue, we can write a for loop that iterates over each key-value pair using .items() and then insert to the new dictionary.
• This logic is applicable for all Python versions. Alternatively, we can simplify it into one line as below, which is commonly used in Python 2.x versions:

# 4.1 codes in one line
print({k:v for d in [d1, d2] for k,v in d.items()})

4.2 Using "+" Operator

print(dict(list(d1.items()) + list(d2.items()))(

• This method is applicable for coding under Python 2.x.

5: Using itertools module

import itertools
print(dict(itertools.chain(d1.items(), d2.items())))

• itertools is a built-in module that manipulates iterable objects. Since dictionaries are also iterables, we can use itertools.chain() to form a larger iterable object, and then use dict() to convert them to dictionaries.
• The logic of using itertools is similar to item() & dict() Method stated before.

6: Using ChainMap library

from collections import ChainMap
print(dict(ChainMap(d1, d2)))

• ChainMap is another module that could be imported in Python to merge dictionaries, with similar effects and logic compared to itertools.
• Disadvantage: When there are duplicate keys between dictionaries, only the first value will be taken.(Using itertools will not have this issue).

Applying these methods to complex scenarios

In reality, dictionaries to be merged are usually more complex than what we had used beforehand (d1 and d2).
To deal with such situations, we shall comprehensively use the methods stated before.

Case 1. Multiple dictionaries (>=3)

If the number of dictionaries can be counted by hands, we can simply use any methods such as "|", "**" or update():

d3 = {'Fav Food': "Pizza", 'Location': "Shanghai"}

# Using Union Operators
print(d1 | d2 | d3)

# Unpacking Method
print({**d1, **d2, **d3})

# update()
d_temp5 = {}
dtemp5.update(d1)
dtemp5.update(d2)
dtemp5.update(d3)
print(dtemp5)

• Outputs are all the same: {'name': 'Claudia', 'country': 'China', 'age': 23, 'gender': 'non-binary', 'Major': 'CS', 'Fav Food': "Pizza", 'Location': "Shanghai"}
• Alternatively, we can simplify and write in for loops with the update() method :

def merge1(*dict):  # Grab all dictionaries into the function 
    new = {} # Create a new dictionary to store merged key-value pairs
    for d in dict:
        new.update(d) # Traverse all the dictionaries with the update() method
    return new
print(merge1(d1,d2,d3))

Case 2. Overlapped keys

Sometimes, we need to merge dictionaries having same keys with different values:

# Overlapped keys: 'order 1' and 'order 2'
d5 = {'order1': 500, 'order2': 300, 'order3': 250}
d6 = {'order1': 600, 'order2': 400, 'order4': 334} 

Depends on different development needs, we may:

• Keep the value we want;
• Collect them together and save as new lists of values.
• Calculate the sum value for the same key and store it as the new value (When the value type is number).

2.1: Keeping the value only in one of the dictionaries:

print(d5 | d6)

Output: {'order1': 600, 'order2': 400, 'order3': 250, 'order4': 334}

• We can see that the output covered the values of 'order1' and 'order 2' in d5 with the ones in d6.
• If we want to keep d5 values, we can reverse their postions in the code:

print(d6 | d5)

Output: {'order1': 500, 'order2': 300, 'order4': 334, 'order3': 250}

2.2: Collecting all values together:

import collections
def merge2(*dicts):
    merge_temp = collections.defaultdict(list)
    for d in dicts:
        for k, v in d.items(): 
            merge_temp[k].append(v)
    return dict(merge_temp)
print(merge2(d5, d6))

Output: {'order1': [500, 600], 'order2': [300, 400], 'order3': [250], 'order4': [334]}

• The defaultdict() method constructs a dictionary with a default value of list. Therefore, we can store all the key-value pairs from both d5 and d6 in it.
• Then, we use .append() method (for adding new items to the lists) to append values with the same keys.
• At last, we need to convert the defaultdict(list) to a divtionary to eliminate the default value.

2.3: Summing up all values together:

d_merge3 = d5.copy()
for k, v in d4.items():
    if k in d5.keys():
        d_merge3[k] += v
    else:
        d_merge3[k] = v
print(d_merge3)

Output: {'order1': 1100, 'order2': 700, 'order4': 334, 'order3': 250}

• Here we used for loops to obtain keys and values in d4, and add the values of overlapped keys in d4 to d_merge3, which is a copied version of d5, with all the key-value pairs of d5 without changing d5 itself.

With this article at OpenGenus, you must have the complete idea of Different ways to merge dictionaries in Python.