Merging and flattening two lists of dictionaries using keys as new fields

I have two lists of dictionaries, each with the same structure. I wish to flatten into a single dictionary taking precedence of list 2, taking the value as the key of a new flat dictionary.

The following code works, but it feels like it's hacking together code that can probably be done via one or two simple comprehensions. Is there a better way than this?

It produces this:

{'SourceIP': 'src2', 
 'DestinationIP': 'dst', 
 'Direction': 'dir', 
 'NEW': 'newvalue'
}

Here is the code:

import operator

default = [
    {"RealField": "SourceIP",   "SuppliedField": "src"},
    {"RealField": "DestinationIP", "SuppliedField": "dst"},
    {"RealField": "Direction", "SuppliedField": "dir"}
]

product_mapping = [
    {"RealField": "SourceIP",   "SuppliedField": "src2"},
    {"RealField": "DestinationIP",   "SuppliedField": "dst2"},
    {"RealField": "NEW",   "SuppliedField": "newvalue"},
]


def dictionary_from_mappings(default_mapping, product_mapping):
    default = [{i["RealField"]:i["SuppliedField"]} for i in default_mapping]
    default_flat = reduce(operator.ior, default, {})
    
    product = [{i["RealField"]:i["SuppliedField"]} for i in product_mapping]
    product_flat = reduce(operator.ior, product, {})
    return default_flat | product_flat

mappings = dictionary_from_mappings(default, product_mapping)
print(mappings)

You can use dict comprehensions to create the flats directly, like this:

def dictionary_from_mappings(default_mapping, product_mapping):  
    default_flat = {i["RealField"]: i["SuppliedField"] for i in default_mapping}  
    product_flat = {i["RealField"]: i["SuppliedField"] for i in product_mapping}
    return default_flat | product_flat  

Your example isn't fully reproducable, but I think this is what you want:

def flat_dict(nested):
    return {x["RealField"]: x["SuppliedField"] for x in nested}

def dictionary_from_mappings(default_mapping, product_mapping):
    return flat_dict(default_mapping) | flat_dict(product_mapping)

The key is to use a second function with a simple dict comprehension and call it twice to eliminate code duplication

Comin' in hot years later as we were looking for example questions to answer and stumbled across this one.

Clearly you have the needed solutions above, but I thought i'd include the work we did for our local example here as a reference or in the event that someone find it helpful.

Few things I did notice / made assumptions on:

> wish to flatten into a single dictionary taking precedence of list 2, taking the value as the key of a new flat dictionary.

I took this to mean: for N-number of lists, when combining the dictionaries contained within them, the last list to be iterated over which contains a key:value that appears in a prior list, override to the newest value. This gets a little confusing given the example output you provided here:

{
 'SourceIP': 'src2', # <-- comes from product_mapping ie: second dict
 'DestinationIP': 'dst', # <-- comes from default
 'Direction': 'dir', # <-- comes from/only appears in default
 'NEW': 'newvalue' # <-- only appears in product_mapping
}

my assumption is it was meant to be:

{
 'SourceIP': 'src2', # <-- comes from product_mapping ie: second dict
 'DestinationIP': 'dst2', # <-- comes from product_mapping ie: second dict
 'Direction': 'dir', # <-- comes from/only appears in default
 'NEW': 'newvalue' # <-- only appears in product_mapping
}

So with that assumption, we wanted to make our answer:

1. not locked into to any hardcoded variables
2. allow to pass in N dictionaries, rather than static number of 2

What we came up with:

def kapow(l):
    """kapow kinda un-nest a list of dicts and create new k:v from values

    Function to essentially do nothing more than generate a new dictionary by iterating over a list of dictionaries,
    whereby the key will be the value of the first item, and the value will be the..well..value from the second item.
    In essence doing away with the key of the second altogether:
    {
       "one" : 1,
       "two" : 2
    } --> {"one" : 2}

    IE: 
                    KEY -----------------------  VALUE
                     |                              |
    {"RealField": "SourceIP",   "SuppliedField": "src"}
    # kinda weird but i get it given the probable use case scenario of the source data

    Parameters
    ----------
    l : list
        Expects a list of dictionaries. 

    Examples
    --------
    >>> _l1 = [
        {"RealField": "SourceIP",   "SuppliedField": "src"},
        {"RealField": "DestinationIP", "SuppliedField": "dst"}
    ]
    >>> kapow(_l1)

    out [1]:  {'SourceIP': 'src', 'DestinationIP': 'dst'}

    """
    return(dict(map(lambda i: i.values(), l)))

def dict_from_mappings(*args):
    """dict_from_mappings for any number of lists of dicts combine into one


    Examples
    --------
    >>> default = [
        {"RealField": "SourceIP",   "SuppliedField": "src"},
        {"RealField": "DestinationIP", "SuppliedField": "dst"},
        {"RealField": "Direction", "SuppliedField": "dir"}
    ]

    >>> product_mapping = [
        {"RealField": "SourceIP",   "SuppliedField": "src2"},
        {"RealField": "DestinationIP",   "SuppliedField": "dst2"},
        {"RealField": "NEW",   "SuppliedField": "newvalue"},
    ]

    >>> a_random_af_third = [
        {"RealField": "SourceIP",   "SuppliedField": "src2"},
        {"RealField": "DestinationIP",   "SuppliedField": "dst2"},
        {"RealField": "NEW",   "SuppliedField": "newvalue"},
        {"RealField": "idk_man", "SuppliedField": "seems like a real specific issue here but yolo"}
    ]

    >>> dict_from_mappings(default, product_mapping, a_random_af_third)
    """
    
    L = [kapow(i) for i in args]

    return(dict(j for i in L for j in i.items()))

Resulting in

dict_from_mappings(default, product_mapping, a_random_af_third)

{'SourceIP': 'src2',
 'DestinationIP': 'dst2',
 'Direction': 'dir',
 'NEW': 'newvalue',
 'idk_man': 'seems like a real specific issue here but yolo'}

This is a similiar approach to the first two solutions, but you can use two loops to loop through each element in the list dictionary and update the new dictionary then return it.

def dictionary_from_mappings(default, product_mapping):
d = {}
for i in default:
d.update({i["RealField"]: i["SuppliedField"]})
p = {}
for i in product_mapping:
p.update({i["RealField"]: i["SuppliedField"]})
return d | p

The result is:

 {'SourceIP': 'src2', 
'DestinationIP': 'dst2', 
'Direction': 'dir', 
'NEW': 'newvalue'}