Method for mapping dictionary values in complex list

Inputs

I have a very complicated list of list.

total_aug_rule_path_list = 
[[[[['#1_0_0', '#2_0_0', '#3_0_0'], ['#1_0_1', '#2_0_1', '#3_0_1']],
   [['#1_0_0', '#2_0_0', '#3_0_0'], ['#1_0_1', '#2_0_1', '#3_0_1']]],
  [[['#1_1_0', '#2_1_0', '#3_1_0', '#4_1_0'],
    ['#1_1_1', '#2_1_1', '#3_1_1', '#4_1_1']]]]]

And i have a dictionary that has each element of the list as a key.

sym2id_dict = {
 '#1_0_0': 1,
 '#1_0_1': 2,
 '#1_1_0': 3,
 '#1_1_1': 4,
 '#2_0_0': 5,
 '#2_0_1': 6,
 '#2_1_0': 7,
 '#2_1_1': 8,
 '#3_0_0': 9,
 '#3_0_1': 10,
 '#3_1_0': 11,
 '#3_1_1': 12,
 '#4_1_0': 13,
 '#4_1_1': 14,}

I’m going to map each element of the list to the value of the dictionary.

output

[[[[[1, 5, 9], [2, 6, 10]], [[1, 5, 9], [2, 6, 10]]],
  [[[3, 7, 11, 13], [4, 8, 12, 14]]]]]

I tried the following to use the for loop as little as possible.

list(map(lambda proofpaths_to_goal : 
list(map(lambda proofpaths_to_template :
list(map(lambda proofpath :
list(map(lambda single_augment : list(map(lambda x : sym2id_dict[x], single_augment)),  
     proofpath)), proofpaths_to_template)), proofpaths_to_goal)),total_aug_rule_path_list))

I would appreciate it if you could let me know if there is a way that is easier or more readable than this method.

Answer

Here are a few alternatives:

Nested list comprehension

[[[[[sym2id_dict[l4] for l4 in l3] for l3 in l2] for l2 in l1] for l1 in l0] for l0 in total_aug_rule_path_list]

Though this is arguably no easier to read.

Using Numpy

This method does not work for your example list because numpy arrays must not be ragged arrays (i.e. all lists that are equally nested must have the same length). However, when you are not using ragged arrays, you can do:

import numpy as np
total_aug_rule_path_list = [[
    [[['#1_0_0', '#2_0_0', '#3_0_0'], ['#1_0_1', '#2_0_1', '#3_0_1']]],
    [[['#1_1_0', '#2_1_0', '#3_1_0'], ['#1_1_1', '#2_1_1', '#3_1_1']]]
]]
sym2id_dict = {...} # your dict here
total_aug_rule_path_list_array = np.array(total_aug_rule_path_list)
print(np.vectorize(sym2id_dict.get)(total_aug_rule_path_list_array))

This applies the sym2id_dict.get function to every string in the array. You can change this to sym2id_dict._getitem__ if you want it to throw an error when the key is not in the dictionary.

Write your own recursive function

Recurse and iterate through lists

This function recurses until the input isn’t a list. This will work on lists like [1, [2, 3]]. If you want it to work on things other than lists, see here.

def vectorised_apply(f, values):
    if isinstance(values,list):
        return [vectorised_apply(f,value) for value in values]
    else:
        return f(values)

print(vectorised_apply(sym2id_dict.get, total_aug_rule_path_list))

Fixed recursion depth

This variation recurses to a fixed depth, so no isinstance checking is needed:

def vectorised_apply_n(f, values, n=0):
    if n == 0:
        return f(values)
    else:
        return [vectorised_apply_n(f, value, n=n-1) for value in values]

print(vectorised_apply_n(sym2id_dict.get, total_aug_rule_path_list, 5))

If you really want, you could use a trick with itertools.accumulate to make this fixed recursion depth function into a single expression, but it’s pretty unpythonic and hard to understand.

from itertools import accumulate

print(
    list(accumulate(
        range(5),  # do 5 times because the list is nested 5 times
        initial=lambda x: sym2id_dict[x], # base case: lookup in the dictionary
        func=lambda rec, _: lambda xs: [rec(x) for x in xs] # recursive case: build a bigger function using a previous function `rec`
    ))[-1](total_aug_rule_path_list)) # get the last function from the list