Skip to content
Advertisement

Checking if a list has duplicate lists

Don

Given a list of lists, I want to make sure that there are no two lists that have the same values and order. For instance with my_list = [[1, 2, 4, 6, 10], [12, 33, 81, 95, 110], [1, 2, 4, 6, 10]] it is supposed to return me the existence of duplicate lists, i.e. [1, 2, 4, 6, 10].

I used while but it doesn’t work as I want. Does someone know how to fix the code:

routes = [[1, 2, 4, 6, 10], [1, 3, 8, 9, 10], [1, 2, 4, 6, 10]]
r = len(routes) - 1
i = 0
while r != 0:
    if cmp(routes[i], routes[i + 1]) == 0:
        print "Yes, they are duplicate lists!"
    r -= 1
    i += 1

Advertisement

Answer

you could count the occurrences in a list comprehension, converting them to a tuple so you can hash & apply unicity:

routes = [[1, 2, 4, 6, 10], [1, 3, 8, 9, 10], [1, 2, 4, 6, 10]]
dups = {tuple(x) for x in routes if routes.count(x)>1}

print(dups)

result:

{(1, 2, 4, 6, 10)}

Simple enough, but a lot of looping under the hood because of repeated calls to count. There’s another way, which involves hashing but has a lower complexity would be to use collections.Counter:

from collections import Counter

routes = [[1, 2, 4, 6, 10], [1, 3, 8, 9, 10], [1, 2, 4, 6, 10]]

c = Counter(map(tuple,routes))
dups = [k for k,v in c.items() if v>1]

print(dups)

Result:

[(1, 2, 4, 6, 10)]

(Just count the tuple-converted sublists – fixing the hashing issue -, and generate dup list using list comprehension, keeping only items which appear more than once)

Now, if you just want to detect that there are some duplicate lists (without printing them) you could

  • convert the list of lists to a list of tuples so you can hash them in a set
  • compare the length of the list vs the length of the set:

len is different if there are some duplicates:

routes_tuple = [tuple(x) for x in routes]    
print(len(routes_tuple)!=len(set(routes_tuple)))

or, being able to use map in Python 3 is rare enough to be mentionned so:

print(len(set(map(tuple,routes))) != len(routes))
Advertisement