Skip to content
Advertisement

Detect duplicate elements python

I get continuously data from a server and can receive the data via the following line of code:

id, type, value = getId(payload)

After that I would like to write them into a file via:

out.write(str(id) + ";" + str(type) + ";" + str(value) + "n")

Now the case is, that the same id can appear multiple times, but the value will be a different one. Therefore I would like to extend the out.write into the following way that the different values are added at the right side but still being referred to the same id:

out.write(str(id) + ";" + str(type) + ";" + str(value) + ";" + str(value1) + ";" + str(value2) + "n")

Does anyone has an idea how to do this in python?

Advertisement

Answer

Using the hints that were already added as comments you can create something similar to this:

from collections import defaultdict

values = defaultdict(set)
types = dict()

for payload in input_stream:
    id, type, value = get(payload)
    values[id].add(value)
    types[id] = type

for id in types.keys():
    out.write(";".join(map(str, [id, types[id]] + list(values[id]))) + "n")

If the values is more of a time series (order is important), then replace set with list.

User contributions licensed under: CC BY-SA
9 People found this is helpful
Advertisement