How can I run multiple processes pool where I process run1-3 asynchronously, with a multi processing tool in python. I am trying to pass the values (10,2,4),(55,6,8),(9,8,7)
for run1,run2,run3
respectively?
import multiprocessing def Numbers(number,number2,divider): value = number * number2/divider return value if __name__ == "__main__": with multiprocessing.Pool(3) as pool: # 3 processes run1, run2, run3 = pool.map(Numbers, [(10,2,4),(55,6,8),(9,8,7)]) # map input & output
Advertisement
Answer
You just need to use method starmap
instead of map
, which, according to the documentation:
Like
map()
except that the elements of the iterable are expected to be iterables that are unpacked as arguments.Hence an iterable of
[(1,2), (3, 4)]
results in[func(1,2), func(3,4)]
.
import multiprocessing def Numbers(number,number2,divider): value = number * number2/divider return value if __name__ == "__main__": with multiprocessing.Pool(3) as pool: # 3 processes run1, run2, run3 = pool.starmap(Numbers, [(10,2,4),(55,6,8),(9,8,7)]) # map input & output print(run1, run2, run3)
Prints:
5.0 41.25 10.285714285714286
Note
This is the correct way of doing what you want to do, but you will not find that using multiprocessing for such a trivial worker function will improve performance; in fact, it will degrade performance due to the overhead in creating the pool and passing arguments and results to and from one address space to another.