I wrote a little program to check a bunch of hashes, and I was wondering if this is a correct application of starmap and if it should be making other processes. My current understanding is that starmap is supposed to divide the task amongst the workers, but it doesn’t seem to do it.
from multiprocessing import Pool from hashlib import sha256 import datetime start = datetime.datetime.now() def hash(text): return sha256(text.encode("utf-8")).hexdigest() string = "Baguette " def f(r, z, string): for i in range(r, z): j = hash(string + str(i)) if j.startswith("000000"): print(i, j[0:6]) if __name__ == '__main__': with Pool(8) as p: p.starmap(f, [[0, 10000000, string]]) p.starmap(f, [[20000000, 30000000, string]]) p.starmap(f, [[30000000, 40000000, string]]) p.starmap(f, [[40000000, 50000000, string]]) print(datetime.datetime.now()-start)
Python apparently has 9 processes open, but only one is using the CPU (11.9%)
Advertisement
Answer
from multiprocessing import Pool from hashlib import sha256 import datetime start = datetime.datetime.now() def hash(text): return sha256(text.encode("utf-8")).hexdigest() string = "Baguette " def f(r, z, string): for i in range(r, z): j = hash(string + str(i)) if j.startswith("000000"): print(i, j[0:6]) if __name__ == '__main__': with Pool(8) as p: p.starmap(f, [[0, 10000000, string], [20000000, 30000000, string], [30000000, 40000000, string], [40000000, 50000000, string]]) print(datetime.datetime.now()-start)
Thank you @flakes for the good answer :)
flakes’ comment: Is my understanding of multiprocessing’s starmap correct?