Basic python threading is not working. What am I missing in this?

I am trying to use python threading and am having problems getting the threads to work independently. They seem to be running in sequential order and waiting for one to finish before starting to process the next thread. I have read other posts suggesting that I need to get more work into the threads to differentiate actual CPU work vs the CPU work of starting and managing the threads, and that a sleep timer could be used to simulate this. So I tried that and then measured the task durations.

So my code is below. It first runs three tasks sequentially with a 2 second timer. This takes about 6 seconds to run as expected. The next section starts three threads and they should run roughly in parallel if my understanding of threading is correct. I have played with the timers to test the overall duration of this section of code, expecting that if one timer is larger than the other two, the code will execute in an interval closest to that larger one. but what I am seeing is that is taking the same amount of time as the three running in sequence – one after the other.

I got onto this because I am writing some code to read an asynchronous queue in the background. After launching the thread to read the queue, my code seems to stop and wait until the queue reader is stopped, which it normally doesn’t as it waits for messages to come in. So what happens is that it never executes the next section of code and it seems to be waiting for the thread to complete.

Also I checked the number of threads active and it remains at the same number, and when I check for the thread ID in the code (not shown) I get the same thread number coming back for every thread.

I am new to python and am using the jupyter compiler environment. Is there a compile option or some other limitation that I am not aware of that is preventing the threading? Am I just not getting the concept? I dont believe that this is related to CPU cores / threading as it would be done through logical thread cores within the python compiled code. I also ran a similar program in a command shell environment and got the same sequential performance.

Cut and paste this code to see what it does. What am I missing?

”’

import threading 
import logging
import datetime 
import time 
import random 

class Parallel: 
    
    def work(self, interval): 
        time.sleep(interval)
        name = self.__repr__()
        print (name, " is complete after ", interval, " seconds")
 


# SetupLogger()
logging.getLogger().setLevel(logging.DEBUG)
logging.debug("thread program start time is %s", datetime.datetime.now())


thread1 = Parallel()
thread2 = Parallel()
thread3 = Parallel()

print ("sequential threads::")
thread1.work(2.0)
thread2.work(2.0)
thread3.work(2.0)

logging.info("parallel threads start time is %s ", datetime.datetime.now())
start = time.time()

work1 = threading.Thread(target=thread1.work(1), daemon=True) 
work1.start()
print ("thread 1 is started and there are ", threading.activeCount(), " threads active")

work2 = threading.Thread(target=thread2.work(2), daemon=False)
work2.start()
print ("thread 2 is started and there are ", threading.activeCount(), " threads active")

work3 = threading.Thread(target=thread3.work(5), daemon=False)
work3.start()
print ("thread 3 is started and there are ", threading.activeCount(), " threads active")

# wait for all to complete 
print ("now wait for all to finish at ", datetime.datetime.now())
work1.join()
work2.join()
work3.join()
end = time.time()
logging.info ("parallel threads end time is %s with %s elapsed", datetime.datetime.now(), str(end-start))

print ("all threads completed at:", datetime.datetime.now())

”’

Answer

In the line that initializes the thread, you are actually executing the function instead of passing its reference to the thread.

thread1.work() —-> this will actually execute the function when the program runs and encounters this statement So when your program reaches this line,

work1 = threading.Thread(target=thread1.work(1), daemon=True)

and encounters target=thread1.work(1), it simply calls the function right there and the actual thread does nothing.

thread1.work is a reference to the function, which you need to pass to your Thread object. So just remove the parenthesis and your code becomes

work1 = threading.Thread(target=thread1.work, daemon=True, args=(1,))

and this will behave as you expect.

Advertisement

Answer