Skip to content
Advertisement

Merge using threads not working in python

I have to merge two lists and every time a full the lists in order to merge them , but what is happening that I did it like this :

   def repeated_fill_buffer(self):
        """
      repeat the operation until reaching the end of file
        """
        # clear buffers from last data
        self.block = [[] for file in self.files]

        filling_buffer_thread = threading.Thread(self.fill_buffer())
        filling_buffer_thread.start()

        # create inverted index thread
        create_inverted_index_thread = threading.Thread(self.create_inverted_index())
        create_inverted_index_thread.start()

        # check if buffers are not empty to merge and start the thread
        if any(self.block):
            self.block = [[] for file in self.files]
            filling_buffer_thread.join()
            create_inverted_index_thread.join()

but what is happening that filling_buffer_thread and create_inverted_index_thread just called one time, and not working again, when I debugged the code I see that

filling_buffer_thread stopped

I don’t know if I explain my question good, but what I want that I can called same thread multi time and run them..

Advertisement

Answer

If there is any operation which is CPU Bound then, using thread is of no use. Because of Python GIL, which prevents multiple byte-code instruction to be executed at a time. use multiprocessing module since, every process has its own GIL.

All number crunching or any operations which depends on CPU for its completion are CPU-Bound. Threads are useful for I/O Bound Operations (like Database Calls, Network Calls)

To summarize your error, your filling_buffer_thread got blocked due to create_inverted_index_thread

User contributions licensed under: CC BY-SA
2 People found this is helpful
Advertisement