I am relatively new to Python and trying to implement a Multiprocessing module for my for loop.
I have an array of Image url’s stored in img_urls which I need to download and apply some Google vision.
if __name__ == '__main__': img_urls = [ALL_MY_Image_URLS] runAll(img_urls) print("--- %s seconds ---" % (time.time() - start_time))
This is my runAll() method
def runAll(img_urls): num_cores = multiprocessing.cpu_count() print("Image URLS {}",len(img_urls)) if len(img_urls) > 2: numberOfImages = 0 else: numberOfImages = 1 start_timeProcess = time.time() pool = multiprocessing.Pool() pool.map(annotate,img_urls) end_timeProcess = time.time() print('n Time to complete ', end_timeProcess-start_timeProcess) print(full_matching_pages) def annotate(img_path): file = requests.get(img_path).content print("file is",file) """Returns web annotations given the path to an image.""" print('Process Working under ',os.getpid()) image = types.Image(content=file) web_detection = vision_client.web_detection(image=image).web_detection report(web_detection)
I am getting this as the warning when I run it and python crashes
objc[67570]: +[__NSPlaceholderDate initialize] may have been in progress in another thread when fork() was called. objc[67570]: +[__NSPlaceholderDate initialize] may have been in progress in another thread when fork() was called. We cannot safely call it or ignore it in the fork() child process. Crashing instead. Set a breakpoint on objc_initializeAfterForkError to debug. objc[67567]: +[__NSPlaceholderDate initialize] may have been in progress in another thread when fork() was called. objc[67567]: +[__NSPlaceholderDate initialize] may have been in progress in another thread when fork() was called. We cannot safely call it or ignore it in the fork() child process. Crashing instead. Set a breakpoint on objc_initializeAfterForkError to debug. objc[67568]: +[__NSPlaceholderDate initialize] may have been in progress in another thread when fork() was called. objc[67568]: +[__NSPlaceholderDate initialize] may have been in progress in another thread when fork() was called. We cannot safely call it or ignore it in the fork() child process. Crashing instead. Set a breakpoint on objc_initializeAfterForkError to debug. objc[67569]: +[__NSPlaceholderDate initialize] may have been in progress in another thread when fork() was called. objc[67569]: +[__NSPlaceholderDate initialize] may have been in progress in another thread when fork() was called. We cannot safely call it or ignore it in the fork() child process. Crashing instead. Set a breakpoint on objc_initializeAfterForkError to debug. objc[67571]: +[__NSPlaceholderDate initialize] may have been in progress in another thread when fork() was called. objc[67571]: +[__NSPlaceholderDate initialize] may have been in progress in another thread when fork() was called. We cannot safely call it or ignore it in the fork() child process. Crashing instead. Set a breakpoint on objc_initializeAfterForkError to debug. objc[67572]: +[__NSPlaceholderDate initialize] may have been in progress in another thread when fork() was called. objc[67572]: +[__NSPlaceholderDate initialize] may have been in progress in another thread when fork() was called. We cannot safely call it or ignore it in the fork() child process. Crashing instead. Set a breakpoint on objc_initializeAfterForkError to debug.
Advertisement
Answer
This error occurs because of added security to restrict multithreading in macOS High Sierra and later versions of macOS. I know this answer is a bit late, but I solved the problem using the following method:
Set an environment variable .bash_profile
(or .zshrc
for recent macOS) to allow multithreading applications or scripts under the new macOS High Sierra security rules.
Open a terminal:
$ nano .bash_profile
Add the following line to the end of the file:
OBJC_DISABLE_INITIALIZE_FORK_SAFETY=YES
Save, exit, close terminal and re-open the terminal. Check to see that the environment variable is now set:
$ env
You will see output similar to:
TERM_PROGRAM=Apple_Terminal SHELL=/bin/bash TERM=xterm-256color TMPDIR=/var/folders/pn/vasdlj3ojO#OOas4dasdffJq/T/ Apple_PubSub_Socket_Render=/private/tmp/com.apple.launchd.E7qLFJDSo/Render TERM_PROGRAM_VERSION=404 TERM_SESSION_ID=NONE OBJC_DISABLE_INITIALIZE_FORK_SAFETY=YES
You should now be able to run your Python script with multithreading.