When my selenium program crashes due to some error, it seems to leave behind running processes.
For example, here is my process list:
carol 30186 0.0 0.0 103576 7196 pts/11 Sl 00:45 0:00 /home/carol/test/chromedriver --port=51789 carol 30322 0.0 0.0 102552 7160 pts/11 Sl 00:45 0:00 /home/carol/test/chromedriver --port=33409 carol 30543 0.0 0.0 102552 7104 pts/11 Sl 00:48 0:00 /home/carol/test/chromedriver --port=42567 carol 30698 0.0 0.0 102552 7236 pts/11 Sl 00:50 0:00 /home/carol/test/chromedriver --port=46590 carol 30938 0.0 0.0 102552 7496 pts/11 Sl 00:55 0:00 /home/carol/test/chromedriver --port=51930 carol 31546 0.0 0.0 102552 7376 pts/11 Sl 01:16 0:00 /home/carol/test/chromedriver --port=53077 carol 31549 0.5 0.0 0 0 pts/11 Z 01:16 0:03 [chrome] <defunct> carol 31738 0.0 0.0 102552 7388 pts/11 Sl 01:17 0:00 /home/carol/test/chromedriver --port=55414 carol 31741 0.3 0.0 0 0 pts/11 Z 01:17 0:02 [chrome] <defunct> carol 31903 0.0 0.0 102552 7368 pts/11 Sl 01:19 0:00 /home/carol/test/chromedriver --port=54205 carol 31906 0.6 0.0 0 0 pts/11 Z 01:19 0:03 [chrome] <defunct> carol 32083 0.0 0.0 102552 7292 pts/11 Sl 01:20 0:00 /home/carol/test/chromedriver --port=39083 carol 32440 0.0 0.0 102552 7412 pts/11 Sl 01:24 0:00 /home/carol/test/chromedriver --port=34326 carol 32443 1.7 0.0 0 0 pts/11 Z 01:24 0:03 [chrome] <defunct> carol 32691 0.1 0.0 102552 7360 pts/11 Sl 01:26 0:00 /home/carol/test/chromedriver --port=36369 carol 32695 2.8 0.0 0 0 pts/11 Z 01:26 0:02 [chrome] <defunct>
Here is my code:
from selenium import webdriver browser = webdriver.Chrome("path/to/chromedriver") browser.get("http://stackoverflow.com") browser.find_element_by_id('...').click() browser.close()
Sometimes, the browser doesn’t load the webpage elements quickly enough so Selenium crashes when it tries to click on something it didn’t find. Other times it works fine.
This is a simple example for simplicity sake, but with a more complex selenium program, what is a guaranteed clean way of exiting and not leave behind running processes? It should cleanly exit on an unexpected crash and on a successful run.
Advertisement
Answer
Whats happening is that your code is throwing an exception, halting the python process from continuing on. As such, the close/quit methods never get called on the browser object, so the chromedrivers just hang out indefinitely.
You need to use a try/except block to ensure the close method is called every time, even when an exception is thrown. A very simplistic example is:
from selenium import webdriver browser = webdriver.Chrome("path/to/chromedriver") try: browser.get("http://stackoverflow.com") browser.find_element_by_id('...').click() except: browser.close() browser.quit() # I exclusively use quit
There are a number of much more sophisticated approaches you can take here, such as creating a context manager to use with the with
statement, but its difficult to recommend one without having a better understanding of your codebase.