Is Selenium slow, or is my code wrong?

0
0

So I’m trying to login to Quora using Python and then scrape some stuff.

I’m using Selenium to login to the site. Here’s my code:

from selenium import webdriver
from selenium.webdriver.common.keys import Keys
driver
= webdriver.Firefox()
driver
.get('http://www.quora.com/')
username
= driver.find_element_by_name('email')
password
= driver.find_element_by_name('password')
username
.send_keys('email')
password
.send_keys('password')
password
.send_keys(Keys.RETURN)
driver
.close()

Now the questions:

  1. It took ~4 minutes to find and fill the login form, which painfully slow. Is there something I can do to speed up the process?

  2. When it did login, how do I make sure there were no errors? In other words, how do I check the response code?

  3. How do I save cookies with selenium so I can continue scraping once I login?

  4. If there is no way to make selenium faster, is there any other alternative for logging in? (Quora doesn’t have an API)

  • You must to post comments
0
0

I had a similar problem with very slow find_elements_xxx calls in Python selenium using the ChromeDriver. I eventually tracked down the trouble to a driver.implicitly_wait() call I made prior to my find_element_xxx() calls; when I took it out, my find_element_xxx() calls ran quickly.

Now, I know those elements were there when I did the find_elements_xxx() calls. So I cannot imagine why the implicit_wait should have affected the speed of those operations, but it did.

  • You must to post comments
Showing 1 result
Your Answer
Post as a guest by filling out the fields below or if you already have an account.
Name*
E-mail*
Website