How to Navigate to a New Webpage In Selenium?

0
0

I have the following code:

driver.get(<some url>)
for element in driver.find_elements_by_class_name('thumbnail'):
element
.find_element_by_xpath(".//a").click() #this works and navigates to new page
element
.find_element_by_link_text('Click here').click() #this doesn't

Which needs to navigate the following HTML (simplified of course) by clicking on a thumbnail, which directs to a new page, and then needs to click the Click here link in that new page:

<!DOCTYPE html>
<html lang="en-US" prefix="og: http://iuytp.me/ns# fb: http://iuytp.me/ns/fb#">
<head>
<meta charset="UTF-8" />
<title>Releases</title>
</head>
<body class="archive category category-releases category-4 custom-background">
<div id="main">
<div id="container" class="one-column">
<div id="content" role="main">
<h1 class="page-title">Releases</h1>
<div id="thumbnail-post-display">
<div id="thumbnail-post" class="post-7158 post type-post status-publish format-standard has-post-thumbnail hentry category-blog category-designer category-releases category-uncategorized">
<div class="thumbnail"><a href="http://records.net/uncategorized/designer-7-inch-bufu-records-co-release/" title="Permanent link to Designer – 7 inch" rel="bookmark"><img width="300" height="300" src="http://records.net/dev/wp-content/uploads/2014/05/dboypledge32-300x300.png" class="attachment-thumbnail wp-post-image" alt="dboypledge3" /></a></div>
<h2><a href="http://records.net/uncategorized/designer-7-inch-bufu-records-co-release/" title="Permanent link to Designer – 7 inch" rel="bookmark">Designer &#8211; 7 inch</a></h2>
</div>
</div><!--end thumbnail post display-->
<div id="thumbnail-post-display">
<div id="thumbnail-post" class="post-7107 post type-post status-publish format-standard has-post-thumbnail hentry category-blog category-releases">
<div class="thumbnail"><a href="http://records.net/releases/people-2014-tour-demos/" title="Permanent link to All My People – 2014 Tour Demos" rel="bookmark"><img width="300" height="300" src="http://records.net/dev/wp-content/uploads/2014/04/01_Doubt-mp3-image-300x300.png" class="attachment-thumbnail wp-post-image" alt="" /></a></div>
<h2><a href="http://records.net/releases/people-2014-tour-demos/" title="Permanent link to All My People – 2014 Tour Demos" rel="bookmark">All My People &#8211; 2014 Tour Demos</a></h2>
</div>
</div><!--end thumbnail post display-->
<div id="thumbnail-post-display">
<div id="thumbnail-post" class="post-7089 post type-post status-publish format-standard has-post-thumbnail hentry category-blog category-releases">
<div class="thumbnail"><a href="http://records.net/releases/sirens-blossom-talk/" title="Permanent link to Syrins – Boss Talk" rel="bookmark"><img width="300" height="300" src="http://records.net/dev/wp-content/uploads/2014/04/sirens_final_smaller-300x300.jpg" class="attachment-thumbnail wp-post-image" alt="sirens_final_smaller" /></a></div>
<h2><a href="http://records.net/releases/sirens-blossom-talk/" title="Permanent link to Syrins – Boss Talk" rel="bookmark">Syrins &#8211; Boss Talk</a></h2>
</div>
</div><!--end thumbnail post display-->
<div id="thumbnail-post-display">
<div id="thumbnail-post" class="post-7073 post type-post status-publish format-standard has-post-thumbnail hentry category-blog category-releases">
<div class="thumbnail"><a href="http://records.net/releases/worlds-strongest-man-scares/" title="Permanent link to World’s Tough Man – Sorry Scares You" rel="bookmark"><img width="300" height="300" src="http://records.net/dev/wp-content/uploads/2014/03/a2312749950_10-300x300.jpg" class="attachment-thumbnail wp-post-image" alt="a2312749950_10" /></a></div>
<h2><a href="http://records.net/releases/worlds-strongest-man-scares/" title="Permanent link to World’s Tough Man – Sorry Scares You" rel="bookmark">World&#8217;s Tough Man &#8211; Sorry Scares You</a></h2>
</div>
</div><!--end thumbnail post display-->
<div id="thumbnail-post-display">
<div id="thumbnail-post" class="post-7046 post type-post status-publish format-standard has-post-thumbnail hentry category-blog category-releases">
<div class="thumbnail"><a href="http://records.net/releases/sundog-space-criminal/" title="Permanent link to Dog – Space Criminal" rel="bookmark"><img width="300" height="300" src="http://records.net/dev/wp-content/uploads/2014/03/Sundog_cover_high_res-300x300.jpg" class="attachment-thumbnail wp-post-image" alt="dog_cover_high_res" /></a></div>
<h2><a href="http://records.net/releases/sundog-space-criminal/" title="Permanent link to Dog – Space Criminal" rel="bookmark">Dog &#8211; Space Criminal</a></h2>
</div>
</div><!--end thumbnail post display-->
<div style="clear:both"></div>
</div><!-- #container -->
</div><!-- #main -->
</div><!-- #wrapper -->
</div><!--#bg-wrapper-->
</body>
</html>

My code however spits out the following error:

Traceback (most recent call last):
...
File "crawler.py", line 17, in main
driver
.find_element_by_link_text('Click here').click()
File "/Library/Python/2.7/site-packages/selenium/webdriver/remote/webdriver.py", line 254, in find_element_by_link_text
return self.find_element(by=By.LINK_TEXT, value=link_text)
File "/Library/Python/2.7/site-packages/selenium/webdriver/remote/webdriver.py", line 662, in find_element
{'using': by, 'value': value})['value']
File "/Library/Python/2.7/site-packages/selenium/webdriver/remote/webdriver.py", line 173, in execute
self
.error_handler.check_response(response)
File "/Library/Python/2.7/site-packages/selenium/webdriver/remote/errorhandler.py", line 164, in check_response
raise exception_class(message, screen, stacktrace)
selenium
.common.exceptions.NoSuchElementException: Message: u'no such element\n (Session info: chrome=35.0.1916.153)\n (Driver info: chromedriver=2.10.267517,platform=Mac OS X 10.9.3 x86_64)'

The problem appears to be that the element isn’t being updated with the contents of the new page. Replacing the problematic line’s element with driver doesn’t work either. What am I doing wrong?

Please note that I must be able to do this for all thumbnails (hence the for loop).

  • You must to post comments
0
0

Turns out you need to store the links you want to navigate to in advance. This is what ended up working for me (found this thread to be helpful):

driver.get(<some url>)
elements
= driver.find_elements_by_xpath("//h2/a")
links
= []
for i in range(len(elements)):
links
.append(elements[i].get_attribute('href'))
for link in links:
print 'navigating to: ' + link
driver
.get(link)
# do stuff within that page here...
driver
.back()
  • You must to post comments
Showing 1 result
Your Answer
Post as a guest by filling out the fields below or if you already have an account.
Name*
E-mail*
Website