Casperjs How To Click Multiple Links In A Table While Collecting Data From The Web /.click() Doesn't Work?

I want to scrape some web data using CasperJS. The data is in a table, in each row there is a link leading to a page with more detail. In the script there is a loop iterating throu

Solution 1:

I see two things here that need to be corrected. First, The for loop in your code doesn't appear to be in the scope of any casperjs methods.

This:

for (var i = 1; i <= result_number; i++)

It should be inside a casper.then method. I notice you have the closing brackets so perhaps you've posted code by copy pasting it in a sloppy manner.

Secondly and most importantly, the tr:nth-child(' + i + ') you'd like to interact with won't work in this way. I don't know why but it doesn't seem to work this straight forwardly. I've tried to do the same thing. My solution was to first of all convert the i to a string instead of a number like so:

pageturn = pageturn + 1;
// Collect <td> contents on each page.var pageturnString = pageturn.toString();
var linknum = 'a.SomeLinkClass:nth-child('+pageturnString+')';

in my case I'm using this to click to change the page, either way you must encapsulate your interaction with the said css selector inside a this.then() method inside the first method, and then a second child method does the rest of the for loop.

Example:

casper.each(pagecount, function() {
    this.then(function() {
        pageturn = pageturn + 1;
        // Collect <td> contents on each page.var pageturnString = pageturn.toString();
        var linknum = 'a.SomeLinkClass:nth-child('+pageturnString+')';
    });

    this.then(function() {
        //Now run for loop here. 
    });
 });

If you don't encapsulate the css selector construction within the this.then() method before it's used in the next method, it won't work. I don't know why but that's the deal. In my code, pagecount could possibly be used instead of your for loop but I'll leave that up to you.

Solution 2:

I've got a page where I'm seeing this in Casper:

[debug] [phantom] Mouse event 'mousedown' on selector: tr:nth-child(2) a CasperError: Cannot dispatch mousedown event on nonexistent selector: tr:nth-child(2) a

As this error is caused by a failed exists, which relies on querySelectorAll, I've played around with that and found that the following sets x2 to null (although x1 isn't null):

this.evaluate(function() {
    var x1 = document.querySelector('tr:nth-child(2) a');
    var x2 = document.querySelector('tr:nth-child(2) a');
    alert(x1 + ', ' + x2);
});

It seems to depend on there being a row that doesn't contain an <a>, as you'd find in a header row. Here's a test page:

http://jsfiddle.net/GKb2g/4/

I'll hopefully post back here once I've found the cause, but in the meantime, you're best off using a selectXPath selector.