how to get first title and url of google search with JSOUP java?

user_10130186 0
Viewed 83

hello im using jsoup for the first time and trying to get the first url and title of my search query. the code runs without any errors however nothing is showing up for some reason. Like im not getting any results in the console for some reason. Can someone explain whats going on ? Thanks !

import java.io.IOException;
import java.util.HashSet;
import java.util.Set;
import java.util.regex.Matcher;
import java.util.regex.Pattern;

import org.jsoup.Jsoup;
import org.jsoup.nodes.Document;
import org.jsoup.nodes.Element;
import org.jsoup.select.Elements;



public class autoClickapp
{



public static void main (String args[]) throws IOException
{  String query = "Milky Way";

        String url = "https://www.google.com/search?q=" + query + "&num=10";

        Document doc = Jsoup
                .connect(url)
                .userAgent("Jsoup client")
                .get();

Elements hTags = doc.select("h1, h2, h3, h4, h5, h6");
        Elements links = doc.select("a[href]");

Elements h2Tags = hTags.select("h3");

            for (int i=0; i <  links.size(); i++) {

                            for (int j=0; j <  h2Tags.size(); j++) {

                               System.out.println(links.get(0).text() + "   " + h2Tags.get(0).text());
                            }
                        }
    }

public static void print(String string) {
        System.out.println(string);
    }


}
jsoup java
1 Answers

i think it's because of the definition card, which shows at the top of any common search term.

so i tried some CSS selectors in my browser with your useragent (Jsoup client) and found and reduced a selector which is:

#main > div > div > div > a

and if that doesn't work, try this selector (works for desktop useragents):

#search > div div.g a:not([class]):not([ping])

hope one of those selectors work

public static final String SELECTOR_1 = "#main > div > div > div > a";
public static final String SELECTOR_2 = "#search > div div.g a:not([class]):not([ping])";


Elements links = doc.select(SELECTOR_1);
for (int i=0; i < links.size(); i++) {
    System.out.println(links.get(i).text())
}
user_24901150
Related