Reading URL goes wrong

user_1042952 0
Viewed 106

I would like to read https://www.instagram.com/mobonews/?__a=1 using Java language. The source code of following URL is equal to:

{"logging_page_id":"profilePage_1410389643","show_suggested_profiles":false,"show_follow_dialog":false,"graphql":{"user":{"biography":"\u200f\u200e\u0645\u0627\u062c\u0631\u0627\u062c\u0648\u06cc\u06cc\u200c\u0647\u0627\u06cc \u0645\u0646 \u062f\u0631 \u062

But the following code returns this instead:

<!DOCTYPE html><html lang="en" class="no-js not-logged-in client-root">    <head>        <meta charset="utf-8">        <meta http-equiv="X-UA-Compatible" content="IE=edge">        <title>Login • Instagram</title>                <meta name="robots" content="noimageindex, noarchive">        <meta name="apple-mobile-web-app-status-bar-style" content="default">        <meta name="mobile-web-app-capable" content="yes">        <meta name="theme-color" content="#ffffff">        <meta id="viewport" name="viewport"

Here is the code I used:

    URL website = new URL(url);
    URLConnection connection = website.openConnection();
    BufferedReader in = new BufferedReader(new InputStreamReader(
                                connection.getInputStream()));

    StringBuilder response = new StringBuilder();
    String inputLine;

    while ((inputLine = in.readLine()) != null) 
        response.append(inputLine);

EDIT:

import java.io.IOException;
import java.io.InputStream;
import java.net.URL;
import java.net.URLConnection;
import org.apache.commons.io.IOUtils;

public class TestReadurlInsta {

    /**
     * @param args the command line arguments
     */
    public static void main(String[] args) throws IOException {
        
  URL u = new URL("https://www.instagram.com/mobonews/?__a=1");

URLConnection con = u.openConnection();
InputStream in = con.getInputStream();
String encoding = con.getContentEncoding(); 
encoding = encoding == null ? "UTF-8" : encoding;
String body = IOUtils.toString(in, encoding);
System.out.println(body);
    
    }

EDIT 2:

It seems that I get the login page of instagram for unknown reasons:

<!DOCTYPE html> <html lang="en" class="no-js not-logged-in client-root">     <head>         <meta charset="utf-8">         <meta http-equiv="X-UA-Compatible" content="IE=edge">          <title> Login • Instagram </title>

In the past I ran same code from same machine and itt was alright but suddenly it gets me into this problem.

EDIT 3:

I ran same code from an online IDE and have received following exception. Seems refused to get a connection and as @Holger said, there might be a prevention from Instagram to access the resource:

Exception in thread "main" java.net.UnknownHostException: www.instagram.com
at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:184)
at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
at java.net.Socket.connect(Socket.java:589)
at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:673)
at sun.security.ssl.BaseSSLSocketImpl.connect(BaseSSLSocketImpl.java:173)
at sun.net.NetworkClient.doConnect(NetworkClient.java:180)
at sun.net.www.http.HttpClient.openServer(HttpClient.java:463)
at sun.net.www.http.HttpClient.openServer(HttpClient.java:558)
at sun.net.www.protocol.https.HttpsClient.<init>(HttpsClient.java:264)
at sun.net.www.protocol.https.HttpsClient.New(HttpsClient.java:367)
at sun.net.www.protocol.https.AbstractDelegateHttpsURLConnection.getNewHttpClient(AbstractDelegateHttpsURLConnection.java:191)
at sun.net.www.protocol.http.HttpURLConnection.plainConnect0(HttpURLConnection.java:1138)
at sun.net.www.protocol.http.HttpURLConnection.plainConnect(HttpURLConnection.java:1032)
at sun.net.www.protocol.https.AbstractDelegateHttpsURLConnection.connect(AbstractDelegateHttpsURLConnection.java:177)
at sun.net.www.protocol.http.HttpURLConnection.getInputStream0(HttpURLConnection.java:1546)
at sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1474)
at sun.net.www.protocol.https.HttpsURLConnectionImpl.getInputStream(HttpsURLConnectionImpl.java:254)
at java.net.URL.openStream(URL.java:1045)
at HelloWorld.main(HelloWorld.java:14)

But is there any solution to this?

httpurlconnection url stream json java
1 Answers

I passed the url as a string to the function and used the bufferedReader. It seems like you just print the body of the page so you don't get the source.

Here's the code I used

public class Solution {
    public static void main(String[] args) throws IOException {
        String str = getContent("https://www.instagram.com/mobonews/?__a=1");
        System.out.println(str);
    }
    public static String getContent(String url) throws IOException {
        URL website = new URL(url);
        URLConnection connection = website.openConnection();
        BufferedReader in = new BufferedReader(new InputStreamReader(
                connection.getInputStream()));

        StringBuilder response = new StringBuilder();
        String inputLine = "";
        
            inputLine = in.readLine();

        while(inputLine != null) {
            response.append(inputLine);
            inputLine = in.readLine();
        }
        return response.toString();
    }
}
user_145359290
Related