C# How can I check if a URL exists/is valid?

Viewed 291345

I am making a simple program in visual c# 2005 that looks up a stock symbol on Yahoo! Finance, downloads the historical data, and then plots the price history for the specified ticker symbol.

I know the exact URL that I need to acquire the data, and if the user inputs an existing ticker symbol (or at least one with data on Yahoo! Finance) it works perfectly fine. However, I have a run-time error if the user makes up a ticker symbol, as the program tries to pull data from a non-existent web page.

I am using the WebClient class, and using the DownloadString function. I looked through all the other member functions of the WebClient class, but didn't see anything I could use to test a URL.

How can I do this?

14 Answers

You could issue a "HEAD" request rather than a "GET"? So to test a URL without the cost of downloading the content:

// using MyClient from linked post
using(var client = new MyClient()) {
    client.HeadOnly = true;
    // fine, no content downloaded
    string s1 = client.DownloadString("http://google.com");
    // throws 404
    string s2 = client.DownloadString("http://google.com/silly");
}

You would try/catch around the DownloadString to check for errors; no error? It exists...


With C# 2.0 (VS2005):

private bool headOnly;
public bool HeadOnly {
    get {return headOnly;}
    set {headOnly = value;}
}

and

using(WebClient client = new MyClient())
{
    // code as before
}

If I understand your question correctly, you could use a small method like this to give you the results of your URL test:

WebRequest webRequest = WebRequest.Create(url);  
WebResponse webResponse;
try 
{
  webResponse = webRequest.GetResponse();
}
catch //If exception thrown then couldn't get response from address
{
  return 0;
} 
return 1;

You could wrap the above code in a method and use it to perform validation. I hope this answers the question you were asking.

Try this (Make sure you use System.Net):

public bool checkWebsite(string URL) {
   try {
      WebClient wc = new WebClient();
      string HTMLSource = wc.DownloadString(URL);
      return true;
   }
   catch (Exception) {
      return false;
   }
}

When the checkWebsite() function gets called, it tries to get the source code of the URL passed into it. If it gets the source code, it returns true. If not, it returns false.

Code Example:

//The checkWebsite command will return true:
bool websiteExists = this.checkWebsite("https://www.google.com");

//The checkWebsite command will return false:
bool websiteExists = this.checkWebsite("https://www.thisisnotarealwebsite.com/fakepage.html");
WebRequest request = WebRequest.Create("http://www.google.com");
try
{
     request.GetResponse();
}
catch //If exception thrown then couldn't get response from address
{
     MessageBox.Show("The URL is incorrect");`
}

A lot of the answers are older than HttpClient (I think it was introduced in Visual Studio 2013) or without async/await functionality, so I decided to post my own solution:

private static async Task<bool> DoesUrlExists(String url)
{
    try
    {
        using (HttpClient client = new HttpClient())
        {
            //Do only Head request to avoid download full file
            var response = await client.SendAsync(new HttpRequestMessage(HttpMethod.Head, url));

            if (response.IsSuccessStatusCode) {
                //Url is available is we have a SuccessStatusCode
                return true;
            }
            return false;
        }                
    } catch {
            return false;
    }
}

I use HttpClient.SendAsync with HttpMethod.Head to make only a head request, and not downlaod the whole file. Like David and Marc already say there is not only http 200 for ok, so I use IsSuccessStatusCode to allow all Sucess Status codes.

Web servers respond with a HTTP status code indicating the outcome of the request e.g. 200 (sometimes 202) means success, 404 - not found etc (see here). Assuming the server address part of the URL is correct and you are not getting a socket timeout, the exception is most likely telling you the HTTP status code was other than 200. I would suggest checking the class of the exception and seeing if the exception carries the HTTP status code.

IIRC - The call in question throws a WebException or a descendant. Check the class name to see which one and wrap the call in a try block to trap the condition.

A lot of other answers are using WebRequest which is now obsolete.

Here is a method that has minimal code and uses currently up-to-date classes and methods.

I have also tested the other most up-voted functions which can produce false positives. I tested with these URLs, which points to the Visual Studio Community Installer, found on this page.

//Valid URL
https://aka.ms/vs/17/release/vs_community.exe

//Invalid URL, redirects. Produces false positive on other methods.
https://aka.ms/vs/14/release/vs_community.exe
using System.Net;
using System.Net.Http;

//HttpClient is not meant to be created and disposed frequently.
//Declare it staticly in the class to be reused.
static HttpClient client = new HttpClient();

/// <summary>
/// Checks if a remote file at the <paramref name="url"/> exists, and if access is not restricted.
/// </summary>
/// <param name="url">URL to a remote file.</param>
/// <returns>True if the file at the <paramref name="url"/> is able to be downloaded, false if the file does not exist, or if the file is restricted.</returns>
public static bool IsRemoteFileAvailable(string url)
{
    //Checking if URI is well formed is optional
    Uri uri = new Uri(url);
    if (!uri.IsWellFormedOriginalString())
        return false;

    try
    {
        using (HttpRequestMessage request = new HttpRequestMessage(HttpMethod.Head, uri))
        using (HttpResponseMessage response = client.Send(request))
        {
            return response.IsSuccessStatusCode  && response.Content.Headers.ContentLength > 0;
        }
    }
    catch
    {
        return false;
    }
}

Just note that this will not work with .NET Framework, as HttpClient.Send does not exist. To get it working on .NET Framework you will need to change client.Send(request) to client.SendAsync(request).Result.

Related