How can I set up a proxy for an unowned site to test front-end changes?

user_14796948 0
Viewed 164

I would like to set up a proxy to facilitate local front-end development.

For example if I go to https://google.com/ all files are loaded from Google, except for some defined set of files like all *.css files, which would come from the file system.

Right now I am using Fiddler to achieve that goal, but it is very manual and not easily set up for other developers. If it is possible with Fiddler, I am assuming it is possible to automate that set up. I have tried lots of configurations and researched

If possible I would like to achieve this with webpack-dev-server, or some other scriptable web framework like Express. The developers have access to their host files to make any local DNS changes that might be required. Ideally the set up would work on Windows and Mac, but Windows is the larger target.

Here is an example of what the webpack devserver config might look like:

devServer: {
  hot: false,
  open: true,
  liveReload: false,
  https: true,
  watchContentBase: true,
  compress: false,
  allowedHosts: [
    'google.com'
  ],
  host: "google.com",    
  proxy: {
    "*.css": {
      target: "http://localhost:3000"
    },
    "*": {
      target: "https://google.com/",
      secure: false,
      changeOrigin: true,
    }
  }
}
webpack-dev-server webpack frontend express node.js
1 Answers

I was able to figure out the various config options I needed to achieve what I wanted. Here are a few variations I went through to get to a usable proxy. All places I have used google.com are replaceable with any external host you would like. The contexts of each proxy uses glob pattern so you can capture/ignore whatever you would like.

The below config will serve you a fully useable https://google.com through https://localhost:3000, except for /test.html and everything in /dist. This works great if your development file structure matches the website file structure. 

devServer: {
    liveReload: true,
    open: false,
    https: true,       
    compress: false,
    port: 3000,
    stats: 'minimal',
    proxy: [
      {
        context: ['**', '!/test.html', '!/dist/**'],
        target: "https://google.com/",
        secure: false,
        changeOrigin: true,
        autoRewrite: true,
        hostRewrite: true,
        logLevel: 'debug'
      }

    ]
  }

This configuration allows you to have the exact same domain name as the external site. This requires you to add an host entry to point the domain at your local machine. This might be required if the site you are dealing with has some redirects or authentication that requires the proper domain name the requests. This also assumes that the files you are trying to replace are in the same structure in your dev environment.

devServer: {
    hot: false,
    liveReload: true,
    open: false,
    https: true,
    compress: false,
    public: 'google.com',       
    host: "google.com",
    //You must match the port the site is served on. On an windows machine you will have to stop IIS to let webpack devServer run on 443
    port: 443,
    stats: 'minimal',
    proxy: [
      {
        context: ['**', '!/test.html', '!/dist/**'],
        //This is the IP the site is running on. 
        target: "https://2607:f8b0:400f:805::2004/",
        secure: false,
        changeOrigin: false,
        autoRewrite: true,
        hostRewrite: true,
        logLevel: 'debug'
      }

    ]
  }

This configuration allows you to rewrite the path of requested files. It uses two proxys to achieve this goal. The first proxy is just like the previous ones, that capture all traffic expect for any specified patterns. The second rewrites the paths of any requests that you specify, and forwards the request with the new path. The pathRewrite property accepts a function, so you can get as complex as you would need to.

devServer: {
    hot: false,
    liveReload: true,
    open: false,
    https: true,
    compress: false,
    public: 'google.com',
    
    host: "google.com",
    //You must match the port the site is served on. On an windows machine you will have to stop IIS to let webpack devServer run on 443
    port: 443,
    stats: 'minimal',
    proxy: [
      {
        context: ['**', '!/test.html', '!/dist/**', '!/pathOfFileToReplace/test.js'],

        //This is the IP the site is running on. 
        target: "https://2607:f8b0:400f:805::2004/",
        secure: false,
        changeOrigin: false,
        autoRewrite: true,
        hostRewrite: true,
        logLevel: 'debug'
      },
      {
        context: [
            '/pathOfFileToReplace/test.js',
        ],
        pathRewrite: {
          "/pathOfFileToReplace/" : "/dist/newPath/"
        },
        target: "https://google.com/",
        secure: false,
        changeOrigin: true,
        autoRewrite: true,
        hostRewrite: true,
        logLevel: 'debug'
      },
    ]
  }

Some notes:

  • I do not recommend this approach unless it is unavoidable. There are a few reason not to do this, but the biggest being there are lots of ways you can open yourself up to malicious attacks.
  • Browsers store DNS caches for a while, so when you update your host entry to proxy a site you should use a completely new private window.
  • You will get certificate errors when you proxy a site using its actual domain name. This is expected, and unavoidable unless you can get the certificate of the site you are proxying.
  • There are alot more options that are available to you to get the perfect set up going. You can find them in https://webpack.js.org/configuration/dev-server/ and https://github.com/chimurai/http-proxy-middleware.
user_147969480
Related