I just went through this today. I'm assuming you already have Git for Windows installed and have generated the ssh keys located in your user's ".ssh" folder.
After that, you must go to the settings in your GitHub account online and look for the "SSH and GPG Keys" option in the menu on the left:

This is where you have to enter your public key by opening the "id_rsa.pub" file and then copying its contents into your GitHub account by clicking "New SSH key":

Then you can just use vscode like you would with a regular HTTP connection. Just Crtl+Shift+P to open vscode's Command Pallet, select git: clone, paste the SSH address of GitHub repo you want. In order to get the SSH address when you click the green "Clone or Download" button on a GitHub repo you must also click "Clone with SSH" in the box that pops up so you copy the SSH address instead of using the HTTPS link.
The only hangup you may have is, the first time you try ssh in vscode it will say, "host cannot be identified" The way to fix this is to simply do a git clone from a git bash command line outside of vscode so you can type "yes" to when it asks you if it's ok to add GitHub's SSH key fingerprint.
ROB@WINDOW10 MINGW64 ~/Documents
$ git clone git@github.com:Rob4226/example.git
Cloning into 'Example Repo'...
The authenticity of host 'github.com (140.82.114.3)' can't be established.
RSA key fingerprint is SHA256:nThbg6kXUpJWGl7E1IGOCspRomTxdCARLviKw6E5SY8.
Are you sure you want to continue connecting (yes/no/[fingerprint])? **yes**
Warning: Permanently added 'github.com,140.82.114.3' (RSA) to the list of known hosts.
For whatever reason the vscode git source control UI doesn't show this message for you to respond so it just throws an error. But after you say "yes" the first time in a git bash command line, ssh will work in vscode from now on without issue. It works great and is so much faster! I never use HTTPS anymore from vscode.
Or, you can manually add GitHub's SSH key fingerprints to your "known_hosts" file in the .ssh folder. They can be found here: https://help.github.com/en/github/authenticating-to-github/githubs-ssh-key-fingerprints
GitHub's SSH key fingerprints
Public key fingerprints can be used to validate a connection to a remote server.
These are GitHub's public key fingerprints (in hexadecimal format):
16:27:ac:a5:76:28:2d:36:63:1b:56:4d:eb:df:a6:48 (RSA)
ad:1c:08:a4:40:e3:6f:9c:f5:66:26:5d:4b:33:5d:8c (DSA)
These are the SHA256 hashes shown in OpenSSH 6.8 and newer (in base64 format):
SHA256:nThbg6kXUpJWGl7E1IGOCspRomTxdCARLviKw6E5SY8 (RSA)
SHA256:br9IjFspm1vxR3iA35FWE+4VTyz1hYVLIE2t1/CeyWQ (DSA)