The problem, in my case, was that I was using Amazon's Cloudflare and Cloudfront's Cloudfront in tandem, and Cloudfront did not like the settings that I had provided Cloudflare.

More specifically, in the Crypto settings on Cloudflare, I had set the "Minimum TLS Settings" to 1.2, without enabling the TLS 1.2 communication setting for the distribution in Cloudfront. This was enough to make Cloudfront declare a 502 Bad Gateway error when it tried to connect to the Cloudflare-protected server.

To fix this, I had to disable SSLv3 support in the Origin Settings for that Cloudfront distribution, and enable TLS 1.2 as a supported protocol for that origin server.

To debug this problem, I used command-line versions of curl, to see what Cloudfront was actually returning when you asked for an image from its CDN, and I also used the command-line version of openssl, to determine exactly which protocols Cloudflare was offering (it wasn't offering TLS 1.0).

tl:dr; make sure everything accepts and asks for TLS 1.2, or whatever latest and greatest TLS everyone is using by the time you read this.

For my particular case it was due to the fact that the Origin ALB behind my CloudFront Behavior had a DEFAULT ACM certificate which was pointing to a different domain name.

To fix this I had to:

1. Go to the ALB
2. Under the Listeners tab, selected my Listener and then Edit
3. Under the Default SSL Certificate, choose the correct Origin Certificate.

Beware the Origin Protocol Policy:

For HTTPS viewer requests that CloudFront forwards to this origin, one of the domain names in the SSL certificate on your origin server must match the domain name that you specify for Origin Domain Name. Otherwise, CloudFront responds to the viewer requests with an HTTP status code 502 (Bad Gateway) instead of returning the requested object.

In most cases, you probably want CloudFront to use "HTTP Only", since it fetches objects from a server probably hosted with Amazon too. No need for additional HTTPS complexity at this step.

Note that this is different to the Viewer Protocol Policy. You can read more about the differences between the two here.

Make sure you have correctly configured your SSL/TLS/Cipher settings. CloudFront will drop HTTPS connections if your origin server is not handshaking with the propper TLS Ciphers.

I recommend the following settings:

# Apache
SSLCipherSuite 'ECDHE+AES:@STRENGTH:+AES256'
SSLCipherSuite 'ECDHE+AES:DHE+AES:@STRENGTH:+AES256:kRSA+3DES'
SSLProtocol all -SSLv3 -TLSv1 -TLSv1.1
SSLProtocol all -SSLv3
SSLHonorCipherOrder on

# Nginx
ssl_ciphers 'ECDHE+AES:@STRENGTH:+AES256';
ssl_ciphers 'ECDHE+AES:DHE+AES:@STRENGTH:+AES256:kRSA+3DES';
ssl_protocols all -SSLv3 -TLSv1 -TLSv1.1;
ssl_protocols all -SSLv3;
ssl_prefer_server_ciphers on;

The @STRENGTH directive will sort the ciphers in order of strength.

SSLHonorCipherOrder on (apache) and ssl_prefer_server_ciphers on (Nginx) will ensure that the order is respected.

You can view the complete list of available ciphers your version of openssl is supporting by running the command on the shell:

$ openssl ciphers -v 'ALL'

You can also list the available ciphers for the above suggested directives in strength order in a similar fashion:

$ openssl ciphers -v 'ECDHE+AES:@STRENGTH:+AES256'

If you are having such issues, I strongly suggest you increase the verbosity of your web server logs, in particular, pertaining to SSL.

In Apache, you must may do this with the following directive in your conf file

LogLevel debug

Here is a list of possible Apache LogLevel directives:

emerg (emergencies - system is unusable)
alert (action must be taken immediately)
crit (critical conditions)
error (error conditions)
warn (warning conditions)
notice (normal but significant condition)
info (informational)
debug (debug-level messages)
trace1 (trace messages)
trace2 (trace messages)
trace3 (trace messages)
trace4 (trace messages)
trace5 (trace messages)
trace6 (trace messages)
trace7 (trace messages, dumping large amounts of data)
trace8 (trace messages, dumping large amounts of data)

Generally speaking, debug will be sufficient for you to identify the negotiation issue (it might be a cipher issue, it might be a cert issue)

A typical reaction from CloudFront not being able to successfully negotiate SSL with your origin server would look something like this:

[ssl:info] [pid 25091] [client xxx.xxx.xxx.xxx:15078] AH01964: Connection to child 1 established (server example.com:443)
[ssl:debug] [pid 25091] ssl_engine_kernel.c(2372): [client xxx.xxx.xxx.xx:15078] AH02043: SSL virtual host for servername example.com found
[ssl:debug] [pid 25091] ssl_engine_io.c(1368): (104)Connection reset by peer: [client xxx.xxx.xxx.xxx:15078] AH02007: SSL handshake interrupted by system [Hint: Stop button pressed in browser?!]
[ssl:info] [pid 25091] [client xxx.xxx.xxx.xxx.15078] AH01998: Connection closed to child 1 with abortive shutdown (server example.com:443)

In this case, Apache interprets CloudFront's dropping of the connection by the "Hint: stop button pressed in browser?!". Yeah, kinda comical.

On a side note, CloudFront can be a tricky beast. If you force HTTPS or direct HTTP to HTTPS in your distribution, you must ensure that the communication between CloudFront and your origin server is running over a valid SSL connection with a valid SSL certificate. This may mean obtaining a certificate over Amazon ACM by linking it over an Elastic Load Balancer (ELB).

... or you may also achieve the desired results by having the ACM certificate linked to your distribution (make sure you have both the apex domain and subdomains listed, ex: example.com and *.example.com). This will propagate the ACM certificate to your origin/target in the distribution, however, your Apache/Nginx server in your origin server will have to have a valid and working SSL certificate, even if it is generated by something like Letsencrypt/Certbot (our some other valid non-self-signed certificate). Make sure you have the full chain configured in your apache *.conf setup.

You may read more about issues pertaining to 502 Bad Gateway or 502 Could Not Satisfy The Request from AWS as well as requiring HTTPS for communication between CloudFront and your custom origin (this may also help). This information about ciphers was extremely helpful, as well as the list of supported ciphers on AWS

Some of these commands may also be useful for debugging your SSL situation:

openssl s_client -connect test.example.com:443 -tls1_1

(you may experiment using -tls1_2 , -tls1_3 or other protocols)

You may also want to try out using the http command line tool (you might have to install it), which would give you an output like such:

$ http https://example.com --head

HTTP/1.1 301 Moved Permanently
Connection: keep-alive
Content-Length: 251
Content-Type: text/html; charset=iso-8859-1
Date: Mon, 20 Dec 2021 19:20:15 GMT
Location: https://example.com
Server: Apache
Strict-Transport-Security: max-age=63072000; includeSubdomains; preload
Via: 1.1 xxxxxxxxxxxxxxxxxxxxxxxxxxxxx.cloudfront.net (CloudFront)
X-Amz-Cf-Id: xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx==
X-Amz-Cf-Pop: GIG51-C2
X-Cache: Miss from cloudfront
X-Content-Type-Options: nosniff
X-Frame-Options: SAMEORIGIN

Good luck debugging, and remember: keep digging deeper, don't give up, and increase the verbosity of your logs!