Extracting TLS secrets in Go

Question

I'm not totally sure whether this is really a Wireshark, Go, or Syncthing question; I tried the Wireshark dev list and the Go dev list but got no response, so I figured I'll try here:

I'm working on a Wireshark Syncthing dissector. Since most of the Syncthing protocols are encapsulated in TLS, I need to provide the TLS secrets to Wireshark.

I read the Wireshark TLS documentation; Syncthing is written in Go, so I patched it to export TLS secrets, like this (this is just Syncthing upstream code with the addition of the two final lines):

// The TLS configuration is used for both the listening socket and outgoing
// connections.

var tlsCfg *tls.Config
if a.cfg.Options().InsecureAllowOldTLSVersions {
    l.Infoln("TLS 1.2 is allowed on sync connections. This is less than optimally secure.")
    tlsCfg = tlsutil.SecureDefaultWithTLS12()
} else {
    tlsCfg = tlsutil.SecureDefaultTLS13()
}
tlsCfg.Certificates = []tls.Certificate{a.cert}
tlsCfg.NextProtos = []string{bepProtocolName}
tlsCfg.ClientAuth = tls.RequestClientCert
tlsCfg.SessionTicketsDisabled = true
tlsCfg.InsecureSkipVerify = true

// The following two lines open a file in the current directory and configure the application to dump its TLS secrets there
// See: https://pkg.go.dev/crypto/tls#example-Config-KeyLogWriter

w, err := os.OpenFile("tls-secrets.txt", os.O_WRONLY|os.O_CREATE|os.O_TRUNC, 0600)
tlsCfg.KeyLogWriter = w

This works, and various stuff is written to the specified file, but providing that file to Wireshark doesn't enable TLS decryption. I examined the file, and I see that it contains CLIENT_HANDSHAKE_TRAFFIC_SECRET, SERVER_HANDSHAKE_TRAFFIC_SECRET, CLIENT_TRAFFIC_SECRET_0, and SERVER_TRAFFIC_SECRET_0 lines, but not the crucial CLIENT_RANDOM lines. Am I doing something wrong or missing something?

user_3011380 · Answer

The root cause is the difference between tls1.2 and tls1.3, the difference could be found here

Per golang tls code

    keyLogLabelTLS12           = "CLIENT_RANDOM"
    keyLogLabelClientHandshake = "CLIENT_HANDSHAKE_TRAFFIC_SECRET"
    keyLogLabelServerHandshake = "SERVER_HANDSHAKE_TRAFFIC_SECRET"
    keyLogLabelClientTraffic   = "CLIENT_TRAFFIC_SECRET_0"
    keyLogLabelServerTraffic   = "SERVER_TRAFFIC_SECRET_0"

For tls1.3, those parameters CLIENT_HANDSHAKE_TRAFFIC_SECRET, SERVER_HANDSHAKE_TRAFFIC_SECRET, CLIENT_TRAFFIC_SECRET_0, and SERVER_TRAFFIC_SECRET_0 could be exported as client secrets
For tls1.2, CLIENT_RANDOM could be exported as client secrets

All of them could be used in Wireshark to decrypt TLS1.2 and TLS 1.3.

Here is one test sample

First, start one HTTPS server

    mux := http.NewServeMux()
    mux.HandleFunc("/", func(w http.ResponseWriter, req *http.Request) {
        if req.URL.Path != "/" {
            http.NotFound(w, req)
            return
        }
    })

    w, err := os.OpenFile("/keypath/https-key.txt", os.O_WRONLY|os.O_CREATE|os.O_TRUNC, 0600)
    if err != nil {
        fmt.Printf("failed to open file err %+v", err)
        return
    }

    cs := make([]uint16, len(cipherSuites))
    copy(cs, cipherSuites)
    var tlsCfg tls.Config
    tlsCfg.Certificates = make([]tls.Certificate, 1)
    tlsCfg.Certificates[0], err = tls.LoadX509KeyPair(*certFile, *keyFile)
    tlsCfg.NextProtos = []string{"h2"}
    tlsCfg.ClientAuth = tls.RequestClientCert
    tlsCfg.SessionTicketsDisabled = true
    tlsCfg.InsecureSkipVerify = true
    tlsCfg.KeyLogWriter = w
    tlsCfg.MinVersion = tls.VersionTLS13
    tlsCfg.CipherSuites = cs
    tlsCfg.PreferServerCipherSuites = true

    srv := &http.Server{
        Addr:      *addr,
        Handler:   mux,
        TLSConfig: &tlsCfg,
    }

    log.Printf("Starting server on %s", *addr)
    err = srv.ListenAndServeTLS("", "")
    log.Fatal(err)

Then, test it through curl with tls1.3 curl -Lv https://localhost:4000 --cacert /crtpath/ca.crt --tlsv1.3

We could find the content of https-key.txt as below

> cat https-key.txt
CLIENT_HANDSHAKE_TRAFFIC_SECRET xxxx yyyyy
SERVER_HANDSHAKE_TRAFFIC_SECRET xxxx yyyyyyyyy
CLIENT_TRAFFIC_SECRET_0 xxxxxxx yyyy
SERVER_TRAFFIC_SECRET_0 xx yyyyyyyyyyyy

Then set the keyFile /keypath/grpc-key.txt to Preferences -> Protocols -> TLS -> (Pre)-Master-Secret log filename in WireShark, now the Wireshark could do TLS decryption

For TLS1.2 test, you could change the server code to tlsCfg.MinVersion = tls.VersionTLS12 and then test it through curl with tls1.2 curl -Lv https://localhost:4000 --cacert /crtpath/ca.crt --tlsv1.2. And check the https-key.txt again, you could find the content could be CLIENT_RANDOM xxxxxx yyyyyyyy.

user_17994624 · Answer

I figured it out, with a lot of help from @zangw. There were two problems:

Although some discussions of TLS decryption in Wireshark mention CLIENT_RANDOM, this is only for TLS 1.2; current Syncthing generally uses TLS 1.3, which involves the other secrets I was seeing (CLIENT_HANDSHAKE_TRAFFIC_SECRET, SERVER_HANDSHAKE_TRAFFIC_SECRET, CLIENT_TRAFFIC_SECRET_0, and SERVER_TRAFFIC_SECRET_0) - see the official documentation of the NSS Key Log Format.
I was starting the capture too late (since I was waiting for the secrets to be written out to the file in order to be able to provide them to Wireshark), and so Wireshark was missing the initial TLS handshake, which resulted in the misidentification of the protocol as TLS 1.2 instead of 1.3 as well as the inability to decrypt the traffic. The correct way to proceed is to start the capture before the TLS negotiation begins, and then subsequently provide Wireshark with the secrets file after it is created. (This may require saving and reloading the capture.)

Wireshark now successfuly decrypts the TLS data; it can be viewed by selecting "Encrypted Application Data" and then clicking on the "Decrypted TLS" tab at the bottom of the window.

Extracting TLS secrets in Go

2 Answers