GRPC Okhttp android client channel with self signed ssl certificate

I have a grpc-js server using self signed ssl certificates.

var credentials = grpc.ServerCredentials.createSsl(
    fs.readFileSync('./node/grpc/ssl/ca.crt'), 
    [{
        cert_chain: fs.readFileSync('./node/grpc/ssl/server.crt'),
        private_key: fs.readFileSync('./node/grpc/ssl/server.key')
    }], 
    true
);

I then tested this setup with a grpc-js client with the following credential setup and this works.

var credentials = grpc.credentials.createSsl(
    fs.readFileSync('./node/grpc/ssl/ca.crt'),
    fs.readFileSync('./node/grpc/ssl/client.key'),
    fs.readFileSync('./node/grpc/ssl/client.crt')
);

I want to replicate this in Android using OkHttpChannelBuilder but it is a bit more complicated. This is what I have so far.

private val mChannel : ManagedChannel
init {
    /**
     * Server certificate to make it trusted.
     */
    val serverCrtFile = applicationContext.resources.openRawResource(R.raw.server)
    val serverCertificate: X509Certificate =
        CertificateFactory.getInstance("X.509").generateCertificate(serverCrtFile) as X509Certificate

    val caKeyStore: KeyStore = KeyStore.getInstance(KeyStore.getDefaultType()).apply {
        load(null, null)
        setCertificateEntry("server", serverCertificate)
    }

    val trustManagerFactory = TrustManagerFactory.getInstance(TrustManagerFactory.getDefaultAlgorithm()).apply {
        init(caKeyStore)
    }

    val sslContext = SSLContext.getInstance("TLS").apply {
        init(null, trustManagerFactory.trustManagers, null)
    }

    mChannel = OkHttpChannelBuilder
        .forAddress(BuildConfig.GRPC_HOST_ADDRESS, BuildConfig.GRPC_HOST_PORT)
        .sslSocketFactory(sslContext.socketFactory)
        .keepAliveTime(10, TimeUnit.SECONDS)
        .useTransportSecurity()
        .keepAliveWithoutCalls(true)
        .build()
}

Everything worked before implementing ssl (so using plaintext() on the channel builder).

The error I get now is io.grpc.StatusRuntimeException: UNAVAILABLE: End of stream or IOException. Can someone please tell me if I am doing something wrong and how I can get a successful connection like between the js server and client.