How to pass passphrase to private key when releasing with standard maven plugin

I am trying to release my maven project using standard maven release plugin and Bitbucket pipelines. I've created my own docker image with private key protected with passphrase. My Bitbucket account allows commits from this docker image and Bitbucket pipeline is able to inject private key passphrase via environment variable when building.

The problem is that maven release plugin ignores everything and asks for passphrase which is an unacceptable manual step.

My settings.xml in /root/.m2 looks like

<server>
    <id>bitbucket.org</id>
    <privateKey>/root/.ssh/id_rsa</privateKey>
    <passphrase>${env.BB_PIPELINES_MAVEN_3_3_9_JDK_8_SSH_PASSPHRASE</passphrase>
</server>

I am using the latest release and scm plugins, respectively 2.5.3 and 1.9.5

In my pom.xml there is server property

<project.scm.id>bitbucket.org</project.scm.id>

and my scm url's are like

<scm>
    <url>https://bitbucket.org/user/repo</url>
    <developerConnection>scm:git:git@bitbucket.org:user/repo</developerConnection
    <connection>scm:git:git@bitbucket.org:user/repo</connection>
</scm>

I was trying to use ssh-agent, and it seems to work, but looks like ssh-agent doesn't allows to pass passphrase in command line as well.

Any ideas how can it be solved?