How can I pass build args with a remote image in docker-compose.yml?

I would like to pull a remote image from a private registry. The image's dockerfile contains some args whose values would be populated via docker-compose.yml. For example:

version: '3.0'

services:
    api:
        image: remoteApiImage
    web:
        image: remoteWebImage
        build: 
            args:
                baseurl: http://remoteApiImage:80

Currently, this does not work as build requires a context. However, if I set a context, it expects a local Dockerfile. Even setting to . without a local Dockerfile will pull the remote image, but the build args are not passed properly.

Is this possible?

Note: I am using Windows Server 2016 containers. Not sure that is relevant to the issue.