Azure pipeline doesn't use specified compiler even though it is installed

We have an azure pipeline with a Microsoft-hosted agent that builds our c++ project for Linux, Mac and Windows. Because we use the std::filesystem library we needed to specify a higher GNU version in order to get it working on Mac and Linux. The issue is that while it works on Mac, Linux is still using gnu 7.5. How can we force Linux to use GNU 9.3 just like Mac?

We force the compiler version by setting the following variables in the .yaml:

  variables:
    CC: gcc-9.3
    CXX: g++-9.3
    CXXFLAGS: -lstdc++fs

For our vmImages we use [platform]-latest, this would mean that both Linux and Mac should have GNU 9.3 installed, as specified here

Here is our complete azure.pipeline.yaml

trigger:
- master

strategy:
  matrix:
    linux:
      imageName: 'ubuntu-latest'
    mac:
      imageName: 'macOS-latest'
    windows:
      imageName: 'windows-latest'

pool:
  vmImage: $(imageName)
  variables:
    CC: gcc-9.3
    CXX: g++-9.3
    CXXFLAGS: -lstdc++fs

steps:
- task: CmdLine@2
  displayName: 'Clear build folder'
  inputs:
    script: 'rm -rf build'

- task: CmdLine@2
  displayName: 'Create build folder'
  inputs:
    script: 'mkdir build'

- task: CmdLine@2
  displayName: 'Setup vcpkg submodule'
  inputs:
    script: 'git submodule init'

- task: CmdLine@2
  displayName: 'Download vcpkg submodule'
  inputs:
    script: 'git submodule update'

- task: Bash@3
  displayName: 'Install vcpkg'
  inputs:
    targetType: 'inline'
    script: 'sh bootstrap-vcpkg.sh'
    workingDirectory: 'extern/vcpkg'

- task: CMake@1
  condition: ne( variables['Agent.OS'], 'Windows_NT' )
  inputs:
    cmakeArgs: '..'
    workingDirectory: 'build'

- task: CMake@1
  condition: eq( variables['Agent.OS'], 'Windows_NT' )
  inputs:
    cmakeArgs: '.. -DCMAKE_GENERATOR_PLATFORM=x64'
    workingDirectory: 'build'

- task: CMake@1
  displayName: 'CMake build'
  inputs:
    cmakeArgs: '--build build'
    workingDirectory: '$(Build.SourcesDirectory)'

- task: CmdLine@2
  displayName: 'Run tests'
  condition: ne( variables['Agent.OS'], 'Windows_NT' )
  inputs:
    script: 'ctest --no-compress-output -T Test'
    workingDirectory: 'build'

- task: CmdLine@2
  condition: eq( variables['Agent.OS'], 'Windows_NT' )
  displayName: 'Run tests'
  inputs:
    script: 'ctest -C Debug --no-compress-output -T Test'
    workingDirectory: 'build'

- task: PublishTestResults@2
  inputs:
    testResultsFormat: 'CTest'
    testResultsFiles: '**/Test*.xml'