How to get Clangd working properly with VS Code on Windows

I can't get clangd to provide autocompletion for any external headers like from SFML.

I have tried doing the same on an Ubuntu VM and everything worked properly. The compile_commands.json has been moved to the project root aswell.

The project tree looks like this right now:

.clangd
--index
----main.cpp.1B607D111B8CF0BE.idx

build
--CMakeFiles etc

src
--main.cpp

CMakeLists.txt

compile_commands.json

The CMakeLists.txt contains this:

cmake_minimum_required(VERSION 3.10)

project(test)

set(CMAKE_CXX_STANDARD 17)

set(CMAKE_EXPORT_COMPILE_COMMANDS ON)

set(src_files
    src/main.cpp)

set(SFML_DIR "D:/Libs/SFML/SFML-2.5.1-windows-vc15-64-bit/SFML-2.5.1/lib/cmake/SFML/")

find_package(SFML COMPONENTS graphics)

add_executable(test ${src_files})

target_link_libraries(test PRIVATE sfml-graphics)

target_include_directories(test PRIVATE D:/Libs/SFML/SFML-2.5.1-windows-vc15-64-bit/SFML-2.5.1/include)

The compile_commands.json contains this:

[
{
  "directory": "C:/Users/Joerg/projects/test/build",
  "command": "C:\\PROGRA~1\\LLVM\\bin\\CLANG_~1.EXE   @CMakeFiles/test.dir/includes_CXX.rsp -g -Xclang -gcodeview -O0 -D_DEBUG -D_DLL -D_MT -Xclang --dependent-lib=msvcrtd   -std=gnu++17 -o CMakeFiles\\test.dir\\src\\main.cpp.obj -c C:\\Users\\Joerg\\projects\\test\\src\\main.cpp",
  "file": "C:/Users/Joerg/projects/test/src/main.cpp"
}
]

I have also tried using MinGW Clang where the corresponding compile_commands.json looked like this:

[
{
  "directory": "C:/Users/Joerg/projects/test/build nin",
  "command": "C:\\msys64\\mingw64\\bin\\clang++.exe      -std=gnu++17 -o CMakeFiles\\test.dir\\src\\main.cpp.obj -c C:\\Users\\Joerg\\projects\\test\\src\\main.cpp",
  "file": "C:/Users/Joerg/projects/test/src/main.cpp"
}
]

I expect clangd to provide autocompletion for the SFML namespace (sf::) as it does for the std namespace (std::). Currently it does not find the Header as it should when I write #include <SFML/Graphics.hpp>. It gives me this error message: 'SFML/Graphics.hpp' file not found clang(pp_file_not_found)