MS Graph API metadata field with term store

I'm trying to update metadata in my drive item. If I create column directly in my Document Library (1 line, Multi lines etc) it's ok but when my columns are linked with term store (managed metadata) i have this error :

GRAVE: Throwable detail: com.microsoft.graph.http.GraphServiceException: Error code: itemNotFound
Error message: The resource could not be found.

GET https://graph.microsoft.com/v1.0/me/drives/[mydrive]/items/017QC4KJLG35SQJ5NC6VALGDTRG7LFJ6IM
SdkVersion : graph-java/v5.5.0

404 : Not Found
[...]

And here is my code :

FieldValueSetRequest updateRequest = client.sites(siteId).lists(listId).items(listItemId).fields().buildRequest();
logger.debug(updateRequest);
FieldValueSet fieldValueSet =  new FieldValueSet();
for(Map.Entry<String, String> element: elements.entrySet()) {
    String key = element.getKey();
    String value = element.getValue();
    logger.debug(key+ "=" + value);
    fieldValueSet.additionalDataManager().put(key, new JsonPrimitive(value));
}
updateRequest.patch(fieldValueSet);