OAuth Google API for Java unable to impersonate user

I would like to impersonate a user and add files to the users Google Drive on their behalf from a server process. I've setup a service account and can successfully access the Drive as the service account adding and listing files, etc. using the following code:

  /** Global instance of the HTTP transport. */
  private static final HttpTransport HTTP_TRANSPORT = new NetHttpTransport();

  /** Global instance of the JSON factory. */
  private static final JsonFactory JSON_FACTORY = new JacksonFactory();

  public static void main(String[] args) {
    try {
        GoogleCredential credential = 
                new GoogleCredential.Builder().setTransport(HTTP_TRANSPORT)
                  .setJsonFactory(JSON_FACTORY)
                  .setServiceAccountId("XXXXX@developer.gserviceaccount.com")
                  .setServiceAccountScopes(DriveScopes.DRIVE)
                  .setServiceAccountPrivateKeyFromP12File(new File("c:/junk/key.p12"))
                  .build();
        Drive drive = new Drive.Builder(HTTP_TRANSPORT, JSON_FACTORY, credential).build();         
        drive.files().list().execute();
  } catch (Exception e) {
     e.printStackTrace();
  }

This works, however only returns files that are associated to what I assume is associated with the service accounts drive (?).

According to the JavaDoc, GoogleCredential can also be used to impersonate a user by adding the service account users email address as follows:

        GoogleCredential credential = 
                new GoogleCredential.Builder().setTransport(HTTP_TRANSPORT)
                  .setJsonFactory(JSON_FACTORY)
                  .setServiceAccountId("XXXXX@developer.gserviceaccount.com")
                  .setServiceAccountScopes(DriveScopes.DRIVE)
                  .setServiceAccountPrivateKeyFromP12File(new File("c:/junk/key.p12"))
                  .setServiceAccountUser("usera@domain.com") //<-- impersonate user a
                  .build();

However, when executing this code, the following exception is thrown:

com.google.api.client.auth.oauth2.TokenResponseException: 400 Bad Request
{
  "error" : "access_denied"
}
at com.google.api.client.auth.oauth2.TokenResponseException.from(TokenResponseException.java:103)
at com.google.api.client.auth.oauth2.TokenRequest.executeUnparsed(TokenRequest.java:303)
at com.google.api.client.auth.oauth2.TokenRequest.execute(TokenRequest.java:323)
at com.google.api.client.googleapis.auth.oauth2.GoogleCredential.executeRefreshToken(GoogleCredential.java:340)
at com.google.api.client.auth.oauth2.Credential.refreshToken(Credential.java:508)
at com.google.api.client.auth.oauth2.Credential.intercept(Credential.java:260)
at com.google.api.client.http.HttpRequest.execute(HttpRequest.java:796)
at com.google.api.client.googleapis.json.GoogleJsonResponseException.execute(GoogleJsonResponseException.java:198)
at com.google.api.client.googleapis.services.GoogleClient.executeUnparsed(GoogleClient.java:237)
at com.google.api.client.http.json.JsonHttpRequest.executeUnparsed(JsonHttpRequest.java:207)
at com.google.api.services.drive.Drive$Files$List.execute(Drive.java:1071)

Am I missing a step or configuration setting?

Thanks, David