Retrofit 2.0 handle error in json response

I'm working on Retrofit 2.0 (which is awesome) to handle API responses. All works well when API answers with success, I return the converted object wished from the json response

Here an example of request:

ServiceAPI.getUser(new Callback<User>() {
    @Override
    public void onResponse(Call<User> call, Response<User> response) {
       User user = response.body(); // user is my POJO
    }

    @Override
    public void onFailure(Call<User> call, final Throwable t) {
        Log.e(TAG, "Error: " + t.getMessage())
    }
});

When response is done, I return the POJO (which is my main purpose), not the json to parse in order to avoid boilerplate.

So here my code to handle this :

// the interface to handle calls
protected interface ServiceAPI {
    @GET("/user/{userId}")
    Call<User> getUser(@Path("userId") String userId);
}

// the GSON part for converting data
Gson gson = new GsonBuilder()
    .registerTypeAdapterFactory(new ItemTypeAdapterFactory())
    .setDateFormat("yyyy'-'MM'-'dd'T'HH':'mm':'ss'.'SSS'Z'")
    .create();

// an interceptor to log requests responses
OkHttpClient okClient = new OkHttpClient.Builder()
    .addInterceptor(new LogJsonInterceptor())
    .build();

// the retrofit builder
retrofit = new Retrofit.Builder()
    .baseUrl(mBaseUrl)
    .client(okClient)
    .addConverterFactory(GsonConverterFactory.create(gson))
    .build();

// the interceptor to log requests
public static class LogJsonInterceptor implements Interceptor {
    @Override
    public okhttp3.Response intercept(Interceptor.Chain chain) throws IOException {
        Request request = chain.request();
        okhttp3.Response response = chain.proceed(request);
        String rawJson = response.body().string();
        Log.d(TAG, rawJson);
        // Re-create the response before returning it because body can be read only once
        return response.newBuilder().body(ResponseBody.create(response.body().contentType(), rawJson)).build();
    }
}

// here the magic to handle json response and get data from "data" json key
public static class ItemTypeAdapterFactory implements TypeAdapterFactory {

    public <T> TypeAdapter<T> create(Gson gson, final TypeToken<T> type) {

        final TypeAdapter<T> delegate = gson.getDelegateAdapter(this, type);
        final TypeAdapter<JsonElement> elementAdapter = gson.getAdapter(JsonElement.class);

        return new TypeAdapter<T>() {

            public void write(JsonWriter out, T value) throws IOException {
                delegate.write(out, value);
            }

            private RestError error;

            public T read(JsonReader in) throws IOException {

                JsonElement jsonElement = elementAdapter.read(in);
                if (jsonElement.isJsonObject()) {
                    JsonObject jsonObject = jsonElement.getAsJsonObject();
                    if (jsonObject.has("data")) {
                        jsonElement = jsonObject.get("data");
                    }
                }

                return delegate.fromJsonTree(jsonElement);
            }
        }.nullSafe();
    }
}

// here the method to call from an activity for example to get an User
public static void getUser(final String userId, final Callback<User> callback) {
    serviceAPI.getUser(userId).enqueue(new Callback<User>() {
        @Override
        public void onResponse(Call<User> call, Response<User> response) {
            callback.onResponse(call, response);
        }

        @Override
        public void onFailure(Call<User> call, Throwable t) {
            callback.onFailure(call, t);
        }
    });
}

// here the call from the Activity
ServiceAPI.getUser(new Callback<User>() {
    @Override
    public void onResponse(Call<User> call, Response<User> response) {
       User user = response.body();
    }

    @Override
    public void onFailure(Call<User> call, final Throwable t) {
        Log.e(TAG, "Error: " + t.getMessage())
    }
});

// when API answers with success (code 200 from headers)
{
"data":
    {
        "id":1,
        "name":"myname",
        "email":"myemail"
    }
}

So all works well here because I get response from "data" and convert the response into my POJO

PROBLEM: But when API answers with an error (also with code 200 from headers), I get this:

{
"error":
    {
        "code":200
        "type":"OAuth_exception",
        "message":"You need an access token to get an user",
    }
}

The problem is that the retrofit "response" is nevertheless successfull and the errorBody is null

So here, I would to convert this into a RestError POJO (below) and send it inside the call when calling getUser method

public class RestError {
    private int code;
    private String message;
    private String type;

    public RestError(int code, String message, String type) {
        this.code = code;
        this.message = message;
        this.type = type;
    }
}

Any idea to fix that?

UPDATE:

I added this in the ItemTypeAdatperFactory

public T read(JsonReader in) throws IOException {

                    JsonElement jsonElement = elementAdapter.read(in);
                    if (jsonElement.isJsonObject()) {
                        JsonObject jsonObject = jsonElement.getAsJsonObject();
                        if (jsonObject.has("data")) {
                            jsonElement = jsonObject.get("data");
                        } else if (jsonObject.has("error")) {
                            jsonElement = jsonObject.get("error");

                            TypeAdapter<RestError> restErrorTypeAdapter = gson.getAdapter(RestError.class);
                            RestError error = restErrorTypeAdapter.fromJsonTree(jsonElement);
                            return (T) error;
                        }
                    }

                    return delegate.fromJsonTree(jsonElement);
                }

And I created a custom Callback like this:

public abstract static class CustomCallback<T> implements Callback<T> {

    public abstract void onError(RestError error);
    public abstract void onSuccess(T body);

    @Override
    public void onResponse(Call<T> call, Response<T> response) {
        if(response.body() instanceof RestError) {
            onError((RestError) response.body());
        } else {
            onSuccess(response.body());
        }
    }

    @Override
    public void onFailure(Call<T> call, Throwable t) {
        Log.e(TAG, t.toString());
    }
}

So now how the call is:

ServiceAPI.getUser(new ServiceAPI.CustomCallback<User>() {
        @Override
        public void onError(RestError error) {
            Log.e(TAG, error.toString());
        }

        @Override
        public void onSuccess(User body) {
            Log.d(TAG, body.toString());  
            User user = body;  
        }

        @Override
        public void onFailure(Call<User> call, Throwable t) {
            Log.e(TAG, t.toString());
        }
    });

So now when an error occurs I get it from onError(), otherwise onSuccess() and seems to do the job

What do you think about?