Deserialize json two-dimensional array to list of objects

I have a problem with deserializing json:

{
"rows": [
        [
          4,
          4,
          5
        ],
        [
          3,
          3,
          3
        ],
        [
          6,
          7,
          7
        ]
      ]
}

My model looks like this:

@Getter
@Builder(builderClassName = "AttemptBuilder", toBuilder = true)
@JsonDeserialize(builder = Attempt.AttemptBuilder.class)
class Action{
    List<Attempt> attempts;

    @JsonPOJOBuilder(withPrefix = "")
    public static class ActionBuilder {
    }
}

@Builder(builderClassName = "Attempt Builder", toBuilder = true)
@JsonDeserialize(builder = Attempt .Attempt Builder.class)
class Attempt {
    List<Integer> attemptList;

    @JsonPOJOBuilder(withPrefix = "")
    public static class AttemptBuilder {
    }
}

I am having error:

JSON decoding error: Cannot deserialize instance of `Attempt$AttemptBuilder` out of START_ARRAY 

When I replace List<Attempt> attempts with List<List<Integer>> attempts everything is working fine, but I would like to have List of custom objects. Does anyone has idea how to fix it?