Display Label with Minimum Threshold

jw-smc 0
Viewed 28

I'm modifying a custom object detector based off this sample provided by google: vision-quickstart I'd like to prevent displaying a detected object altogether if that object doesn't meet a specified confidence level.

Original server part object detection

Then I modified the code to delete the label text with the following:

// ObjectGraphic.java
private static final float CONFIDENCE_THRESHOLD = 0.50f;
...
for (Label label : object.getLabels()) {
  if (label.getConfidence() < CONFIDENCE_THRESHOLD) {continue;}
  canvas.drawText...

Label Text Removed but not the bounding/label boxes enter image description here Obviously, this is one step in the right direction but not the final objective. Here's what I found so far:

// ObjectDetectorProcessor.java

  @Override
  protected void onSuccess(
      @NonNull List<DetectedObject> results, @NonNull GraphicOverlay graphicOverlay) {
    for (DetectedObject object : results) {
      graphicOverlay.add(new ObjectGraphic(graphicOverlay, object));
      // commenting this preventing object overlays from being drawn
      // possibly add graphicOverlay.remove() based on label.confidence, but how do I do that?
    }
  }

// GraphicOverlay.java
  /** Adds a graphic to the overlay. */
  public void add(Graphic graphic) {
    synchronized (lock) {
      graphics.add(graphic);
    }
  }

  /** Removes a graphic from the overlay. */
  public void remove(Graphic graphic) {
    synchronized (lock) {
      graphics.remove(graphic);
    }
    postInvalidate();
  }

Please let me know if I'm not mentioning anything. I'd be more than willing to provide additional information for your questions.

google-mlkit object-detection java
2 Answers

You are already at the right place. In the following method

protected void onSuccess( @NonNull List results, @NonNull GraphicOverlay graphicOverlay) {

You should check the object's label's confidence, and only if the confidence is higher than threshold, you call

graphicOverlay.add(new ObjectGraphic(graphicOverlay, object));

user8ceU0

Solution

Figured it out and no changes are required for any other files. Turned out to be quite simple but I'm just not familiar with java. Not sure if nesting for loops is the optimal solution but it works :/

// ObjectDetectorProcessor.java
import com.google.mlkit.vision.objects.DetectedObject.Label;
...
private static final float CONFIDENCE_THRESHOLD = 0.5f;
...
// Before (right image)
@Override
protected void onSuccess(
    @NonNull List<DetectedObject> results, @NonNull GraphicOverlay graphicOverlay) {
  for (DetectedObject object : results) {
    graphicOverlay.add(new ObjectGraphic(graphicOverlay, object));
  }
}
// After (left image)
protected void onSuccess(
    @NonNull List<DetectedObject> results, @NonNull GraphicOverlay graphicOverlay) {
  for (DetectedObject object : results) {
    for (Label label : object.getLabels()) {
      if (label.getConfidence() < CONFIDENCE_THRESHOLD) {continue;}
      else {graphicOverlay.add(new ObjectGraphic(graphicOverlay, object));}
    }
  }
}

Before (right) and After (left) Additionally, I found that the above results (removing just the label text, but not the boxes) can be reproduced with the following built-in method. Found in PreferenceUtils.java of the mlkit vision-quickstart google sample. Direct link to method below: https://developers.google.com/android/reference/com/google/mlkit/vision/objects/custom/CustomObjectDetectorOptions.Builder#public-customobjectdetectoroptions.builder-setclassificationconfidencethreshold-float-classificationconfidencethreshold

jw-smc0
Related