Android AR CORE on a static image instead of a camera feed

Viewed 618

Imagine a beauty try on experience with two main use cases:

  1. Overlay makeup on a face in realtime with AR
  2. Make a photo and do the same with it

Does ARCore support working with a static photo instead of a camera feed?

I didn't find any info on the topic.
But, since 2 of 3 main features - environmental undestading and light estimation - work without any additional sensors, i suppose that such a thing may be implemented.

2 Answers

For the static image I think you may find you get more accurate results using a dedicated facial image recognition technology, at this time.

Augmented Reality has to do quite a bit of work to figure out where exactly on the screen to place an AR node, given the movement of the camera and possibly of items in the real world also. With current devices and framework releases there can be small amounts of 'drift' etc which you will see being discussed on forums like this.

A static image allows much easier and deterministic feature recognition generally and you should be able to get more precise results using a technology focused on static image analysis and manipulation.

Google, for example, have some good resources on facial recognition on Android including:

These can run in realtime also, so it may be that for a make up example these will provide you an alternative to ARCore also for your real time use case.

OpenCV, the open source video recognition library, also has face recognition capability. You do need to be aware that OpenCV use with Android Studio is not particularly well documented at this time so you may need to look on forums like SO to get started:

I have tested makeup demo app with ARCore Augmented Faces implementation on a static image and it works ok. Although, the main challenge will be that Augmented faces works only with a front facing camera on mobile, so the experience is not optimal.

Related