Augmented Reality with React Native (Points of Interest over the camera)

user_967455 0
Viewed 5044

I'm working on an application for Android & iOS to show points of interest over the camera. ARkit & ARcore has poor compatibility nowdays.

Could you recommend me some library to do this? If it comes with an example, better! I know viro-media, but I don't understand how to do this using that library.

I don't want 3D models, just markers over the camera, similar to the attachment image.

markers over the camera

react-native augmented-reality
2 Answers

To do this with Viro React -- and in AR in general -- the trick is to recognize that there are two coordinate systems:

  1. The local coordinate system of your device, which we'll call 'AR space'. In Viro, this is centered at the user's initial position when the application starts, and is in meters.

  2. Geographic coordinates (latitude and longitude).

To position the overlays, you have to convert your content from geographic coordinates into AR space. This is a two-step process. First project the spherical geographic coordinates onto a 2D plane -- the Web Mercator is great for this. Then translate the projected coordinates by the device's initial projected position.

The device's initial projected position can be derived by projecting its initial geographic position. In Viro React, you can use the Geolocation module to grab this when the user starts the app.

Finally, you'll need to do a similar transformation for the user's bearing: converting from compass direction to device orientation in AR space.

For this to work well you'll likely have to figure out how handle inaccurate geolocation lookups (e.g. what happens if the location retrieved from the device is inaccurate), and may also have to account for drift: over time the two coordinate systems may start to fall out of sync.

The last part, creating the info cards, is easy with Viro -- you either pre-bake the images with text and use ViroImage, or if the cards need to be more dynamic you can use a ViroFlexView.

user_6699880

I am also interested in this one and I'm trying out ViroReact!

I find a bit difficult to understand how to make this work when the lat's and long's have been converted to x-y-values. What should the z-value be?

Let's say you have the lat-lon coordinates [59, 10] as the user location you want to show where [59, 11] is relative to your location. How to you build that in a ViroARScene? 

<ViroNode position={ **userLocationFromLatLonCartesian** }>
   <ViroBox position={ **poiLocationFromLatLonToCartesian** }/>
</ViroNode>

So how do you calculate the scale, position and rotation, so that the object will be visible?

Seem like https://github.com/proj4js/proj4js is a library that could provide conversions from latlon to x-y values

user_11993040
Related