I'm using a face detection service that given a picture returns me face landmarks positions. My goal is to position these landmarks on the picture I'm showing in the screen.
My idea is to use an AbosluteLayout with the Image view and position the Landmarks over the picture. The overlapping works good. The problem is that the points of the landmark refer to the original picture size coordinates and the rendered image has completely different size. I'm using AspectFit for the moment, so full picture is shown in the AbsoluteLayout preserving the aspect ratio, usually with letterboxing.
What I've tried is to take in account the screen density and apply it to the original landmark points like that:
rightEyePoint.X = Convert.ToInt32(rightEyePoint.X / displayDensity);
and it get's closer but not on all pictures (differs from landscape/portrait).
Then I thought that I might want to know the scale factor of the rendered image considering the original one, and by that I could transform all the Landmark points but I don't know how to do that. I tried to create a custom image view and renderer to try to get the scale calculations from it, but wasn't successful as I didn't get to interfere on where the calculations for the native image component are made :
public class CustomImageView : Image
{
protected override SizeRequest OnMeasure(double widthConstraint, double heightConstraint)
{
return base.OnMeasure(widthConstraint, heightConstraint);
}
protected override void OnSizeAllocated(double width, double height)
{
base.OnSizeAllocated(width, height);
}
protected override void OnPropertyChanged([CallerMemberName] string propertyName = null)
{
base.OnPropertyChanged(propertyName);
}
}
public class CustomImageViewRenderer : ImageRenderer
{
public CustomImageViewRenderer(Context context) : base(context) { }
protected override void OnElementChanged(ElementChangedEventArgs<Image> e)
{
base.OnElementChanged(e);
if(e.NewElement != null)
{
var image = Control as ImageView;
}
}
protected override void OnSizeChanged(int w, int h, int oldw, int oldh)
{
base.OnSizeChanged(w, h, oldw, oldh);
}
protected override Task TryUpdateBitmap(Image previous = null)
{
return base.TryUpdateBitmap(previous);
}
}
Then I thought I could calculate myself the current rendered image size but I can't seem to get it. So I only get the xamarin forms Image component size, but not the image rendered in it. At this point I thought that I could caculate, depending on the aspect ratio, the size of the rendered image, as I'm using AspectFit but got stuck in the process.
The axml (sample):
<AbsoluteLayout BackgroundColor="Fuchsia" HorizontalOptions="Fill" VerticalOptions="Fill">
<Controls:PinchToZoomContainer HorizontalOptions="FillAndExpand" VerticalOptions="FillAndExpand" CurrentPosition="{Binding CurrentPhotoPosition}" CurrentScale="{Binding CurrentPhotoScale}" AbsoluteLayout.LayoutBounds="0, 0, 1, 1" AbsoluteLayout.LayoutFlags="All" IsEnabled="{ Binding CurrentStep, Converter={StaticResource checkIntToBooleanValueConverter}, ConverterParameter=2}">
<Image BackgroundColor="Blue" Source="{Binding CurrentPhotoPath}">
<Image.Behaviors>
<behaviors:EventToCommandBehavior
EventName="SizeChanged"
Command="{Binding ImageSizeChangedCommand}"/>
</Image.Behaviors>
</Image>
<!--<Controls:CustomImageView Source="{Binding CurrentPhotoPath}" BackgroundColor="Purple" SizeChanged=""/-->
</Controls:PinchToZoomContainer>
<!-- calculated landmarks -->
<Controls:LandmarkView
Padding="2"
DragMode="TouchAndRelease"
LBounds="{Binding HairStartLandmarkBounds, Mode=TwoWay}"
DragDirection="Vertical"
HorizontalOptions="FillAndExpand"
VerticalOptions="Start"
IsVisible="{ Binding CurrentStep, Converter={StaticResource checkIntToBooleanValueConverter}, ConverterParameter=3}"
x:Name="HairStartLandmark"
ToggleDraggingCommand="{Binding HairStartLandmarkTouchedCommand}">
<Controls:LandmarkView.Content>
<AbsoluteLayout>
<Image BackgroundColor="Red"
WidthRequest="{Binding LandmarkLineWidth}"
AbsoluteLayout.LayoutBounds= "{Binding LandmarkLineXPos}"
AbsoluteLayout.LayoutFlags="YProportional,WidthProportional"/>
<ffimageloading:CachedImage
WidthRequest="{Binding HairStartLandmarkIconSize}"
HeightRequest="{Binding HairStartLandmarkIconSize}"
Source="{Binding HairStartLandmarkIcon, Converter={StaticResource SvgImageSourceConverter}}">
</ffimageloading:CachedImage>
</AbsoluteLayout>
</Controls:LandmarkView.Content>
</Controls:LandmarkView>
</AbsoluteLayout>
So my question is, how can I easily get the rendered image coordinates system origin and scale factor to apply to the landmark points coordinates and position them correctly on the rendered image?
UPDATE:
I add an image of the current status of what I have and what I think I need. The blue background is the Xamarin.Forms Image view background:
Thank you very much