C# Emgu.Cv wrapper: Retrieved (byte) pixel-values of Mat do not match Image<Gray,byte> Bytes values, when cropping ROIs

user_13658211 0
Viewed 829

(I use Emgu.Cv version 4.2.0.3662)

Let's consider the following construction of image objects by cropping a small ROI of width=w, and height=h:

Image<Gray, byte> img = new Image<Gray, byte>(@"D:\temp\tst_img.JPG");
Mat mat = img.Copy().Mat;
var roi = new Rectangle(0, 0, w, h);
img.ROI = roi;
var tst_img = img.Copy();
var tst_mat = new Mat(mat, roi);

(the w of the image file is divisible by 4.) In order to get the byte values of tst_img I can use tst_img.Data (which is a byte[,,1] array) or tst_img.Bytes (which corresponds to the image pixel values as a byte[] array).

In order to get the byte values of the image which tst_mat corresponds to, I firstly tried tst_mat.GetData() (which can be identified as a byte[,] array). I noticed that the values of the tst_mat.GetData() byte array do not match the tst_img.Data values, however their dimensions do match (disregarding the extra 1-dim channel of the latter) . Yet, the following approach gives the expected values: 

tst_mat.ToImage<Gray, byte>().Bytes

I tried to follow some idea of implementing Marshal.Copy as described in https://bugsdb.com/_en/debug/0cad34a2bab07405564a5f09bc14ccf7 so I prepared the following property in a MatExtension

public static byte[] MatData(this Mat mat)
{
        int w = mat.Width;
        int h = mat.Height;
        int l = w * h;
        byte[] output = new byte[l];
        Marshal.Copy(mat.DataPointer, output, 0, l);
        return output;
}

Yet, tst_mat.MatData() do not yield the same byte values as tst_img.Bytes. I suspect that this approach gives values which do not match tst_img.bytes since it doesn't consider the width/stride of the original mat (from which tst_mat was cropped).

Here I provide an explicit tested example, using an roi of size 4*2 (width*height) for the purpose of demonstration:

the test image

I compare the byte values of the following variables:

byte[,,] img_bytes_dat = img_roi.Data;
byte[,] mat_bytes_dat = (byte[,]) mat_roi.GetData();
byte[] img_bytes_arr = img_roi.Bytes;
byte[] mat_bytes_arr = mat_roi.MatData();
byte[] mat2img_bytes_arr = mat_roi.ToImage<Gray, byte>().Bytes;

Looking at img_bytes_dat (or at its flattened version img_bytes_arr) I get following pixel values:

[0, 0, 0] 126

[0, 1, 0] 126

[0, 2, 0] 131

[0, 3, 0] 146

[1, 0, 0] 125 <-

[1, 1, 0] 124

[1, 2, 0] 130

[1, 3, 0] 145

(and mat2img_bytes_arr contains these pixel values). Looking at mat_bytes_dat (mat_bytes_arr has the same pixel values) I get following pixel values:

[0, 0] 126

[0, 1] 126

[0, 2] 131

[0, 3] 146

[1, 0] 154 <-

[1, 1] 156

[1, 2] 157

[1, 3] 153

It is the width of the ROI, which determines the index from which these arrays do not match.

Considering the above I am very confused about the meaning of the Mat.GetData() method, when we apply it to a Mat object which we construct by cropping an ROI of a different Mat object. The resulting array actually has the dimensions of the ROI but the data do not match what one would expect.

what is the correct and most efficient way to get all the pixel values of "tst_mat" as a single array (I don't mind 1d or multi-dimensional)? (without the 2 stages route of "mat2img_bytes_arr" )

mat emgucv c#
1 Answers

First, well done for your diligence and testing. I was able to recreate the exact same behavior and consider this a bug. I would encourage you to report it to Emgu Forum where hopefully you may be better supported.

For you interest, I enclose the EmguCV source for the Mat(Mat,Rectangle) constructor, however it invokes the underlying OpenCV method with little indication of why it might behave this way:

/// <summary>
/// Create a mat header for the specific ROI
/// </summary>
/// <param name="mat">The mat where the new Mat header will share data from</param>
/// <param name="roi">The region of interest</param>
public Mat(Mat mat, Rectangle roi)
           : this(MatInvoke.cveMatCreateFromRect(mat.Ptr, ref roi), true, true){}

To support you in the interim, I would advise against Marshal approaches since I do not believe the Image.Data object is necessarily pinned. Considering you are dealing with an array of unmanaged type, byte, you can be confident that any assignment made from them directly, unless explicitly ref, will assign by value and provide a "copy".

Perhaps the following extension might be helpful to you:

public static TDepth[,] CopyROIData<TDepth>(this Image<Gray, TDepth> image, int channel)
    where TDepth : new()
{
    TDepth[,] result = new TDepth[image.ROI.Height, image.ROI.Width];
    int xOrigin = image.ROI.X;
    int yOrigin = image.ROI.Y;

    TDepth[,,] fullData = image.Data;
    for(int u = 0; u < image.ROI.Width; u++)
    {
        for (int v = 0; v < image.ROI.Height; v++)
        {
            result[v,u] = fullData[v + yOrigin, u + xOrigin, channel];
        }
    }

    return result;
}

Called as:

Image<Gray, byte> myImage = new Image<Gray, byte>(@"D:\temp\tst_img.JPG");
var myROIData = myImage.CopyROIData(0);
user_67960610
Related