(OpenCV / DNN) Face Recognition isn't working - Euklidian distance is always 0

user_14818627 0
Viewed 226

I want to verify if one person is similar to another person. Therefore I want to get the similarity between two faces.

These are the input faces

Mindy Face

Madonna Face

Now I want to push them through the DNN and after that I want to get the Euklidian distance between the two resulting matrices.

I've used the following OpenFace model: https://storage.cmusatyalab.org/openface-models/nn4.small2.v1.t7

This is my code for calculating the distances:

    cv::Mat madonna = cv::imread("/home/der/Madonna_Face.jpg");
    cv::Mat mindy = cv::imread("/home/der/Mindy_Face.jpg");

    cv::resize(madonna, madonna, cv::Size(96, 96));
    cv::resize(mindy, mindy, cv::Size(96, 96));

    cv::Mat madonnaBlob = cv::dnn::blobFromImage(madonna, 1.0 / 255, cv::Size(96, 96), cv::Scalar{0,0,0}, true, false);
    cv::Mat mindyBlob = cv::dnn::blobFromImage(mindy, 1.0 / 255, cv::Size{96, 96}, cv::Scalar{0,0,0}, true, false);

    cv::dnn::Net _net = cv::dnn::readNetFromTorch("/home/der/nn4.small2.v1.t7");

    _net.setInput(madonnaBlob);
    cv::Mat madonnaMat = _net.forward();

    _net.setInput(mindyBlob);
    cv::Mat mindyMat = _net.forward();
    
    std::cout << cv::norm(madonnaMat, mindyMat);

And if I'm doing so the result from cv::norm is 0. The representations are exactly the same:

    std::vector<double> master = madonnaMat;
    std::vector<double> slave = mindyMat;
    for(int i; i < 128; i++) {
        std::cout << master[i] << " # " << slave[i] << std::endl;
    }


Output:
    > -0.0865457 # -0.0865457
    > 0.133816 # 0.133816
    > -0.105774 # -0.105774
    > 0.05389 # 0.05389
    > -0.00391233 # -0.00391233
    > ...

Results:

Madonna Representation: [-0.060358506, 0.14156586, -0.10181303, 0.060315549, 0.0016125928, 0.066964693, -0.044892643, -0.043857966, 0.088271223, 0.047121659, 0.078663647, 0.025775915, 0.062051967, 0.034234334, -0.049976062, 0.045926169, 0.084343202, 0.046965379, -0.092582494, 0.13601208, -0.003582818, -0.15382886, 0.075037867, 0.19894752, -0.041007876, -0.12050319, -0.056161541, -0.018724455, 0.024790274, 0.0092850979, 0.095108159, 0.067354925, 0.06044127, 0.041365273, -0.12024247, 0.18279234, 0.027767293, 0.09874554, -0.16951905, 0.062370241, -0.014530737, 0.015518869, -0.0056175897, -0.066358574, -0.02390888, -0.07608442, 0.13011196, 0.031423025, -0.010443882, 0.12755248, -0.010195011, 0.0051672528, -0.10453289, -0.013270194, 0.096139617, 0.10375636, -0.047089621, 0.050923191, 0.066422582, -0.046726897, -0.1845296, 0.031028474, 0.086226918, -0.27064508, 0.055891197, -0.0053421594, 0.035870265, -0.026942547, -0.17279817, 0.13772435, 0.0071162563, 0.075375959, -0.046405111, 0.12658595, 0.11093359, 0.0030428318, 0.070016958, 0.1725318, -0.056130294, -0.14420295, -0.12438529, 0.056423288, -0.080888703, -0.052004829, -0.06481386, 0.14163122, -0.059617694, -0.026075639, 0.052098148, -0.0055074869, -0.014869845, -0.11943244, 0.068051606, -0.096071519, 0.19727865, -0.016027609, -0.05776047, 0.069935486, -0.020494614, 0.013407955, -0.06065508, -0.056143567, -0.04608072, 0.072748154, -0.035580911, 0.15261506, -0.074352823, -0.081481896, 0.020475708, -0.021581693, -0.16350025, 0.12794609, 0.082243897, 0.015881324, 0.011330541, -0.026391003, 0.086644463, -0.10490314, 0.088207267, 0.17892174, 0.025871141, 0.012454472, 0.010682535, 0.1253885, -0.12909022, 0.082067415, -0.035789803, 0.032903988]
Madonna Size: 1 x 128
Madonna Dims: 2
Mindy Representation: [-0.082645342, 0.14463238, -0.10716592, 0.065654278, 0.0045089996, 0.064019054, -0.047334831, -0.056190431, 0.099919245, 0.048234992, 0.068906084, 0.028518379, 0.057044145, 0.046223734, -0.056203742, 0.033566523, 0.082230642, 0.055683684, -0.080982864, 0.12431844, -0.00075431512, -0.14511517, 0.045022864, 0.20965824, -0.030178605, -0.11852413, -0.066858761, -0.01461118, 0.032898057, 0.02857255, 0.1088237, 0.07066118, 0.044605579, 0.022743503, -0.10785796, 0.20373915, 0.010565795, 0.063950166, -0.18701579, 0.062780239, -0.0042907735, 0.031276166, -0.006556896, -0.038440779, -0.01419229, -0.072688736, 0.13676986, 0.040385362, 0.010314438, 0.095734902, -0.0080824783, 0.011763249, -0.098884396, -0.040797569, 0.10534941, 0.12088351, -0.07317061, 0.063644305, 0.0830286, -0.050620016, -0.18088549, 0.03330183, 0.090282671, -0.25393733, 0.056058947, -0.020288708, 0.049997903, -0.044997148, -0.15860014, 0.15251927, 0.015151619, 0.088731326, -0.028061632, 0.11127418, 0.090425298, 0.0052096732, 0.053858042, 0.18543676, -0.066999368, -0.15851147, -0.11389373, 0.088093147, -0.08713299, -0.048095752, -0.063261949, 0.12453313, -0.051213119, -0.023759408, 0.048403475, -0.012721839, -0.021282939, -0.098075315, 0.066707589, -0.11601795, 0.20438787, -0.015739718, -0.052848384, 0.057336167, -0.01592578, 0.014057826, -0.058749981, -0.043632519, -0.031006066, 0.046038814, -0.065755703, 0.15442967, -0.082077362, -0.099808514, 0.016168201, 0.0046916353, -0.14556217, 0.11152669, 0.062443323, -0.00032889194, 0.0020548289, -0.026999777, 0.096809812, -0.11947374, 0.085579365, 0.16317753, 0.028130196, 0.014577032, 0.0079531483, 0.11340163, -0.15006165, 0.094127603, -0.0440454, 0.033095147]
Mindy Size: 1 x 128
Mindy Dims: 2

Any ideas what I'm doing wrong? Thanks.

face-detection face-recognition computer-vision opencv c++
1 Answers

I've experienced this several times. I couldn't find this explicitly mentioned in the OpenCV documentation, but the cv::dnn::Net::forward function returns a cv::Mat blob with the data member pointing always to the same zone of memory. Therefore on the second forward pass, that zone of memory is overwritten and both madonnaBlob and mindyBlob point there.

As @Christoph Rackwitz pointed out, you need to clone the cv::Mat before running the second inference:

_net.setInput(madonnaBlob);
cv::Mat madonnaMat = _net.forward();
madonnaMat = madonnaMat.clone(); // Copy memory

_net.setInput(mindyBlob);
cv::Mat mindyMat = _net.forward();
user_13894490
Related