Preface
There's a few nice ideas presented here based around the properties of the character contours as polylines. Whilst there is some inherent flaws in this approach due to them being a function of resolution and scale, I would like to offer one further interruption of the same. My algorithm is still susceptible but it may offer a different perspective.
Theory
The method I propose is to compare common characters by the number of inflections in their contours. In this context, what I mean by inflection, is a sign change between the cross products of successive polyline segments as vector. For example; consider a polyline contour of a circle, starting at the mid y coordinate and the x+ most coordinate. If we were to trace the polyline contour CW (clockwise) around the perimeter, each line segment would be incrementally a CW transform of the prior. If at any time a segment turned "away" or "outwards", this transform would be CCW (counter-clockwise) and the cross product will invert. A "rough" circle will therefore have inflections, a "perfect" or "smooth" circle will have none.
Algorithm
The algorithm follows the steps below using the Emgu.CV. C# code below that:
- The images are loaded and converted to binary by means of thresholding
- The binary images then undergo contour detection and these contours are sort by their bounding box, left to right, so and their indices match the occurrence order of the character they contour.
- Each contour is then re-pointed to an equal number of segments in order to normalize for scale and resolution differences between images/characters.
- Each contour is "walked" and its number of inflections counted.
// [Some basic extensions are omitted for clarity]
// Load the images
Image<Rgb, byte> baseLineImage = new Image<Rgb, byte>("BaseLine.png");
Image<Rgb, byte> testCaseImage = new Image<Rgb, byte>("TestCase.png");
// Convert them to Gray Scale
Image<Gray, byte> baseLineGray = baseLineImage.Convert<Gray, byte>();
Image<Gray, byte> testCaseGray = testCaseImage.Convert<Gray, byte>();
// Threshold the images to binary
Image<Gray, byte> baseLineBinary = baseLineGray.ThresholdBinaryInv(new Gray(100), new Gray(255));
Image<Gray, byte> testCaseBinary = testCaseGray.ThresholdBinaryInv(new Gray(100), new Gray(255));
// Some dilation required on the test image so that the characters are continuous
testCaseBinary = testCaseBinary.Dilate(3);
// Extract the the contours from the images to isolate the character profiles
// and sort them left to right so as the indicies match the character order
VectorOfVectorOfPoint baseLineContours = new VectorOfVectorOfPoint();
Mat baseHierarchy = new Mat();
CvInvoke.FindContours(
baseLineBinary,
baseLineContours,
baseHierarchy,
RetrType.External,
ChainApproxMethod.ChainApproxSimple);
var baseLineContoursList = baseLineContours.ToList();
baseLineContoursList.Sort(new ContourComparer());
VectorOfVectorOfPoint testCaseContours = new VectorOfVectorOfPoint();
Mat testHierarchy = new Mat();
CvInvoke.FindContours(
testCaseBinary,
testCaseContours,
testHierarchy,
RetrType.External,
ChainApproxMethod.ChainApproxSimple);
var testCaseContoursList = testCaseContours.ToList();
testCaseContoursList.Sort(new ContourComparer());
var baseLineRepointedContours = RepointContours(baseLineContoursList, 50);
var testCaseRepointedContours = RepointContours(testCaseContoursList, 50);
var baseLineInflectionCounts = GetContourInflections(baseLineRepointedContours);
var testCaseInflectionCounts = GetContourInflections(testCaseRepointedContours);
Inflection Detection/Counting
static List<List<Point>> GetContourInflections(List<VectorOfPoint> contours)
{
// A resultant list to return the inflection points
List<List<Point>> result = new List<List<Point>>();
// Calculate the forward to reverse cross product at each vertex
List<double> crossProducts;
// Points used to store 2D Vectors as X,Y (I,J)
Point priorVector, forwardVector;
foreach (VectorOfPoint contour in contours)
{
crossProducts = new List<double>();
for (int p = 0; p < contour.Size; p++)
{
// Determine the vector to the prior to this vertex
priorVector = p == 0 ?
priorVector = new Point()
{
X = contour[p].X - contour[contour.Size - 1].X,
Y = contour[p].Y - contour[contour.Size - 1].Y
} :
priorVector = new Point()
{
X = contour[p].X - contour[p - 1].X,
Y = contour[p].Y - contour[p - 1].Y
};
// Determine the vector to the next vector
// If this is the lst vertex, loop back to vertex 0
forwardVector = p == contour.Size - 1 ?
new Point()
{
X = contour[0].X - contour[p].X,
Y = contour[0].Y - contour[p].Y,
} :
new Point()
{
X = contour[p + 1].X - contour[p].X,
Y = contour[p + 1].Y - contour[p].Y,
};
// Calculate the cross product of the prior and forward vectors
crossProducts.Add(forwardVector.X * priorVector.Y - forwardVector.Y * priorVector.X);
}
// Given the calculated cross products, detect the inflection points
List<Point> inflectionPoints = new List<Point>();
for (int p = 1; p < contour.Size; p++)
{
// If there is a sign change between this and the prior cross product, an inflection,
// or change from CW to CCW bearing increments has occurred. To and from zero products
// are ignored
if ((crossProducts[p] > 0 && crossProducts[p-1] < 0) ||
(crossProducts[p] < 0 && crossProducts[p-1] > 0))
{
inflectionPoints.Add(contour[p]);
}
}
result.Add(inflectionPoints);
}
return result;
}
Output
L: Baseline Inflections:0 Testcase Inflections:22
E: Baseline Inflections:1 Testcase Inflections:16
M: Baseline Inflections:4 Testcase Inflections:15
P: Baseline Inflections:11 Testcase Inflections:17
E: Baseline Inflections:1 Testcase Inflections:10
R: Baseline Inflections:9 Testcase Inflections:16
Contours (Blue) and Inflections (Red)
