Measure how Straight/Smooth the Text Borders are Rendered in an Image

Viewed 998

I have two images:

enter image description here

enter image description here

I want to measure how straight/smooth the text borders are rendered.

First image is rendered perfectly straight, so it deserves a quality measure 1. On the other hand, the second image is rendered with a lot of variant curves (rough in a way) that is why it deserves a quality measure less than 1. How will I measure it using image processing or any Python function or any function written in other languages?

Clarification :

There are font styles that are rendered originally with straight strokes but there are also font styles that are rendered smoothly just like the cursive font styles. What I'm really after is to differentiate the text border surface roughness of the characters by giving it a quality measure.

I want to measure how straight/smooth the text borders are rendered in an image. Inversely, it can also be said that I want to measure how rough the text borders are rendered in an image.

4 Answers

I don't know any python function, but I would:

1) Use potrace to trace the edges and convert them to bezier curves. Here's a vizualisation: enter image description here

2) Then let's zoom to the top part of the P for example: enter image description here You draw lines perpendicular to the curve for a finite length (let's say 100 pixels). You plot the color intensity (you can convert to HSI or HSV and use one of those channels, or just convert to grayscale and take the pixel value directly) over that line:

enter image description here 3) Then you calculate the standard deviation of the derivative. Small standard deviation means sharp edges, large standard deviation means blurry edges. For a perfect edge, the standard deviation would be zero.

4) For every edge were you drew a perpendicular line, you now have a "smoothness" value. You can then average all the smoothness values per edge, per letter, per word or per image, as you see fit. Also, the more perpendicular lines you draw, the more accurate your smoothness value, but the more computationally intensive.

I would try something simple like creating a 'roughness' metric using a few functions from the opencv library, since it's easy to work with in Python (and C++, as well as other wrappers).

For example (without actual source, since I'm typing on my phone):

  1. Preprocess to create binary images (many standard ways).
  2. Use cv2.findContours to get outlines of the letters.
  3. Use cv2.arcLength on each contour as denominators.
  4. Use cv2.approxPolyDP to simplify each contour.
  5. Use cv2.arcLength on each simplified contour as numerators.
  6. Calculate ratios of simplified over full arc lengths.

In step 5, ratios closer to 1.0 require less simplification, so they're presumably less rough. Ratios closer to 0.0 require a lot of simplification, and are therefore probably very rough. Of course, you'll have to tweak the contour finding code to get appropriate outlines to work with, and you'll need to manage numerical precision to keep the math calculations meaningful, but hopefully the idea is clear enough.

OpenCV also has the useful functions cv2.convexHull and cv2.convexityDefects that you might find interesting in related work. However, they didn't seem appropriate for the letters here, since internal features on letters like M for example would be more challenging to address.

Speaking of rough things, I admit this algorithmic outline is incredibly rough! However, I hope it gives you a useful idea to try that seems straightforward to implement quickly to start getting quantitative feedback.

One idea might be simply to get the average of the number of vertices per character in Python/OpenCV using cv2.CHAIN_APPROX_SIMPLE.

Since you have the same characters and you want to know how straight they are, the CHAIN_APPROX_SIMPLE measures only horizontal and vertical corner vertices. For your first image, there should be much fewer vertices than for your second image.

CHAIN_APPROX_SIMPLE compresses horizontal, vertical, and diagonal segments and leaves only their end points. For example, an up-right rectangular contour is encoded with 4 points.

import cv2
import numpy as np

# read image
img = cv2.imread('lemper1.png')
#img = cv2.imread('lemper2.png')

# convert to grayscale
gray = cv2.cvtColor(img,cv2.COLOR_BGR2GRAY)

# threshold
thresh = cv2.threshold(gray, 0, 255, cv2.THRESH_BINARY+cv2.THRESH_OTSU)[1]

# invert
thresh = 255 - thresh

# get contours and compute average number of vertices per character (contour)
result = img.copy()
contours = cv2.findContours(thresh, cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_SIMPLE)
contours = contours[0] if len(contours) == 2 else contours[1]
num_contours = 0
sum = 0
for cntr in contours:
    cv2.drawContours(result, [cntr], 0, (0,0,255), 1)
    num_vertices = len(cntr)
    sum = sum + num_vertices
    num_contours = num_contours + 1

smoothness = (sum / num_contours)
print(smoothness)

# save resulting images
cv2.imwrite('lemper1_contours.png',result)
#cv2.imwrite('lemper2_contours.png',result)

# show thresh and result    
cv2.imshow("thresh", thresh)
cv2.imshow("contours", result)
cv2.waitKey(0)
cv2.destroyAllWindows()


First image average number of vertices: 49.666666666666664

Second image average number of vertices: 179.14285714285714


So smaller number of vertices means straighter characters.

Preface

There's a few nice ideas presented here based around the properties of the character contours as polylines. Whilst there is some inherent flaws in this approach due to them being a function of resolution and scale, I would like to offer one further interruption of the same. My algorithm is still susceptible but it may offer a different perspective.

Theory

The method I propose is to compare common characters by the number of inflections in their contours. In this context, what I mean by inflection, is a sign change between the cross products of successive polyline segments as vector. For example; consider a polyline contour of a circle, starting at the mid y coordinate and the x+ most coordinate. If we were to trace the polyline contour CW (clockwise) around the perimeter, each line segment would be incrementally a CW transform of the prior. If at any time a segment turned "away" or "outwards", this transform would be CCW (counter-clockwise) and the cross product will invert. A "rough" circle will therefore have inflections, a "perfect" or "smooth" circle will have none.

Algorithm

The algorithm follows the steps below using the Emgu.CV. C# code below that:

  1. The images are loaded and converted to binary by means of thresholding
  2. The binary images then undergo contour detection and these contours are sort by their bounding box, left to right, so and their indices match the occurrence order of the character they contour.
  3. Each contour is then re-pointed to an equal number of segments in order to normalize for scale and resolution differences between images/characters.
  4. Each contour is "walked" and its number of inflections counted.
// [Some basic extensions are omitted for clarity] 
// Load the images
Image<Rgb, byte> baseLineImage = new Image<Rgb, byte>("BaseLine.png");
Image<Rgb, byte> testCaseImage = new Image<Rgb, byte>("TestCase.png");
// Convert them to Gray Scale
Image<Gray, byte> baseLineGray = baseLineImage.Convert<Gray, byte>();
Image<Gray, byte> testCaseGray = testCaseImage.Convert<Gray, byte>();
// Threshold the images to binary
Image<Gray, byte> baseLineBinary = baseLineGray.ThresholdBinaryInv(new Gray(100), new Gray(255));
Image<Gray, byte> testCaseBinary = testCaseGray.ThresholdBinaryInv(new Gray(100), new Gray(255));
// Some dilation required on the test image so that the characters are continuous
testCaseBinary = testCaseBinary.Dilate(3);

// Extract the the contours from the images to isolate the character profiles
// and sort them left to right so as the indicies match the character order
VectorOfVectorOfPoint baseLineContours = new VectorOfVectorOfPoint();
Mat baseHierarchy = new Mat();
CvInvoke.FindContours(
    baseLineBinary,
    baseLineContours,
    baseHierarchy,
    RetrType.External,
    ChainApproxMethod.ChainApproxSimple);
var baseLineContoursList = baseLineContours.ToList();
baseLineContoursList.Sort(new ContourComparer());

VectorOfVectorOfPoint testCaseContours = new VectorOfVectorOfPoint();
Mat testHierarchy = new Mat();
CvInvoke.FindContours(
    testCaseBinary,
    testCaseContours,
    testHierarchy,
    RetrType.External,
    ChainApproxMethod.ChainApproxSimple);
var testCaseContoursList = testCaseContours.ToList();
testCaseContoursList.Sort(new ContourComparer());

var baseLineRepointedContours = RepointContours(baseLineContoursList, 50);
var testCaseRepointedContours = RepointContours(testCaseContoursList, 50);

var baseLineInflectionCounts = GetContourInflections(baseLineRepointedContours);
var testCaseInflectionCounts = GetContourInflections(testCaseRepointedContours);

Inflection Detection/Counting

static List<List<Point>> GetContourInflections(List<VectorOfPoint> contours)
{
    // A resultant list to return the inflection points
    List<List<Point>> result = new List<List<Point>>();

    // Calculate the forward to reverse cross product at each vertex
    List<double> crossProducts;
    // Points used to store 2D Vectors as X,Y (I,J)
    Point priorVector, forwardVector;
    foreach (VectorOfPoint contour in contours)
    {
        crossProducts = new List<double>();
        for (int p = 0; p < contour.Size; p++)
        {
            // Determine the vector to the prior to this vertex
            priorVector = p == 0 ?
            priorVector = new Point()
            {
                X = contour[p].X - contour[contour.Size - 1].X,
                Y = contour[p].Y - contour[contour.Size - 1].Y
            } :
            priorVector = new Point()
            {
                X = contour[p].X - contour[p - 1].X,
                Y = contour[p].Y - contour[p - 1].Y
            };

            // Determine the vector to the next vector
            // If this is the lst vertex, loop back to vertex 0
            forwardVector = p == contour.Size - 1 ?
            new Point()
            {
                X = contour[0].X - contour[p].X,
                Y = contour[0].Y - contour[p].Y,
            } :
            new Point()
            {
                X = contour[p + 1].X - contour[p].X,
                Y = contour[p + 1].Y - contour[p].Y,
            };

            // Calculate the cross product of the prior and forward vectors
            crossProducts.Add(forwardVector.X * priorVector.Y - forwardVector.Y * priorVector.X);
        }

        // Given the calculated cross products, detect the inflection points
        List<Point> inflectionPoints = new List<Point>();
        for (int p = 1; p < contour.Size; p++)
        {                   
            // If there is a sign change between this and the prior cross product, an inflection,
            // or change from CW to CCW bearing increments has occurred. To and from zero products
            // are ignored
            if ((crossProducts[p] > 0 && crossProducts[p-1] < 0) ||
                (crossProducts[p] < 0 && crossProducts[p-1] > 0))
            {
                inflectionPoints.Add(contour[p]);
            }
        }
        result.Add(inflectionPoints);
    }
    return result;
}

Output

L: Baseline Inflections:0 Testcase Inflections:22
E: Baseline Inflections:1 Testcase Inflections:16
M: Baseline Inflections:4 Testcase Inflections:15
P: Baseline Inflections:11 Testcase Inflections:17
E: Baseline Inflections:1 Testcase Inflections:10
R: Baseline Inflections:9 Testcase Inflections:16

Contours (Blue) and Inflections (Red)

BaseLineOutput TestCaseOutput

Related