Receipt Parser using Google Cloud Vision API

TL;DR : I want to extract items, total price, payment type, outlet name, transaction date, from a receipt using Google Vision API.

$imageAnnotator = new ImageAnnotatorClient(['credentials' => 'cloud-vision-keyfile.json']);
$image = file_get_contents($image_path);

$response = $imageAnnotator->textDetection($image);
$texts = $response->getTextAnnotations();

foreach ($texts as $text) {
    // read through all the text scanned
    // some code to sanitize and extract $text->getDescription() ....
}

The receipt parser API from GCP is a limited access feature.

So my goal is to find a better approach to get these fields :

Tenant name

Currently I implement fuzzy search on the title.

$outlet_block = explode(PHP_EOL, $texts[0]->getDescription(), 5);
$outlet_candidates = array_slice($outlet_block, 0, 4);

Receipt date

I use date_parse to get a string that looks like a date, and check if the date is not older than a week.

$parsed_date = date_parse($filtered_text);
if (
    $parsed_date['year'] && 
    $parsed_date['month'] &&
    $parsed_date['day']
)
{
    // ... and not old enough
    $date_candidate = implode('-', [
        $parsed_date['year'],
        $parsed_date['month'],
        $parsed_date['day']
    ]);
    $str_date_candidate = strtotime($date_candidate);
    if ($str_date_candidate >= strtotime($oldest_transaction_date) && $str_date_candidate <= strtotime('now'))
    {
        $date_candidates[] = date('Y-m-d', strtotime($date_candidate));
    }

Total Amount

I found an interesting article from Lutz Kunneke to get a pretty same logic to get the "total". First, I search for a string that similar to "total", "total amount", etc. and read through the rest of the text "at the same line" as that string.

if (in_array($filtered_text, $totals_arr))
{
    $is_total_found = true;
    $vertices = $text->getBoundingPoly()->getVertices();


    // top left, top right, bottom right, bottom left
    $y_min = min($vertices[0]->getY(), $vertices[1]->getY());
    $y_max = max($vertices[2]->getY(), $vertices[3]->getY());

    $y_min_totals[] = $y_min;
    $y_max_totals[] = $y_max;
}
elseif ($is_total_found) // check all number after the "total" text is found
{
    $text_number = $filtered_text;

    // trim everything unnecessary to get the number
    $text_number = preg_replace('/^(rp|idr|usd)|(\.00|\,00)$/', '', $text_number);
    $text_number = (int) filter_var(str_replace([',', '.'], '', $text_number), FILTER_SANITIZE_NUMBER_INT);

    if ($text_number == 0)
        continue;

    if ($text_number < max($price_candidates))
    {
        continue;
    }

    // check Y position
    $vertices = $text->getBoundingPoly()->getVertices();
    $y_min = min($vertices[0]->getY(), $vertices[1]->getY());
    $y_max = max($vertices[2]->getY(), $vertices[3]->getY());

    $y_center = ($y_min + $y_max) / 2;

    foreach ($y_min_totals as $key => $y_min_total)
    {
        // if y_center is inside the y_min_total and y_max_total
        if ($y_center >= $y_min_total && $y_center <= $y_max_totals[$key])
        {
            // we have a candidate
            $price_candidates[] = $text_number;
        }
    }
}

Items / Product list

Currently I have no idea how to start.

Payment Type

Still no clue.

Image orientation

I get it by comparing the second text vertice and the center of the block

$vertices = $text->getBoundingPoly()->getVertices();
$x_center = ($vertices[0]->getX() + $vertices[1]->getX() + $vertices[2]->getX() + $vertices[3]->getX()) / 4;
$y_center = ($vertices[0]->getY() + $vertices[1]->getY() + $vertices[2]->getY() + $vertices[3]->getY()) / 4;

$x0 = $vertices[0]->getX();
$y0 = $vertices[0]->getY();

return (bool) ($x0 < $x_center && $y0 < $y_center);

I hope I can get some enlightenment. Thank you.

https://apps.prabowomurti.com/receipt