How to use OpenCV to overlay an image AND text/shapes?

Question

I ran into a problem regarding OpenCV's transparent overlays. This is my code so far:

import cv2

cap = cv2.VideoCapture('Sample_Vid.mp4')
stat_overlay = cv2.imread('overlay.png')
fps = 21

if cap.isOpened():
    while cap.isOpened():
        ret, frame = cap.read()
        overlay = frame.copy()
        output = frame.copy()

        cv2.rectangle(overlay, (0, 0), (730, 50), (0, 0, 0), -1)
        cv2.putText(overlay, fps, (1230, 20), cv2.FONT_HERSHEY_DUPLEX, 0.5, (255, 255, 255), 1)
        cv2.addWeighted(overlay, 1.0, output, 0, 0, output)

        cv2.imshow('frame', output)

So I have my frame with a rectangle and the FPS displayed on it. Now I want to overlay my stat_overlay image FIRST and THEN the text and shape, because they are meant to be dynamic. In every explanation I read it was told doing it with cv2.addWeighted(stat_overlay, 1.0, output, 0, 0, output), but I already have one command like that used by the dynamic overlays and if I insert a second one above it, it does not work. Any ideas how to solve this?

Thanks for the answers in advance!

user_4926757 · Answer

The command you are using: cv2.addWeighted(overlay, 1.0, output, 0, 0, output), uses alpha = 1.0, and beta = 0, so there is no transparency.
You are basically copying overlay image into output image.

AddWeighted documentation:

cv2.addWeighted(src1, alpha, src2, beta, gamma[, dst[, dtype]])
src1 – first input array.
alpha – weight of the first array elements.
src2 – second input array of the same size and channel number as src1.
beta – weight of the second array elements.
dst – output array that has the same size and number of channels as the input arrays.

You can also use the following code for overlaying the text:

output = frame.copy()
cv2.rectangle(output, (0, 0), (730, 50), (0, 0, 0), -1)
cv2.putText(output, fps, (1230, 20), cv2.FONT_HERSHEY_DUPLEX, 0.5, (255, 255, 255), 1)

For overlaying stat_overlay you can use a solution like Alpha blending code sample.

I don't know if 'overlay.png' is in RGB or RGBA format.
In case image has an alpha channel, you can use it as transparency plane.
If the image is RGB, you may create your desired alpha plane.

In case 'overlay.png' is a small image (like a logo), you probably don't need any of this, you can "place" the small image on the output image.

I created a self contained code sample, that based on the alpha blending sample.
In order to make the code self contained, the code uses:

ffmpeg-python for generating synthetic video (for testing).
The code, draws a red circle that replaces 'overlay.png'

Here is the code:

import ffmpeg
import cv2
import numpy as np

in_filename = 'Sample_Vid.mp4' # Input file for testing (".264" or ".h264" is a convention for elementary h264 video stream file)

## Build synthetic video, for testing:
################################################
# ffmpeg -y -r 10 -f lavfi -i testsrc=size=192x108:rate=1 -c:v libx264 -crf 23 -t 50 test_vid.264

width, height = 640, 480

(
    ffmpeg
    .input('testsrc=size={}x{}:rate=1'.format(width, height), f='lavfi')
    .output(in_filename, vcodec='libx264', crf=23, t=5)
    .overwrite_output()
    .run()
)
################################################


cap = cv2.VideoCapture('Sample_Vid.mp4')
#stat_overlay = cv2.imread('overlay.png')

# Create image with green circle, instead of reaing a file
# The image is created as RGBA (the 4'th plane is the transparency).
stat_overlay = np.zeros((height, width, 4), np.uint8)
cv2.circle(stat_overlay, (320, 240), 80, (0, 0, 255, 255), thickness=20) # Draw red circle (with alpha = 255) 

# https://www.learnopencv.com/alpha-blending-using-opencv-cpp-python/
stat_alpha = stat_overlay[:, :, 3] # Take 4'th plane as alpha channel
stat_alpha = cv2.cvtColor(stat_alpha, cv2.COLOR_GRAY2BGR) # Duplicate alpha channel 3 times (to match output dimensions)

# https://www.learnopencv.com/alpha-blending-using-opencv-cpp-python/
# Normalize the alpha mask to keep intensity between 0 and 1
stat_alpha = stat_alpha.astype(float) / 255

stat_overlay = stat_overlay[:, :, 0:3] # Get RGB channels

fps = 21


if cap.isOpened():
    while cap.isOpened():
        ret, frame = cap.read()
        if ret:            
            output = frame.copy()

            # https://www.learnopencv.com/alpha-blending-using-opencv-cpp-python/
            # Alpha blending:
            foreground = stat_overlay.astype(float)
            background = output.astype(float)

            # Multiply the foreground with the alpha matte
            foreground = cv2.multiply(stat_alpha, foreground)

            # Multiply the background with ( 1 - alpha )
            background = cv2.multiply(1.0 - stat_alpha, background)

            # Add the masked foreground and background.
            output = cv2.add(foreground, background).astype(np.uint8)

            cv2.rectangle(output, (0, 0), (230, 50), (0, 0, 0), -1)
            cv2.putText(output, str(fps), (123, 20), cv2.FONT_HERSHEY_DUPLEX, 0.5, (255, 255, 255), 1)

            cv2.imshow('frame', output)
            cv2.waitKey(1000)

        else:
            break

cv2.destroyAllWindows()

Result (last frame):

How to use OpenCV to overlay an image AND text/shapes?

1 Answers