Numpy Array Multiplication Differnce in Regular vs when using For loops

Viewed 429

I wanted to apply multiply blend on an image with a completely yellow image. The image is: Eiffel Tower Paris

The yellow image is created using:

img_paris = img1 = cv2.imread("/content/drive/My Drive/Datasets/Images/paris.jpg")
yellow_image = np.ones(img_paris.shape) * 255 

yellow_image[:,:,0] *= 0

I tried two techniques: Firstly I used the regular multiplication

imgc = img_paris.copy()

imgc = (imgc * yellow_image)

This resulted in: Eiffel Multiplied

Then I used for loops to multiply individual elements

for x in range(yellow_image.shape[0]):
  for y in range(yellow_image.shape[1]):
    imgc[x,y] = (imgc[x,y] * yellow_image[x,y])

which resulted in Eiffel Multiplied 2

The result of second looks like the image is getting inverted somehow.

Both techniques should result in similar results. Why is the image getting inverted in the for loop technique? I want to use the For loops for more control. Can someone tell me why is this happening? Why are both techniques result in different results?

3 Answers

The images are different, because the former image contains floating point data and the data type of the later image is uint8.

The instruction imgc = (imgc * yellow_image) generates a new array with floating point data since the data type of yellow_image is floating point.

Create a yellow_image with datatype uint8 to sove the issue:

yellow_image = np.ones(img_paris.shape, dtype=np.uint8) * 255 
yellow_image[:,:,0] *= 0
imgc *= yellow_image

Alternatively use numpy.ndarray.astyp to create a copy of the array and to cast to uint8:

imgc = (imgc * yellow_image)

imgc = (imgc * yellow_image).astype(np.uint8)

or use numpy.multiply, by specifying a casting rule and a type:

imgc = np.multiply(imgc, yellow_image, casting='unsafe', dtype=np.uint8)

The problem is in a totally unexpected place: in the creation of yellow_image. np.ones by default creating an array of floating-point type: hence yellow_image is created with float-type elements. Check:

>>> yellow_image.dtype
dtype('float64')

Hence when you do the following: imgc * yellow_image, the resulting array data-type is promoted to the one with higher precision (the floating-point type, of course) and hence imgc has elements like:

array([[[    0., 24225., 12750.],
        [    0., 23715., 12240.],
        [    0., 23460., 11985.],
        ...,

which are of floating-point type.

To avoid this problem, and to not go through the hassle of writing datatypes explicitly, use:

yellow_image = np.ones_like(img_paris) * 255

np.ones_like creates an array of ones with the shape of the array passed to it, and the exact same dtype - removing your worries. Now check:

>>> yellow_image.dtype
dtype('uint8')

Now consider imgc1 = imgc * yellow_image and imgc2 is the output of the loop. Check:

>>> np.allclose(imgc1, imgc2)
True

Problem solved.

N.B. - To answer your question about why the image is inverted:

When the multiplication results in floating point types, it hold large numbers (24225., 12750., etc) as pixel colour values. When you write an image using this array, all these numbers are clipped to the maximum possible pixel colour value: 255. Thus, most of the image you see is yellow, since all the 'overflowing' values have been clipped to 255, leading to the brightest possible yellow shade.

On the other hand, when the multiplication is done forcing the uint8 type, any values larger than 255 'roll back' to the smallest possible value of an unsigned, 8-bit integer: 0. So if the value is 487, the dtype restriction will force it to be 0 + (287 - 255) - 1 = 31. This is overflow. So a large number can end up becoming very small - the math is simple, look it up. Hence you get an inverted sort of an image (unexpected dark pixels).

Both techniques should result in similar results. Why is the image getting inverted in the for loop technique?

Because you should've done this way, converting data type:

imgc = np.uint64(img_paris.copy()) # <-- convert datatype

for x in range(yellow_image.shape[0]):
    for y in range(yellow_image.shape[1]):
        imgc[x,y] = (imgc[x,y] * yellow_image[x,y])

Explanation part 1 (dtype)

This is because dtype changes when reassigning the full matrix, while it does not happen reassigning slices:

a = np.array([[1]], np.uint8)
b = np.array([[1]], np.float64)

a[0] = a[0] * b[0] # assigning slices dtype of a does not change
print(a.dtype) #=> uint8

a = a * b # while assigning the full matrix it does
print(a.dtype) #=> float64

If you print your dtype along the way you see this:

yellow_image_1 = np.ones(img_paris.shape) * 255
print(yellow_image_1.dtype) #=> float64
yellow_image_1[:,:,0] *= 0
print(yellow_image_1.dtype) #=> float64

imgc_1 = img_paris.copy()
print(imgc_1.dtype) #=> uint8
imgc_1 = (imgc_1 * yellow_image_1)
print(imgc_1.dtype) #=> float64

And this:

yellow_image_2 = np.ones(img_paris.shape) * 255
print(yellow_image_2.dtype)  #=> float64
yellow_image_2[:,:,0] *= 0
print(yellow_image_2.dtype) #=> float64

imgc_2 = img_paris.copy()
print(imgc_2.dtype) #=> uint8
for x in range(yellow_image_2.shape[0]):
    for y in range(yellow_image_2.shape[1]):
        imgc_2[x,y] = (imgc_2[x,y] * yellow_image_2[x,y])
print(imgc_2.dtype) #=> uint8

So you end up with different dtype matrix.


Explanation part 2 (OpenCV BGR)

As mentioned, keep in mind that OpenCv uses the BGR format and each pixel value goes from 0 to 255, says np.uint8.

So, if you are using for example matplotlib, for showing the image you must swap the B and R channels:

img_paris = cv2.imread('3ClnT.jpg')
plt.imshow(img_paris[:,:,::-1])

While this is not required if you are saving using cv2.imwrite() or cv2imshow(), for example:

cv2.imwrite('paris.jpg', img_paris)

That said, you can generate your solid yellow image using this one liner command:

yellow_image = np.ones_like(img_paris) * (0, 255, 255)

And show it or save it:

plt.imshow(yellow_image[:,:,::-1])
cv2.imwrite('solid_yellow.jpg', yellow_image)

Now, your result from the multiplication paris_yellow = img_paris * yellow_image results in a value greater than 255:

Clipping input data to the valid range for imshow with RGB data ([0..1] for floats or [0..255] for integers).

So, when you multiply you end up with a max pixel value which can be 255 * 255 = 65025.

You then need:

  • convert terms of multiplication to a datatype that supports integer up to 65025
  • After multiplication, normalize then convert back to uint8

This is an example:

paris_yellow_2 = np.int64(img_paris) * np.int64(yellow_image) # <- use int64 terms
max_px_val = np.amax(paris_yellow_2) # <-- Max pixel alue
paris_yellow_2 = np.uint8((paris_yellow_2/max_px_val) * 255) # <- normalize and convert back to uint8
plt.imshow(paris_yellow_2[:,:,::-1])

And this is the result:

enter image description here


Other option, which gives a different result is to multiply the G and R channels for a coefficient greater than 1 then crop values >255. In this case you need to use a float dtype:

paris_yellow_3 = np.float64(img_paris) * (1, 3, 3)
paris_yellow_3[paris_yellow_3 > 255] = 255 # <- crops to 255 pixels values > 255
paris_yellow_3 = paris_yellow_3.astype(np.uint8) # <- back to uint8

In this case B is multiplied by 1 (no change), G and R are multiplied by 3, giving the following result:

enter image description here

Related