Get image from headless Pygame instance

user_8732955 0
Viewed 184

I'm working with game based on Pygame. In the abstract - I need to run it in headless mode. By now, I'm using:

os.environ['SDL_VIDEODRIVER'] = 'dummy'

And the problem is, that I can't take the screen from game to NumPy array. The one solution which I found actually works, but only with saving the image to the file, what totally misses the point - but maybe it is a hint. I've tried things like parsing Surface to string:

pil_string_image = pygame.image.tostring(screen,"RGBA",False)
im = Image.frombytes("RGBA",(400, 400), pil_string_image)       
cv2.imshow('screen', np.asarray(im))

but afterall I've got only grey screen and that's all.

The reason is, that I need to prepare the game to run in Collab or in text env because I'm doing ML things using game screen.

Is there any way to take the screen properly? Or maybe I could trigger the headless mode in another way, which lets me take image simply like that?

pygame.surfarray.array3d(pygame.display.get_surface())
opencv pygame python-imaging-library numpy python
2 Answers

It seems screen is a pygame.surface object, presumably returned from pygame.display.set_mode(..). You can call the surfarray methods using screen as the first argument.

E.g.

>>> import os
>>> import pygame
pygame 1.9.6
Hello from the pygame community. https://www.pygame.org/contribute.html
>>> WIDTH, HEIGHT = 640, 480
>>> os.environ["SDL_VIDEODRIVER"] = "dummy"
>>> pygame.init()
(6, 0)
>>> screen = pygame.display.set_mode((WIDTH, HEIGHT))
>>> screen.fill(pygame.color.Color("black"))
<rect(0, 0, 640, 480)>
>>> print(pygame.surfarray.array3d(screen)[0])
[[0 0 0]
 [0 0 0]
 [0 0 0]
 ...
 [0 0 0]
 [0 0 0]
 [0 0 0]]
>>> pygame.draw.line(screen, pygame.color.Color("blueviolet"), (0,0), (0,480))
<rect(0, 0, 1, 481)>
>>> print(pygame.surfarray.array3d(screen)[0])
[[146  36 255]
 [146  36 255]
 [146  36 255]
 ...
 [146  36 255]
 [146  36 255]
 [146  36 255]]
user_22808900

I'm really stupid... Remember kids, when you're trying to diagnose if image was read correctly with OpenCV, sometimes 

    cv2.waitKey(1)

would be your best friend.

Thanks for: cv2.imshow command doesn't work properly in opencv-python

No response from the cv2 window gave me concern if displaying is working correctly...

user_87329550
Related