return the position of all circles from an image

Question

Disclaimer: I'm not really a good python programmer, I'm an android developer, so if you see anything dumb, don't be suprised

The example images I'm working with:

before modifying:

after:

I'm trying to get the positions of all circles in the second image (only 5 of them) to later use pyautogui to move the cursor to the positions and press them, however the cv2.HoughCircles returns a lot of positions and I'm not sure why is that, is there any way to get only the desired (X,Y) positions as a list?

current result: (not even sure whether those are positions tbh)

[[[ 577.   603.   196.2]
  [ 579.   593.   188. ]
  [ 475.   319.    37.6]
  [ 613.   391.   123.4]
  [ 575.   613.   205.6]
  [ 415.   469.   164.2]
  [ 617.   371.   135.6]
  [ 575.   259.    38.6]
  [ 583.   581.   176.2]
  [ 587.   385.   118.8]
  [ 475.   443.    37.2]
  [ 285.   381.   315.2]
  [ 589.   357.   114. ]
  [ 579.   503.    37.6]
  [ 571.   391.   125.2]
  [ 621.   383.   125.2]
  [ 539.   379.   123.6]
  [ 585.   347.   123.2]
  [ 575.   357.   113.4]
  [ 409.   459.   169.2]
  [ 571.   347.   121.4]
  [ 571.   407.   113. ]
  [ 591.   611.   201. ]
  [ 541.   393.   120.4]
  [ 683.   375.   186. ]
  [ 685.   443.    36.2]
  [ 255.   391.   326.4]
  [ 563.   355.   112. ]
  [ 587.   401.   104.6]
  [ 577.   627.   201. ]
  [ 617.   311.   183.2]
  [ 523.   669.   264.6]
  [ 615.   611.   216. ]
  [ 575.   379.   109. ]
  [ 335.   753.   446.2]
  [ 531.   395.   110. ]
  [ 477.   511.   101.4]
  [ 603.   613.   215. ]
  [ 135.   431.   440.2]
  [ 197.   439.   380.6]
  [ 555.   411.   112.8]
  [ 301.   435.   277.2]
  [ 579.   637.   185.6]
  [ 425.   487.   157.6]
  [ 577.   369.    99.2]
  [ 151.   435.   424. ]
  [ 207.   319.   411. ]
  [ 587.   373.   106.4]
  [ 493.   519.    90.2]
  [ 227.   435.   352.4]
  [ 291.   435.   284.2]
  [ 385.   505.   186.6]
  [ 943.   613.   536.8]
  [ 515.   677.   270. ]
  [ 241.   439.   340.8]
  [ 971.   613.   563.2]
  [ 931.   613.   524.4]
  [ 223.   323.   396.8]
  [ 487.   509.   101.8]
  [ 291.   315.   256.2]
  [ 175.   439.   400.4]
  [ 579.   659.   205. ]
  [ 495.   529.    88.6]
  [ 291.   447.   286.6]
  [ 403.   509.   165.6]
  [ 475.   529.    93.8]
  [   3.   319.   586. ]
  [  21.   323.   568. ]
  [ 743.   319.   282. ]
  [ 589.   641.   193.2]
  [   7.   443.   572. ]
  [ 161.   319.   436. ]
  [ 165.   439.   410.6]
  [ 497.   507.   100.8]
  [  31.   323.   557.8]
  [1021.   443.   445.2]
  [  51.   435.   530. ]
  [ 681.    33.   445.8]
  [  27.   449.   553.8]
  [  39.   435.   538.8]
  [ 233.   447.   344.4]
  [  49.   323.   540. ]
  [  79.   317.   514.2]
  [ 997.   443.   421.6]
  [ 211.   443.   365.2]
  [ 691.   221.   204.2]
  [ 575.   649.   194.8]
  [ 275.   447.   274. ]
  [ 515.   525.   101.4]
  [  59.   319.   531.6]
  [ 985.   147.   458.8]
  [ 803.   319.   327.2]
  [ 475.   243.   111.2]
  [ 673.   515.   109.4]
  [ 515.   515.    97.8]
  [ 877.   319.   385.6]
  [ 391.   421.   119.6]
  [  69.   435.   512.6]
  [ 107.   435.   468.8]
  [1009.   447.   434.2]
  [ 987.   443.   411.8]
  [ 579.   689.   232.4]
  [ 637.   383.   140.6]
  [ 641.   613.   138.4]
  [ 627.   329.   115.8]
  [ 661.   505.   100.2]
  [ 503.   515.    91. ]
  [ 571.   667.   208.4]
  [ 375.   435.   132.2]
  [ 681.   575.   140. ]
  [ 483.   519.    87.8]
  [ 255.   443.   323.8]
  [ 187.   319.   320.8]
  [ 283.   259.   302.4]
  [ 677.   227.   189.6]
  [ 647.   549.   115.4]
  [ 523.   371.   105.8]
  [ 497.   241.    92.6]
  [ 475.   539.    89. ]
  [ 595.   681.   216.6]
  [ 359.   443.   132.6]
  [ 329.   443.   125.2]
  [ 971.   443.   396. ]
  [ 275.   435.   304.6]
  [ 295.   505.   263. ]
  [ 691.   211.   209.4]
  [ 487.   253.   102.2]
  [ 685.   157.   252.4]
  [ 601.   697.   248.8]
  [ 633.   677.   248. ]
  [ 681.   215.   200.8]
  [ 385.   439.   126.6]
  [ 657.   271.   186. ]
  [ 669.   611.   133.4]
  [ 681.   533.    98. ]
  [ 467.   379.    68.4]
  [ 987.   323.   507. ]
  [ 181.   447.   327.6]
  [ 275.   253.   281.2]
  [ 653.   477.    77.8]
  [ 949.   319.   472.8]
  [ 579.   713.   254.4]
  [ 771.   323.   231. ]
  [ 939.   319.   460. ]
  [ 949.   443.   374.2]
  [ 585.   673.   220.2]
  [ 685.   307.   121.4]
  [ 621.   549.   132.8]
  [ 615.   323.   106.2]
  [ 335.   619.    96.8]
  [ 487.   535.    82.8]
  [ 533.   475.    61.2]
  [ 265.   447.   281.6]
  [ 755.   319.   221.8]
  [ 589.   663.   213.2]
  [ 691.   245.   209.2]
  [ 683.   235.   199.8]
  [ 357.   431.   124.4]
  [ 431.   361.    91.4]
  [ 515.   489.    73.6]
  [  45.   443.   431.6]
  [ 197.   449.   314.4]
  [ 681.   267.   212.2]
  [ 653.   253.   156.2]
  [ 705.   549.   141.4]
  [ 475.   233.    97.8]
  [ 509.   233.    83. ]
  [ 691.   255.   192.8]
  [ 341.   431.   139.6]
  [ 345.   443.   129. ]
  [ 523.   481.    62.8]
  [ 211.   453.   296.4]
  [ 571.   703.   291.6]
  [ 669.   253.   182.8]
  [ 635.   319.   121.6]
  [ 505.   529.    89.8]
  [ 661.   515.    83.6]
  [ 483.   549.    70.8]
  [ 531.   293.    57.6]
  [ 153.   447.   354.4]
  [ 681.   585.   132.4]
  [ 397.   431.   114.4]
  [ 633.   557.   111.8]
  [ 677.   319.   108. ]
  [ 661.   261.   171.6]
  [ 409.   421.   130.4]
  [ 475.   253.   101.6]
  [ 487.   237.   100.6]
  [ 523.   469.    58.6]
  [  15.   453.   463.4]
  [  53.   449.   425.6]
  [ 275.   319.   268.8]
  [ 579.   731.   265. ]
  [ 453.   559.   128.2]
  [ 635.   439.    57.6]
  [  67.   447.   412.6]
  [ 221.   141.   385.8]
  [ 589.   721.   252. ]
  [ 571.   677.   212.2]
  [ 673.   239.   180.8]
  [ 691.   555.   147.2]
  [ 677.   617.   139.8]
  [ 483.   593.   113.6]
  [ 487.   227.    90.6]
  [ 521.   459.    59.4]
  [ 601.   449.    47. ]
  [ 137.   443.   356. ]
  [ 409.    99.   232. ]
  [ 321.   263.   222.4]
  [ 467.   147.   160. ]
  [ 465.   523.    90.6]
  [ 515.   215.    78.2]
  [ 523.   447.    57.6]
  [ 613.   443.    36. ]
  [ 233.   323.   275.2]
  [ 577.   761.   265.8]
  [ 785.   323.   241.8]
  [ 335.   259.   225.4]
  [ 677.   633.   154. ]
  [ 635.   453.    58.4]
  [1005.   549.   506. ]
  [  81.   449.   394.2]
  [ 221.   447.   290.4]
  [ 571.   779.   283.6]
  [ 311.   255.   232.6]
  [ 681.   203.   211.4]
  [ 681.   255.   186.4]
  [ 699.   617.   164. ]
  [ 451.   515.   113.4]
  [ 465.   537.    90. ]
  [ 425.   353.    89.6]
  [ 639.   483.    69.8]
  [ 683.   347.   132. ]
  [ 695.   319.   116. ]
  [ 397.   443.   110. ]
  [ 455.   379.    70.6]
  [ 515.   535.    66. ]
  [  39.   451.   439.2]
  [ 101.   449.   381.6]
  [ 291.   253.   251.4]
  [ 595.   709.   239.6]
  [ 467.   137.   167.6]
  [ 719.   361.   167.2]
  [ 625.   619.   156. ]
  [ 437.   557.   115.2]
  [ 667.   549.    95.8]
  [ 517.   499.    78. ]
  [ 487.   365.    62.2]
  [ 621.   465.    53.2]
  [ 889.   509.   383.8]
  [ 857.   503.   353.6]
  [ 575.   745.   280.2]
  [ 653.   335.   108.8]
  [ 505.   247.    87.8]
  [ 995.   549.   496.8]
  [ 923.   439.   348. ]
  [ 165.   449.   331. ]
  [ 483.   763.   281. ]
  [ 685.   719.   275.4]
  [ 681.   181.   229.8]
  [ 755.   329.   218.2]
  [ 689.   613.   159. ]
  [ 409.   411.   107.4]
  [ 653.   535.   100.8]
  [ 465.   509.    77.8]
  [  83.   439.   406.4]
  [  91.   445.   391. ]
  [ 101.   319.   387. ]
  [ 681.   777.   299.2]
  [ 571.   721.   259.8]
  [ 301.   257.   242.4]
  [ 471.   563.    83.8]
  [ 563.   311.    55. ]
  [ 115.   549.   409. ]
  [ 849.   319.   241. ]
  [ 839.   323.   232.8]
  [ 685.   671.   193.2]
  [ 687.   277.   146.6]
  [ 371.   447.   122.8]
  [ 681.   601.   120.6]
  [ 493.   549.    70.8]
  [ 625.   431.    50.8]
  [ 535.   435.    47. ]
  [ 509.   391.    44.8]
  [ 685.   645.   166.4]
  [ 445.   549.   104.2]
  [ 655.   319.    93. ]
  [ 595.   329.    86. ]
  [ 497.   361.    60.4]
  [ 479.   387.    56.4]
  [ 695.   677.   197. ]
  [ 465.   159.   152.2]
  [ 729.   311.   149.6]
  [ 381.   453.   127.2]
  [ 685.   335.    96. ]
  [ 683.   709.   267.2]
  [ 409.   205.   139.4]
  [ 685.   295.   133.4]
  [ 313.   447.   123.8]
  [ 675.   557.   102. ]
  [ 461.   255.   100.6]
  [ 645.   323.    92. ]
  [ 465.   549.    86.2]]]

and here's the code:

import pyautogui
import cv2


def process(img):
    img_gray = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)
    _, thresh = cv2.threshold(img_gray, 128, 255, cv2.THRESH_BINARY)
    img_blur = cv2.GaussianBlur(thresh, (5, 5), 2)
    img_canny = cv2.Canny(img_blur, 0, 0)
    img_hough = cv2.HoughCircles(img_canny, cv2.HOUGH_GRADIENT, 2, minDist=10)
    print(img_hough)
    return img_canny


# region
firstThirdOfTheScreensWidth = int(pyautogui.size()[0] / 1.7)
heightOfTheScreen = pyautogui.size()[1]
starting_point_x = 100
starting_point_y = 200
region = (starting_point_x, starting_point_y,
          (firstThirdOfTheScreensWidth - starting_point_x), (heightOfTheScreen - (starting_point_y + 100)))

pyautogui.screenshot(region=region, imageFilename="whatTheBotSees.png")
processed_image = process(cv2.imread("whatTheBotSees.png"))

Max Behling · Answer

I was able to get the correct circles by setting the min and max radius in HoughCircles to 35 and 53.

import numpy as np
import pyautogui
from cv2 import cv2

def process(img, minr=35, maxr=53):
    img_gray = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)
    _, thresh = cv2.threshold(img_gray, 128, 255, cv2.THRESH_BINARY)
    img_blur = cv2.GaussianBlur(thresh, (5, 5), 2)
    img_canny = cv2.Canny(img_blur, 0, 0)
    circles = cv2.HoughCircles(img_canny, cv2.HOUGH_GRADIENT, 2, minDist=10, minRadius=minr, maxRadius=maxr)
    if len(circles) > 0:
        circles = np.uint16(np.around(circles))
        return img_canny, circles
    else:
        return img_canny, None


raw_image = cv2.imread("whatTheBotSees.jpg")
processed_image, circles = process(raw_image)

#draw circles:
if circles is not None:
    for i in circles[0, :]:
        print("(center), radius:  (",i[0],  i[1],")", i[2])
        cv2.circle(raw_image, (i[0], i[1]), i[2], (0, 255, 0), 2)
        cv2.circle(raw_image, (i[0], i[1]), 2, (0, 0, 255), 3)

cv2.imshow("Processed Image", raw_image)
cv2.waitKey(0)
cv2.destroyAllWindows()

output list of circles:

(center), radius:  ( 579 259 ) 38
(center), radius:  ( 471 445 ) 38
(center), radius:  ( 681 443 ) 38
(center), radius:  ( 473 321 ) 37
(center), radius:  ( 579 505 ) 40

This is not a robust solution due to the the hand tuned radius parameters, but it might be good enough if all the images you are processing in this way are mostly identical to the test image you provided. If you need to identify those circles at other sizes or on other screens, you may need to add some more logic to identify the correct circles.

return the position of all circles from an image

1 Answers