How to detect objects in the Surf Game in Microsoft Edge with object detection? [closed]

I'm trying to develop an AI for the Surf game in Microsoft Edge (will be published on GitHub later).

My code uses PIL.ImageGrab to take screenshots, so now I need the program to detect objects on the screen. For example (the High Visibility Mode (HV) is enabled):

And another with HV disabled:

So how can I detect and classify these objects into 8 classes (described in the code) with object detection?

Your code should be formatted like follows (PyTorch is preferred):

from enum import Enum
from typing import Tuple
from PIL import Image

# Position for each object
Rect = Tuple[int, int, int, int] # (top, left, bottom, right)

class GameObject(Enum):
    # Enum for the 8 types of objects in the game
    surfer    = 0 # Other surfers to avoid (Marked in black)
    obstacle  = 1 # Obstacles, e.g. islands (Also marked in black)
    slowdown  = 2 # Objects that slow you down (Marked in red)
    changeDir = 3 # Objects that change your direction (Also marked in red)
    monster   = 4 # The monster that try to catch you (Marked in red or black)
    dog       = 5 # The dog that protects you from monsters (Marked in green)
    heart     = 6 # Add a heart to the surfer (Marked in green)
    boost     = 7 # Gives you a chance to boost or let you boost now (Marked in green)

def recognizeObjects(img: Image.Image) -> Tuple[GameObject, Rect]:
    '''
    Parameter [img]: the image to recognize objects
    Returns a tuple of objects and its position.
    '''
    pass

from Recent Questions - Stack Overflow https://ift.tt/3COw3tJ
https://ift.tt/3CJ8kuL