Issue with setting scale on python matplotlib axes

I'm having a bit of an issue with matplotlib I can't seem to figure out. My graph plots fine but if I try to set the axes limits manually all the points disappear (see examples below).

Below is the code and before and after shots of what the plots look like when I add the manual axis scaling. As you can see the axis does change but the markers disappear.

I have verified that the data I am plotting falls within the ranges I am setting so not sure what is going on. Would appreciate any thoughts you may have.

def generateGraph(bore, type, xMin, xMax, yMin, yMax, legend):     #type: 1=Normalised, 2=Actual or 3=Drawdown
    fig, ax = plt.subplots()      #declare instance of plot
   
    markerSize = 1 
   #Plot observed data
    if type == 1 or type == 2:
        observedPt = ax.scatter(bore.listObsTime, bore.listObsHead, s=markerSize)
        observedLine = ax.plot(bore.listObsTime, bore.listObsHead)
    else:
        observedPt = ax.scatter(bore.listObsTime, bore.listObsDrawDown)
        observedLine = ax.plot(bore.listObsTime, bore.listObsDrawDown)
    
        
    for i in range(bore.numFilesLoaded):
        if type == 1:
            modelledPt = ax.scatter(bore.listModelledTimeArrays[i], bore.listModelledHeadNormalisedArrays[i], s=markerSize)
            modelledLine = ax.plot(bore.listModelledTimeArrays[i], bore.listModelledHeadNormalisedArrays[i])
        elif type ==2:
            modelledPt = ax.scatter(bore.listModelledTimeArrays[i], bore.listModelledHeadArrays[i], s=markerSize)
            modelledLine = ax.plot(bore.listModelledTimeArrays[i], bore.listModelledHeadArrays[i])
        else:
            modelledPt = ax.scatter(bore.listModelledTimeArrays[i], bore.listModelledDrawDownArrays[i], s=markerSize)
            modelledLine = ax.plot(bore.listModelledTimeArrays[i], bore.listModelledDrawDownArrays[i])
    
    ax.set_xlim(xMin, xMax)
    ax.set_ylim(yMin, yMax)
  
    plt.show()

Here is how the graph plots with the set_xlim and set y_lim commented out:

Graph with xlim and ylim commented out

And here is what happens when I leave them in and try to set reasonable limits manually:

Same graph with xlim and ylim set to reasonable values