Not getting good results with OpenCV in Python

I'm trying to use OpenCV with python in anaconda.

I'm following this webpage: https://towardsdatascience.com/deep-learning-based-super-resolution-with-opencv-4fd736678066

I did the butterfly image and got fantastic results. It looks like a high-res photo. However, every other image I upscale does not at all, they all just look zoomed in. Why is this? The code I'm using is this:

import cv2
from cv2 import dnn_superres
import sys
import pathlib
import os
from os import path

# Create an SR object
sr = dnn_superres.DnnSuperResImpl_create()

# Read image

findfile = "people"

print("Searching for " + findfile)

for file in pathlib.Path(sys.path[0]).iterdir():
    if file.is_file():
        if findfile in str(file):
            foundfile = str(file)
            break

print("Found file " + foundfile)
image = cv2.imread(foundfile, 1)

# Read the desired model
path = sys.path[0]+"/EDSR_x4.pb"
sr.readModel(path)

# Set the desired model and scale to get correct pre- and post-processing
sr.setModel("edsr", 4)

# Upscale the image
result = sr.upsample(image)

# Save the image
findfile_basename = os.path.basename(findfile)
suffix = pathlib.Path(foundfile).suffix
print("Writing to upscaled_"+findfile_basename + suffix)
cv2.imwrite(sys.path[0]+"/upscaled_"+findfile_basename+suffix, result)

Why does the butterfly one work so well and the others dont? Do I need to train something or do something extra?