Understanding slow Django performance

user_5509839 0
Viewed 531

My webserver is scaling and I'm using Sentry Performance to try and better understand where things are slow. One thing that I don't quite understand is where the slowness might be coming from when the total reported query times are dramatically different than the time it takes to get a response.

For example, one of my endpoints is taking 40s to finally get a response to the user. You'll see the total response time took an incredible 44,000ms even though all the work was done in about 1s (which is still slow, but not 44s slow).

Edit: as a general note, all of my endpoints are doing this– the peculiar thing is those dashed dots at the end of the timeline where any of the actual Django / database hits begin. It just sits blank for a whole minute before Django begins doing anything. Is this a potential guincorn or Uvicorn configuration error?

Additional info:

  • Running gunicorn w/ uvicorn: poetry run gunicorn -w 17 myApp.asgi:application -k myApp.uvicorn.AppUvicornWorker --log-level=debug
  • I'm getting about a hundred requests a second, which the server should be more than able to handle– it has 32gb of ram and 8 cpu cores.

AppUvicornWorker:

class AppUvicornWorker(UvicornWorker):
    CONFIG_KWARGS = {"loop": "uvloop", "http": "httptools", "lifespan": "off"}
  • Views are just basic, completely standard Django Rest Framework ListViewSets queried:
class UserSerializer(serializer.ModelSerializer):
    class Meta:
        model = User
        fields = ["id", "username", "name"]

class CommentSerializer(serializer.ModelSerializer):
    user = UserSerializer()

    class Meta:
        model = Comment
        fields = ["id", "text", "user"]


class CommentsViewSet(viewsets.ViewSet):
    def list(self, request):
        queryset = Comments.objects.all()
        serializer = CommentSerializer(queryset, many=True)
        return Response(serializer.data)

enter image description here

uvicorn gunicorn django http python
1 Answers

One of the reasons that this is slow is because for each Comment, it has to make a separate query to fetch the User data. You can boost efficiency with .select_related(…) [Django-doc]:

class CommentsViewSet(viewsets.ViewSet):
    
    def list(self, request):
        queryset = Comments.objects.select_related('user')
        serializer = CommentSerializer(queryset, many=True)
        return Response(serializer.data)

This will retrieve the data of the user in the same query, and thus prevent the N+1 query problem.

Note: normally a Django model is given a singular name, so Comment instead of Comments.

user_675790
Related