How to get values from different tables from DB faster?

Viewed 46

so our DB was designed very badly. There is no foreign key used to link multiple tables I need to fetch complete information and export it to csv. the challenge is the information need to be queried from multiple tables (say for e.g, usertable only stored sectionid in the table, in order to get section detail, I would have to query from section table and match it with sectionid acquired from usertable).

So i did this using serializer, because the fields are multiples.

So the problem with my current method is that its so slow because it needs to query for each object(queryset) to match with other tables using uuid/userid/anyid.

this is my views

class FileDownloaderSerializer(APIView):

    def get(self, request, **kwargs):

            filename = "All-users.csv"
            f = open(filename, 'w')                
            datas = Userstable.objects.using(dbname).all()                
            serializer = UserSerializer( datas, context={'sector': sector}, many=True)                                        
            df=serializer.data

        df.to_csv(f, index=False, header=False)
        f.close()

        wrapper = FileWrapper(open(filename))
        response = HttpResponse(wrapper, content_type='text/csv')
        response['Content-Length'] = os.path.getsize(filename)
        response['Content-Disposition'] = "attachment; filename=%s" % filename

        return response

so notice that i need one file exported which is .csv.

this is my serializer

class UserSerializer(serializers.ModelSerializer):
   class Meta:
        model = Userstable
        fields = _all_
   section=serializers.SerializerMethodField()

   def get_section(self, obj):
        return section.objects.using(dbname.get(pk=obj.sectionid).sectionname

   department =serializers.SerializerMethodField()
   def get_department(self, obj):
        return section.objects.using(dbname).get(pk=obj.deptid).deptname

im showing only two tables here, but in my code i have total of 5 different tables

I tried to limit 100 rows and it is successful, i tried to fecth 300000 and it took me 3 hours to download csv. certainly not efficient. How can i solve this?

0 Answers
Related