Sending a Python Dictionary Through a Single POST method in Django REST Framework

user_13030120 0
Viewed 1916

I have been attempting to implement this method several ways in which seems like it should be pretty straightforward. I have a web scraper that scrapes stock data and stores it into a Python dictionary in a python script. This then makes a post request to my Django API which is handled under my views using GenericAPIView. This post request sends a QueryDict and I am able to post a single instance but not multiple at once.

Yes, I have set many=True on my serializer and attempted to override these methods. Each solution I have made is inefficient and I am frustrated not finding a clear and simple solution. This is my first time working with the Django Rest Framework so bear with me and thanks for your help in advance!

CryptoView (Mostly followed a tutorial on Medium.com to improve my method)

class CryptoView(ListCreateAPIView):
    queryset = Crypto.objects.all()
    serializer_class = CryptoSerializer

    def get(self, request, *args, **kwargs):
        return self.list(request, *args, **kwargs)

    def post(self, request, *args, **kwargs):
        return self.create(request, *args, **kwargs)

class SingleCryptoView(RetrieveUpdateDestroyAPIView):
    queryset = Crypto.objects.all()
    serializer_class = CryptoSerializer

Cryptotest.py

crypto = []


# For Loop - Yahoo Finance requires us to crawl through specific
# attributes to find data
for listing in soup.find_all('tr', attrs={'class':'simpTblRow'}):
    listing_dict = {}
    for name in listing.find_all('td', attrs={'aria-label':'Name'}):
        listing_dict["name"] = name.text
    for price in listing.find_all('td', attrs={'aria-label':'Price (Intraday)'}):
        listing_dict["price"] = price.text.replace(',', '')
    for change in listing.find_all('td', attrs={'aria-label':'Change'}):
        listing_dict["change"] = change.text
    for percentChange in listing.find_all('td', attrs={'aria-label':'% Change'}):
        listing_dict["percentChange"] = percentChange.text.replace('%', '')
    index += 1

    crypto.append(listing_dict)

headers = {'Content-type': 'application/json'}
requests.post(url=API_ENDPOINT, json=json.dumps(crypto), headers=headers)

CryptoSerializer (Was testing with the init accepting many)

class CryptoSerializer(serializers.ModelSerializer):
    def __init__(self, *args, **kwargs):
        many = kwargs.pop('many', True)
        super(CryptoSerializer, self).__init__(many=many, *args, **kwargs)

    class Meta:
        model = Crypto
        fields = ('name', 'price', 'change', 'percentChange')

    def create(self, validated_data):
        return Crypto.objects.create(**validated_data)

    def update(self, instance, validated_data):
        instance.name = validated_data.get('name', instance.name)
        instance.price = validated_data.get('price', instance.price)
        instance.change = validated_data.get('change', instance.change)
        instance.volume = validated_data.get('percentChange', instance.volume)

        instance.save()
        return instance

UPDATE: Able to send the data as a JSON array to the post method. Now getting the error of Bad Request from the serializer which fails on the serializer.is_valid() check. If I remove the raise_exception=True then my create() gets no data passed in. This is how my JSON array is formatted if it is correct I don't know: [{"id": 0, "name": "Bitcoin USD", "price": "5314.87", "change": "+54.02", "percentChange": "+1.03"}, Thanks again for the help it is much appreciated as I am excited to keep improving this project!

UPDATE (2): As requested, here is the exact JSON data passed to the serializer with the create() method. This is not all of the data but it is the exact format [{"id": 107, "name": "FirstCoin USD", "price": "0.0017", "change": "-0.0012", "percentChange": "-41.52"}, {"id": 108, "name": "MCAP USD", "price": "0.0046", "change": "0.0000", "percentChange": "0.00"}, {"id": 109, "name": "ATBCoin USD", "price": "0.0011", "change": "+0.0002", "percentChange": "+17.10"}, {"id": 110, "name": "Exchange Union USD", "price": "0.6857", "change": "+0.0026", "percentChange": "+0.38"}]Bad Request: /crypto/ This data is not present when attemping to print(request.data) within my serializer. All that is returned is {}. Here is my Crypto Model Models.py

# Template Model
class Crypto(models.Model):
    name = models.CharField(max_length=30)           # Name of the stock
    price = models.FloatField()                        # Opening stock price
    change = models.FloatField()                      # Closing stock price
    percentChange = models.FloatField()                     # Amount of sales

    def __str__(self):
        return self.name
django-rest-framework post rest django python
1 Answers

OK, first read this example to know more about creating multiple at once.

Your problem is simply with the format you passed to post.

In JSON when you are passing multiple objects then we expect something like that

[
  {
    "name":"John",
    "age":30,
    "cars":[ "Ford", "BMW", "Fiat" ]
  },

  {
    "name":"Matt",
    "age":25,
    "cars":["BMW", "Fiat" ]
  },
]

As you can see it's a JSON array.

In DRF when you are creating a serializer you write something to convert from JSON to one model instance and vice versa. So when we have multiple objects like the above example and we want to do whatever with it, we add the keyword many=True to our serializer so it knows that the passed data has many objects, not just one, so it loops over it and converts for each one.

Let's get back to your created data:

crypto = {
    "id": [],
    "name": [],
    "price": [],
    "change": [],
    "percentChange": []
}

Which is a dictionary and each key has an array, you can see now why it doesn't work at all as it doesn't match what the serializer is expecting

Of course, if this is the format you want to pass, we can work on the serializer and write some code and change its fields so it handles it, but I feel it's better to construct the data as an array of JSON and pass it.

You can use zip() to loop on two lists or more at the same time, so you create the complete object at once, or whatever you want there are too many ways.

UPDATE: As a suggestion, the simplest you can do to make an array of dictionaries is,

crypto = []


# For Loop - Yahoo Finance requires us to crawl through specific
# attributes to find data
for listing in soup.find_all('tr', attrs={'class':'simpTblRow'}):
    listing_dic={}
    for name in listing.find_all('td', attrs={'aria-label':'Name'}):
        listing_dic["name"] = name.text
    for price in listing.find_all('td', attrs={'aria-label':'Price (Intraday)'}):
        listing_dic["price"] = price.text.replace(',', '')
    for change in listing.find_all('td', attrs={'aria-label':'Change'}):
        listing_dic["change"] = change.text
    for percentChange in listing.find_all('td', attrs={'aria-label':'% Change'}):
        listing_dic["percentChange"] = percentChange.text.replace('%', ''))
    crypto.append(listing_dic)

Note: I assumed every child for loop is running only once because it is one row, as I'm not a soup expert I will let you handle this.

Then you can pass data or use json.dump() or whatever, it's ready for use now

UPDATE 2: apply json.dump() on every dict inside the loop, then apply it agin on the array.

crypto.append(json.dump(listing_dic))

then data=json.dump(stocks)

user_49844930
Related