Issue With Getting Values from a List of Objects into a Readable Dataframe

Viewed 97

I've been running into this same problem for the last few days, googling everything and searching for answers on this forum, but nothing I'm trying is seeming to work. I've been following tutorials other people have posted and they're able to do this very simply, but I run into multiple problems when I try it from multiple angles. I've got my head so tangled up from all the things I've tried that I'm not even sure what's happening or what I've done anymore. This isn't all of the code, but this should be the only relevant code because the other parts do other functions, but I apologize if I missed anything.

Basically, I'm grabbing historical financial candlestick data from a website, trying to put it into a Pandas dataframe, and then use that dataframe to make charts with Plotly. I get the data as 'result', PyCharm outputs the data just fine into the 'Run' box, but now I need to save that data, so I have Pandas turn 'result' into a dataframe as 'priceData', and convert that to 'pricedata.csv'. When I open that CSV file, all I get is a list from 0 to 1439 (I'm importing 1440 1 minute candlesticks at a time), and each candlestick only shows the object reference (<binance_f.model.candlestick.Candlestick object at 0x0D55EF50>, or similar). That's obviously not what I'm after, I need the data within the candlesticks, which should be made up of 12 pieces of data (open time, open, low, close, high, etc...). When the chart function runs, it comes back with "AttributeError: 'DataFrame' object has no attribute 'high'", which I assume is because it's accessing that candlestick object ID and not the values.

I can get the specific values by going in-depth, and calling for it to save result[0].high, or result[0].low, etc. But then I have to iterate through 1440 candlesticks of data and write/save them all separately, and then bring together the high/low/open/close/time/etc of each of them to be able to plot, which is far more complicated than what the tutorials I've watched do. They're able to literally just use the dataframe right away to plot with no troubles, they don't run into the attribute error, it's like it just identifies the columns correctly for them. But when I look at the columns of 'result', there's 1440 columns (again one for each candlestick), but I would think it should be 12 columns for the 12 different bits of data each candlestick is made up of. I've tried transposing the columns and rows, but that doesn't work either.

Even if I try to get the whole candlestick data for just one object, by specifying "result[0]" without specifying the .high/.low/etc, I run into the same attribute error. Some things recommended to specify what the columns are, so that's what the hashed out "priceData.columns" is for, where I identify what each column is. But then I get "Length mismatch: Expected axis has 1440 elements, new values have 12 elements".

I'm really confusing myself and going in circles at this point, can anyone help point me in the right direction and tell me what I'm screwing up on? Thank you in advance to anyone who takes the time to even read this, or has any direction they can offer.

from binance_f import RequestClient
from binance_f.model import *
from binance_f.constant.test import *
from binance_f.base.printobject import *
import pandas as pd
import numpy as np
import json
import plotly.io as pio

pd.options.plotting.backend = 'plotly'
pio.renderers.default = "browser"

request_client = RequestClient(api_key=g_api_key, secret_key=g_secret_key, url="https://fapi.binance.com")

result = request_client.get_candlestick_data(symbol="BTCUSDT", interval=CandlestickInterval.MIN1, 
                                                startTime=1609473600000, endTime=None, limit=1440)

priceData = pd.DataFrame([result])
priceData.to_csv('priceData.csv')
print(pd.read_csv("E:\Creative\GitHub Projects\Binance_Futures_python\example\market\priceData.csv"))
#priceData.columns = ['opentime', 'open', 'high', 'low', 'close', 'vol', 'closetime', 'quoteassetvol', #'numberoftrades','takerbuybase', 'takerbuyquote', 'ignore']

def chart():
    pricedata = pd.read_csv('E:\Creative\GitHub 
        Projects\Binance_Futures_python\example\market\priceData.csv')
    highprice = pricedata.high
    lowprice = pricedata.low
    openprice = pricedata.open
    closeprice = pricedata.close
    fig = dict({
        "data": [{"type": "candlestick",
                  "open": [openprice],
                  "close": [closeprice],
                  "low": [lowprice],
                  "high": [highprice]}],
        "layout": {"title": {"text": "Candlestick Charts"}}
    })
    pio.show(fig)


chart()
1 Answers

The way you are currently getting the candlestick data, result is a list of Candlestick objects, and each object has attributes that can be individually accessed, such as result[0].high as you pointed out. It sounds like you want to unpack all of these attributes and put them into a DataFrame.

To obtain all of the attributes of an object, such as all 12 attributes of the Candlestick object in a dictionary, you can use result[0].__dict__ which returns: {'openTime': 1609473600000, 'open': '29302.11', 'high': '29356.04', 'low': '29302.10', 'close': '29344.00', 'volume': '170.018', 'closeTime': 1609473659999, 'quoteAssetVolume': '4988200.62513', 'numTrades': 1045, 'takerBuyBaseAssetVolume': '139.291', 'takerBuyQuoteAssetVolume': '4086412.01157', 'ignore': '0'}

To get a list of these dictionaries from result, you can use a list comprehension: [candlestick_obj.__dict__ for candlestick_obj in result], and pd.DataFrame allows you to construct a DataFrame from such a list of dictionaries so priceData = pd.DataFrame([candlestick_obj.__dict__ for candlestick_obj in result]) will create a DataFrame with columns for the keys in each of the dictionaries with 1 row for each list item.

           openTime      open      high  ...  takerBuyBaseAssetVolume  takerBuyQuoteAssetVolume  ignore
0     1609473600000  29302.11  29356.04  ...                  139.291              4.086412e+06       0
1     1609473660000  29344.00  29369.00  ...                   95.555              2.804598e+06       0
2     1609473720000  29353.57  29356.75  ...                   43.821              1.285575e+06       0
3     1609473780000  29339.05  29340.01  ...                   26.836              7.870962e+05       0
4     1609473840000  29334.63  29334.63  ...                    6.004              1.760785e+05       0
...             ...       ...       ...  ...                      ...                       ...     ...
1435  1609559700000  29357.15  29370.00  ...                   35.133              1.031685e+06       0
1436  1609559760000  29370.00  29380.07  ...                   25.957              7.624015e+05       0
1437  1609559820000  29378.76  29382.00  ...                   20.663              6.070590e+05       0
1438  1609559880000  29371.32  29374.25  ...                   15.185              4.459265e+05       0
1439  1609559940000  29363.07  29364.68  ...                    4.084              1.199246e+05       0

The rest of your code is fine, and your chart function produces a candlestick chart as expected.

from binance_f import RequestClient
from binance_f.model import *
from binance_f.constant.test import *
from binance_f.base.printobject import *
import pandas as pd
import numpy as np
import json
import plotly.io as pio

pd.options.plotting.backend = 'plotly'
pio.renderers.default = "browser"

request_client = RequestClient(api_key=g_api_key, secret_key=g_secret_key, url="https://fapi.binance.com")

result = request_client.get_candlestick_data(symbol="BTCUSDT", interval=CandlestickInterval.MIN1, 
                                                startTime=1609473600000, endTime=None, limit=1440)

## you want the attributes of the Candlestick object
priceData = pd.DataFrame([candlestick_obj.__dict__ for candlestick_obj in result])
priceData.to_csv('priceData.csv')
# print(pd.read_csv("E:\Creative\GitHub Projects\Binance_Futures_python\example\market\priceData.csv"))
# priceData.columns = ['opentime', 'open', 'high', 'low', 'close', 'vol', 'closetime', 'quoteassetvol', #'numberoftrades','takerbuybase', 'takerbuyquote', 'ignore']

def chart():
    pricedata = pd.read_csv('priceData.csv', index_col=0)
    # print(pricedata.head())
    highprice = pricedata['high']
    lowprice = pricedata['low']
    openprice = pricedata['open']
    closeprice = pricedata['close']
    fig = dict({
        "data": [{"type": "candlestick",
                  "open": openprice,
                  "close": closeprice,
                  "low": lowprice,
                  "high": highprice}],
        "layout": {"title": {"text": "Candlestick Charts"}}
    })
    pio.show(fig)

chart()

enter image description here

Related