Binance ccxt fetch ohlcv exceeding 1000 limit

user_13771261 0
Viewed 5163

Trying to create a loop that will get all 5m interval ohlcv data for coins in pair_list between start and date (3 months)

Based on some discussions and snippets I've come up with the following, however, it doesn't work

import ccxt
import config
import pandas as pd
pd.set_option('display.max_rows', 1000)
pd.set_option('display.max_columns', 500)
pd.set_option('display.width', 1000)


import warnings
warnings.filterwarnings('ignore')

import numpy as np
#np.printoptions(precision=2, suppress=True)

from datetime import datetime, timedelta, date, time

import datetime as dt

import datedelta
import calendar
import dateutil.relativedelta
from dateutil.relativedelta import relativedelta


start = str(int(dt.datetime(2021, 5, 1).replace(hour=1).timestamp() * 1000))
end = str(int(dt.datetime(2021, 2, 1).replace(hour=1).timestamp() * 1000))

for i in pair_list:

    startDate = end
    while startDate>start:

        bars[i] = 'exchange.fetch_ohlcv(binancePair[i], timeframe=timeframe, '
        if startDate is not None:
            bars[i] += 'since='+ str(startDate) +', limit=1000'
            d2[i] = pd.DataFrame(bars[i][:-1], columns =['timestamp', 'open', 'high', 'low', 'close', 
            'volume' ])
            d[i] = pd.concat([d2[i], d[i]], axis=0, ignore_index=True, keys=None)
            startDate[i] = d[i].timestamp[0] 

            d[i].reset_index(drop=True, inplace=True)   
            d[i]['timestamp'] = pd.to_datetime(d[i]['timestamp'], unit='ms')
binance python
1 Answers

This isn't exactly the answer you are looking for, but it will definitely solve your problem (ccxt):

    limit=None

Code would look something like this(ccxt):

    start_date = int(datetime.datetime(2018, 1, 1, 10, 20).timestamp() * 1000)
    candles = exchange.fetch_ohlcv('BTC/USDT', timeframe='1d', since=start_date, 2012', limit=None)

That should get candles to present time, then you could easily amend pandas df.

I know the Binance exchange documents state 1000 limit, though when using their api they still allow large amounts of data, for instance this code works on Binance (binance api):

    candles = client.get_historical_klines("BTCUSDT", Client.KLINE_INTERVAL_1DAY, "1 Jan, 2018", "22 May, 2021")

I know what you want to do specifically, as some exchanges do need the limit. Though the Binance exchange those code snippets will work.

I'll make time in a few days to answer the original question, until then i hope this helps.

user_68152010
Related