Extract datapoints from an interactive chart using python?

user_11693768 0
Viewed 69

is it possible to extract the datapoints from the chart in this link?

https://ycharts.com/companies/AAPL/market_cap

The chart is located at //*[@id="dataChartCanvass1"]

Not the table below the chart.

I tried to look at the source for the website but I can only see the datapoints in the table.

Is it possible using python and requests? Where should I start?

python-requests web-scraping charts python
1 Answers

You can simulate their Ajax call to get the chart codepoints, for example:

import json
import requests
import pandas as pd

api_url = "https://ycharts.com/charts/fund_data.json"

params = {
    "securities": "id:AAPL,include:true,,",  # <-- ticker here
    "calcs": "id:market_cap,include:true,,",
    "correlations": "",
    "format": "real",
    "recessions": "false",
    "zoom": "5",
    "startDate": "",
    "endDate": "",
    "chartView": "",
    "splitType": "single",
    "scaleType": "linear",
    "note": "",
    "title": "",
    "source": "false",
    "units": "false",
    "quoteLegend": "true",
    "partner": "",
    "quotes": "",
    "legendOnChart": "true",
    "securitylistSecurityId": "",
    "displayTicker": "false",
    "ychartsLogo": "",
    "useEstimates": "false",
    "maxPoints": "918",
}

data = requests.get(api_url, params=params).json()
# uncomment to see all data:
# print(json.dumps(data, indent=4))

df = pd.DataFrame(
    data["chart_data"][0][0]["raw_data"], columns=["date", "value"]
)
df["date"] = pd.to_datetime(df["date"] / 1000, unit="s")
df["value"] = df["value"].astype(int)
print(df)

Prints:

          date    value
0   2016-08-29   575593
1   2016-09-06   580335
2   2016-09-09   555710
3   2016-09-16   619239
4   2016-09-23   607331
5   2016-09-30   603253
6   2016-10-07   608643
7   2016-10-14   627239
8   2016-10-21   621747
9   2016-10-28   606390
10  2016-11-04   580368
11  2016-11-11   578182

...and so on.
user_100359850
Related