Is dc.js (used with crossfilter and d3.js) still a good option for Big data visualization on a browser page?

user_18045902 0
Viewed 133

I'm try to build an online dashboard to visualize a large csv dataset and i want to be sure i'm following the right path.

Thank you everybody.

bigdata crossfilter dc.js d3.js csv
1 Answers

Crossfilter will speedily handle up to around 500K rows of data, maybe more depending on column complexity.

Around that size you also have to consider the time that it will take to download the data to the browser, affecting when the charts will appear in the page.

If your data is bigger than that, and you still want to use dc.js, you have two options:

  • Pre-aggregate your data: instead of counting rows with group.reduceCount(), use group.reduceSum() and have a column with pre-summed integers. Of course, you will not be able to drill into pre-aggregated data, so this is only effective if you can accept some granularity of the dimensions of your charts.

  • Use a server-side replacement for crossfilter, such as elastic-dc. There are other solutions floating around but I think Deepak has the most complete solution.

user_6761950
Related