R `summary` function closest equivalent in python

user_1727392 0
Viewed 759

Is there any kind of helper to find the min, max (and ideally standard deviation) of each dimension in a multidimensional array within numpy? I'm looking for something like the summary() function in R.

My data is essentially a huge 2D array (list of lists), in which the sublists contain n dimensional values. E.g. currently I have data with 3 dimensional attributes x,y,z:

a = np.random.rand(100,3)

For each of those dimensions (x,y,z) I want to know the min, max, mean, and std.

I know one can loop through the axes and measure these values, e.g.: 

for i in range(a.shape[-1]):
  vals = a[:,i]
  print(np.min(vals), np.max(vals), np.std(vals))

I find myself writing the code to do that almost every time I have a new dataset. Any way to expedite this operation would be hugely helpful!

numpy arrays python
2 Answers

Without pandas:

from scipy import stats
import numpy as np

a = np.random.rand(100,3)
summary = stats.describe(a, axis = 0)

print(summary.mean)
print(summary.minmax)
...

Using pandas:

import pandas as pd

summary_across_rows = pd.DataFrame(a).describe() # across axis=0
print(summary)
                0           1           2
count  100.000000  100.000000  100.000000
mean     0.495204    0.573827    0.476202
std      0.275131    0.246189    0.271626
min      0.005202    0.037195    0.023595
25%      0.295210    0.399358    0.258712
50%      0.512023    0.562181    0.417322
75%      0.710216    0.790970    0.712047
max      0.998371    0.997717    0.980840

Note: for the summary across the other dimension you need:
summary_across_columns = pd.DataFrame(a.T).describe() # across axis=1
user_50250090

Without pandas:

from scipy import stats

stats.describe(lst)
stats.scoreatpercentile(lst,(5,10,50,90,95))

Here is an example:

from scipy import stats
import numpy as np

stdev = 10
mu =  10
a=stdev*np.random.randn(100)+mu
stats.describe(a)

[OUT1]: DescribeResult(nobs=100, minmax=(-13.180682481878286, 40.6109521437826), mean=10.352380786199149, variance=103.27168865119998, skewness=0.13852516641657087, kurtosis=0.2691915766145532)

stats.scoreatpercentile(a,(5,10,50,90,95))

[OUT2]: array([-7.21731609, -3.22696662, 10.39364637, 21.78527621, 24.20685179])
user_59866510
Related