KNDM
0
I have a dataset and I need to filter bad/invalid data for my compute. I tried percentile but that will filter genuine value
Eg: 1, 2,8,10,20,25,55,100,100000,98,99,95
Here, 100000 is corrupted/bad, When I call Max () function, I expect 100 instead of 100000.
datatable(val:int)[1, 2,8,10,20,25,55,100,100000,98,99,95]
| summarize val = make_list(val)
| extend anomaly_score = series_outliers(val)
| mv-expand val to typeof(int), anomaly_score to typeof(real);
//| where anomaly_score between (-1.5 .. 1.5)
| val | anomaly_score |
|---|---|
| 1 | -0.045592692382452886 |
| 2 | -0.017097259643419842 |
| 8 | 0 |
| 10 | 0 |
| 20 | 0 |
| 25 | 0 |
| 55 | 0 |
| 100 | 0.0028495432739035469 |
| 100000 | 2846.6965801726751 |
| 98 | 0 |
| 99 | 0 |
| 95 | 0 |