How to enable numerical feature drift?

user_11672069 0
Viewed 395

I want to enable numerical feature drift without setting a domain. Ex: I am scoring customers based on age and in my training set I had a uniform distribution of the variable. Now, in my serving data, all customers are in their 50's (a domain won't catch such shift change). Is there any option to flag such behavior that would call for a re-training in tfdv ?

I tried giving tfdv two distributions N(0,1) and N(10,1) but no anomalies where detected.

EDIT : Drift only works for Categorical features.

tensorflow-data-validation
3 Answers

Drifts for numerical features is in development right now and are going to be supported in the next version of tfdv (after 0.24.1). To do that you will have to use JSD instead of infinity norm.

user_144142620
user_46803340

Drift Comparator is used for, kind of Time Series Data, to compare, for example, Yesterday's and Today's Data.

In your case, I think you should be using the Skew Comparator, as it will find the difference in Distribution/Skew between Training and Serving Data. Code for that function is shown below:

serving_stats = tfdv.generate_statistics_from_tfrecord(data_location=serving_data_path)

tfdv.get_feature(schema, 'payment_type').skew_comparator.infinity_norm.threshold = 0.01

skew_anomalies = tfdv.validate_statistics(
        statistics=train_stats, schema=schema, serving_statistics=serving_stats)

You may have to play around with the Threshold Value (mentioned as 0.01) for flagging the Anamolies.

user_115304620
Related