When performing a clip operation on a dataframe...
df = df.clip(lower_bound, upper_bound)
... I'd like to get the number of elements that has been replaced, that is: the number of items below the lower_bound and above the upper_bound.
What is the most efficient way of doing that?
Try with
df_new = df.clip(lower_bound, upper_bound)
Low = (df_new>df).sum().sum()
Up = (df_new<df).sum().sum()
If you're looking for the total number of replaced values, going to NumPy domain should be efficient:
np.count_nonzero(clipped.to_numpy() != df.to_numpy())
which compares frames' underlying NumPy arrays and counts nonzeros i.e., counts Trues to get the number of replaced values.
(if you don't want to import NumPy for this, an equivalent is (clipped.to_numpy() != df.to_numpy()).sum().)
An option (suggested by Ananya C.) to count the number of clipped values for a particular column (column_name):
num_of_clipped_values = ((df[column_name] < (lower_bound)) | (df[column_name] > (upper_bound))).sum()
(not tested for efficiency against other solutions)