Imbalanced classes in multi-class classification problem

there are various options to build weights for un unbalance classification problems. one of the most common is to use directly the class counts in train to estimate sample weights. this option is easily computed by sklearn. The 'balanced' mode uses the values of y to automatically adjust weights inversely proportional to class frequencies.

what we try to do in the example below is to 'incorporate' the compute_sample_weight method in fitting our DNNClassifier. as label distribution, I used the same expressed in the question

import numpy as np
import pandas as pd
import tensorflow as tf
from sklearn.utils.class_weight import compute_sample_weight

train_size = 1000
test_size = 200
columns = 30

## create train data
y_train = np.random.choice([0,1,2,3], train_size, p=[0.15, 0.35, 0.28, 0.22])
x_train = pd.DataFrame(np.random.uniform(0,1, (train_size,columns)).astype('float32'))
x_train.columns = [str(i) for i in range(columns)]

## create train weights
weight = compute_sample_weight(class_weight='balanced', y=y_train)
x_train['weight'] = weight.astype('float32')

## create test data
y_test = np.random.choice([0,1,2,3], test_size, p=[0.15, 0.35, 0.28, 0.22])
x_test = pd.DataFrame(np.random.uniform(0,1, (test_size,columns)).astype('float32'))
x_test.columns = [str(i) for i in range(columns)]

## create test weights
x_test['weight'] = np.ones(len(y_test)).astype('float32') ## set them all to 1

## utility functions to pass data to DNNClassifier
def train_input_fn():
    dataset = tf.data.Dataset.from_tensor_slices((dict(x_train), y_train))
    dataset = dataset.shuffle(1000).repeat().batch(10)
    return dataset

def eval_input_fn():
    dataset = tf.data.Dataset.from_tensor_slices((dict(x_test), y_test))
    return dataset.shuffle(1000).repeat().batch(10)

## define DNNClassifier
classifier = tf.estimator.DNNClassifier(
    feature_columns=[tf.feature_column.numeric_column(str(i), shape=[1]) for i in range(columns)],
    weight_column = tf.feature_column.numeric_column('weight'),
    hidden_units=[10],
    n_classes=4,
)

## train DNNClassifier
classifier.train(input_fn=lambda: train_input_fn(), steps=100)

## make evaluation
eval_results = classifier.evaluate(input_fn=eval_input_fn, steps=1)

considering that our weights are built as a function of the target we have to set them to 1 in our test data because the labels are unknown.

I would highly suggest you to use undersampling (if you have enough data to do so) or oversampling with SMOTE, in imblearn library, you can find it here

As an experienced ML engineer, I can say that none of the "weighting" methods will ever work for you, XGBoost has a parameter called scale_pos_weight, or you could use logistic regression with class_weight="balanced" but they're quite insignificant because the problem is not about the estimator, it's about your data. So I would highly suggest you to play with your data instead of assigning weights.

you can try the following formula to balanced all classes:

weight_for_class_X = total_samples_size / size_of_class_X / num_classes

for exampe:

num_CLASS_0: 10000   
num_CLASS_1: 1000
num_CLASS_2: 100

wgt_for_0 = 11100 / 10000 / 3 = 0.37  
wgt_for_1 = 11100 / 1000 / 3 = 3.7
wgt_for_2 = 11100 / 100 / 3 = 37

# so after one epoch training the total weights of each class will be:
total_wgt_of_0 = 0.37 * 10000 = 3700
total_wgt_of_1 = 3.7 * 1000 = 3700
total_wgt_of_2 = 37 * 100 = 3700