Constrained Logistic Regression (outcome variable) in Python

user_16679189 0
Viewed 240

In order to predict future stock movements, we use a logistic regression in Python. We do so by converting the daily return into a weekly return.Next, we determine if the return is going up or down, by using this code:

# calculate weekly log returns and market direction
stock['returns'] = np.log(stock / stock.shift(5))
stock.dropna(inplace=True)
stock['direction'] = np.sign(stock['returns']).astype(int)

We have three direction signals:

 0  = hold the stock / do noting
 1  = buy the stock
-1 = sell the stock

Below is the code to determine the long direction, if the stock is still increasing then hold the stock, otherwise sell the stock.

stock['long direction'] = 0
for val, group in itertools.groupby(enumerate(stock['direction']), itemgetter(1)):
    # this is tuple unpacking, irrelevant is a list of values that aren't the last one, and last is the one we care about.
    [*irrelevent, last] = group
    stock['long direction'].iloc[last[0]] = -last[1]

del stock['direction'] 

The output is as follows:

    Date        close       returns     long direction
    2021-12-08  1068.95     -0.024068   0
    2021-12-09  1003.79     -0.077418   1
    2021-12-10  1017.03      0.002028   -1
    2021-12-13  966.40      -0.043137   0
    2021-12-14  958.51      -0.092831   0
    2021-12-15  975.98      -0.090989   0
    2021-12-16  926.91      -0.079681   0
    2021-12-17  932.57      -0.086698   1

We have used a logistic regression to predict the future movements, but we don't know how to add a constraint, which prevents short selling. Hence, we don't want a -1, -1 direction in a row, and we don't want a 1, 1 direction in a row. A 0,0,0,0, signal is fine, which means that we have to hold the stock. We have a large imbalance in our dataset, the signal 0 is predominant. We have dealt with this issue by using class_weight.

Below is the current code:

stock = stock.dropna()
X = stock.loc[:, stock.columns != 'long direction']
y = stock['long direction']

X_train, X_test, y_train, y_test = sklearn.model_selection.train_test_split(X, y, test_size = 0.05, random_state = 5, shuffle=False)

model1 = LogisticRegression(random_state=0, multi_class='multinomial', penalty='none', solver='newton-cg', class_weight={-1:0.35, 0:0.3, 1:0.75}).fit(X_train, y_train)
preds = model1.predict(X_test)

#print the tunable parameters (They were not tuned in this example, everything kept as default)
params = model1.get_params()
print(params)

Below is the output of the code, where the signal -1,-1, is the predictor (outcome or Y variables), but we don't want that the model predicts -1,-1 signals consecutively.

           long direction   pred
Date        
2021-10-11  0               -1
2021-10-12  0               -1
2021-10-13  0               -1
2021-10-14  0               -1
2021-10-15  0               -1

How can we add the constraint, that the model knowns that the outcome variables (Y) -1,-1 and a 1,1 signals in a row are not possible? I can only find information on the internet about constraints on the input variables (X).

logistic-regression scikit-learn machine-learning python
1 Answers

There are a few approaches to do this.

The first approach is to modify the recommendation after it is given by the logistic regression, so if it returns -1 and then -1 in a row you have some other mechanism in place which decides what to "change" that second -1 into (this could be anything: a simple decision rule you create yourself, going back in the projections to use previous long values, etc.). This approach feels like more of bodge than an elegant solution. If you prefer a solution that is more elegant, continue reading for the second approach.

The second approach is to change your model so that it includes that fact. The foundational idea of Machine Learning algorithms like logistic regression is to be able to do the prediction for you automatically without you having to create and define explicit rules. In congruence with this, you can modify your training data such that it takes into account the previous signal as part of its input data, and then the training output data will also use this fact. If your training data always has the previous indicator as one of its inputs, and whenever that previous indicator is -1 the output is never -1 (because you do not want two -1 in a row), then the model will learn not to give you a -1 on new inference data with the previous indicator was -1.

A third approach that comes to mind which is perhaps in between the first and second approach is to have three different trained models: one model trained whenever the previous direction was -1, one model when it was 0, and one model when it was 1. The first and third models do not even have the option -1 and 1 respectively in their label output space so they can not go wrong in that way, and then second model would have all three of -1, 0, and 1.

user_160436320
Related