We can log the training of libraries such as gensim using
import logging
logger = logging.getLogger()
logger.setLevel(logging.INFO)
This doesn't work when running of T-SNE (from sklearn.manifold). Since this can take a long time, logging of some form would be nice. I have looked through the docs, but cannot seem to find another way to log the training of this. Is there a way to log this, or is it not possible?
It's because that sklearn code don't have any logging. The contributing guide don't have any standard approach for this.
However you can use the verbose flag to get more information about what the class is doing.
If you use the flag like this:
import numpy as np
from sklearn.manifold import TSNE
X = np.array([[0, 0, 0], [0, 1, 1], [1, 0, 1], [1, 1, 1]])
tsne = TSNE(n_components=2, verbose=1) # note the verbose flag
X_embedded = tsne.fit_transform(X)
X_embedded.shape(4, 2)
The output will be
[t-SNE] Computing 3 nearest neighbors...
[t-SNE] Indexed 4 samples in 0.000s...
[t-SNE] Computed neighbors for 4 samples in 0.000s...
[t-SNE] Computed conditional probabilities for sample 4 / 4
[t-SNE] Mean sigma: 1125899906842624.000000
[t-SNE] KL divergence after 250 iterations with early exaggeration: 31.763723
[t-SNE] KL divergence after 650 iterations: 0.048727
That code is adapted from sklearn TSNE code example.
Note that output is the standard output, not a file of something like that. All messages from verbose flag is written using the print function.