I have the following tuple stored in a variable called newsfeed, which contains a row vector in the form of a CSR sparse matrix of shape (1, 9635) representing a news article and an integer from 0-4 (inclusive) representing the topic of the news article, where:
{0: "business", 1: "entertainment", 2: "politics", 3: "sport", 4: "tech"}
Example of the top 10 newsfeed tuple:
Index: 0 (<1x9635 sparse matrix of type '<class 'numpy.float64'>'
with 122 stored elements in Compressed Sparse Row format>, 0)
Index: 1 (<1x9635 sparse matrix of type '<class 'numpy.float64'>'
with 176 stored elements in Compressed Sparse Row format>, 0)
Index: 2 (<1x9635 sparse matrix of type '<class 'numpy.float64'>'
with 136 stored elements in Compressed Sparse Row format>, 2)
Index: 3 (<1x9635 sparse matrix of type '<class 'numpy.float64'>'
with 65 stored elements in Compressed Sparse Row format>, 2)
Index: 4 (<1x9635 sparse matrix of type '<class 'numpy.float64'>'
with 159 stored elements in Compressed Sparse Row format>, 2)
Index: 5 (<1x9635 sparse matrix of type '<class 'numpy.float64'>'
with 106 stored elements in Compressed Sparse Row format>, 1)
Index: 6 (<1x9635 sparse matrix of type '<class 'numpy.float64'>'
with 98 stored elements in Compressed Sparse Row format>, 0)
Index: 7 (<1x9635 sparse matrix of type '<class 'numpy.float64'>'
with 106 stored elements in Compressed Sparse Row format>, 0)
Index: 8 (<1x9635 sparse matrix of type '<class 'numpy.float64'>'
with 271 stored elements in Compressed Sparse Row format>, 3)
Index: 9 (<1x9635 sparse matrix of type '<class 'numpy.float64'>'
with 143 stored elements in Compressed Sparse Row format>, 0)
I need to build an Adaptive Information Filter (AIF) class that uses its classifier self.clf to select news articles of interest and the corresponding topics, and yield them as a tuple (e.g., yield news_vecs, topics, where news_vecs is a 2D CSR sparse matrix of shape (N, 9635) and topics is an np.ndarray of shape (N,). And N is dynamically determined by the classifier.)
This is what I have so far my AIF class skeleton:
import scipy.sparse
import numpy as np
class AIF:
def __init__(self, random_state=None):
self.clf = None # Any classifier of my choice
self.random_state = 42 # in case you need one
def filtering(self, newsfeed, batch_sz=100):
# Some additional variables may be needed
# Step into the newsfeed
for idx, (news_vec, topic) in enumerate(newsfeed):
# Yield news_vecs and topics when a full batch has arrived
# For simplicity, you can ignore any incomplete batches after the newsfeed is exhausted
def gather_feedback(self, feedback, news_vecs, topics):
# feedback may be a degenerate np.ndarray if news_vecs and topics are degenerate
return # Feedback