New row not being added to existing dataframe pandas

Viewed 16

I've faced a problem while working with some data and saving it to .csv file. To avoid using deprecated pd.append() method I wanted to use pd.concat() to add new data to my df, but it seems as nothing is being added.

I have a class which has some values and a function measure_cpu() which should measure and save data from class Server.

Class:

class Server:
  def __init__(self, port):
    self.port = port
    self.file_name = '_'.join(['cpu', str(n_of_servers), str(port), '.csv'])
    file_names.append(self.file_name)
    self.server_process = subprocess.Popen([server_path, '--cert', 'keys/ca-cert.pem', '--key', 'keys/server-key.pem', '--no_transcoding', '--port', str(port)],
      cwd = root_dir)
    self.publisher_process = subprocess.Popen([client_path, '--in_folder', path_to_assets, '--kpi', '--address', '192.168.65.30', '--port', str(port)])
    self.server_pid = self.server_process.pid
    self.publisher_pid = self.publisher_process.pid
    self.df = pd.DataFrame(columns=headerList)
    data_frames.append(self.df)
    processes.append(self.server_pid)
    processes.append(self.publisher_pid)

And function to measure some stuff where I pass class Server instance:

def measure_cpu(s):
  p = psutil.Process(s.server_pid)
  cpu_usage = p.cpu_times()
  mem_usage = p.memory_info()
  timestamp=time.time()
  df_new_row = pd.DataFrame.from_records({ 'timestamp': [timestamp], 'mem_usage.rss': [mem_usage.rss],
                'mem_usage.vms': [mem_usage.vms], 'mem_usage.shared': [mem_usage.shared], 'mem_usage.text': [mem_usage.text],
                'mem_usage.data': [mem_usage.data], 'user': [cpu_usage.user], 'system': [cpu_usage.system], 
                'children_user': [cpu_usage.children_user], 'children_system': [cpu_usage.children_system]})
  s.df = pd.concat([s.df, df_new_row])

And main function where Server instance is created:

this_server = Server(ports[i])
servers.append(this_server)
(...)
measure_cpu(servers[i])

My problem is - when I'm debugging, I can see that in df_new_row there is new data to be saved, but after using pd.concat() the initial s.df (from my class instance) is still showing as empty DataFrame.

Why it's not being concatenated, what am I doing wrong?

Thanks for any help.

0 Answers
Related