Value Error:could not convert string to float: b'*********'

Question

I am running the following the following python codes: It has been working well until today

import pandas as pd
import dbfread

data = dbfread.DBF(data.DBF, load = True)
df = pd.DataFrame(data)

I got the error message below: Not sure what's wrong?

    ValueError                                Traceback (most recent call last)
    ~\Anaconda3\lib\site-packages\dbfread\field_parser.py in parseN(self, field, data)
        157         try:
    --> 158             return int(data)
        159         except ValueError:
    
    ValueError: invalid literal for int() with base 10: b'*********'
    
    During handling of the above exception, another exception occurred:
    
    ValueError                                Traceback (most recent call last)
     in 
    ----> 1 data = dbfread.DBF('LNMSTR.DBF', load = True)
          2 df = pd.DataFrame(data)
    
    ~\Anaconda3\lib\site-packages\dbfread\dbf.py in __init__(self, filename, encoding, ignorecase, lowernames, parserclass, recfactory, load, raw, ignore_missing_memofile)
        131 
        132         if load:
    --> 133             self.load()
        134 
        135     @property
    
    ~\Anaconda3\lib\site-packages\dbfread\dbf.py in load(self)
        166         """
        167         if not self.loaded:
    --> 168             self._records = list(self._iter_records(b' '))
        169             self._deleted = list(self._iter_records(b'*'))
        170 
    
    ~\Anaconda3\lib\site-packages\dbfread\dbf.py in _iter_records(self, record_type)
        306                                  for field in self.fields]
        307                     else:
    --> 308                         items = [(field.name,
        309                                   parse(field, read(field.length))) \
        310                                  for field in self.fields]
    
    ~\Anaconda3\lib\site-packages\dbfread\dbf.py in (.0)
        307                     else:
        308                         items = [(field.name,
    --> 309                                   parse(field, read(field.length))) \
        310                                  for field in self.fields]
        311 
    
    ~\Anaconda3\lib\site-packages\dbfread\field_parser.py in parse(self, field, data)
         73             raise ValueError('Unknown field type: {!r}'.format(field.type))
         74         else:
    ---> 75             return func(field, data)
         76 
         77     def parse0(self, field, data):
    
    ~\Anaconda3\lib\site-packages\dbfread\field_parser.py in parseN(self, field, data)
        162             else:
        163                 # Account for , in numeric fields
    --> 164                 return float(data.replace(b',', b'.'))
        165 
        166     def parseO(self, field, data):
    
    ValueError: could not convert string to float: b'*********'

i would appreciate it if anyone can help me understand this and fix. Is it the problem from my DBF database or is it from my code?

update: I can run the same codes at home's computer without errors. But it couldn't go thru at work's computer. Do you think it might be something else unrelated to the database itself? It's weird.

user_208880 · Answer

Check out How do I convert a .dbf file into a Pandas DataFrame? for a function to convert dbf tables to Pandas DataFrames.

To answer your question, the bytes stored for that column of that record is a bunch of asterisks (*), which are obviously not numbers. Some programs, such as FoxPro, will store asterisks when unable to store the actual value.

My dbf library will return None in that circumstance.

user_6495889 · Answer

From the documentation, since you are importing data using the pandas.DataFrame, the docs show to me that you are using the "DataType, inference" parameter. This would mean that the data you are importing is likely being read as an integer when you are passing in another data point. If the data-set you are reading from has mixed data type (i.e. some of the data you are importing is int and some float, it might cause an issue).

At the bottom of your error, you will see ValueError: could not convert string to float: b'*********', which is likely because you are trying to import a string in place of a float (in this case, it looks literally like the string of multiple asterisks: which could be a column-header delimiter). Maybe remove this from the data string before you attempt to load it into the DataFrame, and see if it works (check this by printing out the data variable before you load it into the DataFrame function).

You can read more about it here: https://pandas.pydata.org/pandas-docs/stable/reference/api/pandas.DataFrame.dtypes.html#pandas.DataFrame.dtypes

Value Error:could not convert string to float: b'*********'

2 Answers