Convert size notation with units (100kb, 32MB) to number of bytes in Python

user_7719864 0
Viewed 2633

I need to convert a human readable notation of the upload size limit (e.g. 100kb,32MB, etc.) using Python. The converted value should be expressed as a number of bytes.

Example

convert_size_to_bytes("32MB") # should return 33554432
convert_size_to_bytes("100kB") # should return 102400
convert_size_to_bytes("123B") # should return 123
convert_size_to_bytes("123") # should return 123
type-conversion python
3 Answers

I liked the simple approach taken by @Robson, but I noticed it has a number of corner cases that it didn't handle. The version below is largely the same, but it adds/fixes the following:

  • Adds support for byte, kilobyte, etc.
  • Provides support for singular units, e.g., "1 byte"
  • Adds support for yottabytes, zetabytes, etc. The version above would crash for these.
  • Adds support for spaces before, after, and in the input.
  • Adds support for floats ("5.2 mb")
  • Bigger docstring

Here's the code:

def convert_size_to_bytes(size_str):
    """Convert human filesizes to bytes.

    Special cases:
     - singular units, e.g., "1 byte"
     - byte vs b
     - yottabytes, zetabytes, etc.
     - with & without spaces between & around units.
     - floats ("5.2 mb")

    To reverse this, see hurry.filesize or the Django filesizeformat template
    filter.

    :param size_str: A human-readable string representing a file size, e.g.,
    "22 megabytes".
    :return: The number of bytes represented by the string.
    """
    multipliers = {
        'kilobyte':  1024,
        'megabyte':  1024 ** 2,
        'gigabyte':  1024 ** 3,
        'terabyte':  1024 ** 4,
        'petabyte':  1024 ** 5,
        'exabyte':   1024 ** 6,
        'zetabyte':  1024 ** 7,
        'yottabyte': 1024 ** 8,
        'kb': 1024,
        'mb': 1024**2,
        'gb': 1024**3,
        'tb': 1024**4,
        'pb': 1024**5,
        'eb': 1024**6,
        'zb': 1024**7,
        'yb': 1024**8,
    }

    for suffix in multipliers:
        size_str = size_str.lower().strip().strip('s')
        if size_str.lower().endswith(suffix):
            return int(float(size_str[0:-len(suffix)]) * multipliers[suffix])
    else:
        if size_str.endswith('b'):
            size_str = size_str[0:-1]
        elif size_str.endswith('byte'):
            size_str = size_str[0:-4]
    return int(size_str)

And I wrote a bunch of tests for values that we're scraping:

class TestFilesizeConversions(TestCase):

    def test_filesize_conversions(self):
        """Can we convert human filesizes to bytes?"""
        qa_pairs = [
            ('58 kb', 59392),
            ('117 kb', 119808),
            ('117kb', 119808),
            ('1 byte', 1),
            ('117 bytes', 117),
            ('117  bytes', 117),
            ('  117 bytes  ', 117),
            ('117b', 117),
            ('117bytes', 117),
            ('1 kilobyte', 1024),
            ('117 kilobytes', 119808),
            ('0.7 mb', 734003),
            ('1mb', 1048576),
            ('5.2 mb', 5452595),
        ]
        for qa in qa_pairs:
            print("Converting '%s' to bytes..." % qa[0], end='')
            self.assertEqual(convert_size_to_bytes(qa[0]), qa[1])
            print('✓')
user_649110
Related