String Slice Syntax on PySpark Column Object Behaves Like Substring?

PySpark 3.1.2

I am using input_file_name() to add a column with partition information to my DataFrame. More specifically, I'm parsing the return value (a Column object) to extract a substring of the file path. Like so:

from pyspark.sql.functions import input_file_name
df = spark.read.parquet(*path_list).withColumn("partition", input_file_name()[81:13])

The part of the file path I'm interested in is a yyyy_mm_dd_hh value. If the return type of input_file_name() were a string, I would extract it with standard Python string slice syntax: returned_column[80:93]. Experimentation revealed that I can extract this data via input_file_name()[81:13], effectively the same as calling substring thus: substring(returned_string, 81, 13).

So, there are two behaviors I found unexpected in this situation:

1. Indices start at 1 in PySpark (rather than the 0 used in most of Python)
2. PySpark (or at least the input_file_name() method) treats slice syntax as equivalent to the substring(str, pos, len) method, rather than the more conventional [start:stop].

Perusing the source code of Column, it looks like this might be why the slice syntax works this way on Column objects:

def __getitem__(self, k):
    if isinstance(k, slice):
        if k.step is not None:
            raise ValueError("slice with step is not supported.")
        return self.substr(k.start, k.stop)
    else:
        return _bin_op("apply")(self, k)

I am an analyst rather than a developer, so I may be out of my depth here. Could someone more knowledgeable:

• Confirm whether I've correctly understood the index and slice syntax behavior in PySpark?
• Provide any background on why these design decisions were taken, despite being seemingly at odds with Python convention?

Thank you very much!