Spark create dataframe ERROR: IntegerType can not accept object nan in type <class 'float'>

I have an RDD I am transforming to a DF with a specified schema (inferring won't work)

                scoring_df = scoring_rdd.toDF(scoring_schema)

However when I attempt to convert I get the error TypeError field dip: IntegerType can not accept object nan in type <class 'float'>

The field dip gets assigned to either np.NaN or and integer based on a difference of two other integers. It seems that when dip is np.NaN this is causing an issue, although I have no idea why because the following is what I specified in the schema for dip

        {
            "name": "dip",
            "type": "integer",
            "nullable": true,
            "metadata": {}
        },

DIP is a nullable integer structtype, which I thought means it should allow np.NaN values. Anyone have an idea as to what could be the issue.

I am running the code on an amazon AWS EMR with spark version Spark 2.4.8