Currently it looks like boto3 has an upper limit of 12 hours as per here. Is there a way you can increase the S3 session duration for more than 12 hours?
After 12 hours any call to show data on spark dataframe lands into this exception: Error Code: 403 Forbidden; Request ID: tx0000055c5a217ed148183-0063170ae4-189e9643-de-bs01; S3 Extended Request ID: null), S3 Extended Request ID
sdf.show(3, False)
22/09/06 08:55:00 WARN TaskSetManager: Lost task 3.0 in stage 464.0 (TID 77378) (100.66.45.57 executor 128): java.io.IOException: com.amazonaws.services.s3.model.AmazonS3Exception: Forbidden (Service: Amazon S3; Status Code: 403; Error Code: 403 Forbidden; Request ID: tx0000055c5a217ed148183-0063170ae4-189e9643-de-bs01; S3 Extended Request ID: null), S3 Extended Request ID: null at com.ibm.stocator.fs.cos.COSAPIClient.getFileStatus(COSAPIClient.java:540) at com.ibm.stocator.fs.ObjectStoreFileSystem.getFileStatus(ObjectStoreFileSystem.java:513) at org.apache.parquet.hadoop.util.HadoopInputFile.fromPath(HadoopInputFile.java:39) at org.apache.parquet.hadoop.ParquetFileReader.readFooter(ParquetFileReader.java:448) at org.apache.spark.sql.execution.datasources.parquet.ParquetFileFormat.footerFileMetaData$lzycompute$1(ParquetFileFormat.scala:271)