I am running glueetl (Glue version 2.0) job using Python with the below configuration for logging. I have continuous logging enabled.
I get INFO entries from the driver present in /aws-glue/jobs/output log group, however there are no INFO entries present for executors, neither in /aws-glue/jobs/output nor in /aws-glue/jobs/logs-v2 log groups.
I ran experiment where setting log level to WARN and logging WARN statements made the WARN entries be present, but in /aws-glue/jobs/error log group.
import sys
import logging
MSG_FORMAT = '%(asctime)s %(levelname)s %(name)s: %(message)s'
DATETIME_FORMAT = '%Y-%m-%d %H:%M:%S'
logging.basicConfig(format=MSG_FORMAT, datefmt=DATETIME_FORMAT)
logger = logging.getLogger(getResolvedOptions(sys.argv, ["JOB_NAME"])["JOB_NAME"])
logger.setLevel(logging.INFO)