Is it possible to delete the aws Security Group Default Egress allow all Rule with Cloudformation?

The CloudFormation documentation has a dedicated section for that:

• Remove Default Rule:

When you specify a VPC security group, Amazon EC2 creates a default egress rule that allows egress traffic on all ports and IP protocols to any location. The default rule is removed only when you specify one or more egress rules. If you want to remove the default rule and limit egress traffic to just the localhost (127.0.0.1/32), use the following example.

sgwithoutegress:
  Type: AWS::EC2::SecurityGroup
  Properties:
    GroupDescription: Limits security group egress traffic
    SecurityGroupEgress:
    - CidrIp: 127.0.0.1/32
      IpProtocol: "-1"
    VpcId:
      Ref: myVPC

Not that this does not remove physically the egress rule from the security group. This will lead to the following egress rule being created:

Instead of the default one: