Get 403 Error when trying to use CloudFront OAI for S3 bucket access

I have a S3 bucket which hosts my react website. I banned public access to the bucket, and configured an OAI to access the S3 bucket.

I can see that I have granted the OAI with IAM policy in the S3 bucket permission, and also the OAI ID matches:

{
   "Effect":"Allow",
   "Principal":{
      "AWS":"arn:aws:iam::cloudfront:user/CloudFront Origin Access Identity E2M5555555"
   },
   "Action":"s3:GetObject",
   "Resource":"arn:aws:s3:::dummywebsite.com-12345-us-east-1/*"
}

However, I still get 403 permission error from the cloudfront url. Any idea what may cause this?

Context

Here is my CDK code:

   // Amazon S3 bucket to host the store  website artifact
    const websiteBucket = new S3.Bucket(this, "dummywebsite", {
      bucketName: `${props.websiteDomain}-${props.env.account}-${props.env.region}`,
      websiteIndexDocument: "index.html",
      websiteErrorDocument: "error.html",
      removalPolicy: CDK.RemovalPolicy.DESTROY,
      autoDeleteObjects: true,

      accessControl: S3.BucketAccessControl.PRIVATE,
      encryption: S3.BucketEncryption.S3_MANAGED,
      publicReadAccess: false,
      blockPublicAccess: S3.BlockPublicAccess.BLOCK_ALL,
    });

    const hostedZone = Route53.HostedZone.fromLookup(this, "HostedZoneId", {
      domainName: props.websiteDomain,
    });

    const certificateManagerCertificate = new ACM.Certificate(
      this,
      "CertificateManagerCertificate",
      {
        domainName: props.websiteDomain,
        validation: ACM.CertificateValidation.fromDns(hostedZone),
      }
    );

    // Create a special CloudFront user called an origin access identity (OAI)
    // and associate it with the CloudFront distribution.
    const cloudFrontOAI = new CloudFront.OriginAccessIdentity(
      this,
      "dummyWebsiteOriginAccessIdentityID"
    );
    const cloudfrontUserAccessPolicy = new IAM.PolicyStatement();
    cloudfrontUserAccessPolicy.addActions("s3:GetObject");
    cloudfrontUserAccessPolicy.addPrincipals(cloudFrontOAI.grantPrincipal);
    cloudfrontUserAccessPolicy.addResources(websiteBucket.arnForObjects("*"));
    websiteBucket.addToResourcePolicy(cloudfrontUserAccessPolicy);

    const cloudFrontDistribution = new CloudFront.Distribution(
      this,
      "CloudFrontDistribution",
      {
        domainNames: [props.websiteDomain],
        defaultBehavior: {
          origin: new CloudFrontOrigins.S3Origin(websiteBucket, {
            // CloudFront can use the OAI to access the files in the S3 bucket and serve them to users.
            // Users can’t use a direct URL to the S3 bucket to access a file there.
            originAccessIdentity: cloudFrontOAI,
          }),
          compress: true,
          allowedMethods: CloudFront.AllowedMethods.ALLOW_GET_HEAD,
          cachedMethods: CloudFront.CachedMethods.CACHE_GET_HEAD,
          viewerProtocolPolicy:
            CloudFront.ViewerProtocolPolicy.REDIRECT_TO_HTTPS,
          cachePolicy: CloudFront.CachePolicy.CACHING_OPTIMIZED,
        },
        errorResponses: [
          {
            httpStatus: 403,
            responsePagePath: "/index.html",
            responseHttpStatus: 200,
            ttl: CDK.Duration.minutes(0),
          },
          {
            httpStatus: 404,
            responsePagePath: "/index.html",
            responseHttpStatus: 200,
            ttl: CDK.Duration.minutes(0),
          },
        ],
        priceClass: CloudFront.PriceClass.PRICE_CLASS_ALL,
        enabled: true,
        certificate: certificateManagerCertificate,
        minimumProtocolVersion: CloudFront.SecurityPolicyProtocol.TLS_V1_2_2021,
        httpVersion: CloudFront.HttpVersion.HTTP2,
        defaultRootObject: "index.html",
        enableIpv6: true,
      }
    );

    new Route53.ARecord(this, "Route53RecordSet", {
      recordName: props.websiteDomain,
      zone: hostedZone,
      target: Route53.RecordTarget.fromAlias(
        new Route53Targets.CloudFrontTarget(cloudFrontDistribution)
      ),
    });

Update

I figured out what went wrong, just not find out what kind of CDK code change I need to make yet.

The issue is same as mentioned here: https://aws.amazon.com/premiumsupport/knowledge-center/cloudfront-access-to-amazon-s3/.

"Important: Before you begin, be sure that the Amazon S3 origin of your CloudFront distribution is configured as a REST API endpoint (AWSDOC-EXAMPLE-BUCKET.s3.amazonaws.com). This resolution doesn't apply to S3 origins that are configured as a website endpoint (AWSDOC-EXAMPLE-BUCKET.s3-website-us-east-1.amazonaws.com)."

This is exactly the problem. Below code generate s3 website endpoint instead of S3 REST API endpoint.

origin: new CloudFrontOrigins.S3Origin(websiteBucket, {
          originAccessIdentity: cloudFrontOAI,
        }),

Follow the instructions in above link, I'm able to solve the problem manually.

I have not find out the CDK code solution yet. Looks like maybe a CDK bug: https://github.com/aws/aws-cdk/issues/19539.