Angular - date.clone is not a function at DatePickerAdapter.clone happens only when "First time of compiling"

I got a tricky situation.

I'm using Angular Material Datepicker with material-moment-adapter, and I ALREADY set the type of data to be "moment"

mode.component.ts

import moment from 'moment';

export class TimesheetDayModeComponent {
  chosenDate = new FormControl(moment());
}

And in the template

mode.component.html

<input matInput [formControl]="chosenDate" [matDatepicker]="picker" />
<mat-datepicker #picker></mat-datepicker>

app.module.ts

import { DateAdapter } from '@angular/material/core';
import { MatDatepickerModule } from '@angular/material/datepicker';
import { MatMomentDateModule } from "@angular/material-moment-adapter";
import { MatNativeDateModule } from '@angular/material/core';
import { MomentDateAdapter } from '@angular/material-moment-adapter';

@NgModule({
    ...
    imports: [
        MatDatepickerModule,
        MatNativeDateModule,
        MatMomentDateModule,
    ],
    providers: [
        {
          provide: DateAdapter,
          useClass: MomentDateAdapter
        },
    ]

}

But the error "ERROR TypeError: date.clone is not a function at DatePickerAdapter.clone" still happened, and it only happened when the code is compiled for the first time.

Happend when first time

But if I recompile the code without changing anything in the code, the error is gone.

After recompiling

Has anyone encountered the same situation before?

And my version of the packages

"dependencies": {
    "@agm/core": "^3.0.0-beta.0",
    "@angular-slider/ngx-slider": "^2.0.3",
    "@angular/animations": "^11.2.10",
    "@angular/cdk": "^11.0.0",
    "@angular/common": "^11.2.10",
    "@angular/compiler": "^11.2.10",
    "@angular/core": "^11.2.10",
    "@angular/flex-layout": "^11.0.0-beta.33",
    "@angular/forms": "^11.2.10",
    "@angular/material": "^11.0.0",
    "@angular/material-moment-adapter": "^11.0.0",
    "@angular/platform-browser": "^11.2.10",
    "@angular/platform-browser-dynamic": "^11.2.10",
    "@angular/router": "^11.2.10",
    "core-js": "^3.0.0",
    "file-saver": "^2.0.5",
    "hammerjs": "^2.0.8",
    "lodash": "^4.17.21",
    "moment": "^2.29.1",
    "moment-timezone": "^0.5.33",
    "ngx-bar-rating": "^2.0.0",
    "ngx-color": "^6.0.0",
    "ngx-image-cropper": "^4.0.1",
    "ngx-input-file": "^10.0.0",
    "ngx-mask": "^12.0.0",
    "ngx-toastr": "^13.0.0",
    "ngx-ui-switch": "^10.0.0",
    "numbro": "^2.3.6",
    "rxjs": "^6.6.7",
    "tslib": "^2.0.0",
    "uuid": "^8.3.2",
    "zone.js": "^0.11.0"
  },

Any advice is appreciated. Thanks.