GORM model foreign key to different Postgres schema

I have 2 separate Go APIs that connect to the same Postgres database into different schemas. Each user has all access to all schemas.

• In the first API I have the User model with schema1.Users table
• In the 2nd I have the Task model with schema2.Tasks table

type User struct {
 gorm.Model
}

type Task struct {
 gorm.Model
 CreatedBy uint
 User User `gorm:"foreignKey:CreatedBy;references:User"`
}

When I try to run the 2nd API, the migrations create a Users table in schema2 and connects the foreign key there.

If I use the line below I get an error

 User User `gorm:"foreignKey:CreatedBy;references:schema1.User"`

[error] invalid field found for struct github.com/xyz/models.Tasks's field User: define a valid foreign key for relations or implement the Valuer/Scanner interface

Any ideas how to connect the different schemas?