Golang GORM invalid association

I'm trying to write a very simple belongsTo association with GORM, but with primary keys that isn't Id.

My structs are as such:

type State struct {
    FIPS string `gorm:"type:char(2);primary_key;column:FIPS"`
    Name string `gorm:"not null"`
    Area float64 `gorm:"type:real;not null"`
}

type ZipCode struct {
    ZipCode   string `gorm:"type:char(5);primary_key;"`
    Name      string `gorm:"not null"`
    State     State `gorm:"ForeignKey:StateFIPS;AssociationForeignKey:FIPS"`
    StateFIPS string `gorm:"type:char(2);column:state_FIPS;not null"`
}

and with the following code:

var zc ZipCode
var s State
db.Model(&zc).Related(&s)

I get the error: [2017-05-18 14:26:13] invalid association [] and a find on the zipcode doesn't load the state. Does GORM not like non-Id primary keys or am I missing something?