JPA @ManyToOne joining on part of composite primary key

I have two tables that I want to join. Lets say TableA and TableB.

TableA

@Entity
@Table(name = "TableA")
@Data
@Builder
@AllArgsConstructor
@NoArgsConstructor
public class TableA {

    @Id
    @EmbeddedId
    private TableA.PrimaryId id = new TableA.PrimaryId();

    @Column(name = "COL1")
    private Integer col1;   
   
    @ManyToOne
    @JoinColumn(name = "ID1")

    private TableB tableB;

    @Data
    @Embeddable
    @Builder
    @AllArgsConstructor
    @NoArgsConstructor
    public static class PrimaryId implements Serializable {
        @Column(name = "ID1")
        private Integer id1;

        @Column(name = "ID2")
        private Integer id2;
    }
}

TableB

@Entity
@Table(name = "TableB")
@Data
@Builder
@AllArgsConstructor
@NoArgsConstructor
public class TableB {
    @Id
    @EmbeddedId
    private PrimaryId id = new PrimaryId();

    @Column(name = "COLUMN1")
    private String column1;

    @Data
    @Embeddable
    @Builder
    @AllArgsConstructor
    @NoArgsConstructor
    public static class PrimaryId implements Serializable {
        @Column(name = "ID1")
        private Integer id1;

        @Column(name = "KEY2")
        private Integer key2;

        @Column(name = "KEY3")
        private Integer key3;

        @Column(name = "KEY4")
        private Long key4;
    }
}

The problem is that TableB has composite primary key (4 columns), but I need to do join only on 1 column (ID1). It isnt a standart join, it is a partial primary key join.

So it causes an error:

nested exception is org.hibernate.AnnotationException: A Foreign key refering TableB from TableA has the wrong number of column. should be 4

If I try this

@ManyToOne
@JoinColumn(name = "ID1", referencedColumnName = "ID1", updatable=false, insertable=false)

I get another error

nested exception is org.hibernate.AnnotationException: referencedColumnNames(ID1) of TableA.TableB referencing TableB not mapped to a single property

I've looked for an answer in similar posts but found nothing useful :(

So I appreciate any help...