AWS VPC CIDR in both 10.0.0.0/8 and 192.168.0.0/16

user_1637003 0
Viewed 488

I have VPC A with CIDR 10.A.0.0/16 and VPC B with CIDR 10.B.0.0/16. I have VPC A and B peered and updated the route tables and from a server in 10.B.0.0/16 can ping a server in 10.A.0.0/16 and vice versa.

The applications on VPC A also use some IPs in the 192.168.0.0/16 range. Not something I can easily change, but I need to be able to reach 192.168.0.0/16 on VPC A from VPC B.

I've tried adding 192.168.0.0/16 to the route table used for VPC B and setting the target of the peered connection. That does not work, I believe because 192.168.0.0/16 is not in the CIDR block for VPC A.

I'm unable to add 192.168.0.0/16 as a secondary CIDR in VPC A because it is restricted. See CIDR block association restrictions and related question. I understand it is restricted, but why is it restricted? RFC1918 doesn't seem to say anything against using more than one of the private address spaces.

I've also tried making a Transit Gateway, attaching both VPCs, and adding a static route to the Transit Gateway Route Table for 192.168.0.0/16 that targets the VPC A attachment. But still cannot reach that range from within VPC B.

Is there another way to peer to both 10.0.0.0/8 and 192.168.0.0/16 CIDR blocks on the same VPC?

Updated, background info

The VPCs are used by two different kubernetes clusters. The older one uses project-calico that uses the default cluster CIDR 192.168.0.0/16 and pod IPs get assigned in that range. The newer one is an EKS cluster and pod IPs are assigned from the VPC's CIDR range. During the transition period I've got the two clusters' VPCs peered together.

Route Table

The route table for the private subnet for VPC A

10.A.0.0/16    local
10.B.0.0/16    pcx-[VPC A - VPC B peering connection]
0.0.0.0/0      nat-[gateway for cluster A]

Route table for the private subnet for VPC B

10.B.0.0/16    local
10.A.0.0/16    pcx-[VPC A - VPC B peering connection]
192.168.0.0/16 pcx-[VPC A - VPC B peering connection]
0.0.0.0/0      nat-[gateway for cluster B]

This does not work, of course, because 192.168.0.0/16 is not in VPC A's CIDR block, nor can it be added.

amazon-vpc kubernetes cidr amazon-web-services
1 Answers

Calico creates an overlay network using the specified cluster CIDR (192.168.x.x) on top of VPC (A) CIDR, so pods/services in this k8s cluster can communicate. The overlay network routing information is neither expose nor usable for AWS route table. This is different from k8s cluster running in VPC (B) which uses VPC CNI that leverage on the VPC CIDR as the cluster CIDR.

Calico BGP Peering offers a way here but it is not going to be an easy route for this case.

Calico nodes can exchange routing information over BGP to enable reachability for Calico networked workloads (Kubernetes pods or OpenStack VMs).

If you must achieve pod to pod communication in different k8s clusters and networks but not via Ingress/LB, migrate one of the k8s cluster CNI to be the same as the other so you can fully leverage on their unique peering capabilities.

user_147047990
Related