How can I register a protobuf schema with references in other packages in Kafka schema registry?

Question

I'm running Kafka schema registry version 5.5.2, and trying to register a schema that contains a reference to another schema. I managed to do this when the referenced schema was in the same package with the referencing schema, with this curl command:

curl -X POST -H  "Content-Type: application/vnd.schemaregistry.v1+json" \
--data '{"schemaType":"PROTOBUF","references": [{"name": "other.proto","subject": "other.proto","version": 1}],"schema":"syntax = \"proto3\"; package com.acme; import \"other.proto\";

message MyRecord {
  string f1 = 1;
  OtherRecord f2 = 2;
 }
"}' \
http://localhost:8081/subjects/test-schema/versions

However, when I changed the package name of the referred schema, like this:

syntax = "proto3";
package some.other.package;

message OtherRecord {
  int32 other_id = 1;
}

I got {"error_code":42201,"message":"Either the input schema or one its references is invalid"} when I tried to register the referring schema, no matter what I put under the references name/subject. That's one of my trials:

curl -X POST -H  "Content-Type: application/vnd.schemaregistry.v1+json" \
--data '{"schemaType":"PROTOBUF","references": [{"name": "other.proto","subject": "other.proto","version": 1}],"schema":"syntax = \"proto3\"; package com.acme; import \"some.package.other.proto\";

message MyRecord {
  string f1 = 1;
  OtherRecord f2 = 2;
 }
"}' \
http://localhost:8081/subjects/test-shcema/versions

user_5334094 · Answer

First you should registrer your other proto to the schema registry.

Create a json (named other-proto.json) file with following syntax:

{
  "schemaType": "PROTOBUF",
  "schema": "syntax = \"proto3\";
package com.acme;

message OtherRecord {
  int32 other_id = 1;
}
"
}

Then

curl -XPOST -H 'Content-Type:application/vnd.schemaregistry.v1+json' http://localhost:8081/subjects/other.proto/versions --data @other-proto.json

Now your schema registry know this schema as the reference of subject other.proto. If you create another json file (named testproto-value.json) as

{
  "schemaType": "PROTOBUF",
  "references": [
    {
      "name": "other.proto",
      "subject": "other.proto",
      "version": 1
    }
  ],
  "schema": "syntax = \"proto3\";
package com.acme;

import \"other.proto\";

message MyRecord {
  string f1 = 1;
  .com.acme.OtherRecord f2 = 2;
}
"
}

And post it to schema registry:

curl -XPOST -H 'Content-Type:application/vnd.schemaregistry.v1+json' http://localhost:8081/subjects/testproto-value/versions --data @testproto-value.json

It will fix your issue. Because the reference in testproto-value.json is known by the registry into other.proto subject.

How can I register a protobuf schema with references in other packages in Kafka schema registry?

1 Answers