I am looking at the relevant library for PySpark to get the schema registry from Kafka and decode the data. Does anyone know what is the code/library convert from scala to pyspark in scala-code?
2 Answers
You can use requests package to send requests to schema-registry restAPI and get the schema of your topic and also if you are listening to some specific topics you can cache the schema of them on spark and use them
Pyspark can import and use any JVM Spark class. Any Scala other Java examples you find, therefore should just work
Related