How to validate ksql script?

user_11908311 0
Viewed 205

I would like to know if there is a way for checking if a .ksql script is syntactically correct?

I know that you can send a POST request to the server, this however would also execute the containing ksql commands. I would love to have some kind of endpoint where you can pass your statement and it returns you either an error code or an OK like:

curl -XPOST <ksqldb-server>/ksql/validate -d '{"ksql": "<ksql-statement>"}' .

My question aims for a way to check the synatx in an automated fashion without the need to cleaning up everything afterwards.

Thanks for you help!

Note: I am also well aware that I could run everything separately using, e.g., a docker-compose file and tear everything down again. This however is quite resource heavy and and harder to maintain.

ksqldb apache-kafka
2 Answers
user_75326270

By now I've found a way to test for my use case. I had a ksqldb cluster already in place with all other systems needed for the Kafka ecosystem (Zookeeper, Broker,...). I had to compromise a little but and go through the effort of deploying everything but here is my approach:

  1. Use proper naming (let it be prefixed with test or whatever suits your use case) for your streams, tables,... the queries' sink property should include the prefixed topic in order to find it easily, or you simply assign an QUERY_ID (https://github.com/confluentinc/ksql/pull/6553).
  2. Deploy the streams, tables,... to your ksqldb server using its rest API. Since I was programming in Python, I made use of the ksql package using pip (https://github.com/bryanyang0528/ksql-python).
  3. Cleanup the ksqldb server by filtering for the naming that you assigned to the ksql resources and run the corresponding DROP or TERMINATE statement. Consider also, you will have dependencies that result in multiple streams/tables reusing a topics. The statements can be looked up in the official developer guide (https://docs.ksqldb.io/en/latest/developer-guide/ksqldb-reference/quick-reference/).
  4. If you had errors in step 2, step 3 should have cleaned up the leftovers so that you can adjust your ksql scripts until they run through smoothly.

Note: I could not make any assumptions on how the streams,... look like. If you can, I would prefer the ksql-test-runner.

user_119083110
Related