We are running Flink jobs that consume data from Kafka, perform an SQL query (using the SQL API) and write the output to Kafka.
Our jobs are configured to run forever, so they should never be FINISHED (unless they are stopped with a savepoint). However, we have noticed that with some (invalid) SQL queries, the job can start but becomes FINISHED later on. That is the case when we include something like field <> NULL. The documentation of Flink says:
value1 <> value2 Returns TRUE if value1 is not equal to value2; returns UNKNOWN if value1 or value2 is NULL.
It is not clear to us what UNKNOWN is and how the job is expected to react when that happens. In any case, we would expect it to fail instead of finishing silently. Could it be a bug in Flink? Is it the expected behaviour?