I want to send streaming data from a socket using python to Spark Structured Streaming locally. When I tried this with Netcat It worked. but It doesn't work when I make a Server Socket instead of Netcat.

Basic Info

• I am working on mac os.
• spark version is 3.1.2
• I coded in two different jupyter notebook pages in the same machine.
  (1) Server Socket (2) Spark Structured Steaming
• I made a server socket with python.

Here is what I did and response.

• Turned a server socket on.
  
• Launched Spark Structured Streaming
  -> Connection is established in this stage.
  
• Send a text to Spark Structured Streaming.
  -> There is not any response on the cmd.
  
• Turned the server socket off
  -> Spark Structured Streaming shows me the result of what I sent from the Server Socket.

Code

Server Socket

import socket
server = socket.socket()
host = '0.0.0.0'
port = 9999
server.bind((host, port))
server.listen(2)
client_socket, addr = server.accept()
print("connection established.")

# Sending data
client_socket.sendall("Text".encode())

Spark Structured Streaming

from pyspark.sql import SparkSession
from pyspark.sql.functions import explode
from pyspark.sql.functions import split

spark = SparkSession \
    .builder \
    .appName("StructuredNetworkWordCount") \
    .getOrCreate()

# Create DataFrame representing the stream of input lines from connection to localhost:9999
lines = spark \
    .readStream \
    .format("socket") \
    .option("host", "0.0.0.0") \
    .option("port", 9999) \
    .load()

# Split the lines into words
words = lines.select(
   explode(
       split(lines.value, " ")
   ).alias("word")
)

# Generate running word count
wordCounts = words.groupBy("word").count()

 # Start running the query that prints the running counts to the console
query = wordCounts \
    .writeStream \
    .outputMode("complete") \
    .format("console") \
    .start()
query.awaitTermination()