Relation of Executors and Bolts in Storm Stream API

user_13847890 0
Viewed 52

I have a complex storm query that is written in Storms Stream API and consists out of multiple operations, like map, filter, window and so on.

When I run this topology on my remote cluster, I am just able to see that several executors are executed on my machines, but in the Storm UI there appearing far less bolts than operators (=bolts) I have implemented before. I set the task parallelism to 1, also the num of tasks per operator to 1.

Example: For a query running with a source, 2 x map, a sink, there is just one spout and one bolt displayed.

So my questions are:

Is there any chaining / co-locating of operators happening in storm? How could I sepereate each operator from the Streaming API to one worker machine?

apache-storm stream-processing distributed-computing java
1 Answers

I recommend checking the log files of operators. Print something in each bolt and spout and see how they react in the logs. Besides, there are some tags in storm conf like topology.ras.one.component.per.worker: true, topology.ras.one.executor.per.worker: true and topology.ras.acker.executors.per.worker: 1 that may work for you.

user_51693450
Related