Spark history logs decompress manually

user_2786156 0
Viewed 1577

I have some problems with spark history server and therefore tried to uncompress lz4 logs manually. I installed liblz4-tool and downloaded application log application_1510049252849_0303.lz4. But when uncompressing it with lz4 ~/application_1510049252849_0303.lz4 I got the following error:

Error 44 : Unrecognized header : file cannot be decoded

And this I have for every spark history lz4 files. What is the solution? Is there a way to open it manually?

lz4 apache-spark scala java
2 Answers

lz4 no need to decompress while placing into spark history server. Just you can place your event logs into event log directory. It will automatically pick that application and show it in the Spark History Server UI.

user_19189620
Related