Long-term Recurrent Convolutional Networks paper reproduction error in LSTM?

user_2527680 0
Viewed 638

I am trying to reproduce the Long-term Recurrent Convolutional Networks paper.

I have used their given code. And followed their instructions and generated the single frame model. But when trying to train the LSTM hybrid network it fails. I have already made necessary changes as mentioned in the instructions.

The command i run is the caffe train -solver lstm_solver_flow.prototxt -weights singleframe_flow/snaps/snapshots_singleFrame_flow_v2_iter_50000.caffemodel the output I get is 

I0323 18:16:30.685951  9123 net.cpp:205] This network produces output loss
I0323 18:16:30.685967  9123 net.cpp:446] Collecting Learning Rate and Weight Decay.
I0323 18:16:30.685976  9123 net.cpp:218] Network initialization done.
I0323 18:16:30.685982  9123 net.cpp:219] Memory required for data: 817327112
I0323 18:16:30.686339  9123 solver.cpp:42] Solver scaffolding done.
I0323 18:16:30.686388  9123 caffe.cpp:86] Finetuning from singleframe_flow/snaps/snapshots_singleFrame_flow_v2_iter_50000.caffemodel
I0323 18:16:33.377488  9123 solver.cpp:247] Solving lstm_joints
I0323 18:16:33.377518  9123 solver.cpp:248] Learning Rate Policy: step
I0323 18:16:33.391726  9123 solver.cpp:291] Iteration 0, Testing net (#0)
Traceback (most recent call last):
  File "/home/anilil/projects/lstm/lisa-caffe-public/examples/LRCN_activity_recognition/sequence_input_layer.py", line 220, in forward
    new_result_data = [None]*len(self.batch_advancer.result['data']) 
KeyError: 'data'
terminate called after throwing an instance of 'boost::python::error_already_set'
*** Aborted at 1458753393 (unix time) try "date -d @1458753393" if you are using GNU date ***
PC: @     0x7f243731bcc9 (unknown)
*** SIGABRT (@0x23a3) received by PID 9123 (TID 0x7f24389077c0) from PID 9123; stack trace: ***
    @     0x7f243731bd40 (unknown)
    @     0x7f243731bcc9 (unknown)
    @     0x7f243731f0d8 (unknown)
    @     0x7f2437920535 (unknown)
    @     0x7f243791e6d6 (unknown)
    @     0x7f243791e703 (unknown)
    @     0x7f243791e976 (unknown)
    @     0x7f2397bb5bfd caffe::PythonLayer<>::Forward_cpu()
    @     0x7f243821d87f caffe::Net<>::ForwardFromTo()
    @     0x7f243821dca7 caffe::Net<>::ForwardPrefilled()
    @     0x7f243822fd77 caffe::Solver<>::Test()
    @     0x7f2438230636 caffe::Solver<>::TestAll()
    @     0x7f243823837b caffe::Solver<>::Step()
    @     0x7f2438238d5f caffe::Solver<>::Solve()
    @           0x4071c8 train()
    @           0x405701 main
    @     0x7f2437306ec5 (unknown)
    @           0x405cad (unknown)
    @                0x0 (unknown)
run_lstm_flow.sh: line 8:  9123 Aborted                 (core dumped) GLOG_logtostderr=1 $TOOLS/caffe train -solver lstm_solver_flow.prototxt -weights singleframe_flow/snaps/snapshots_singleFrame_flow_v2_iter_50000.caffemodel
Done.

This is my changed sequence_input_layer.py and prototext files. My input train and test txts to the network is of this format.

I think the main problem is the ##rearrange the data: The LSTM takes inputs as [video0_frame0, video1_frame0,...] but the data is currently arranged as [video0_frame0, video0_frame1, ...]

I was not able to solve this is confused me quite a bit. But I might be wrong.

caffe machine-learning python
2 Answers

I know I am a bit late, hope my answer helps someone in future. I too faced the same error. The problem was with scikit-image version. I was using some version newer than scikit-image 0.9.3. Following line

processed_image = transformer.preprocess('data_in',data_in)

uses scikit-image library functions being imported by lisa-caffe-public-lstm_video_deploy/python/caffe/io.py. With newer version, transformer.preprocess() was crashing, and therefore processImageCrop() function didnot return and thus KeyError: 'data'.

In short, you should use scikit-image-0.9.3 to get rid of your error :) Hope that helps :)

user_60260510
Related