Train deeplabv3+ with multi gpu?

Viewed 528

I tried to fine tune pre-trained model which is provided by tensorflow using 2 gpu So I set flags like below

python train.py --logtostderr
--train_split='train'
--model_variant='xception_65'
--atrous_rates=12
--atrous_rates=24
--atrous_rates=36
--output_stride=8
--decoder_output_stride=4
--train_crop_size=513
--train_crop_size=513
--train_batch_size=2
--training_number_of_steps=20000
--fine_tune_batch_norm=false
--tf_initial_checkpoint="/storage/models-master/research/deeplab/datasets/pascal_voc_seg/init_models/deeplabv3_pascal_train_aug/model.ckpt" --train_logdir="/storage/models-master/research/deeplab/datasets/pascal_voc_seg/exp/train_on_train_set/train/0831_2"
--dataset_dir="/storage/models-master/research/deeplab/datasets/pascal_voc_seg/tfrecord"
--image_pyramid=0.5
--image_pyramid=0.25
--image_pyramid=1.75
--num_replicas=1
--num_clones=2
--num_ps_tasks=1

But I got an error message like below

Cannot assign a device for operation 'parallel_read/filenames/Greater': Operation was explicitly assigned to /job:worker/device:CPU:0 but available devices are [ /job:localhost/replica:0/task:0/device:CPU:0, /job:localhost/replica:0/task:0/device:GPU:0, /job:localhost/replica:0/task:0/device:GPU:1 ]. Make sure the device specification refers to a valid device. [[Node: parallel_read/filenames/Greater = Greater[T=DT_INT32, _device="/job:worker/device:CPU:0"](parallel_read/filenames/Size, parallel_read/filenames/Greater/y)]]

It seems operations have been assigned '/job:worker/device:CPU:0' but my setting is 2 gpus, how can I this problem?

0 Answers
Related