I have trained a network on deeplabcut for multianimal training on zebrafish dataset. The video contains 14 zebrafish. I extracted 10 frames. I trained the restnet-50 for 1000 iterations with batch size 8.
deeplabcut.train_network(path_config_file, shuffle=1, trainingsetindex=0, gputouse=None, autotune=False,maxiters= 1000)
When evaluating the network I get an nan values for fish13, fish3, fish8. I don't get why I am getting nan values.
Results for 1000 training iterations, training fraction of 95, and shuffle 1:
Train error: 10.07 pixels. Test error: 10.02 pixels.
With pcutoff of 0.6:
Train error: 3.16 pixels. Test error: 2.84 pixels.
##########################################
Average Euclidean distance to GT per individual (in pixels; test-only)
individuals
fish1 2.258812
fish10 1.943449
fish11 2.069003
fish12 3.125527
fish13 NaN
fish14 6.967174
fish2 0.161736
fish3 NaN
fish4 1.597071
fish5 2.447644
fish6 3.080594
fish7 3.711668
fish8 NaN
fish9 3.863770
Average Euclidean distance to GT per bodypart (in pixels; test-only)
bodyparts
head 2.838768
Done and results stored for snapshot: snapshot-1000
After evaluation I analyzed the video. During analyzing, I get a runtime warning and a OS error that tracklets are empty
deeplabcut.analyze_videos(path_config_file,[videofile_path], videotype='.mp4')
Video Analyzed. Saving results in D:\Projects\DeepLabCut-master\Track-ZebFish-2022-09-15\videos...
Using snapshot-1000 for model D:\Projects\DeepLabCut-master\Track-ZebFish-2022-09-15\dlc-models\iteration-0\TrackSep15-trainset95shuffle1
Processing... D:\Projects\DeepLabCut-master\Track-ZebFish-2022-09-15\videos\zebrafishtrain-multi.mp4
Analyzing D:\Projects\DeepLabCut-master\Track-ZebFish-2022-09-15\videos\zebrafishtrain-multiDLC_resnet50_TrackSep15shuffle1_1000.h5
0it [00:00, ?it/s]D:\Projects\DeepLabCut-master\DeepLabCut-master\deeplabcut\pose_estimation_tensorflow\lib\trackingutils.py:402: RuntimeWarning: invalid value encountered in double_scalars
r = w / float(h)
2000it [00:09, 215.54it/s]
The tracklets were created (i.e., under the hood deeplabcut.convert_detections2tracklets was run). Now you can 'refine_tracklets' in the GUI, or run 'deeplabcut.stitch_tracklets'.
Processing... D:\Projects\DeepLabCut-master\Track-ZebFish-2022-09-15\videos\zebrafishtrain-multi.mp4
OSError: Tracklets are empty.
Below is the config file:
# Project definitions (do not edit)
Task: Track
scorer: ZebFish
date: Sep15
multianimalproject: true
identity: false
# Project path (change when moving around)
project_path: D:\Projects\DeepLabCut-master\Track-ZebFish-2022-09-15
# Annotation data set configuration (and individual video cropping parameters)
video_sets:
D:\Projects\DeepLabCut-master\Track-ZebFish-2022-09-15\videos\zebrafishtrain-multi.mp4:
crop: 60, 428, 28, 242
individuals:
- fish1
- fish2
- fish3
- fish4
- fish5
- fish6
- fish7
- fish8
- fish9
- fish10
- fish11
- fish12
- fish13
- fish14
uniquebodyparts: []
multianimalbodyparts:
- head
bodyparts: MULTI!
# Fraction of video to start/stop when extracting frames for labeling/refinement
start: 0
stop: 1
numframes2pick: 10
# Plotting configuration
skeleton: []
skeleton_color: black
pcutoff: 0.6
dotsize: 12
alphavalue: 0.7
colormap: rainbow
# Training,Evaluation and Analysis configuration
TrainingFraction:
- 0.95
iteration: 0
default_net_type: resnet_50
default_augmenter: multi-animal-imgaug
default_track_method: box
snapshotindex: -1
batch_size: 8
# Cropping Parameters (for analysis and outlier frame detection)
cropping: false
#if cropping is true for analysis, then set the values here:
x1: 0
x2: 640
y1: 277
y2: 624
# Refinement configuration (parameters from annotation dataset configuration also relevant in this stage)
corner2move2:
- 50
- 50
move2corner: true
Here is the pose_cfg file:
all_joints:
- - 0
all_joints_names:
- head
alpha_r: 0.02
apply_prob: 0.5
batch_size: 8
contrast:
clahe: true
claheratio: 0.1
histeq: true
histeqratio: 0.1
convolution:
edge: false
emboss:
alpha:
- 0.0
- 1.0
strength:
- 0.5
- 1.5
embossratio: 0.1
sharpen: false
sharpenratio: 0.3
crop_sampling: hybrid
crop_size:
- 400
- 400
cropratio: 0.4
dataset: training-datasets\iteration-0\UnaugmentedDataSet_TrackSep15\Track_ZebFish95shuffle1.pickle
dataset_type: multi-animal-imgaug
decay_steps: 30000
display_iters: 10
global_scale: 0.8
init_weights: D:\Projects\DeepLabCut-master\DeepLabCut-master\deeplabcut\pose_estimation_tensorflow\models\pretrained\resnet_v1_50.ckpt
intermediate_supervision: false
intermediate_supervision_layer: 12
location_refinement: true
locref_huber_loss: true
locref_loss_weight: 0.05
locref_stdev: 7.2801
lr_init: 0.0005
max_input_size: 1500
max_shift: 0.4
metadataset: training-datasets\iteration-0\UnaugmentedDataSet_TrackSep15\Documentation_data-Track_95shuffle1.pickle
min_input_size: 64
mirror: false
multi_stage: false
multi_step:
- - 0.0001
- 7500
- - 5.0e-05
- 12000
- - 1.0e-05
- 200000
net_type: resnet_50
num_idchannel: 0
num_joints: 1
num_limbs: 0
optimizer: adam
pafwidth: 20
pairwise_huber_loss: false
pairwise_loss_weight: 0.1
pairwise_predict: false
partaffinityfield_graph: []
partaffinityfield_predict: false
pos_dist_thresh: 17
pre_resize: []
project_path: D:\Projects\DeepLabCut-master\Track-ZebFish-2022-09-15
rotation: 25
rotratio: 0.4
save_iters: 100
scale_jitter_lo: 0.5
scale_jitter_up: 1.25
weigh_only_present_joints: false