What would be the function of such a sequence folding layer and how would the architecture need to be changed?
Using sequenceFolding/sequenceUnFoldingLayer works as follows experimentally, the results in the red box in the figure are consistent.
%% Testing the sequenceFoldingLayer/sequenceUnfoldingLayer internal flow
% The difference between using the sequence layer and not using the sequenceFolding/sequenceUnFoldingLayer to extract features
% Common parameters used below
%
% 测试sequenceFoldingLayer/sequenceUnfoldingLayer内部具体操作流程
% 用和不用sequence layer的区别,使用sequenceFolding/sequenceUnFoldingLayer提取出特征结果比较
% 下面使用的共有参数
inputSize = [28 28 1];
filterSize = 5;
numFilters = 20;
numHiddenUnits = 200;
numClasses = 10;
inputData = rand(28,28,1,30);% h*w*c*n
convL = convolution2dLayer(filterSize,numFilters,'Name','conv',...
'Weights',ones(filterSize,filterSize,1,numFilters),...
'Bias',ones(1,1,numFilters));
bnL= batchNormalizationLayer('Name','bn',...
'TrainedVariance',ones(1,1,numFilters),...
'TrainedMean',zeros(1,1,numFilters),...
'Offset',zeros(1,1,numFilters),...
'Scale',ones(1,1,numFilters));
%% first stage, use sequence layer
layers = [ ...
sequenceInputLayer(inputSize,'Name','input')
sequenceFoldingLayer('Name','fold')
convL
bnL
reluLayer('Name','relu')
sequenceUnfoldingLayer('Name','unfold')
flattenLayer('Name','flatten')
lstmLayer(numHiddenUnits,'OutputMode','last','Name','lstm',...
'InputWeights',ones(800,11520),...
'RecurrentWeights',ones(800,200),...
'Bias',ones(800,1));
fullyConnectedLayer(numClasses, 'Name','fc',...
'Weights',ones(numClasses,200),...
'Bias',ones(numClasses,1));
softmaxLayer('Name','softmax')
classificationLayer('Name','classification')];
lgraph = layerGraph(layers);
lgraph = connectLayers(lgraph,'fold/miniBatchSize','unfold/miniBatchSize');
assembleNet = assembleNetwork(lgraph);
%% second stage: not use sequence layer
% The sequenceFoldingLayer is essentially a per-frame graph acquisition feature as follows
layers2 = [ ...
imageInputLayer(inputSize,'Name','input','Normalization','none')
convL
bnL
reluLayer('Name','relu')
fullyConnectedLayer(numClasses, 'Name','fc',...
'Weights',ones(numClasses,11520),...
'Bias',ones(numClasses,1));
softmaxLayer('Name','softmax')
classificationLayer('Name','classification')];
lgraph2 = layerGraph(layers2);
assembleNet2 = assembleNetwork(lgraph2);
%% visualize
analyzeNetwork(lgraph)
analyzeNetwork(lgraph2)
%% comparison
out1 = activations(assembleNet,inputData,'unfold','OutputAs','channels');
out2 = activations(assembleNet2,inputData,'relu','OutputAs','channels');
t1 = out1{1};
t2 = out2;
%% result is same
t1(1:10)
t2(1:10)
