I'm using the transformers library of HuggingFace. As far as I know changing the number of hidden layers in the config file leads to loading the first x layers of the pre-trained BERT. I want to load the even layers (or the last x layers) of the pre-trained BERT and then fine-tune them for a classification task.
An example for classification tasks can be found here : run_glue.py
Thanks in advance