How to seek to a specific frame in a h.264 video?

I need to decode a video from specific frame by FFmpeg, I know h.264 video has I-frame/P-frame/B-frame, when I use av_seek_frame() to seek to a specific frame, it could only help me to find the nearest previous I-frame, so I tried to use the AVPacket.dts and a while loop to navigate to the location of the specific frame, and then start to demux the video frame.

auto DurationTime = static_cast<uint64_t>(m_pAVFormatContext->streams[m_VideoStreamIndex]->duration);
auto FrameNum = m_pAVFormatContext->streams[m_VideoStreamIndex]->nb_frames;
av_seek_frame(m_pAVFormatContext, m_VideoStreamIndex, DurationTime / FrameNum * (vSpecificFrame), AVSEEK_FLAG_FRAME | AVSEEK_FLAG_BACKWARD);
int TempValue = 0;
while ((TempValue = av_read_frame(m_pAVFormatContext, &m_AVPacket)) >= 0 && m_AVPacket.stream_index != m_VideoStreamIndex)
{
    av_packet_unref(&m_AVPacket);
}
while(m_AVPacket.dts / (DurationTime / FrameNum) < vSpecificFrame)
{
    if(m_AVPacket.data)
        av_packet_unref(&m_AVPacket);
    while ((TempValue = av_read_frame(m_pAVFormatContext, &m_AVPacket)) >= 0 && m_AVPacket.stream_index != m_VideoStreamIndex)
    {
        av_packet_unref(&m_AVPacket);
    }
}
if(m_AVPacket.data)
    av_packet_unref(&m_AVPacket);

But the question is when I demux the next frame, I can get it's pos, data, size and all other information, but the image is unable to display. Before demuxing the next I-frame, all frames between this specific frame with the next I-frame has the same case. Did I use the wrong method? Or whether there has any other method to seek to the specific frame of a video? Thanks very much.