bpf_btf_get_fd_by_id return bad file descriptor?

I am debugging a tracer that forked a new process to run and trace the tracee's bpf map relative operation. I can only get the tracee map_fd and I need to get the relative btf information. The rough code snippet is as follows.

int pidfd = pidfd_open(tracee_pid, 0);
int tracer_fd = pidfd_get_fd(pidfd, tracee_fd, 0);
struct bpf_map_info info = {};
int len = sizeof(info);
bpf_obj_get_info_by_fd(tracer_fd, &info, &len);
close(pidfd);
close(tracer_fd);

int btf_fd = bpf_btf_get_fd_by_id(info.btf_id);
if (btf_fd == -1)
    printf("btf_fd: %s\n", strerror(btf_fd));
struct btf* btf = fetch_btf_from_fd(btf_fd, NULL);

bpf_btf_get_fd_by_id return "bad file descriptor" error.

My debugging attempt:
I use gdb to debug and get info.btf_id is 91, so I wrote a demo that uses 91 as an input parameter. It works and returns btf_fd is 4. I use gdb to debug the tracer and find that tracer_fd is also 4. I don't know if they are related.