I am using the memleak BCC tool to trace memory allocations in my Rust code but I notice a lot of my stack traces seem either incomplete or have missing symbol names. Reading Brendan Gregg's BPF Performance Tools book, it seems the two common causes of this are omitting stack frame pointers and stripped debug symbols from the binary.
I believe I have set the correct settings in my Rust build to fix these two: compiling my code as RUSTFLAGS="-Cforce-frame-pointers=yes" cargo build --release for the former and having the following section in my Cargo.toml for the latter:
[profile.release]
debug = true
This is what file gives me on the binary that is being run/traced:
$ file /runner
/runner: ELF 64-bit LSB shared object, x86-64, version 1 (SYSV), dynamically linked, interpreter /lib64/l, for GNU/Linux 2.6.32, BuildID[sha1]=7238c7fe5546d7420cf8c85a3d3af79136dab6e5, with debug_info, not stripped
Not sure what else might be going wrong here. Some examples of the output of memleak with the missing symbols:
16 bytes in 1 allocations from stack
bytes::bytes::Bytes::copy_from_slice::h77f998d9367a6e1d+0x27
_$LT$actix_web..handler..ExtractResponse$LT$T$C$S$GT$$u20$as$u20$core..future..future..Future$GT$::poll::hb2514b3babf3ddcd+0x4e9
_$LT$futures_util..future..future..map..Map$LT$Fut$C$F$GT$$u20$as$u20$core..future..future..Future$GT$::poll::h9193cccfdb6726fa+0x23
[unknown]
[unknown]
and missing stack traces:
16778510 bytes in 1 allocations from stack
alloc::vec::Vec$LT$T$GT$::extend_from_slice::h17e22096d3c66dad+0x4f
_$LT$hyper..client..conn..Connection$LT$T$C$B$GT$$u20$as$u20$core..future..future..Future$GT$::poll::h96fcb741ed4c6751+0x58
For this latter bit, the main thing that confuses me is why does the stack trace start with Connection as opposed to showing me what caused the Connection call? I wonder if this might be due to details of how network requests are tracked on the stack and/or how Rust futures are implemented?