multiple definition of `memset' when building static rust library to be used for Nordic SDK program

Question

Note: I do not see that this is a duplicate of the other question. It's in my question, that I checked the suggetion provided there and this doesn't help here (as shown below). If you see this differently, I would like to know which kind of information you are thinking of, when you write “If the solution does not work, you should make clear why not and what you have tried to solve it already.” I'm happy to provide everything that is needed to understand the problem.

That's way I wrote this question to ask for additional suggestions what I can test or where I could search for additional solutions.

I know that this question is a long one. But the text I wrote is actually as short as possible. It's just the shell output I provide, that makes it long. So I think it actually is quite readable.

I try to build a static library in Rust, that I want to use in an embedded program, that runs on a Nordic nRF52832 bluetooth low energy micro controller.

The problem is that the static library that Rust generates contains symbols, that are also defined by libc_nano:

/usr/lib/gcc/arm-none-eabi/7.3.1/../../../arm-none-eabi/bin/ld: ../../modem/target/thumbv7em-none-eabihf/release/libtbmodem.a(compiler_builtins-ace0cbd4c713eeda.compiler_builtins.8gdoevk6-cgu.12.rcgu.o): in function `memset':
/checkout/src/rustc/compiler_builtins_shim/../../libcompiler_builtins/src/mem.rs:47: multiple definition of `memset'; /usr/lib/gcc/arm-none-eabi/7.3.1/../../../arm-none-eabi/lib/thumb/v7e-m/fpv4-sp/hard/libc_nano.a(lib_a-memset.o):/build/newlib-4qXI0C/newlib-3.0.0.20180802/build_nano/arm-none-eabi/thumb/v7e-m/fpv4-sp/hard/newlib/libc/string/../../../../../../../../../newlib/libc/string/memset.c:41: first defined here
collect2: error: ld returned 1 exit status

Looking at the output of nm is see, that indeed the library compiled by rust does define memset and some other typical libc functions:

$ nm target/thumbv7em-none-eabihf/release/libtbmodem.a|grep mem
         U __aeabi_memclr
         U __aeabi_memcpy
         U __aeabi_memclr
         U __aeabi_memcpy
         U __aeabi_memcpy
         U __aeabi_memmove
         U __aeabi_memcpy
         U __aeabi_memcpy
         U __aeabi_memmove
         U __aeabi_memcpy
         U memcmp
         U __aeabi_memcpy
00000000 T memcmp
00000000 T memcpy
00000000 T memmove
00000000 T memset
00000000 W __aeabi_memclr
00000000 W __aeabi_memclr4
00000000 W __aeabi_memclr8
00000000 W __aeabi_memcpy
00000000 W __aeabi_memcpy4
00000000 W __aeabi_memcpy8
00000000 W __aeabi_memmove
00000000 W __aeabi_memmove4
00000000 W __aeabi_memmove8
00000000 W __aeabi_memset
00000000 W __aeabi_memset4
00000000 W __aeabi_memset8
         U memcmp
00000000 T _ZN4core5slice6memchr6memchr17h3803f373e8c2e0b4E
00000000 T _ZN4core5slice6memchr7memrchr17hed16b4d75d82bdf8E
         U __aeabi_memcpy
         U memcmp
         U _ZN4core5slice6memchr6memchr17h3803f373e8c2e0b4E
         U __aeabi_memclr4
         U __aeabi_memcpy4
         U memcmp
         U __aeabi_memcpy
         U __aeabi_memset
         U memcmp
         U _ZN4core5slice6memchr6memchr17h3803f373e8c2e0b4E
         U __aeabi_memclr4
         U __aeabi_memcpy4
         U __aeabi_memset
         U __aeabi_memclr4
         U __aeabi_memcpy4
         U memcmp

Interestingly these colliding symbols aren't generated by Rust, when I compile for my host architecture x86_64-unknown-linux-gnu:

$ nm target/x86_64-unknown-linux-gnu/release/libtbmodem.a |grep mem
                 U memcpy
                 U memset
                 U memcpy
                 U memset
                 U memcpy
                 U memmove
                 U memcpy
                 U memcpy
                 U memmove
                 U memcmp
                 U memcpy
                 U memcpy
0000000000000000 T _ZN17compiler_builtins3mem6memcmp17hd848cdf5fbc51bd1E
0000000000000000 T _ZN17compiler_builtins3mem6memcpy17he0a8277ca9da5208E
0000000000000000 T _ZN17compiler_builtins3mem6memset17h0fcc19dd786ae994E
0000000000000000 T _ZN17compiler_builtins3mem7memmove17h75c47f31c61a641fE
0000000000000000 T _ZN4core5slice6memchr6memchr17h3803f373e8c2e0b4E
0000000000000000 T _ZN4core5slice6memchr7memrchr17hed16b4d75d82bdf8E
                 U memcmp
                 U memcpy
                 U _ZN4core5slice6memchr6memchr17h3803f373e8c2e0b4E
                 U memcmp
                 U memcpy
                 U memset
                 U memcmp
                 U memcpy
                 U _ZN4core5slice6memchr6memchr17h3803f373e8c2e0b4E
                 U memcpy
                 U memset
                 U memcmp
                 U memcpy

What is the reason for Rust to define these symbols when compiling for ARM but not when compiling for x86_64?

I know, that the same error is also discussed in "multiple definition of `memcmp" error when linking Rust staticlib with embedded C program, but the solution provided there doesn't work for me. Actually I already have the static library produced by rust as the last argument to the linker call:

arm-none-eabi-gcc -DBUILD=\"date +%F\" -DBUILDY=date +%Y -DBUILDM=date +%-m -DBUILDD=date +%-d -mcpu=cortex-m4 -mthumb -mabi=aapcs -mfloat-abi=hard -mfpu=fpv4-sp-d16 -ffunction-sections -fdata-sections -fno-strict-aliasing -fno-builtin --short-enums -Iconfig -std=c99 -DNRF52 -DNRF52832_XXAA -DBOARD_CUSTOM -DBLE_STACK_SUPPORT_REQD -DNRF_SD_BLE_API_VERSION=6 -DNRF_DFU_SVCI_ENABLED -DNRF_DFU_TRANSPORT_BLE=1 -DS132 -DSOFTDEVICE_PRESENT -DSWI_DISABLE0 -Wall -Werror -DFLOAT_ABI_HARD -DCONFIG_GPIO_AS_PINRESET -DNRF52_PAN_74 -I/home/matthias/source/tbblue/libtrailer/src @sdk_include_paths -Og -g3 -DDEBUG -DDEBUG_NRF -mthumb -mabi=aapcs -mcpu=cortex-m4 -Wl,--gc-sections --specs=nano.specs -lc -lnosys -lm -Wl,-Map=tbplatform.map,--wrap,_sbrk -L/home/matthias/source/tbblue/libtrailer/src -L /home/matthias/source/tbblue/nRF5_SDK//components/toolchain/gcc -T tbplatform.ld -o tbplatform .sdk/modules/nrfx/mdk/gcc_startup_nrf52.o nrfplatform.o base64.o bleinithelper.o ble_dfu_service.o broadcast.o config_service.o fw_version.o main.o mac_address.o modem.o nrf_ble_qwrs.o on_board_flash.o service.o system.o compartment_util.o saved_config.o temperature_util.o tick_util.o digitalinput/digital.o spi/spi.o spi/can/can_controller.o spi/can/can_interrupt.o spi/can/can_timer.o spi/can/can.o spi/uart/spi_uart.o uart/datacold500_uart.o uart/datacold600_uart.o uart/euroscan_uart.o uart/ibox_uart.o uart/uart.o .sdk/components/ble/ble_advertising/ble_advertising.o .sdk/components/ble/ble_services/ble_dfu/ble_dfu.o .sdk/components/ble/ble_services/ble_dfu/ble_dfu_unbonded.o .sdk/components/ble/common/ble_advdata.o .sdk/components/ble/common/ble_conn_params.o .sdk/components/ble/common/ble_conn_state.o .sdk/components/ble/common/ble_srv_common.o .sdk/components/ble/nrf_ble_gatt/nrf_ble_gatt.o .sdk/components/ble/peer_manager/gatt_cache_manager.o .sdk/components/ble/peer_manager/gatts_cache_manager.o .sdk/components/ble/peer_manager/id_manager.o .sdk/components/ble/peer_manager/peer_database.o .sdk/components/ble/peer_manager/peer_data_storage.o .sdk/components/ble/peer_manager/peer_id.o .sdk/components/ble/peer_manager/peer_manager.o .sdk/components/ble/peer_manager/pm_buffer.o .sdk/components/ble/peer_manager/pm_mutex.o .sdk/components/ble/peer_manager/security_dispatcher.o .sdk/components/ble/peer_manager/security_manager.o .sdk/components/boards/boards.o .sdk/components/libraries/atomic/nrf_atomic.o .sdk/components/libraries/atomic_fifo/nrf_atfifo.o .sdk/components/libraries/atomic_flags/nrf_atflags.o .sdk/components/libraries/balloc/nrf_balloc.o .sdk/components/libraries/bootloader/dfu/nrf_dfu_svci.o .sdk/components/libraries/crc16/crc16.o .sdk/components/libraries/experimental_log/src/nrf_log_default_backends.o .sdk/components/libraries/experimental_log/src/nrf_log_backend_rtt.o .sdk/components/libraries/experimental_log/src/nrf_log_backend_serial.o .sdk/components/libraries/experimental_log/src/nrf_log_backend_uart.o .sdk/components/libraries/experimental_log/src/nrf_log_frontend.o .sdk/components/libraries/experimental_log/src/nrf_log_str_formatter.o .sdk/components/libraries/experimental_memobj/nrf_memobj.o .sdk/components/libraries/experimental_section_vars/nrf_section_iter.o .sdk/components/libraries/fds/fds.o .sdk/components/libraries/fifo/app_fifo.o .sdk/components/libraries/fstorage/nrf_fstorage.o .sdk/components/libraries/fstorage/nrf_fstorage_sd.o .sdk/components/libraries/pwr_mgmt/nrf_pwr_mgmt.o .sdk/components/libraries/strerror/nrf_strerror.o .sdk/components/libraries/timer/app_timer.o .sdk/components/libraries/uart/app_uart_fifo.o .sdk/components/libraries/uart/retarget.o .sdk/components/libraries/util/app_error_weak.o .sdk/components/libraries/util/app_util_platform.o .sdk/components/softdevice/common/nrf_sdh.o .sdk/components/softdevice/common/nrf_sdh_ble.o .sdk/components/softdevice/common/nrf_sdh_soc.o .sdk/external/fprintf/nrf_fprintf.o .sdk/external/fprintf/nrf_fprintf_format.o .sdk/external/segger_rtt/SEGGER_RTT.o .sdk/external/segger_rtt/SEGGER_RTT_Syscalls_GCC.o .sdk/external/segger_rtt/SEGGER_RTT_printf.o .sdk/external/tiny-AES128/aes.o .sdk/integration/nrfx/legacy/nrf_drv_spi.o .sdk/integration/nrfx/legacy/nrf_drv_uart.o .sdk/modules/nrfx/drivers/src/nrfx_gpiote.o .sdk/modules/nrfx/drivers/src/nrfx_spis.o .sdk/modules/nrfx/drivers/src/nrfx_uart.o .sdk/modules/nrfx/drivers/src/nrfx_uarte.o .sdk/modules/nrfx/drivers/src/prs/nrfx_prs.o .sdk/modules/nrfx/drivers/src/nrfx_spim.o .sdk/modules/nrfx/mdk/system_nrf52.o -ltrailer ../../modem/target/thumbv7em-none-eabihf/release/libtbmodem.a

(It's the file libtbmodem.a.)

What else can I do to get the library linked?

user_805673 · Answer

I had the same problem as you today and I can offer a possible answer (it works in my case) and a partial response why this works:

TL;DR: My solution was to place my Rust library before libc on the linker line. In your case, this would mean to change your call to:

arm-none-eabi-gcc [...] specs=nano.specs -ltbmodem -lc -lnosys [...]

Explanation: Examining my Rust library shows, as in your case, that symbols for memcpy, memset and so forth are exported from my static library. Note that those are contained in an object with several other memory related functions:

$ nm librustlib.a
[...]
compiler_builtins-17b2875688712538.compiler_builtins.eth5fjl6-cgu.100.rcgu.o:
00000000 T bcmp
00000000 T memcmp
00000000 T memcpy
00000000 T memmove
00000000 T memset
00000000 T _ZN17compiler_builtins3mem40__llvm_memcpy_element_unordered_atomic_117h2c6de8c0ed769de1E
[... seven "compiler_builtins" functions more omitted ...]
00000000 T _ZN17compiler_builtins3mem41__llvm_memmove_element_unordered_atomic_417h971099a5edfec01eE
[...]

Some of those functions are also exported from libc, for example:

$ nm /usr/arm-none-eabi/lib/thumb/v7e-m+fp/hard/libc_nano.a
[...]
lib_a-memcpy.o:

lib_a-memcpy-stub.o:
00000000 t $t
00000001 T memcpy
[...]

Specifically, the five functions without name-mangling (the functions with normal names) can be found in libc_nano.a.

To understand if link order makes a difference here, I have read Library Order in Static Linking which contains the following interesting section:

When the linker encounters a new library, things are a bit more interesting. The linker goes over all the objects in the library. For each one, it first looks at the symbols it exports.

If any of the symbols it exports are on the undefined list, the object is added to the link and the next step is executed. Otherwise, the next step is skipped.

If the object has been added to the link, it's treated as described above - its undefined and exported symbols get added to the symbol table.

Finally, if any of the objects in the library has been included in the link, the library is rescanned again - it's possible that symbols imported by the included object can be found in other objects within the same library.

So when the linker scans a library, every contained object is handled separately and can be used or be omitted from the final result, depending on whether it seems to be required to the linker, i.e. whether it defines a symbol that is still undefined. In the article above, this is stated as:

Also note that when a library is examined, an object file within it can be left out of the link if it does not provide symbols that the symbol table needs. This is a very important feature of static linking. The C library I mentioned before makes a heavy use of this feature, by mostly splitting itself to an-object-per-function.

For linking my Rust library with C code (and thus libc) this means specifically:

If the Rust library is placed earlier than libc on the command-line, the "compiler_builtins"-object defining the memory-related functions will be linked in the final result. When libc is examined afterwards, the memory-related functions are already defined so that the corresponding objects from libc will be omitted.
If the Rust library is placed later on the command-line, the memory-related functions from libc will be used. Then, the linker also uses the object from the Rust library, most probably because of the name-mangled functions are required. This leaves us with having memcpy and the like defined twice which is an error.

I hope this information is still helpful or can be helpful to anyone who stumbles upon the same problem in the future.

multiple definition of `memset' when building static rust library to be used for Nordic SDK program

1 Answers