A working linux kernel + gem5 config for FS boot up in x86 SMP

user_3382741 0
Viewed 607

I'm trying to bring up a gem5 FullSystem (FS) simulation of Linux kernel 2.6.22.9 (as the binary was provided by gem5) and also with a custom Linux kernel 3.4.112 on a TimingSimpleCPU. While both of these work in a single core x86 FS simulated machine, they fail to boot up in a multi-core simulated machine.

I'm lost on how to even begin debugging. I have tried connecting to the remote gdb port provided by gem5 TimingSimpleCPU for each processor on ports 7000, 7001 and so on. I see that on a dual core boot up, after a point, core 0 gets stuck on schedule() call and core 1 always stays on idle() and never schedules() anything until core 0 also gets stuck on the schedule() call.

What is a proper way to go about debugging gem5 and its compatibility with Linux Kernel for multi core full system boot up on a TimingSimpleCPU X86 arch? I'm thinking there could be issues relating to spinlock support or the APIC.

gem5 x86-64 x86 multiprocessing linux
1 Answers

X86 2 core Linux kernel 5.1, TimingSimpleCPU, gem5 08c79a194d1a3430801c04f37d13216cc9ec1da3 happened to work on this setup: https://github.com/cirosantilli/linux-kernel-module-cheat/tree/6aa2f783a8a18589ae66e85f781f86b08abb3397#gem5-buildroot-setup-getting-started Boot completes and cat /proc/cpuinfo says 2 CPUs.

The final run command was:

./run --cpus 2 --emulator gem5 -- --cpu-type TimingSimpleCPU

Everything is specified in that repo, including how to build gem5, the Linux kernel, and how to run them.

Then, with a mere flick of a switch, the same works on aarch64 as well if you are curious:

./run --arch aarch64 --cpus 2 --emulator gem5 -- --cpu-type TimingSimpleCPU

I then added the options --caches --l2cache as per OP's comment, and now I reproduce the failure, to which I don't have a solution:

./run --cpus 2 --emulator gem5 -- --cpu-type TimingSimpleCPU --caches --l2cache

Boot hangs, the last terminal message is:

pci 0000:00:04.0: legacy IDE quirk: reg 0x1c: [io  0x0376]

and a bit above we can see the suspicious message:

[Firmware Bug]: CPU1: APIC id mismatch. Firmware: 1 APIC: 0

ARM boot with the extra options worked however:

./run --arch aarch64 --cpus 2 --emulator gem5 -- --cpu-type TimingSimpleCPU --caches --l2cache

However, I later tried with more cache options:

/run --arch aarch64 --emulator gem5 --cpu 2 --run-id 2 -- --cpu-type=HPI --caches --l2cache --l1d_size=64kB --l1i_size=64kB --l2_size=256kB

and it also failed as explained at: https://github.com/cirosantilli/linux-kernel-module-cheat/tree/99180e6616331b7385b09147f11f67962f9facc4#gem5-arm-multicore-hpi-boot-fails ...

How to debug such problems to get things working in general is an extremely difficult problem that requires understanding enough Linux kernel + X86 ISA + gem5, where enough is undefined. This learning process is closely intertwined with enabling just the right log options / focusing on the right part of the code. That setup just happened to work out of "luck".

user_91607620
Related