How do I see how many slices are in the last level cache?

user_14328098 0
Viewed 522

I read an introduction to last level cache in a paper. In the introduction, it is said that the last level cache has many slices. Each slice is like a traditional set-associated cache. Find the corresponding cache set by set index and slice id (as shown in the figure below).

enter image description here

I want to check how many slices my server has. And the information of each slice (how many cache sets, how many cachelines in each cache set). The method I searched on google is to view the cache information under the folder /sys/devices/system/cpu/cpu0/cache/index3.

But after checking, I found that the introduction under this folder is no different from l1 and l2. The files under this folder have

coherency_line_size  level           physical_line_partition  shared_cpu_list  size  uevent
id                   number_of_sets  power                    shared_cpu_map   type  ways_of_associativity

How can I check how many slices there are on the server? Is the number_of_sets shown here a slice cache set?

I am using a server. The version is:Linux version 4.15.0-122-generic (buildd@lcy01-amd64-010) (gcc version 5.4.0 20160609 (Ubuntu 5.4.0-6ubuntu1~16.04.12)) #124~16.04.1-Ubuntu SMP.

cpu-cache intel x86 cpu-architecture caching
1 Answers

Haswell and Broadwell

The following processor collections use the server uncore microarchitecture:

  • Core X
  • Xeon E5
  • Xeon E7

On these processors, the number of L3 cache slices can be obtained as follows:

Step 1: Run the following command on a Linux terminal:

setpci | grep Power

This shows you a list of PCI device functions of the PCU of each processor package in your system. An example output on dual Xeon E5 v4 processor system looks like this:

df:1e.0 System peripheral: Intel Corporation Xeon E7 v4/Xeon E5 v4/Xeon E3 v4/Xeon D Power Control Unit (rev 01)
df:1e.1 System peripheral: Intel Corporation Xeon E7 v4/Xeon E5 v4/Xeon E3 v4/Xeon D Power Control Unit (rev 01)
df:1e.2 System peripheral: Intel Corporation Xeon E7 v4/Xeon E5 v4/Xeon E3 v4/Xeon D Power Control Unit (rev 01)
df:1e.3 System peripheral: Intel Corporation Xeon E7 v4/Xeon E5 v4/Xeon E3 v4/Xeon D Power Control Unit (rev 01)
df:1e.4 System peripheral: Intel Corporation Xeon E7 v4/Xeon E5 v4/Xeon E3 v4/Xeon D Power Control Unit (rev 01)
df:1e.0 System peripheral: Intel Corporation Xeon E7 v4/Xeon E5 v4/Xeon E3 v4/Xeon D Power Control Unit (rev 01)
df:1e.2 System peripheral: Intel Corporation Xeon E7 v4/Xeon E5 v4/Xeon E3 v4/Xeon D Power Control Unit (rev 01)
ff:1e.0 System peripheral: Intel Corporation Xeon E7 v4/Xeon E5 v4/Xeon E3 v4/Xeon D Power Control Unit (rev 01)
ff:1e.1 System peripheral: Intel Corporation Xeon E7 v4/Xeon E5 v4/Xeon E3 v4/Xeon D Power Control Unit (rev 01)
ff:1e.2 System peripheral: Intel Corporation Xeon E7 v4/Xeon E5 v4/Xeon E3 v4/Xeon D Power Control Unit (rev 01)
ff:1e.3 System peripheral: Intel Corporation Xeon E7 v4/Xeon E5 v4/Xeon E3 v4/Xeon D Power Control Unit (rev 01)
ff:1e.4 System peripheral: Intel Corporation Xeon E7 v4/Xeon E5 v4/Xeon E3 v4/Xeon D Power Control Unit (rev 01)
ff:1e.0 System peripheral: Intel Corporation Xeon E7 v4/Xeon E5 v4/Xeon E3 v4/Xeon D Power Control Unit (rev 01)
ff:1e.2 System peripheral: Intel Corporation Xeon E7 v4/Xeon E5 v4/Xeon E3 v4/Xeon D Power Control Unit (rev 01)

The device functions on bus df are on one processors and those on bus ff are on the other socket. The only piece of information needed is the bus number. Take bus df for example.

Step 2: Run the following command:

sudo setpci -s df:1e.3 98.l

An example out is 46000f2e. Bits 0-23 of this value represent a bit vector where a bit value of zero indicates an enabled L3 cache slice and a bit value of zero indicates a disabled cache slice. You can't actually disable any slices yourself; only the PCU can do that during package C-state transitions. During normal operation, all the available slices would be enabled. Therefore, the number of set bits is the number of slices. In this example, the bit vector 000f2e has 8 bits set, so the number of slices on the selected processor is 8.

Usually, all Intel processors in the same shared memory system are homogeneous, but you can repeat the above states for each processor if you want.

In general, there can be up to 24 slices, each up to 2.5 MiB in size.

Sandy Bridge and Ivy Bridge

The following processor collections use the server uncore microarchitecture:

  • Core X
  • Xeon E5
  • Xeon E7

Step 1: The same as before.

Step 2: Run the following command:

sudo setpci -s XX:0a.3 94.l

where XX is the bus number from Step 1. Bits 0-14 represent the cache slice bit vector. In general, there can be up to 15 slices, each up to 2.5 MiB in size.

I'm not sure whether this method works on the Sandy Bridge processors and Ivy Bridge's Core X series, but there is no harm in checking if it works.

Skylake, Cascade Lake, and Cooper Lake

The following processor collections use the server uncore microarchitecture:

  • Core X
  • Xeon SP
  • Xeon W
  • Xeon D

Step 1: The same as before.

Step 2: Run the following command (thanks to @JohnDMcCalpin):

sudo setpci -s XX:1e.3 9c.l

where XX is the bus number from Step 1. Bits 0-27 represent the cache slice bit vector. In general, there can be up to 28 slices, each 1.375 MiB in size.

All processor models with server uncore released by Intel have L3 caches consisting of 1.375 MiB slices. The number of slices is the total cache size divided by 1.375 MiB. I'm not aware of any exceptions.

Ice Lake

The following processor collections use the server uncore microarchitecture:

  • Xeon SP

It seems to me that the slice size is 1.5 MiB on these processors.

Nehalem and Westmere

The following processor collections use the server uncore microarchitecture with a distributed L3 cache:

  • Xeon 6500
  • Xeon 7500
  • Xeon E7

The slice size can be up to 3 MiB on these processors.

All other Intel processors with a server uncore design

They don't use a distributed cache architecture, so the concept of slice doesn't exist on these processors.

user_42306180
Related