Why do I get NaN in table cells? (Python - Pandas)

user_14891020 0
Viewed 94

can anyone help me understand why the below code provides a table with a load of NaN in the cells.. Ive looked thru the data, some has (), some has "< b r >" tags and some is just plain text but still gives NaN and other strings seem fine. Cant fathom why.

import requests
import pandas as pd

headers = {'User-Agent': 'Mozilla/5.0'}
url = "https://www.cpu-world.com/CPUs/Bulldozer/AMD-FX-Series%20FX-8100.html"
r = requests.get(url, headers=headers)
df_list = pd.read_html(r.text, attrs={"class": "spec_table"})
print(df_list)

enter image description here

pandas python-3.x html
2 Answers

This does not directly answer your question but may solve your problem. They have provided a link "Get specs" at the top that provides html text. You can wrap this in a StringIO buffer to get a formatted table:

from io import StringIO

html_string = """<br><style type='text/css'>table.spec_table { border: 0; font-family: verdana, helvetica, arial, sans-serif; font-size: 12px; } table.spec_table td { padding: 3px 5px; line-height: 135%;  } table.spec_table td.spec_hd { font-weight: bold; padding: 20px 5px 10px 5px; } table.spec_table td.spec_hd span { min-width: 33%; background-color: #FFFFFF; padding: 3px 3px !important; border: 0; border-bottom: 1px solid #000000; } table.sh_table { font-family: verdana, helvetica, arial, sans-serif; font-size: 12px; } table.sh_table td, table.sh_table th { padding: 3px 5px; border-left: solid 1px #FFFFFF; text-align: left; } table.sh_table td:first-child { border-left: 0; } table.center_td td { text-align: center } .sh_subhead { color: #707070; font-size: 90%; background-color: #D8D8D8; font-weight: bold } tr.sh_subhead td { border-left: 1px solid #C0C0C0; } table.spec_table ul { padding-left: 11px; } ul.ul_nobull li { list-style-type: none; text-indent: -11px; } </style>
<table width="100%" cellpadding="0" cellspacing="0" border="0" class="spec_table">

<tbody><tr><td colspan="2" class="spec_hd"><span>General information</span></td></tr>
<tr><td valign="top" width="35%">Type</td><td><a href="https://www.cpu-world.com/CPUs/CPU.html">CPU / Microprocessor</a></td></tr>
<tr bgcolor="#F0F0F0"><td valign="top">Market segment</td><td>Desktop</td></tr>
<tr><td valign="top">Family</td><td id="SPECTD_FAMILY"><div id="SPECDV_FAMILY" class="spec_menu"></div><a href="https://www.cpu-world.com/CPUs/Bulldozer/TYPE-FX-Series.html">AMD FX-Series</a></td></tr>
<tr bgcolor="#F0F0F0"><td valign="top">Model number</td><td><b><a href="https://www.cpu-world.com/CPUs/Bulldozer/AMD-FX-Series%20FX-8100.html">FX-8100</a></b></td></tr>
<tr><td valign="top">CPU part numbers</td><td id="SPECTD_PN"><div id="SPECDV_PN" class="spec_menu"></div><ul class="ul_nobull">
<li><b style="color: #808080">FD8100WMW8KGU</b> is an OEM/tray microprocessor</li>
<li><b style="color: #808080">FD8100WMGUSBX</b> is a boxed microprocessor with fan and heatsink</li>
</ul></td></tr>
<tr bgcolor="#F0F0F0"><td valign="top"><b>Frequency</b></td><td><b>2800 MHz</b></td></tr>
<tr><td valign="top">Maximum turbo frequency</td><td>3100 MHz (more than 4 cores)<br>
3700 MHz (4 cores or less)</td></tr>
<tr bgcolor="#F0F0F0"><td valign="top"><b>Bus speed</b></td><td><b>One 2600 MHz 16-bit HyperTransport link</b></td></tr>
<tr><td valign="top">Package</td><td>940-pin organic micro Pin Grid Array (UOC940)<br>
Pb-free</td></tr>
<tr bgcolor="#F0F0F0"><td valign="top"><b>Socket</b></td><td><b>Socket AM3+</b></td></tr>
<tr><td valign="top">Size</td><td>1.57" x 1.57" / 4cm x 4cm</td></tr>
<tr bgcolor="#F0F0F0"><td valign="top">Introduction date</td><td><a href="https://www.cpu-world.com/Releases/Desktop_CPU_releases_(2011).html#December">4th quarter 2011</a></td></tr>
<tr><td colspan="2" class="spec_hd"><span>Architecture / Microarchitecture</span></td></tr>
<tr><td valign="top">Microarchitecture</td><td>Bulldozer</td></tr>
<tr bgcolor="#F0F0F0"><td valign="top">Platform</td><td>Scorpius</td></tr>
<tr><td valign="top">Processor core</td><td><a href="https://www.cpu-world.com/Cores/Zambezi.html">Zambezi</a></td></tr>
<tr bgcolor="#F0F0F0"><td valign="top">Core stepping</td><td>OR-B2</td></tr>
<tr><td valign="top">CPUID</td><td>600F12</td></tr>
<tr bgcolor="#F0F0F0"><td valign="top">Manufacturing process</td><td>0.032 micron Hi-K Metal Gate process with silicon-on-insulator (SOI) technology</td></tr>
<tr><td valign="top">Die</td><td>316mm<sup>2</sup></td></tr>
<tr bgcolor="#F0F0F0"><td valign="top"><b>Data width</b></td><td><b>64 bit</b></td></tr>
<tr><td valign="top"><b>The number of CPU cores</b></td><td><b>8</b></td></tr>
<tr bgcolor="#F0F0F0"><td valign="top"><b>The number of threads</b></td><td><b>8</b></td></tr>
<tr><td valign="top">Floating Point Unit</td><td>Integrated (shared between each pair of cores)</td></tr>
<tr bgcolor="#F0F0F0"><td valign="top">Level 1 cache size</td><td>4 x 64 KB 2-way set associative shared instruction caches<br>
8 x 16 KB 4-way set associative data caches</td></tr>
<tr><td valign="top"><b>Level 2 cache size</b></td><td><b>4 x 2 MB 16-way set associative shared exclusive caches</b></td></tr>
<tr bgcolor="#F0F0F0"><td valign="top"><b>Level 3 cache size</b></td><td><b>8 MB 64-way set associative shared cache</b></td></tr>
<tr><td valign="top">Cache latency</td><td>3 (L1 cache)</td></tr>
<tr bgcolor="#F0F0F0"><td valign="top">Multiprocessing</td><td>Uniprocessor</td></tr>
<tr><td valign="top">Extensions and Technologies</td><td><ul><li> MMX instructions</li>
<li> Extensions to MMX</li>
<li> SSE / Streaming SIMD Extensions</li>
<li> SSE2 / Streaming SIMD Extensions 2</li>
<li> SSE3 / Streaming SIMD Extensions 3</li>
<li> SSSE3 / Supplemental Streaming SIMD Extensions 3</li>
<li> SSE4 / SSE4.1 + SSE4.2 / Streaming SIMD Extensions 4</li>
<li> SSE4a</li>
<li> AES / Advanced Encryption Standard instructions</li>
<li> AVX / Advanced Vector Extensions</li>
<li> FMA4 / 4-operand Fused Multiply-Add instructions</li>
<li> XOP / eXtended Operations instructions</li>
<li> AMD64 / AMD 64-bit technology</li>
<li> AMD-V / AMD Virtualization technology</li>
<li> Turbo Core 2.0 technology</li>
</ul></td></tr>
<tr bgcolor="#F0F0F0"><td valign="top">Security Features</td><td><ul><li> EVP / Enhanced Virus Protection</li>
</ul></td></tr>
<tr><td valign="top">Low power features</td><td><ul><li> Core C1, C1E, C6 and CC6 states</li>
<li> Package S3, S4 and S5 states</li>
<li> PowerNow!</li>
</ul></td></tr>
<tr><td colspan="2" class="spec_hd"><span>Integrated peripherals / components</span></td></tr>
<tr><td valign="top">Integrated graphics</td><td>None</td></tr>
<tr bgcolor="#F0F0F0"><td valign="top">Memory controller</td><td>The number of controllers: 1<br>
Memory channels: 2<br>
Channel width (bits): 72<br>
Supported memory: DDR3-1866<br>
DIMMs per channel: 2<br>
Maximum memory bandwidth (GB/s): 29.9</td></tr>
<tr><td valign="top">Other peripherals</td><td><ul><li> HyperTransport 3.1 technology</li>
</ul></td></tr>
<tr><td colspan="2" class="spec_hd"><span>Electrical / Thermal parameters</span></td></tr>
<tr><td valign="top">V core</td><td>0.95V - 1.4125V</td></tr>
<tr bgcolor="#F0F0F0"><td valign="top">Maximum operating temperature</td><td>70°C</td></tr>
<tr><td valign="top"><b>Thermal Design Power</b></td><td><b>95 Watt</b></td></tr>
<tr><td colspan="2" class="spec_hd"><span>Notes on AMD FX-8100</span></td></tr>
<tr><td colspan="2"><div id="JSc_26" class=""><ul><li>The processor has unlocked clock multiplier</li><li>NorthBridge controller frequency is 2 GHz</li></ul></div></td></tr>
</tbody></table>
"""

df = pd.read_html(StringIO(html_string))[0]

output:

                                                    0                                                  1
0                                 General information                                General information
1                                                Type                               CPU / Microprocessor
2                                      Market segment                                            Desktop
3                                              Family                                      AMD FX-Series
4                                        Model number                                            FX-8100
5                                    CPU part numbers  FD8100WMW8KGU is an OEM/tray microprocessor FD...
6                                           Frequency                                           2800 MHz
7                             Maximum turbo frequency  3100 MHz (more than 4 cores) 3700 MHz (4 cores...
8                                           Bus speed            One 2600 MHz 16-bit HyperTransport link
9                                             Package  940-pin organic micro Pin Grid Array (UOC940) ...
10                                             Socket                                        Socket AM3+
11                                               Size                          1.57" x 1.57" / 4cm x 4cm
12                                  Introduction date                                   4th quarter 2011
13                   Architecture / Microarchitecture                   Architecture / Microarchitecture
14                                  Microarchitecture                                          Bulldozer
15                                           Platform                                           Scorpius
16                                     Processor core                                            Zambezi
17                                      Core stepping                                              OR-B2
18                                              CPUID                                             600F12
19                              Manufacturing process  0.032 micron Hi-K Metal Gate process with sili...
20                                                Die                                             316mm2
21                                         Data width                                             64 bit
22                            The number of CPU cores                                                  8
23                              The number of threads                                                  8
24                                Floating Point Unit     Integrated (shared between each pair of cores)
25                                 Level 1 cache size  4 x 64 KB 2-way set associative shared instruc...
26                                 Level 2 cache size  4 x 2 MB 16-way set associative shared exclusi...
27                                 Level 3 cache size           8 MB 64-way set associative shared cache
28                                      Cache latency                                       3 (L1 cache)
29                                    Multiprocessing                                       Uniprocessor
30                        Extensions and Technologies  MMX instructions  Extensions to MMX  SSE / Str...
31                                  Security Features                    EVP / Enhanced Virus Protection
32                                 Low power features  Core C1, C1E, C6 and CC6 states  Package S3, S...
33                Integrated peripherals / components                Integrated peripherals / components
34                                Integrated graphics                                               None
35                                  Memory controller  The number of controllers: 1 Memory channels: ...
36                                  Other peripherals                      HyperTransport 3.1 technology
37                    Electrical / Thermal parameters                    Electrical / Thermal parameters
38                                             V core                                    0.95V - 1.4125V
39                      Maximum operating temperature                                               70°C
40                               Thermal Design Power                                            95 Watt
41                               Notes on AMD FX-8100                               Notes on AMD FX-8100
42  The processor has unlocked clock multiplierNor...  The processor has unlocked clock multiplierNor...
user_50329410

I can't be sure but one possible reason could be that the website contains much more than just a data table. Which is not allowing the library to read from the website what you actually want. What can be done, in a quick way, is copy the table and then run the following command in your notebook

  Df = pd.read_clipboard()

This would help you, most probably.

user_145336070
Related