pivot_wider (see mozway's answer) is probably best here from a pure pandas perspective, but if you need more flexibility, you could also melt and pivot:
import pandas as pd
# recreating your dataframe
df = pd.DataFrame(['AA-24', '0', '700', '2100', '300', '1159', '2877', '30', '30', '47', '10', '5'],
index= ['Num', 'TP1(USD)', 'TP2(USD)', 'TP3(USD)', 'VReal1(USD)', 'VReal2(USD)', 'VReal3(USD)', 'TiV1(EUR)', 'TiV2(EUR)', 'TiV3(EUR)', 'TR', 'TR-Tag']).T
# reshaping the data
(df.melt(id_vars=['Num','TR', 'TR-Tag'])
.assign(col=lambda x: x['variable'].str[:2], idx=lambda x: x['variable'].str.extract("([0-9])"))
.pivot(values='value', columns='col', index='idx')
.rename(columns={'TP': 'Price', 'VR': 'Net', 'Ti': 'Range'})
)
Perhaps surprisingly, this is also faster than wide_to_long. Benchmarking gives 7.76 ms ± 841 µs per loop for this method.
The wide_to_long approach from mozway:
(pd
.wide_to_long(df.set_axis(df.columns.str.replace(r'\([A-Z]{3}\)$', '', regex=True),
axis=1),
stubnames=['TP', 'VReal', 'TiV'], i='Num', j='ID')
.reset_index('ID')
.drop(columns=['TR', 'TR-Tag'])
.rename(columns={'TP': 'Price', 'VReal': 'Net', 'TiV': 'Range'})
)
benchmarks at 30.4 ms ± 3.07 ms per loop on my machine.
Umar.H's answer using stack is the faster than both:
df1 = df.filter(regex='TP|VR|TV')
df1.columns = df1.columns\
.str.replace('(\d+)', r' \1' ,regex=True).str.split(' ',expand=True)
df1.stack(1).rename(columns={'TP': 'Price', 'VR': 'Net', 'TV': 'Range'})
Runs at 6.07 ms ± 156 µs per loop
If you don't mind the additional import, sammywemmy's answer using pyjanitor's pivot_wider offers speed and an elegant syntax.
(df
.select_columns('TP*', 'VR*', 'Ti*')
.pivot_longer(index = None,
names_to = ('.value', 'ID'),
names_pattern = ('(.+)(\d).+'))
.rename(columns = {'TP':'Price', 'VReal':'Net', 'TiV':'Range'})
)
benchmarks at 11.2 ms ± 229 µs per loop
and the names pattern approach:
df.pivot_longer(index = None,
names_to = ('Price', 'Net', 'Range'),
names_pattern = ('TP.*', 'VR.*', 'Ti.*'),
ignore_index = False)
is the fastest of the lot as tested, coming in at 3.53 ms ± 95 µs per loop.
(It is worth noting that this dataset is probably too small to care about speed, and the order may not be the same on larger datasets)