Create new columns based on rank order in Pandas

df = df.set_index(["ID", "Rank"])
df = df.unstack("Rank")
df.columns = df.columns.map(lambda col: "_".join(map(str, col)))

where we first take ID and Rank to index side and then unstack the Rank to columns. Then joining the ranks with the column names Hour and Prob that were already there gives

     Hour_1  Hour_2  Hour_3  Hour_4  Prob_1  Prob_2  Prob_3  Prob_4
ID
123      12      13      11      10     1.0     0.9     0.6     0.4
456      10      11      13      12     0.8     0.5     0.2     0.1

but since you need Rank again instead of Hour, we lastly do:

df.columns = df.columns.str.replace("Hour", "Rank")

and get

>>> df

     Rank_1  Rank_2  Rank_3  Rank_4  Prob_1  Prob_2  Prob_3  Prob_4
ID
123      12      13      11      10     1.0     0.9     0.6     0.4
456      10      11      13      12     0.8     0.5     0.2     0.1

In case of duplicate entries in the unstack argument (i.e., Rank here), we can append whilst setting index. Then duplicate issue will be gone but a lot of NaNs would appear:

>>> df   # 2nd rows rank was made 4 for duplication

ID  Hour    Prob       Rank
  
 123  10     0.4         4
 123  11     0.6         4
 123  12     1.0         1
 123  13     0.9         2
 456  10     0.8         1
 456  11     0.5         2
 456  12     0.1         4
 456  13     0.2         3

# same operations, *except* `append=True`
>>> df = df.set_index(["ID", "Rank"], append=True)
>>> df = df.unstack("Rank")
>>> df.columns = df.columns.map(lambda col: "_".join(map(str, col)))
>>> df.columns = df.columns.str.replace("Hour", "Rank")
>>> df

       Rank_1  Rank_2  Rank_3  Rank_4  Prob_1  Prob_2  Prob_3  Prob_4
  ID
0 123     NaN     NaN     NaN    10.0     NaN     NaN     NaN     0.4
1 123     NaN     NaN     NaN    11.0     NaN     NaN     NaN     0.6
2 123    12.0     NaN     NaN     NaN     1.0     NaN     NaN     NaN
3 123     NaN    13.0     NaN     NaN     NaN     0.9     NaN     NaN
4 456    10.0     NaN     NaN     NaN     0.8     NaN     NaN     NaN
5 456     NaN    11.0     NaN     NaN     NaN     0.5     NaN     NaN
6 456     NaN     NaN     NaN    12.0     NaN     NaN     NaN     0.1
7 456     NaN     NaN    13.0     NaN     NaN     NaN     0.2     NaN

Now pivot_table can aggregate these; say we'd like to sum the values in duplications:

>>> df.pivot_table(index="ID", aggfunc="sum")

     Prob_1  Prob_2  Prob_3  Prob_4  Rank_1  Rank_2  Rank_3  Rank_4
ID
123     1.0     0.9     0.0     1.0    12.0    13.0     0.0    21.0
456     0.8     0.5     0.2     0.1    10.0    11.0    13.0    12.0

Note that result is the same for the IDthat didn't have any duplicates; for the other, it is 0 for rank 3 that didn't exist; and summation of values for rank 4.

You can do it with tidyverse way in python with datar:

>>> from datar.all import f, tribble, pivot_wider, rename_with
>>> 
>>> df = tribble(
...     f.ID,  f.Hour,  f.Prob,  f.Rank,
...     123,   10,      0.4,     4,
...     123,   11,      0.6,     3,
...     123,   12,      1.0,     1,
...     123,   13,      0.9,     2,
...     456,   10,      0.8,     1,
...     456,   11,      0.5,     2,
...     456,   12,      0.1,     4,
...     456,   13,      0.2,     3,
... )
>>> 
>>> df >> pivot_wider(
...     names_from=f.Rank, 
...     values_from=[f.Hour, f.Prob], 
...     names_glue='{_value}_{Rank}'
... ) >> rename_with(
...     lambda col: col.replace('Hour', 'Rank')
... )
       ID  Rank_1  Rank_2  Rank_3  Rank_4    Prob_1    Prob_2    Prob_3    Prob_4
  <int64> <int64> <int64> <int64> <int64> <float64> <float64> <float64> <float64>
0     123      12      13      11      10       1.0       0.9       0.6       0.4
1     456      10      11      13      12       0.8       0.5       0.2       0.1

I am the author of the datar package. Feel free to submit issues if you have any questions.