Fill NaN values within groups by number in pandas

I would first select the unique NAMEs for the NaN rows:

m = df['VALUES'].isna()
names = df.loc[m, 'NAME'].unique()

then create a mapping for these:

mapping = dict(zip(names, list(range(1,len(names)+1))))

then fill your VALUES for the NaN rows with the mapping:

df.loc[m, 'VALUES'] = df.loc[m, 'NAMES'].map(mapping)

Update to fill the VALUES based on the GROUPS as I understand from your comment:

So we select the rows with NaN VALUES again. Now we do a groupby and keep the original df index using transform. To add a list, we need to know the length of the group. I therefore added the size columns.

df = pd.DataFrame({'Groups': {0: 'G1', 1: 'G1', 2: 'G1', 3: 'G1', 4: 'G2', 5: 'G2', 6: 'G2', 7: 'G3', 8: 'G3', 9: 'G3', 10: 'G4', 11: 'G4', 12: 'G4', 13: 'G4'}, 'NAME': {0: 'A', 1: 'B', 2: 'C', 3: 'C', 4: 'D', 5: 'E', 6: 'D', 7: 'F', 8: 'G', 9: 'H', 10: 'I', 11: 'I', 12: 'J', 13: 'K'}, 'VALUES': {0: 1.0, 1: 2.0, 2: 3.0, 3: 3.0, 4: np.nan, 5: np.nan, 6: np.nan, 7: np.nan, 8: np.nan, 9: np.nan, 10: 8.0, 11: 8.0, 12: 89.0, 13: 65.0}})
    sizes = df.groupby(['Groups']).size()
    df['Size']=df['Groups'].map(sizes)
    m = df['VALUES'].isna()

Next you want to give a duplicate occurance of Group and NAME (so a groupby on Group and NAME) the same number (like G2 and D) => therefore we select the first occurance of such rows and map this to the combination of Group and NAME:

df.loc[m, 'VALUES_new']  = df.loc[m].groupby(['Groups'])['Size'].transform(lambda x:list(range(1,len(x)+1)))
mapping = df.loc[m].groupby(['Groups', 'NAME'])['VALUES_new'].first().copy()
df.set_index(['Groups', 'NAME'], inplace=True)
m = df['VALUES'].isna()
df.loc[m,'VALUES'] = df.loc[m].index.map(mapping)
df.reset_index(inplace=True)
df.drop(columns=['Size', 'VALUES_new'], inplace=True)
df['VALUES']=df['VALUES'].astype(int)

Just to see what happens with the individual groups you could run this:

df = pd.DataFrame({'Groups': {0: 'G1', 1: 'G1', 2: 'G1', 3: 'G1', 4: 'G2', 5: 'G2', 6: 'G2', 7: 'G3', 8: 'G3', 9: 'G3', 10: 'G4', 11: 'G4', 12: 'G4', 13: 'G4'}, 'NAME': {0: 'A', 1: 'B', 2: 'C', 3: 'C', 4: 'D', 5: 'E', 6: 'D', 7: 'F', 8: 'G', 9: 'H', 10: 'I', 11: 'I', 12: 'J', 13: 'K'}, 'VALUES': {0: 1.0, 1: 2.0, 2: 3.0, 3: 3.0, 4: np.nan, 5: np.nan, 6: np.nan, 7: np.nan, 8: np.nan, 9: np.nan, 10: 8.0, 11: 8.0, 12: 89.0, 13: 65.0}})
m = df['VALUES'].isna()
grouped = df.loc[m].groupby(['Groups']) #groupby object

for group in grouped:
    print(group[0]) # str with the group name
    dfgroup = group[1] # dataframe of the group
    values = list(range(1,len(dfgroup)+1))
    dfgroup['VALUES'] = values
    print(dfgroup)

Try converting each groups' names to category type, then grab the cat codes and add 1:

import numpy as np
import pandas as pd

d = {'Groups': {0: 'G1', 1: 'G1', 2: 'G1', 3: 'G1', 4: 'G2', 5: 'G2', 6: 'G2',
                7: 'G3', 8: 'G3', 9: 'G3', 10: 'G4', 11: 'G4', 12: 'G4',
                13: 'G4'},
     'NAME': {0: 'A', 1: 'B', 2: 'C', 3: 'C', 4: 'D', 5: 'E', 6: 'D', 7: 'F',
              8: 'G', 9: 'H', 10: 'I', 11: 'I', 12: 'J', 13: 'K'},
     'VALUES': {0: 1.0, 1: 2.0, 2: 3.0, 3: 3.0, 4: np.nan, 5: np.nan,
                6: np.nan, 7: np.nan, 8: np.nan, 9: np.nan, 10: 8.0,
                11: 8.0, 12: 89.0, 13: 65.0}}

df = pd.DataFrame(d)

# Mask for Where VALUES is NaN
m = df['VALUES'].isna()
# Groupby 'Groups'
df.loc[m, 'VALUES'] = df[m].groupby('Groups', as_index=False, sort=False).apply(
    # Convert 'NAME' to a category and grab the cat codes
    # add 1 to start with 1 instead of 0
    lambda g: g['NAME'].astype('category').cat.codes + 1
).values

# Convert to int to match output
df['VALUES'] = df['VALUES'].astype(int)

print(df)

df:

   Groups NAME  VALUES
0      G1    A       1
1      G1    B       2
2      G1    C       3
3      G1    C       3
4      G2    D       1
5      G2    E       2
6      G2    D       1
7      G3    F       1
8      G3    G       2
9      G3    H       3
10     G4    I       8
11     G4    I       8
12     G4    J      89
13     G4    K      65