How can I remove the item in list using the comparison between list and index in pandas?

Use filtering in groupby+apply:

idx = set(all_cites_dog['Dog_Number'])
all_cites_dog = (all_cites_dog.groupby('Dog_Number')['Cites_Dogs']
                             .apply(lambda x: list([y for y in x if y in idx])))

print (all_cites_dog)
Dog_Number
DOG123    [DOG127]
DOG126          []
DOG127    [DOG123]
Name: Cites_Dogs, dtype: object

For better performance first filter by boolean indexing and isin and then groupby, last add missing not matched empty values:

s = (all_cites_dog[all_cites_dog['Cites_Dogs'].isin(all_cites_dog['Dog_Number'].unique())]
             .groupby('Dog_Number')['Cites_Dogs']
             .apply(list))

idx = np.setdiff1d(all_cites_dog['Dog_Number'].unique(), s.index)
s1 = pd.Series([[]] * len(idx), index=idx)
print (s1)
DOG126    []
dtype: object

s = s.append(s1).sort_index()
print (s)
DOG123    [DOG127]
DOG126          []
DOG127    [DOG123]
dtype: object

You can use apply and use a list comprehension to keep the elements in the index:

l = all_cites_dog.index
all_cites_dog.apply(lambda x: [i for i in x if i in l])

Dog_Number
DOG123    [DOG127]
DOG126          []
DOG127    [DOG123]
Name: Cites_Dogs, dtype: object

You can filter on an isin check.

(df.set_index('Dog_Number')
   .query("Cites_Dogs in index")
   .reindex(df.Dog_Number.unique()))

           Cites_Dogs
Dog_Number           
DOG123         DOG127
DOG126            NaN
DOG127         DOG123

If further reduction is needed, you can chain groupby.

(df.set_index('Dog_Number')
   .query("Cites_Dogs in index")
   .reindex(df.Dog_Number.unique())
   .groupby(level=0)['Cites_Dogs']
   .apply(pd.Series.tolist))

Dog_Number
DOG123    [DOG127]
DOG126       [nan]
DOG127    [DOG123]
Name: Cites_Dogs, dtype: object

Another option is groupby and apply with a set membership check.

s = set(df.Dog_Number)
df.groupby('Dog_Number').Cites_Dogs.apply(lambda x: x[x.isin(s)].tolist())

Dog_Number
DOG123    [DOG127]
DOG126          []
DOG127    [DOG123]
Name: Cites_Dogs, dtype: object

You can follow the broad steps:

1. Filter your dataframe according to Cites_Dogs.
2. Perform your groupby + apply with list.
3. Reindex your dataframe as per unique dog numbers.
4. Replace NaN values with empty lists for consistency.

Here's a demonstration:

unq_dogs = df['Dog_Number'].unique()

res = df.loc[df['Cites_Dogs'].isin(unq_dogs]\
        .groupby('Dog_Number')['Cites_Dogs'].apply(list)\
        .reindex(unq_dogs)\
        .fillna(pd.Series([[] for _ in range(len(unq_dogs))], index=unq_dogs))\
        .reset_index()

print(res)

  Dog_Number Cites_Dogs
0     DOG123   [DOG127]
1     DOG126         []
2     DOG127   [DOG123]

Try if this works just one liner solution:

df = pd.DataFrame({'Cites_Dogs':  ['DOG45555' ,'DOG127' , 'DOG7760' ,'DOG45','DOG559','DOG760','DOG123','DOG789','DOG860'],
               'Dog_Number': ['DOG123', 'DOG123', 'DOG126', 'DOG126', 'DOG126', 'DOG126', 'DOG127', 'DOG127', 'DOG127']})
a = ['DOG123', 'DOG126', 'DOG127']

df['Cites_Dogs'][~df['Cites_Dogs'].isin(a)] = np.nan

df.replace([np.nan], '', inplace=True)

df = df.groupby('Dog_Number')['Cites_Dogs'].apply(list)

# and output looks like this
Dog_Number
DOG123      [, DOG127]
DOG126        [, , , ]
DOG127    [DOG123, , ]
Name: Cites_Dogs, dtype: object

Thanks!