Column value will increment and reset for changes in another two columns

you can try groupby+transform with a lambda

df['count'] = df.groupby("id")['val'].transform(lambda x: x.ne(x.shift()).cumsum())

print(df)

   id  val  count
0  p1    1      1
1  p1    1      1
2  p1    2      2
3  p2    3      1
4  p2    5      2
5  p3    1      1
6  p3    2      2
7  p3    1      3

Solution based on Your initial try:

df['count'] = df.assign(dif = (df.val.diff() != 0)).groupby(['id']).dif.cumsum()

result:

   id  val  count
0  p1    1    1.0
1  p1    1    1.0
2  p1    2    2.0
3  p2    3    1.0
4  p2    5    2.0
5  p3    1    1.0
6  p3    2    2.0
7  p3    1    3.0

Taking into account comments of @ALollz, alternative solution:

df['count'] = df.assign(dif = (df.val.diff() != 0).astype(int)).groupby(['id']).dif.apply(lambda x : x.cumsum().rank(method='dense'))

Now counting begins always from 1 within groups.

I will do

df.val.diff().ne(0).cumsum().groupby(df.id).transform(lambda x : x.factorize()[0]+1)
Out[149]: 
0    1
1    1
2    2
3    1
4    2
5    1
6    2
7    3
Name: val, dtype: int32

df2 = df.groupby(['id','val']).size().reset_index() #get unique combinations
df2 = df2.groupby('id').val.cumcount()

Read more about cumcount in the docs