How to select the first occurrence of a repeated item in sequence in a list using traditional python or using pandas/numpy/sciy

Here's one for performance using array-slicing, similar to @piRSquared's second solution but without any appending/concatenation -

a = np.array(series)
out = np.flatnonzero((a[2:] == a[1:-1]) & (a[1:-1] != a[:-2]))+1

Sample run -

In [28]: a = np.array(series)

In [29]: np.flatnonzero((a[2:] == a[1:-1]) & (a[1:-1] != a[:-2]))+1
Out[29]: array([ 5, 11, 17, 21])

Runtime test (for working solutions)

Approaches -

def piRSquared1(series):
    d = np.flatnonzero(np.diff(series) == 0)
    w = np.append(True, np.diff(d) > 1)
    return d[w].tolist()

def piRSquared2(series):
    s = np.array(series)
    return np.flatnonzero(
        np.append(s[:-1] == s[1:], True) &
        np.append(True, s[1:] != s[:-1])
    ).tolist()

def Zach(series):
    s = pd.Series(series)
    i = [g.index[0] for _, g in s.groupby((s != s.shift()).cumsum()) if len(g) > 1]
    return i

def jezrael(series):
    s = pd.Series(series)
    s1 = s.shift(1).ne(s).cumsum()
    m = ~s1.duplicated() & s1.duplicated(keep=False)
    s2 = m.index[m].tolist()
    return s2    

def divakar(series):
    a = np.array(series)
    x = a[1:-1]
    return (np.flatnonzero((a[2:] == x) & (x != a[:-2]))+1).tolist()

For the setup, we are simply tiling the sample input a number of times.

Timings -

Case #1 : Large set

In [34]: series0 = [2,3,7,10,11,16,16,9,11,12,14,16,16,16,5,7,9,17,17,4,8,18,18]

In [35]: series = np.tile(series0,10000).tolist()

In [36]: %timeit piRSquared1(series)
    ...: %timeit piRSquared2(series)
    ...: %timeit Zach(series)
    ...: %timeit jezrael(series)
    ...: %timeit divakar(series)
    ...: 
100 loops, best of 3: 8.06 ms per loop
100 loops, best of 3: 7.79 ms per loop
1 loop, best of 3: 3.88 s per loop
10 loops, best of 3: 24.3 ms per loop
100 loops, best of 3: 7.97 ms per loop

Case #2 : Much larger set (on top 2 solutions)

In [40]: series = np.tile(series0,1000000).tolist()

In [41]: %timeit piRSquared2(series)
1 loop, best of 3: 823 ms per loop

In [42]: %timeit divakar(series)
1 loop, best of 3: 823 ms per loop

Now, those two solutions differ only in the way appending is avoided in the latter one. Let's take a closer look at them and run on a smaller dataset -

In [43]: series = np.tile(series0,100).tolist()

In [44]: %timeit piRSquared2(series)
10000 loops, best of 3: 89.4 µs per loop

In [45]: %timeit divakar(series)
10000 loops, best of 3: 82.8 µs per loop

Thus, it reveals that the concatenation/append avoiding in the latter solution helps a lot when dealing with smaller datasets, but at much larger datasets, they become comparable.

Marginal improvement on larger dataset is possible with one concatenation there. Thus, the last step could be re-written as :

np.flatnonzero(np.concatenate(([False],(a[2:] == a[1:-1]) & (a[1:-1] != a[:-2]))))

You could use shift

In [3815]: s = pd.Series(series)

In [3816]: cond = (s == s.shift(-1))

In [3817]: cond.index[cond]
Out[3817]: Int64Index([5, 11, 12, 17, 21], dtype='int64')

Or, diff

In [3828]: cond = s.diff(-1).eq(0)

In [3829]: cond.index[cond]
Out[3829]: Int64Index([5, 11, 12, 17, 21], dtype='int64')

For list output use tolist

In [3833]: cond.index[cond].tolist()
Out[3833]: [5, 11, 12, 17, 21]

Details

In [3823]: s.head(10)
Out[3823]:
0     2
1     3
2     7
3    10
4    11
5    16
6    16
7     9
8    11
9    12
dtype: int64

In [3824]: cond.head(10)
Out[3824]:
0    False
1    False
2    False
3    False
4    False
5     True
6    False
7    False
8    False
9    False
dtype: bool

np.diff & np.flatnonzero
This answer uses np.diff and tests for when that difference is zero. At those points, we know we have duplication. We use np.flatnonzero to give us the positions of where those differences are zero. However, we only want the first position of consecutive differences. So we use np.diff again to filter out only the first of the series of duplicates. This time we use the results as a boolean mask.

d = np.flatnonzero(np.diff(series) == 0)
w = np.append(True, np.diff(d) > 1)
d[w]

array([ 5, 11, 17, 21])

np.flatnonzero
This is a superior answer in my opinion. We build a boolean array evaluating when a value is equal to the next but not equal to the previous. We leverage np.flatnonzero to tell us the positions of the True values.

I also find the symmetry of the answer appealing.

s = np.array(series)

np.flatnonzero(
    np.append(s[:-1] == s[1:], True) &
    np.append(True, s[1:] != s[:-1])
)

array([ 5, 11, 17, 21])

First create unique groups by shift with cumsum and then get mask for first duplicates and filter by boolean indexing:

s = pd.Series([2,3,7,10,11,16,16,9,11,12,14,16,16,16,5,7,9,17,17,4,8,18,18])

s1 = s.shift(1).ne(s).cumsum()
m = ~s1.duplicated() & s1.duplicated(keep=False)
s2 = m.index[m].tolist()
print (s2)
[5, 11, 17, 21]

print (s1)
0      1
1      2
2      3
3      4
4      5
5      6
6      6
7      7
8      8
9      9
10    10
11    11
12    11
13    11
14    12
15    13
16    14
17    15
18    15
19    16
20    17
21    18
22    18
dtype: int32

print (m)
dtype: int32
0     False
1     False
2     False
3     False
4     False
5      True
6     False
7     False
8     False
9     False
10    False
11     True
12    False
13    False
14    False
15    False
16    False
17     True
18    False
19    False
20    False
21     True
22    False
dtype: bool

Since we appear to be competing on speed, and it's not likely anyone will beat Divakar / piRsquared without cheating around the pandas/numpy/scipy requirement, here's my numba solution:

from numba import jit
import numpy as np

@jit
def rpt_idx(s):
    out = []
    j = True
    for i in range(len(s)):
        if s[i] == s[i+1]:
            if j:
                out.append(i)
                j = False
        else:
            j = True
    return out

rpt_idx(series)
Out: array([ 5, 11, 17, 21])

Probably totally overkill to pull out jit for such a trivial case, but it does give a big speedup

%timeit rpt_idx(series)
The slowest run took 10.50 times longer than the fastest. This could mean that an intermediate result is being cached.
100000 loops, best of 3: 1.99 µs per loop

%timeit divakar(series)
The slowest run took 7.73 times longer than the fastest. This could mean that an intermediate result is being cached.
100000 loops, best of 3: 12.5 µs per loop

series_ = np.tile(series,10000).tolist()

%timeit divakar(series_)
100 loops, best of 3: 20.1 ms per loop

%timeit rpt_idx(series_)
100 loops, best of 3: 5.84 ms per loop

You can imitate Python's itertools.groupby simply enough, and group adjacent duplicates together.

>>> import pandas
>>> s = pandas.Series([2, 3, 7, 10, 11, 16, 16, 9, 11, 12, 14, 16, 16, 16, 5, 7, 9, 17, 17, 4, 8, 18, 18])
>>> for _, group in s.groupby((s != s.shift()).cumsum()):
...     if len(group) > 1:
...         print(group.index[0])
5
11
17
21

Or as a list:

>>> [g.index[0] for _, g in s.groupby((s != s.shift()).cumsum()) if len(g) > 1]
[5, 11, 17, 21]