Build a list of array without ',' with sliding window

Question

I have an array a as follow:

import numpy as np
a= np.array([[1, 3, 5, 7, 8, 7, 1],
   [11, 13, 51, 17, 18, 17, 10]])

I want to build a list of that array with a sliding window. Here is the output that I want:

I have using the following code, however it does not provide the output which I want:

lag           = 3
out = []
for i in range(2):
    eachrow  =[]
    for col in range(a.shape[1]-lag):
        X_row = []
        xtmp = a[i, col:col+lag]
        X_row.append(xtmp)
        ytmp = a[i, col+lag]  
        X_row.append(ytmp)
        eachrow.append(X_row)
    out.append(eachrow)

Any help appreciated. Thanks.

user_1740577 · Answer

You can use numpy.lib.stride_tricks.sliding_window_view and numpy.apply_along_axis like below:

numpy.lib.stride_tricks.sliding_window_view is New in version 1.20.0. you need update your numpy

a = np.array([[1, 3, 5, 7, 8, 7, 1],[11, 13, 51, 17, 18, 17, 10]])
b = np.lib.stride_tricks.sliding_window_view(a.ravel(),4)

def create_array(row):
    return np.array([row[:3],np.array(row[-1])], dtype=object)

c = np.apply_along_axis(create_array, 1, b)
print(c)

Output:

[[array([1, 3, 5]) array(7)]
 [array([3, 5, 7]) array(8)]
 [array([5, 7, 8]) array(7)]
 [array([7, 8, 7]) array(1)]
 [array([8, 7, 1]) array(11)]
 [array([ 7,  1, 11]) array(13)]
 [array([ 1, 11, 13]) array(51)]
 [array([11, 13, 51]) array(17)]
 [array([13, 51, 17]) array(18)]
 [array([51, 17, 18]) array(17)]
 [array([17, 18, 17]) array(10)]]

user_17769815 · Answer

There are two issues I see:

The last element in the window isn't enclosed in an np.array.

Then, a quick fix would be to change this line:
```
 X_row.append(ytmp)
```
to
```
X_row.append(np.array([ytmp]))
```
which produces the desired output.
The result is two dimensions, since you create a separate sublist for each row in the array, and then append that sublist to the result. To resolve, change:
```
out.append(eachrow)
```
to
```
out.extend(eachrow)
```

Admin · Answer

You can use numpy.lib.stride_tricks.sliding_window_view for a fast, vectorized solution:

x = np.lib.stride_tricks.sliding_window_view(a, (1,3))[:, :-1]
x.shape = (*x.shape[:2], *x.shape[3:])

y = a[:, -x.shape[1]:, None]

Output:

>>> x
array([[[ 1,  3,  5],
        [ 3,  5,  7],
        [ 5,  7,  8],
        [ 7,  8,  7]],

       [[11, 13, 51],
        [13, 51, 17],
        [51, 17, 18],
        [17, 18, 17]]])

>>> y
array([[[ 7],
        [ 8],
        [ 7],
        [ 1]],

       [[17],
        [18],
        [17],
        [10]]])

Now, just use zip + list:

out = [list(zip(x[i], y[i])) for i in range(len(y))]

Output:

>>> out
[[(array([1, 3, 5]), array([7])),
  (array([3, 5, 7]), array([8])),
  (array([5, 7, 8]), array([7])),
  (array([7, 8, 7]), array([1]))],
 [(array([11, 13, 51]), array([17])),
  (array([13, 51, 17]), array([18])),
  (array([51, 17, 18]), array([17])),
  (array([17, 18, 17]), array([10]))]]

user_901925 · Answer

Your code produces:

In [3]: out
Out[3]: 
[[[array([1, 3, 5]), 7],
  [array([3, 5, 7]), 8],
  [array([5, 7, 8]), 7],
  [array([7, 8, 7]), 1]],
 [[array([11, 13, 51]), 17],
  [array([13, 51, 17]), 18],
  [array([51, 17, 18]), 17],
  [array([17, 18, 17]), 10]]]

That's a list of length 2. Within that lists.

If we make an array from that - with object dtype, we get a 3d array, where some elements are arrays, and some are integers:

In [6]: arr = np.array(out, object)
In [7]: arr
Out[7]: 
array([[[array([1, 3, 5]), 7],
        [array([3, 5, 7]), 8],
        [array([5, 7, 8]), 7],
        [array([7, 8, 7]), 1]],

       [[array([11, 13, 51]), 17],
        [array([13, 51, 17]), 18],
        [array([51, 17, 18]), 17],
        [array([17, 18, 17]), 10]]], dtype=object)
In [8]: arr.shape
Out[8]: (2, 4, 2)

Change one line of your code to

     X_row.append(np.array([ytmp]))

In [11]: np.array(out,object)
Out[11]: 
array([[[array([1, 3, 5]), array([7])],
        [array([3, 5, 7]), array([8])],
        [array([5, 7, 8]), array([7])],
        [array([7, 8, 7]), array([1])]],

       [[array([11, 13, 51]), array([17])],
        [array([13, 51, 17]), array([18])],
        [array([51, 17, 18]), array([17])],
        [array([17, 18, 17]), array([10])]]], dtype=object)

or displayed with the str/print array formatting:

In [12]: print(_)
[[[array([1, 3, 5]) array([7])]
  [array([3, 5, 7]) array([8])]
  [array([5, 7, 8]) array([7])]
  [array([7, 8, 7]) array([1])]]

 [[array([11, 13, 51]) array([17])]
  [array([13, 51, 17]) array([18])]
  [array([51, 17, 18]) array([17])]
  [array([17, 18, 17]) array([10])]]]

We could reshape that to a (8,2) array (still object dtype):

In [14]: print(Out[11].reshape(-1,2))
[[array([1, 3, 5]) array([7])]
 [array([3, 5, 7]) array([8])]
 [array([5, 7, 8]) array([7])]
 [array([7, 8, 7]) array([1])]
 [array([11, 13, 51]) array([17])]
 [array([13, 51, 17]) array([18])]
 [array([51, 17, 18]) array([17])]
 [array([17, 18, 17]) array([10])]]

Since the inner most arrays have a mixed size - some 3 some 1, the result can only be object dtype - or list of lists. That's isn't optimal for array calculations.

Commas are part of the display, along with [] and words like array. Together they give us clues as to the underlying objects, whether they are lists or arrays. Equally important are the shape and dtype (if the object is an array) or length if a list.

===

Another answer uses a striding_tricks function. Here's that method in more detail. While x is a view, slicing and reshaping will make copies, so it's hard to say whether this is any faster. For this small example I bet your code is faster, but for larger case it might not be.

In [16]: np.lib.stride_tricks.sliding_window_view(a,(1,4))
Out[16]: 
array([[[[ 1,  3,  5,  7]],

        [[ 3,  5,  7,  8]],

        [[ 5,  7,  8,  7]],

        [[ 7,  8,  7,  1]]],


       [[[11, 13, 51, 17]],

        [[13, 51, 17, 18]],

        [[51, 17, 18, 17]],

        [[17, 18, 17, 10]]]])
In [17]: x = np.lib.stride_tricks.sliding_window_view(a,(1,4))
In [18]: x.shape
Out[18]: (2, 4, 1, 4)

That's a 4d view of the original 1d array.

Your size 3 'arrays' can be sliced from that:

In [19]: x[:,:,0,:3]
Out[19]: 
array([[[ 1,  3,  5],
        [ 3,  5,  7],
        [ 5,  7,  8],
        [ 7,  8,  7]],

       [[11, 13, 51],
        [13, 51, 17],
        [51, 17, 18],
        [17, 18, 17]]])
In [20]: x[:,:,0,:3].reshape(-1,3)
Out[20]: 
array([[ 1,  3,  5],
       [ 3,  5,  7],
       [ 5,  7,  8],
       [ 7,  8,  7],
       [11, 13, 51],
       [13, 51, 17],
       [51, 17, 18],
       [17, 18, 17]])

and the 1 element column:

In [21]: x[:,:,0,-1].reshape(-1,1)
Out[21]: 
array([[ 7],
       [ 8],
       [ 7],
       [ 1],
       [17],
       [18],
       [17],
       [10]])

These 2 arrays may be more useful than your object out.

The arrays shown in [14] could be split into 2 similar arrays:

In [27]: np.stack(arr.reshape(-1,2)[:,0])
Out[27]: 
array([[ 1,  3,  5],
       [ 3,  5,  7],
       [ 5,  7,  8],
       [ 7,  8,  7],
       [11, 13, 51],
       [13, 51, 17],
       [51, 17, 18],
       [17, 18, 17]])
In [28]: arr.reshape(-1,2)[:,1].astype(int)
Out[28]: array([ 7,  8,  7,  1, 17, 18, 17, 10])

Build a list of array without ',' with sliding window

4 Answers