I'd like to create two list (appearantly it doesnt need to be a list, can be tuple, array, numpy.array, see edit below) of a certain partern, lets call them list_1 and list_2
list_1 has the patern that can be described as (i is an integer)
{i}, {i, i + 17}, {i, i+17 , i + 17 + 17}, {i, i + 17, i + 17 + 17, i + 17 + 17 + 17},
But the pattern occurs twice, which means that the resulting list will look like
list_1 = [i, i, i + 17, i, i + 17 , i + 34, i, i + 17, i + 34, i + 51,
i, i, i + 17, i, i + 17 , i + 34, i, i + 17, i + 34, i + 51]
Currently I'm doing it as (with i = 2 in this example)
some_limit = 5
list_1 = [17 * x + i for b in range(some_limit + 1) for x in range(b)]
list_1 += list_1
result
[2, 2, 19, 2, 19, 36, 2, 19, 36, 53, 2, 19, 36, 53, 70, 2, 2, 19, 2, 19, 36, 2, 19, 36, 53, 2, 19, 36, 53, 70]
When some_limit is a big number this takes time. Is there a faster way?
list_2 has a pattern that can be described as (j is an integer)
{j} {j+1, j+1}, {j+2, j+2, j+2}, {j+3, j+3, j+3, j+3}
This pattern also occours twice but with a shift which means that the resulting list will look like
list_2 = [j, j+1, j+1, j+2, j+2, j+2, j+3, j+3, j+3, j+3,
j+shift, j+1+shift, j+1+shift, j+2+shift, j+2+shift, j+2+shift, j+3+shift, j+3+shift, j+3+shift, j+3+shift,
Currently I'm doing it as (with j = 0 in this example)
some_limit = 5
arithemic_list = [k for k in range(some_limit + 1)]
rows_index_temp = [item -1 + some_limit * j * 2 for item, count in zip(arithemic_list, arithemic_list) for k in range(count)]
rows_index_temp += [some_limit + elem for elem in rows_index_temp]
result
[0, 1, 1, 2, 2, 2, 3, 3, 3, 3, 4, 4, 4, 4, 4, 5, 6, 6, 7, 7, 7, 8, 8, 8, 8, 9, 9, 9, 9, 9]
When some_limit is a big number this takes time. Is there a faster way?
Edit
This is used to create sparse matrices for the optimization package cvxopt. A matrix is built by providing elements with their row and column indicies. For example a 3x3 identity matrix is created by 3 lists, elements = [1,1,1], rows = [0, 1, 2] and columns = [0, 1, 2]. My matrix is very large which implies that the lists will be very large.
From the documentation
cvxopt.spmatrix(x, I, J[, size[, tc ] ]) I and J are sequences of integers (lists, tuples, array arrays, . . . ) or integer matrices (matrix objects with typecode 'i'), containing the row and column indices of the nonzero entries. The lengths of I and J must be equal. If they are matrices, they are treated as lists of indices stored in column-major order, i.e., as lists list(I), respectively, list(J)
https://readthedocs.org/projects/cvxopt/downloads/pdf/1.2.0/
Example Lets say we would like to create the following matrix:
[ 1.00e+00 0 0 0 0 ]
[ 2.00e+00 0 0 3.00e+00 0 ]
[ 0 0 0 0 4.00e+00]
We are only interested in the non-zero-elements and we can group them as
element row column
1 0 0
2 1 0
3 1 3
4 2 4
Thus, by having
elements = [1,2,3,4,0]
rows = [0,1,1,2, 3]
columns = [0,0, 3, 4, 4]
print(spmatrix(elements, rows, columns))
[ 1.00e+00 0 0 0 0 ]
[ 2.00e+00 0 0 3.00e+00 0 ]
[ 0 0 0 0 4.00e+00]
Notice that the order doesnt mather, as long as the element,row,column are grouped together. Thus this would be equivalent.
elements = [4,2,3,1]
rows = [2,1,1,0]
columns = [4,0, 3,0]
print(spmatrix(elements, rows, columns))
[ 1.00e+00 0 0 0 0 ]
[ 2.00e+00 0 0 3.00e+00 0 ]
[ 0 0 0 0 4.00e+00]
What am I trying to create? (some_limit = 3)
This is basically a concatination of two lower triangual matrix, with a constant shift between its elements. So how would we describe this matrix? If we focus on the "upper" lower triangual part which contains -1:s.
We can group as
element row column
-1 0 2
-1 1 2
-1 1 19
-1 2 2
-1 2 19
-1 2 36
The "lower" triangular part with only 1:s can be grouped as
We can group as
element row column
1 3 2
1 4 2
1 4 19
1 5 2
1 5 19
1 5 36
Together this become
element row column
-1 0 2
-1 1 2
-1 1 19
-1 2 2
-1 2 19
-1 2 36
1 3 2
1 4 2
1 4 19
1 5 2
1 5 19
1 5 36
Or,
element row column
-1 0 2
-1 1 2
-1 1 2 + 17
-1 2 2
-1 2 2 + 17
-1 2 2 + 2*17
1 1 + 2 2
1 2 + 2 2
1 2 + 2 2 + 17
1 3 + 2 2
1 3 + 2 2 + 17
1 3 + 2 2 + 2*17
Since the order doesnt mather, as long as element, row, column is grouped togethere, there might be a "better" pattern which I havent consider.
The size of this matrix is directly correlated with the number of elements in the list describing the elements, rows, and columns. I wish to have some_limit >=4343 => the lenght of each of the list elements, rows, and columns will be (4343*(4343 + 1)/2)*2 which explodes quite fast...
some_limit
4343
len(list_1)
18865992
#size_of_matrix
<34744x73831 sparse matrix, tc='d', nnz=75463969>