Remove array from a list if another array with the same entry at a certain position is already in said list

Viewed 47

My question might sound a little confusing but what I am trying to do is find an efficient method to iterate over a list that contains arrays and remove arrays from the list if they have the same entries at certain positions. Let's say I have a list with 3x2 arrays and want to make the list unique regarding the last two elements in the bottom row for example. What I came up with so far is the following:

import numpy as np

my_array_list = [np.array([[1,2,3],[4,5,6]]), np.array([[9,8,7],[6,5,4]]), 
               np.array([[2,3,4],[5,6,7]]), np.array([[1,7,8],[0,5,6]])]

while i < len(my_array_list):
        j = i + 1
        while j < len(my_array_list):
            if my_array_list[i][1,1] == my_array_list[j][1,1] and my_array_list[i][1,2] == my_array_list[j][1,2]:
                del my_array_list[j]
            else:
                j += 1
        i += 1 
print(my_array_list)
>>> my_array_list = [np.array([[1,2,3],[4,5,6]]), np.array([[9,8,7],[6,5,4]]), 
              np.array([[2,3,4],[5,6,7]])] ## Since 5,6 is already in the last two colums of the first array the last array got deleted

This loop does what I want but the problem is that it is very slow. The data I am having will be generated from a Monte Carlo Simulation thus there will most likely be millions of arrays in my list. I was wondering if there was a faster way to do this like to somehow remember which combinations have already been encountered so I only have to loop over the list once and not len(my_array_list)-times for every single combination.

Thanks in advance, any help would be appreciated. Cheers!

0 Answers
Related