Prevent identical pairs when shuffling and slicing Ruby array

Viewed 433

I'd like to prevent producing pairs with the same items when producing a random set of pairs in a Ruby array.

For example:

[1,1,2,2,3,4].shuffle.each_slice(2).to_a

might produce:

[[1, 1], [3, 4], [2, 2]]

I'd like to be able to ensure that it produces a result such as:

[[4, 1], [1, 2], [3, 2]]

Thanks in advance for the help!

4 Answers
arr = [1,1,2,2,3,4]
loop do
  sliced = arr.shuffle.each_slice(2).to_a
  break sliced if sliced.none? { |a| a.reduce(:==) }
end

Here are three ways to produce the desired result (not including the approach of sampling repeatedly until a valid sample is found). The following array will be used for illustration.

arr = [1,4,1,2,3,2,1]

Use Array#combination and Array#sample

If pairs sampled were permitted to have the same number twice, the sample space would be

arr.combination(2).to_a
  #=> [[1, 4], [1, 1], [1, 2], [1, 3], [1, 2], [1, 1], [4, 1], [4, 2],
  #    [4, 3], [4, 2], [4, 1], [1, 2], [1, 3], [1, 2], [1, 1], [2, 3],
  #    [2, 2], [2, 1], [3, 2], [3, 1], [2, 1]]

The pairs containing the same value twice--here [1, 1] and [2, 2]--are not wanted so they are simple removed from the above array.

sample_space = arr.combination(2).reject { |x,y| x==y }
  #=> [[1, 4], [1, 2], [1, 3], [1, 2], [4, 1], [4, 2], [4, 3],
  #    [4, 2], [4, 1], [1, 2], [1, 3], [1, 2], [2, 3], [2, 1],
  #    [3, 2], [3, 1], [2, 1]]

We evidently are to sample arr.size/2 elements from sample_space. Depending on whether this is to be done with or without replacement we would write

sample_space.sample(arr.size/2)
  #=> [[4, 3], [1, 2], [1, 3]]

for sampling without replacement and

Array.new(arr.size/2) { sample_space.sample }
  #=> [[1, 3], [4, 1], [2, 1]]

for sampling with replacement.

Sample elements of each pair sequentially, Method 1

This method, like the next, can only be used to sample with replacement.

Let's first consider sampling a single pair. We could do that by selecting the first element of the pair randomly from arr, remove all instances of that element in arr and then sample the second element from what's left of arr.

def sample_one_pair(arr)
  first = arr.sample
  [first, second = (arr-[first]).sample]
end

To draw a sample of arr.size/2 pairs we there execute the following.

Array.new(arr.size/2) { sample_one_pair(arr) }
   #=> [[1, 2], [4, 3], [1, 2]]

Sample elements of each pair sequentially, Method 2

This method is a very fast way of sampling large numbers of pairs with replacement. Like the previous method, it cannot be used to sample without replacement.

First, compute the cdf (cumulative distribution function) for drawing an element of arr at random.

counts = arr.group_by(&:itself).transform_values { |v| v.size }
  #=> {1=>3, 4=>1, 2=>2, 3=>1}

def cdf(sz, counts)
  frac = 1.0/sz
  counts.each_with_object([]) { |(k,v),a|
    a << [k, frac * v + (a.empty? ? 0 : a.last.last)] }
end

cdf_first = cdf(arr.size, counts)
  #=> [[1, 0.429], [4, 0.571], [2, 0.857], [3, 1.0]]

This means that there is a probability of 0.429 (rounded) of randomly drawing a 1, 0.571 of drawing a 1 or a 4, 0.857 of drawing a 1, 4 or 2 and 1.0 of drawing one of the four numbers. We therefore can randomly sample a number from arr by obtaining a (pseudo-) random number between zero and one (p = rand) and then determine the first element of counts_cdf, [n, q] for which p <= q:

def draw_random(cdf)
  p = rand
  cdf.find { |n,q| p <= q }.first
end

draw_random(counts_cdf) #=> 1
draw_random(counts_cdf) #=> 4
draw_random(counts_cdf) #=> 1
draw_random(counts_cdf) #=> 1
draw_random(counts_cdf) #=> 2
draw_random(counts_cdf) #=> 3

In simulation models, incidentally, this is the standard way of generating pseudo-random variates from discrete probability distributions.

Before drawing the second random number of the pair we need to modify cdf_first to reflect that fact that the first number cannot be drawn again. Assuming there will be many pairs to generate randomly, it is most efficient to construct a hash cdf_second whose keys are the first values drawn randomly for the pair and whose values are the corresponding cdf's.

cdf_second = counts.keys.each_with_object({}) { |n, h|
  h[n] = cdf(arr.size - counts[n], counts.reject { |k,_| k==n }) }
  #=> {1=>[[4, 0.25], [2, 0.75], [3, 1.0]],
  #    4=>[[1, 0.5], [2, 0.833], [3, 1.0]],
  #    2=>[[1, 0.6], [4, 0.8], [3, 1.0]],
  #    3=>[[1, 0.5], [4, 0.667], [2, 1.0]]}

If, for example, a 2 is drawn for the first element of the pair, the probability is 0.6 of drawing a 1 for the second element, 0.8 of drawing a 1 or 4 and 1.0 of drawing a 1, 4, or 3.

We can then sample one pair as follows.

def sample_one_pair(cdf_first, cdf_second)
  first = draw_random(cdf_first)
  [first, draw_random(cdf_second[first])]
end

As before, to sample arr.size/2 values with replacement, we execute

Array.new(arr.size/2) { sample_one_pair }
  #=> [[2, 1], [3, 2], [1, 2]]

With replacement, you may get results like:

unique_pairs([1, 1, 2, 2, 3, 4]) # => [[4, 1], [1, 2], [1, 3]]

Note that 1 gets chosen three times, even though it's only in the original array twice. This is because the 1 is "replaced" each time it's chosen. In other words, it's put back into the collection to potentially be chosen again.

Here's a version of Cary's excellent sample_one_pair solution without replacement:

def unique_pairs(arr)
  dup = arr.dup

  Array.new(dup.size / 2) do
    dup.shuffle!

    first = dup.pop
    second_index = dup.rindex { |e| e != first }
    raise StopIteration unless second_index
    second = dup.delete_at(second_index)

    [first, second]
  end
rescue StopIteration
  retry
end

unique_pairs([1, 1, 2, 2, 3, 4]) # => [[4, 3], [1, 2], [2, 1]]

This works by creating a copy of the original array and deleting elements out of it as they're chosen (so they can't be chosen again). The rescue/retry is in there in case it becomes impossible to produce the correct number of pairs. For example, if [1, 3] is chosen first, and [1, 4] is chosen second, it becomes impossible to make three unique pairs because [2, 2] is all that's left; the sample space is exhausted.

This should be slower than Cary's solution (with replacement) but faster (on average) than the posted solutions (without replacement) that require looping and retrying. Welp, chalk up another point for "always benchmark!" I was wrong about all most of my assumptions. Here are the results on my machine with an array of 16 numbers ([1, 1, 2, 2, 3, 4, 5, 5, 5, 6, 7, 7, 8, 9, 9, 10]):

cary_with_replacement
                         93.737k (± 2.9%) i/s -    470.690k in   5.025734s
mwp_without_replacement
                        187.739k (± 3.3%) i/s -    943.415k in   5.030774s
mudasobwa_without_replacement
                        129.490k (± 9.4%) i/s -    653.150k in   5.096761s

EDIT: I've updated the above solution to address Stefan's numerous concerns. In hindsight, the errors are obvious and embarrassing! On the plus side, the revised solution is now faster than mudasobwa's solution, and I've confirmed that the two solutions have the same biases.

You can check if there any mathes and shuffle again:

a = [1,1,2,2,3,4]
# first time shuffle
sliced = a.shuffle.each_slice(2).to_a
# checking if there are matches and shuffle if there are
while sliced.combination(2).any? { |a, b| a.sort == b.sort } do
    sliced = a.shuffle.each_slice(2).to_a
end

It is unlikely, be aware about possibility of infinity loop

Related