How to list all permutations without repetition?

Viewed 112

The current post is a follow-up question to this linked one:

Shuffle a deck of 7 hypothetical trading cards and list out the orders they can come in a Google Sheet

Surprise! My problem child is actually for Shin Megami Tensei 3 permutations -- for a video game. I thought putting it into terms of trading cards would make more sense to the layman. Forgive me.

Big Permutations... Now Handle It!!!

Okay. I messed around with a data set of 7 unique monsters (an initial set that's easy to obtain early in the video game). They can't be duplicated in my party but can be combined to make different ones. At the start, there are just 8 slots available for monsters in the video game.

This project focuses on building all the permutations of a "fusion chain" that attempts to take these monsters and arrange them into unique orders for a later combination within this chain.

It starts with A+B and then cleans that list to eliminate any B+A scenarios from the initial pairings (fusing A+B or B+A makes the same result). Then, the fusions just tack on C, D, E, F, G, and H (currently broken) to the result of the previous fusion until no more possible fusions remain (having only a single monster in my party).

The problem is this: the query or other functions within the permutation cell throw the error "The resulting array was too large" when attempting to list permutations for sorting 8 monsters at once -- even before the fusions can happen. I have isolated the issue to this formula (a bit long):

=iferror(if(counta($A$2:$A$13)>=2,arrayformula(query(query(split(flatten(flatten(flatten(flatten(flatten(flatten(
filter($F$2:$F,$F$2:$F<>"")&if(counta($A$2:$A$13)>=3,","&transpose(
filter($A$2:$A$13,$A$2:$A$13<>"")),""))&if(counta($A$2:$A$13)>=4,","&transpose(
filter($A$2:$A$13,$A$2:$A$13<>"")),""))&if(counta($A$2:$A$13)>=5,","&transpose(
filter($A$2:$A$13,$A$2:$A$13<>"")),""))&if(counta($A$2:$A$13)>=6,","&transpose(
filter($A$2:$A$13,$A$2:$A$13<>"")),""))&if(counta($A$2:$A$13)>=7,","&transpose(
filter($A$2:$A$13,$A$2:$A$13<>"")),""))&if(counta($A$2:$A$13)>=8,","&transpose(
filter($A$2:$A$13,$A$2:$A$13<>"")),"")),","),
"where Col1 <> Col2"&
if(counta($A$2:$A$13)>=3," and Col1 <> Col3 and Col2 <> Col3"&
if(counta($A$2:$A$13)>=4," and Col1 <> Col4 and Col2 <> Col4 and Col3 <> Col4"&
if(counta($A$2:$A$13)>=5," and Col1 <> Col5 and Col2 <> Col5 and Col3 <> Col5 and Col4 <> Col5"&
if(counta($A$2:$A$13)>=6," and Col1 <> Col6 and Col2 <> Col6 and Col3 <> Col6 and Col4 <> Col6 and Col5 <> Col6"&
if(counta($A$2:$A$13)>=7," and Col1 <> Col7 and Col2 <> Col7 and Col3 <> Col7 and Col4 <> Col7 and Col5 <> Col7 and Col6 <> Col7"&
if(counta($A$2:$A$13)>=8," and Col1 <> Col8 and Col2 <> Col8 and Col3 <> Col8 and Col4 <> Col8 and Col5 <> Col8 and Col6 <> Col8 and Col7 <> Col8",),),),),),),0),"where Col1 <>''",0)),"not enough data"),)

And the first range this formula was looking at is here in its previously stable form (column F):

unique init pairs
Pixie,Shikigami
Kodama,Pixie
Hua Po,Pixie
Datsue-Ba,Pixie
Angel,Pixie
Fomorian,Pixie
Kodama,Shikigami
Hua Po,Shikigami
Datsue-Ba,Shikigami
Angel,Shikigami
Fomorian,Shikigami
Hua Po,Kodama
Datsue-Ba,Kodama
Angel,Kodama
Fomorian,Kodama
Datsue-Ba,Hua Po
Angel,Hua Po
Fomorian,Hua Po
Angel,Datsue-Ba
Datsue-Ba,Fomorian
Angel,Fomorian

It was provided by a sort of "cleaner" formula I made but that isn't the problem.

The overall input I was testing is like this (in column A) and is also the input for the cleaner formulas for the initial pairs:

available
Pixie
Shikigami
Kodama
Hua Po
Datsue Ba
Angel
Fomorian
High Pixie

And the expected output... is really big. Here's a sample of the first lines to get an idea (hosted in H2 of the original sheet):

A B C D E F G H
Pixie Shikigami Kodama Hua Po Datsue Ba Angel Fomorian High Pixie
Pixie Shikigami Kodama Hua Po Datsue Ba Fomorian Angel High Pixie
Pixie Shikigami Kodama Hua Po Angel Datsue Ba Fomorian High Pixie
Pixie Shikigami Kodama Hua Po Angel Fomorian Datsue Ba High Pixie
Pixie Shikigami Kodama Hua Po Fomorian Datsue Ba Angel High Pixie
Pixie Shikigami Kodama Hua Po Fomorian Angel Datsue Ba High Pixie
Pixie Shikigami Kodama Datsue Ba Hua Po Angel Fomorian High Pixie
and so on...

I am currently at a loss for how to fix this problem. I would like to fit at least 8 starting monsters within my sheets for analysis, if not a full 12 for the end of the game.

There is probably a better, more compact way to generate these permutations than the way I have. I would probably like to boot up Excel to try this on my suped-up system and then see where it breaks offline. Yet, I want more efficient formulae to work around my "array too large" issues in Google Sheets. It's where I work best and where I have many other projects.

PS: Shout out to Erik for his help with the full sheet and for teaching me some new techniques with LAMBDA and Named Functions. It helped with eventually solving my issue. Stack's policy restricts me from sharing the full spreadsheet for good reason (concerns about bad actors and all that). So, I made edits to this post to make it better contained. Again, sorry for the mess.

1 Answers

There are different algorithms to implement this. See Permutation in computing:

The straight forward and the easiest approach is create a sequence of numbers with BASE equal to the number of items to choose from. For eg, if there are 7 items to choose from, create a sequence like this:

BASE 7(=ARRAYFORMULA(BASE(SEQUENCE(25),7,7)))
0000001
0000002
0000003
0000004
0000005
0000006
0000010
0000011
0000012
0000013
0000014
0000015
0000016
0000020
0000021
0000022
0000023
0000024
0000025
0000026
0000030
0000031
0000032
0000033
0000034
....

Notice at each position, there are 7 variables(0 to 6) and there are 7 positions. Once we get all the numbers for PERMUTATIONA(7,7), it's a simple matter of removing all the duplicates only getting numbers, where all numbers in each position are unique, i.e., COUNTUNIQUE per number = 7(eg:0124536). Here's a implementation:

=ARRAYFORMULA(LAMBDA(n,QUERY(BYROW(SPLIT(REGEXREPLACE(TO_TEXT(BASE(SEQUENCE(PERMUTATIONA(n,n)-1),n,n)),"\B","."),"."),LAMBDA(r, IF(COUNTUNIQUE(r)<>n,"",JOIN(,r)))),"where not Col1='' ",0))(5))

Unfortunately, Google arbitrarily limited execution to less than a few seconds. So, this formula is unable to get all permutations for more than n=5.

The next in the list is using factorial(Lehmer's code) to get the permutations. See permutations here. Note how there's a direct relation between a sequence of numbers and permutation.

decimal factoradic permutation
0 0:0:0! (0,1,2)
1 0:1:0! (0,2,1)
2 1:0:0! (1,0,2)
3 1:1:0! (1,2,0)
4 2:0:0! (2,0,1)
5 2:1:0! (2,1,0)

Table from https://wikipedia.org/wiki/Factorial_number_system Licensed under CC-BY-SA 3.0

I implemented this algorithm and I've hit Google's limit again at n=5. (Code not shown here).

Up next, we have Lexicographic ordering. Algorithm is as follows:

The following algorithm generates the next permutation lexicographically after a given permutation. It changes the given permutation in-place.

Find the largest index k such that a[k] < a[k + 1]. If no such index exists, the permutation is the last permutation. Find the largest index l greater than k such that a[k] < a[l]. Swap the value of a[k] with that of a[l]. Reverse the sequence from a[k + 1] up to and including the final element a[n].

For example, given the sequence [1, 2, 3, 4] (which is in increasing order), and given that the index is zero-based, the steps are as follows:

Index k = 2, because 3 is placed at an index that satisfies condition of being the largest index that is still less than a[k + 1] which is 4. Index l = 3, because 4 is the only value in the sequence that is greater than 3 in order to satisfy the condition a[k] < a[l]. The values of a[2] and a[3] are swapped to form the new sequence [1, 2, 4, 3]. The sequence after k-index a[2] to the final element is reversed. Because only one value lies after this index (the 3), the sequence remains unchanged in this instance. Thus the lexicographic successor of the initial state is permuted: [1, 2, 4, 3].

Quoted from https://en.wikipedia.org/wiki/Permutation Licensed under CC-BY-SA 3.0

Thanks to Google's latest support for recursion and named functions, I implemented this and I was able to get up to n=6(720 items) within a single formula, but I still hit the Google's recursion limit at n=7(5040 items). Having said that, it's still possible to get all the 5k permutations one by one without a array formula(and maybe even n=8(40320 items) depending on what your device can handle).

1.Pixie 2.Shikigami 3.Kodama 4.Hua Po 5.Datsue Ba 6.Angel 7.Fomorian
1.Pixie 2.Shikigami 3.Kodama 4.Hua Po 5.Datsue Ba 7.Fomorian 6.Angel
1.Pixie 2.Shikigami 3.Kodama 4.Hua Po 6.Angel 5.Datsue Ba 7.Fomorian
1.Pixie 2.Shikigami 3.Kodama 4.Hua Po 6.Angel 7.Fomorian 5.Datsue Ba
1.Pixie 2.Shikigami 3.Kodama 4.Hua Po 7.Fomorian 5.Datsue Ba 6.Angel
1.Pixie 2.Shikigami 3.Kodama 4.Hua Po 7.Fomorian 6.Angel 5.Datsue Ba
1.Pixie 2.Shikigami 3.Kodama 5.Datsue Ba 4.Hua Po 6.Angel 7.Fomorian
1.Pixie 2.Shikigami 3.Kodama 5.Datsue Ba 4.Hua Po 7.Fomorian 6.Angel
1.Pixie 2.Shikigami 3.Kodama 5.Datsue Ba 6.Angel 4.Hua Po 7.Fomorian
1.Pixie 2.Shikigami 3.Kodama 5.Datsue Ba 6.Angel 7.Fomorian 4.Hua Po
1.Pixie 2.Shikigami 3.Kodama 5.Datsue Ba 7.Fomorian 4.Hua Po 6.Angel
1.Pixie 2.Shikigami 3.Kodama 5.Datsue Ba 7.Fomorian 6.Angel 4.Hua Po
1.Pixie 2.Shikigami 3.Kodama 6.Angel 4.Hua Po 5.Datsue Ba 7.Fomorian
1.Pixie 2.Shikigami 3.Kodama 6.Angel 4.Hua Po 7.Fomorian 5.Datsue Ba
1.Pixie 2.Shikigami 3.Kodama 6.Angel 5.Datsue Ba 4.Hua Po 7.Fomorian
1.Pixie 2.Shikigami 3.Kodama 6.Angel 5.Datsue Ba 7.Fomorian 4.Hua Po
1.Pixie 2.Shikigami 3.Kodama 6.Angel 7.Fomorian 4.Hua Po 5.Datsue Ba
1.Pixie 2.Shikigami 3.Kodama 6.Angel 7.Fomorian 5.Datsue Ba 4.Hua Po
1.Pixie 2.Shikigami 3.Kodama 7.Fomorian 4.Hua Po 5.Datsue Ba 6.Angel
1.Pixie 2.Shikigami 3.Kodama 7.Fomorian 4.Hua Po 6.Angel 5.Datsue Ba
1.Pixie 2.Shikigami 3.Kodama 7.Fomorian 5.Datsue Ba 4.Hua Po 6.Angel
1.Pixie 2.Shikigami 3.Kodama 7.Fomorian 5.Datsue Ba 6.Angel 4.Hua Po
1.Pixie 2.Shikigami 3.Kodama 7.Fomorian 6.Angel 4.Hua Po 5.Datsue Ba
1.Pixie 2.Shikigami 3.Kodama 7.Fomorian 6.Angel 5.Datsue Ba 4.Hua Po

Showing the first few permutations for n=7. For the formula to work, it's important to note that there must be a inherent ascending order in the list. I added prefixes:1., 2., etc to 1.Pixie, 2.Shikigami... and so on to enforce ascending order. It is possible to the order within the formula itself, but it's not implemented.

  • A1:G1:

    1.Pixie 2.Shikigami 3.Kodama 4.Hua Po 5.Datsue Ba 6.Angel 7.Fomorian
  • A2:

    =GET_NEXT_LEX(A1:G1)
    

Drag fill or auto fill down as much as needed (40k or 5k rows). The advantage of using this method is, you can continue where you left off. If you need 2 million permutations, and Google sheets cannot handle more than 1 million. You can put the first million in one spreadsheet and continue the next million in another(all you need is the last permutation from the previous spreadsheet).

Named functions:

Create these functions

Main function:

  • GET_NEXT_LEX(arr):
=ARRAYFORMULA(
  TRANSPOSE(
    LAMBDA(arr,     
      LAMBDA(k,       
        LAMBDA(sarr,k,{SPLICE(sarr,k+1,2^999);REVERSE(SPLICE(sarr,1,k+1))})  
          (SWAP(arr,k,XMATCH(TRUE,INDEX(arr,k)<SPLICE(arr,1,k+1),,-1)+k),k)
      )(XMATCH(TRUE,POP(arr)<SHIFT(arr),,-1))
    )(TRANSPOSE(arr))
  )
)

Helper functions:

Functions similar to or

  • SPLICE(arr,i,j)
=FILTER(arr,LAMBDA(seq,(seq<i)+(seq>=j))(SEQUENCE(ROWS(arr))))
  • REVERSE(arr)
=POP(REDUCE(,arr,LAMBDA(a,c,{c;a})))
  • SWAP(arr,i,j)
=SORT(arr,LAMBDA(keys,SWITCH(keys,i,j,j,i,keys))(SEQUENCE(ROWS(arr))),1)
  • POP(arr)
=ARRAY_CONSTRAIN(arr,ROWS(arr)-1,1)
  • SHIFT(arr)
=FILTER(arr,{0;SEQUENCE(ROWS(arr)-1)})
Related