I am trying to automatically extract analogies from a word2vec model in Python. My basic approach is as follows:
- Enumerate all of the pairs of vectors (n^2) and get their difference.
- For each difference, add it to every vector (n^3) and find the closest match to the result (n^4).
- Subtract the difference vector from the closest match and see if we get back to the original test vector, to verify that we have a genuine relationship.
There are some things that can be done to speed this up a bit; if adding a difference to a test vector produces a result that's way off of the unit hypersphere, the closest in-model vector is probably spurious, so we can skip that. And once a relationship has been found, we can skip later re-testing similar differences between all of the pairs that we already added to that relation. But it's still excruciatingly slow!
I know this brute search works in principle, as, having run it for about 12 hours, it does manage to automatically discover analogy sets like son:grandson::daughter:granddaughter and less-obvious-but-it-checks-out-when-I-google-the-words ones like scinax:oreophryne::amalda:gymnobela. But it takes between several seconds and a few minutes to check every candidate difference, and with over 4 billion vector differences in a model with a 90-ish-thousand-word vocabulary... that will take millions of hours!
So, is there any way to speed this up? Is there a non-brute-force solution to finding natural clusters of similar differences between vectors that might represent coherent analogy sets?