If let(:thing) is supposed to be memoised per-example, why am I seeing the value when running a second example?

We have this kind of pattern in a spec:

describe Thing do
  sanity0 = '../test-data/sanity.mbox'
  let(:sanity1) {
    File.join(@scratch, 'sanity1.mbox').tap { |f|
      FileUtils.cp sanity0, f
    }
  }

  after(:example) do
    FileUtils.rm sanity1
  end

  before(:example) do
    # does stuff using sanity1
    sanity1
  end

  uses_case(proc { [sanity0, sanity1] })

  describe '#method1' do
    it 'does something' do
      # ...
    end

    it 'does another thing' do
      # ...
    end
  end
  # ... more methods ...

end

So sanity1 is used from a before(:example), initialised, and then eventually torn down in the after(:example).

The uses_case helper method there is shorthand for a before/after pair which creates more on-disk stuff and then deletes it. The list of files passed in is usually passed as a proc, because otherwise you'd be calling let stuff from the wrong place.

def uses_case(files)
  before do

    # instance_exec is here because RSpec whines if you call the proc
    # directly instead of in the context of the example.
    files = instance_exec(&files) if files.is_a?(Proc)

    # Uses files to initialise case
  end

  after do
    # cleanup stuff
  end
end

The problem is, when the second example runs, it fails:

 [java]       Java::JavaIo::FileNotFoundException:
 [java]         File does not exist: /var/folders/d5/p7yfxkhx4673nd7z7lz_t36c0000gn/T/scratch20170525-83837-1r7t2g/sanity1.mbox

So it's like the value from the first test somehow floated into the second test... but the cleanup logic was run after that first test, so the file is now gone.

If I inject output lines into the appropriate spots:

 [java] Thing
 [java]   #method1
 [java] let(:sanity1)
 [java] cleanup(:sanity1)
 [java]     does something
 [java]     does another thing (FAILED - 1)
 [java]   #method2
 [java]     does something (FAILED - 2)
 [java]     does another thing (FAILED - 3)

So clearly it really is only calling it once at the context level.

How can this be the case? If I write a standalone case which just has the examples and no other framework, it behaves in exactly the opposite way, calling both blocks for every single example.

Edit: Further reduction, I managed to get it to happen standalone:

require 'rspec'

def uses_case(files)
  before do
    files = instance_exec(&files) if files.is_a?(Proc)
  end
end

describe 'something' do
  let(:in_describe) { $stderr.puts('let(:in_describe)'); 'd' }

  uses_case(proc { [in_describe] })

  context 'when context 1' do
    it 'does something' do
      $stderr.puts('in example 1-1')
    end
    it 'does something else' do
      $stderr.puts('in example 1-2')
    end
  end
end

Output of that:

$ rspec test_spec.rb 
let(:in_describe)
in example 1-1
.in example 1-2
.

If I add more diagnostics to that, it gets weirder:

--- in before for uses_case ---
self = #<RSpec::ExampleGroups::Something::WhenContext1:0x1722011b>
__memoized = #<RSpec::Core::MemoizedHelpers::ThreadsafeMemoized:0x5b3f61ff>
             contents: {}
--- in let(:in_describe) block ---
self = #<RSpec::ExampleGroups::Something::WhenContext1:0x1722011b>
__memoized = #<RSpec::Core::MemoizedHelpers::ThreadsafeMemoized:0x5b3f61ff>
             contents: {}

after instance_exec
__memoized = #<RSpec::Core::MemoizedHelpers::ThreadsafeMemoized:0x5b3f61ff>
             contents: {:in_describe=>"d"}

in example 1-1
.--- in before for uses_case ---
self = #<RSpec::ExampleGroups::Something::WhenContext1:0x45a4b042>
__memoized = #<RSpec::Core::MemoizedHelpers::ThreadsafeMemoized:0x327af41b>
             contents: {}
after instance_exec
__memoized = #<RSpec::Core::MemoizedHelpers::ThreadsafeMemoized:0x327af41b>
             contents: {}

in example 1-2
.

So it looks like, the second time around, it is a new hash, and the hash is empty, but the let block is not called.