Idiomatically, should python context managers acquire resources at initialization, or when entered?

In general, I really like the explicitness of Python's context managers (CMs), but have been having trouble dealing with their inherent ambiguity. Specifically, without looking at a CM's code, it's often not possible to figure out when it will acquire its resources.

For instance, the object returned from open() is RAII - it acquires its file handle immediately upon initialization.

f = open('file.txt', 'r')  # opens file or raises exception
with f as stream:
   [...]

I've seen this pattern in many places: the resource is acquired immediately on initialization; entering the context does nothing; and exiting the context releases the resource.

Contrarily, the contextlib.contextmanager decorator creates CMs that do the opposite: they do nothing upon initialization, and defer all setup until the context is entered. In Python, this is certainly safer than RAII, because it guarantees cleanup (well, unless the caller directly calls __enter__).

@contextlib.contextmanager
def iter_lines(path):
   with open(path, 'r') as stream:
      yield stream.readlines()

line_iter = iter_lines("file.txt")  # does NOT open file, always succeeds
with line_iter as stream:  # opens file or raises exception
   [...]

Is there a preferred idiomatic way to deal with this ambiguity? I can think of a couple ideas:

1. Accept the ambiguity and hope that the docstring tells you what kind of CM you're dealing with; or
2. Only initialize CMs in a with statement, nowhere else. If you can't construct and enter the CM on the same line, create a lambda that can be called to construct the object and pass that. The 2nd example above would become:

line_iter = lambda: iter_lines("file.txt")
[...]
with line_iter() as stream:
   [...]

Is there an idiom that I'm missing - some "right way" to use context managers - that resolves this ambiguity?