When can/can't a submodule be directly referenced from a module without an import?

I'm trying to figure out when it's valid to reference a submodule like:

>>> import foo
>>> foo.bar

and when I have to explicitly import it:

>>> from foo import bar
>>> bar

This example uses scipy, but just as an example: this also happens with some modules of the base library that have a lot of submodules (like xml).

A case where I can't reference the submodule directly:

>>> import scipy
>>> scipy
<module 'scipy' from 'C:\\Users\\apocheir\\AppData\\Local\\Continuum\\anaconda3\\lib\\site-packages\\scipy\\__init__.py'>
>>> scipy.sparse
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
AttributeError: module 'scipy' has no attribute 'sparse'

A case where I can:

>>> from scipy import sparse
>>> sparse
<module 'scipy.sparse' from 'C:\\Users\\apocheir\\AppData\\Local\\Continuum\\anaconda3\\lib\\site-packages\\scipy\\sparse\\__init__.py'>
>>> sparse.csr
<module 'scipy.sparse.csr' from 'C:\\Users\\apocheir\\AppData\\Local\\Continuum\\anaconda3\\lib\\site-packages\\scipy\\sparse\\csr.py'>

It seems like it has to do with whether it's a directory with __init__.py or a single file. (I haven't tried any implicit namespace packages.) What's weird is that after that from scipy import sparse statement, the submodule DOES appear as an attribute of the module:

>>> scipy.sparse
<module 'scipy.sparse' from 'C:\\Users\\apocheir\\AppData\\Local\\Continuum\\anaconda3\\lib\\site-packages\\scipy\\sparse\\__init__.py'>

Versions: Python 3.6.3, Anaconda 5.0.1, SciPy 1.0.0.