Bash control flow using || on function, with set -e

If I put set -e in a Bash script, the script will exit on future errors. I'm confused about how this works with functions. Consider the following, which will only print one to standard out:

set -e # Exit on error
fun(){
    echo one
    non_existing_command
    echo two
}
fun

Clearly, the non_existing_command is an error and so the script exits before the second echo. Usually one can use the or operator || to run another command if and only if the first command fails. That is, I would suspect the following to print out both one and three, but not two:

set -e # Exit on error
fun(){
    echo one
    non_existing_command
    echo two
}
fun || echo three

What I get however is one and two. That is, the || operator prevents the exit (as it should) but it chooses to continue with the function body and disregard the right-hand command.

Any explanation?