Why do loops in Julia introduce their own scope

Firstly, loops in Julia only introduce a new scope of the sort that hides variables existing outside the loop (as per your complaint) if the scope outside the loop is global scope. So, for instance

function foo()
    a = 0
    # Loop does not hide existing variable `a`, will work just fine
    for i = 100
        a += i^2
    end
    return a
end

julia> foo()
10000

in other words

# Anywhere other than global scope
a = 0
for i = 100
    a += i^2
end
a == 10000 # TRUE

This is because in Julia, as in many many other languages, global scope may be considered harmful. At the very least, a for loop with global scope would encounter significant performance penalties. For instance, consider the following:

julia> a = 0
0

julia> @time for i=1:100
          # Technically this "global" keyword is superfluous since we're running this at the repl, but doesn't hurt to be explicit
          global a += rand()^2
       end
  0.000022 seconds (200 allocations: 3.125 KiB)

julia> function bar()
           a = 0
           for i=1:100
               a += rand()^2
           end
           return a
       end
bar (generic function with 1 method)

julia> @time bar()
  0.000002 seconds
33.21364180865362

Note the massive difference in allocations (the bottom version has zero) and the ~10x time difference.

Now, you may have noticed I used a special keyword global there in the global example, but since this was being run in the REPL, that doesn't actually do anything other than make it explicit what is happening.

That brings us to the other significant difference you have noticed: when run in the REPL, for loops appear not to introduce a new scope, even though the REPL is certainly global scope. This is because it turns out to be a huge pain when debugging to have to add a bunch of global qualifiers to code you have copy-pasted from somewhere deeper in your program (say within a function, where loops do not hide outside variables). So for the sake of convenience when debugging, the REPL effectively adds those global keywords for you, making the presumption that if you cared about performance you wouldn't just be pasting raw loops into the REPL, and if you are just pasting raw loops into the REPL, you're probably debugging or something.

In a script, however, it is presumed that you do care about performance, so you will get an error if you try to use a global variable within a loop without explicitly declaring it as such.

The details are substantially more complicated, as the other answer explains in more technically correct terms. Some of this complication, as far as I know, is due to a reversal on the decision of whether or not global variables should or should not be accessible by default within a loop in the REPL that happened around the time of Julia v0.7.

for loops in Julia introduce a so called local (soft) scope, see https://docs.julialang.org/en/v1/manual/variables-and-scoping/#man-scope-table.

The rules for local (soft) scope are (quoting):

If x is not already a local variable and all of the scope constructs containing the assignment are soft scopes (loops, try/catch blocks, or struct blocks), the behavior depends on whether the global variable x is defined:

1. if global x is undefined, a new local named x is created in the scope of the assignment;
2. if global x is defined, the assignment is considered ambiguous:
   • in non-interactive contexts (files, eval), an ambiguity warning is printed and a new local is created;
   • in interactive contexts (REPL, notebooks), the global variable x is assigned.

So your statement:

why doing a=1 inside the loop doesn't affect variable a outside the loop

is only true in non-interactive contexts if the for loop is not inside a hard local scope (typically if for loop is in a global scope), and the variable you assign to is defined in global scope. However, you will get a warning then.

Now the crucial part of your question is I think:

My question is why was Julia implemented with this behaviour. Is there a benefit to this from the prespective of the user?

The answer is that for loop creates a new binding for a variable that is defined within its scope. To see the consequence consider the following code (I assume that variable x is not defined in enclosing scope so that x is defined in local scope):

julia> v = []
Any[]

julia> for i in 1:2
       x = i
       push!(v, () -> x)
       end

julia> v[1]()
1

julia> v[2]()
2

Whe have created two anonymous functions and all works as you probably expected.

Now let us check what would happen in Python:

>>> v = []
>>> for i in range(1, 3):
...     x = i
...     v.append(lambda: x)
...
>>> v[0]()
2
>>> v[1]()
2

The result might surprise you. Both anonymous functions return 2. This is a consequence of not creating a local variable with a new binding in each iteration of the loop.

However, if in Julia you were working in REPL and x were defined in global scope you would get:

julia> x = 0
0

julia> v = []
Any[]

julia> for i in 1:2
       x = i
       push!(v, () -> x)
       end

julia> v[1]()
2

julia> v[2]()
2

just like in Python.

The other consideration, as explained in the other answer is performance. But most likely performance critical code is written inside a function anyway, and the discussed performance considerations are only relevant in global scope.

EDIT

This is a design choice of Matlab, quoting from https://research.wmz.ninja/articles/2017/05/closures-in-matlab.html:

When an anonymous function is created, the immediate values of the referenced local variables will be captured. Hence if any changes to the referenced local variables made after the creation of this anonymous function will not affect this anonymous function.

So as you can see in Matlab there is a difference of anonymous function vs. a closure, which does something different:

When a nested function is created, the immediate values of the referenced local variables will not be captured. When the nested function is called, it will use the current values of the referenced local variables.

In Julia there is no such difference as you can see in the examples above.

And quoting the documentation of Matlab https://www.mathworks.com/help/matlab/matlab_prog/anonymous-functions.html:

Because a, b, and c are available at the time you create parabola, the function handle includes those values. The values persist within the function handle even if you clear the variables:

(but I think it is not as explicit as the explanation I linked above)