What is the space-time complexity of the scan operator in APL?

You are right in your algorithmic description Scan, and in the general case, it is indeed O(N²). However, by far the most common uses of it is with a known set of scalar primitives (including +, ∨, ∧, ⌈, ⌊, <, ≤, ≠). These are recognised by the interpreter, which then uses special code O(n) (or less, as they might leave early) code.

We can easily demonstrate this by comparing the performance of + with the functionally identical +∘⊢ (plus, where the right argument is pre-processed by the identity function):

      'cmpx'⎕CY'dfns'
      a←?2000⍴127
      cmpx'+∘⊢\a'
1.8E¯1
      a←?4000⍴127
      cmpx'+∘⊢\a'
7.6E¯1

We can see that +∘⊢\ took about 4 times as long (0.2 s → 0.8 s) when we doubled the number of small integers from 2000 to 4000. Whereas:

      a←?2000⍴127
      cmpx'+\a'
1.5E¯6
      a←?4000⍴127
      cmpx'+\a'
2.9E¯6

+\ only doubles (15 ms → 29 ms) when going from 2000 to 4000 small integers. Also note the extreme performance difference between the optimised case and the non-optimised case.

Scan gets its order of evaluation from Reduce. Iverson explains in Conventions Governing Order of Evaluation:

4. In the definition

      F/x ≡ x1 F x2 F x3 F ... F x⍴x

   the right-to-left convention leads to a more useful definition for nonassociative functions F than does the left-to-right convention. For example, -/x denotes the alternating sum of the components of x , whereas in a left-to-right convention it would denote the first component minus the sum of the remaining components. Thus if d is the vector of decimal digits representing the number n , then the value of the expression 0=9|+/d determines the divisibility of n by 9 ; in the right-to-left convention, the similar expression 0=11|-/d determines divisibility by 11 .