Why variable used in lambda expression should be final or effectively final

When an instance of a lambda expression is created, any variables in the enclosing scope that it refers are copied into it. Now, suppose if that were allowed to modify, and now you are working with a stale value which is there in that copy. On the other hand, suppose the copy is modified inside the lambda, and still the value in the enclosing scope is not updated, leaving an inconsistency. Thus, to prevent such occurrences, the language designers have imposed this restriction. It would probably have made their life easier too. A related answer for an anonymous inner class can be found here.

Another point is that you will be able to pass the lambda expression around and if it is escaped and a different thread executes it, while current thread is updating the same local variable, then there will be some concurrency issues too.

It is for the same reason the anonymous classes require the variables used in their coming out from the scope of themselves must be read-only -> final.

final int finalInt = 0;
int effectivelyFinalInt = 0;
int brokenInt = 0;
brokenInt = 0;

Supplier<Integer> supplier = new Supplier<Integer>() {
    @Override
    public Integer get() {
        return finalInt;                        // compiles
        return effectivelyFinalInt;             // compiles
        return brokenInt;                       // doesn't compile
    }
};

Lambda expressions are only shortcuts for instances implementing the interface with only one abstract method (@FunctionalInterface).

Supplier<Integer> supplier = () -> brokenInt;   // compiles
Supplier<Integer> supplier = () -> brokenInt;   // compiles
Supplier<Integer> supplier = () -> brokenInt;   // doesn't compile

I struggle to read the Java Language specification to provide support to my statements below, however, they are logical:

• Note that evaluation of a lambda expression produces an instance of a functional interface.

• Note that instantiating an interface requires implementing all its abstract methods. Doing as an expression produces an anonymous class.

• Note that an anonymous class is always an inner class.

• Each inner class can access only final or effectively-final variables outside of its scope: Accessing Members of an Enclosing Class

  In addition, a local class has access to local variables. However, a local class can only access local variables that are declared final. When a local class accesses a local variable or parameter of the enclosing block, it captures that variable or parameter.

I'd like to preface this answer by saying what I show below is not actually how lambdas are implemented. The actual implementation involves java.lang.invoke.LambdaMetafactory if I'm not mistaken. My answer makes use of some inaccuracies to better demonstrate the point.

Let's say you have the following:

public static void main(String[] args) {
  String foo = "Hello, World!";
  Runnable r = () -> System.out.println(foo);
  r.run();
}

Remember that a lambda expression is shorthand for declaring an implementation of a functional interface. The lambda body is the implementation of the single abstract method of said functional interface. At run-time an actual object is created. So the above results in an object whose class implements Runnable.

Now, the above lambda body references a local variable from the enclosing method. The instance created as a result of the lambda expression "captures" the value of that local variable. It's almost (but not really) like you have the following:

public static void main(String[] args) {
  String foo = "Hello, World!";

  final class GeneratedClass implements Runnable {
    
    private final String generatedField;

    private GeneratedClass(String generatedParam) {
      generatedField = generatedParam;
    }

    @Override
    public void run() {
      System.out.println(generatedField);
    }
  }

  Runnable r = new GeneratedClass(foo);
  r.run();
}

And now it should be easier to see the problems with supporting concurrency here:

1. Local variables are not considered "shared variables". This is stated in §17.4.1 of the Java Language Specification:

   Memory that can be shared between threads is called shared memory or heap memory.

   All instance fields, static fields, and array elements are stored in heap memory. In this chapter, we use the term variable to refer to both fields and array elements.

   Local variables (§14.4), formal method parameters (§8.4.1), and exception handler parameters (§14.20) are never shared between threads and are unaffected by the memory model.

   In other words, local variables are not covered by the concurrency rules of Java and cannot be shared between threads.

2. At a source code level you only have access to the local variable. You don't see the generated field.

I suppose Java could be designed so that modifying the local variable inside the lambda body only writes to the generated field, and modifying the local variable outside the lambda body only writes to the local variable. But as you can probably imagine that'd be confusing and counterintuitive. You'd have two variables that appear to be one variable based on the source code. And what's worse those two variables can diverge in value.

The other option is to have no generated field. But consider the following:

public static void main(String[] args) {
  String foo = "Hello, World!";
  Runnable r = () -> {
    foo = "Goodbye, World!"; // won't compile
    System.out.println(foo);
  }
  new Thread(r).start();
  System.out.println(foo);
}

What is supposed to happen here? If there is no generated field then the local variable is being modified by a second thread. But local variables cannot be shared between threads. Thus this approach is not possible, at least not without a likely non-trivial change to Java and the JVM.

So, as I understand it, the designers put in the rule that the local variable must be final or effectively final in this context in order to avoid concurrency problems and confusing developers with esoteric problems.