Why doesn't the string change when modifying string array element

Strings are immutable. That means their value never changes. Their references can get reassigned, which is what happens here.

A timeline in the comments:

// Create two Strings. [string1]="abc", [string2]="def"
// Assign [string1] to arr[0]. 
// Assign [string2] to arr[1].
String[] arr={"abc","def"};

// Get value from arr[1] = [string2]
// Assign [string2] to s
String s=arr[1];

// Get value from arr[1] = [string2]
// create [string3] = "123r"
// create [string4] = [string2] + [string3]
// assign [string4] to arr[1]
arr[1]+="123r";

// get value from arr[0] = [string1]
// print [string1] = "abc"
// get value from arr[1] = [string4]
// print [string4] = "def123r"

for (String i:arr) {System.out.print(i);}

System.out.println(); // prints "abcdef123r"

// get value from s = [string2]
// print value "def"
System.out.println(s); //prints "def"

I say reference since from what I understand Java is pass by value and a memory reference is that "value"

Almost correct.
A reference is an address. The value can be found in the memory at that adress.

So what you have is:
Variable = human readable. apple.
reference = memory address. Computer readable.
value = some bytes in memory at a given address.

So when you do

 String s = arr[1];

You are assigning the memory address that was linked to the variable arr[1] to the variable s. The value in RAM is untouched and doesn't change.

When you do

 arr[1] += "123r";

You are creating a whole new String.
Expanded this is what happens step for step.

arr[1] += "123r";
arr[1] = arr[1] + "123r"; 
arr[1] = "def" + "123r";
arr[1] = "def123r";

So arr[1] gets assigned the memory address/reference of the result of the operation after the =
This action however has no relation at all to the variable s, as this variable holds the address of the original String, and there is no code to update that reference/memory address.

In a nutshell, the behaviour you are experiencing is expected. In Java, Strings are designed to be immutable, that is, their values CANNOT change.

What does String immutability mean?

Study the code snippet below and try to predict what the output would be without running the code.

 String s = "Stack";  
 s.concat(" overflow"); 
 System.out.println(s);

What do you expect the output to be? Stack overflow right? NOPE! the output is Stack. You can test and see for yourself.

This is exactly how a String behaves.

Of course, we can modify the codebase above to work the way we want by doing this:

 String s = "Stack";  
 s = s.concat(" overflow"); 
 System.out.println(s);

This way, a new object is created and the output becomes Stack overflow.

I hope this helps. Merry coding! Edited

we know，string is const variable in java.

1. String[] arr={"abc","def"}

   const string variable pool: "abc", "def"

   arr[0] -> "abc"

   arr[1] -> "def"

2. String s=arr[1]

   const string variable pool: "abc", "def"

   arr[0] -> "abc"

   arr[1] -> "def"

   s -> "def"

3. arr[1]+="123r"

   const string variable pool: "abc", "def", "def123r", "123r"

   arr[0] -> "abc"

   arr[1] -> "def123r"

   s -> "def"