Why is field setter not called when retrieving object from cache in salesforce?

Let's say I have the following class.

public class Person {
  public String accountNumber {
    get;
    set {
      accountNumber = value;
      last4AccountNumber = value.right(4);
    }
  }
  public transient String last4AccountNumber;

  public Person() {
  }
  public Person(String accountNumber) {
    this.accountNumber = accountNumber;
  }
}

Note that last4AccountNumber is transient and is set when accountNumber is set.

When I do JSON serialize/deserialize of person object, last4AccountNumber is properly set.

Person p = new Person('999999999');

p = (Person) JSON.deserialize(JSON.serialize(p), Person.class);
System.assertEquals('9999', p.last4AccountNumber); // this passes

If I put the person object in Cache and later retrieve it (in another execution), last4AccountNumber is not set

Person p = new Person('999999999');
Cache.Org.put('local.PersonStore.999999999', p);

PersonStore is a cache partition

Person p = (Person) Cache.Org.get('local.PersonStore.999999999');
System.assertEquals('999999999', p.accountNumber); // this passes
System.assertEquals('9999', p.last4AccountNumber); // this fails meaning setter for accountNumber is not called

Executed separately

My question is why is the setter for accountNumber field not called when retrieving the person object from cache in salesforce, assuming some form of serialization to take place when caching and deserialization when retrieving from cache? Or is my assumption wrong?

I am assuming serialization to take place when caching because last4AccountNumber equals null in the retrieved person object.