Copy-on-Write and varargin in MATLAB

Viewed 286

MATLAB documentation has the section Avoid Unnecessary Copies of Data in which the following statement can be found:

Copy-on-Write

If a function does not modify an input argument, MATLAB does not make a copy of the values contained in the input variable.

There is no word about varargin in that context. I tried to work out a function capable of monitoring the memory usage without success. So I'm here to ask: does the copy-on-write feature work with varargin?

Suppose the function function Y = f(x,y,z) versus the function function Y = f(varargin). In the first case, the function call f(a,b,c) will not make a copy of a, b and c (regardless the type of the variables). In the second case, the behavior of the function call f(a,b,c) is not clear. Will MATLAB point out varargin{1} to a, varargin{2} to b and varargin{3} to c without explicitly creating the cell array, or is varargin an explicit concatenation of a, b and c (and therefore the memory will store copies of the three variables inside the cell array)?

3 Answers

This is a more involved subject that it may seem at first. Partly because it isn't documented fully in MATLAB docs, and partly because the sharing mechanisms behind the scenes have changed over the years. First I will describe briefly what a MATLAB variable is. Then I will describe the various sharing mechanisms that MATLAB uses. And finally I will describe how these sharing mechanisms are used in MATLAB behind the scenes.

A MATLAB variable:

A MATLAB variable is basically a C struct called an mxArray with various fields for holding information such as size, class, storage class, and data pointers. The address of this C struct is typically known as the "Structure Address" of the variable, and the data pointers known typically as "Pr", "Pi", "Ir", "Jc", etc. For later versions of MATLAB complex data is interleaved and there is no Pi pointer. For the intrinsic numeric, logical, and char classes, the data lives directly behind the Pr and Pi data pointers (and Ir and Jc pointers for sparse variable indexing). For OOP classdef class variables, there is a proprietary structure behind the data pointers where the actual data lives and the user does not have direct access to this (a fundamental flaw IMO that limits the usefulness of OOP classdef variables in mex routines).

Variable sharing:

MATLAB shares variables in these ways:

Deep Copy: The variable in question is not sharing anything with any other variable.

Shared Data Copy: Multiple variables can have different Structure Addresses but have the same data pointers. E.g., this is what typically results from a direct whole variable assignment or a reshape of a full variable. There used to be a field in the mxArray (CrossRef) that was part of a linked list of all of these variables. Later versions of MATLAB only have a counter to tell you how many variables are part of the list, but the list itself is no longer accessible to the user.

Reference Copy: Multiple variables can have the exact same Structure Address. A field in the mxArray (refcount) indicates how many variables are sharing the same Structure Address. This is what is typically used for cell or struct variable elements.

Parent Copy: Not really a copy in and of itself like above, but in nested structs and cell arrays, variables can end up being shared with variables in other parts of the variable or other variables because of upstream sharing. There is no indication of this in the mxArray itself. I.e., the CrossRef and refcount will appear to be unshared, but sharing is actually taking place.

Handle Copy: If OOP classdef variables are derived from handle, then multiple variables are essentially shared. There will be no indication of this in the mxArray itself, and these variables do not follow the normal "copy-on-write" or "lazy-copy" rules.

When is sharing used?

This is where it gets sticky. The rules are not published, and have changed over the years. The best I can do is give examples:

-- Shared Data Copy Examples --

A = B; % direct whole variable assignment (earlier versions of MATLAB)

A{1} = B; % assigning from workspace into cell or struct (earlier versions of MATLAB)

A = reshape(B,whatever); % reshape of full variable

B{1} % cell or struct element in expression or assignment

fun(B); % function arguments are passed as shared data copies of original

A = typecast(B,'whatever'); % later versions of MATLAB only. Early versions did deep copy.

-- Reference Copy Examples --

A = B; % direct whole variable assignment (later versions of MATLAB)

A{1} = B{1}; % assignment among cell or struct elements

A = 1:5; % literal assignment of small variable can result in background reference copy

-- Parent Copy Example --

A.x = 5; B = A; % A.x is sharing with B.x through the parent A and B sharing.

The original question:

Non-Mex Function arguments are passed into functions via some type of copy mechanism. Whether it is literal variables or varargin, typically shared data copies are used (either for the explicit arguments or as the result of building the varargin cell array). The only exception I have seen to this is sometimes nested functions can pass a deep copy of a scalar variable instead of a shared data copy. So the "copy-on-write" or "lazy-copy" mechanism applies to both literal arguments and varargin arguments inside of a function, because in both cases you are actually working with shared data copies (or perhaps reference copies in later versions of MATLAB) of the originals inside the function. A caveat to this is that if you use special syntax in your function calls you can get the MATLAB parser to recognize that you are trying to modify a variable "in-place" and avoid the deep copy which would otherwise occur.

Mex Function arguments have been somewhat different. Older versions of MATLAB always used to pass in the original variable Structure Address, but later versions of MATLAB use the same rules as Non-Mex functions and pass in shared data copies (although scalars might be passed in as deep copies).

So the "copy-on-write" or "lazy-copy" mechanism in functions is really nothing special. A shared data copy or reference copy of the original variable was passed in. So if no changes are made to it, no deep copy will be made inside the function. If you do change elements of the argument variable, then a deep copy will be made first (i.e., unshared). But this is the behavior that happens at any level of MATLAB ... if you change an element of a shared variable then a deep copy must be made first. It is the same rule applied whether you are inside a function or not ... if the variable is shared and you change an element then a deep copy will be made first.

varargin is a cell array. When you place an object into a cell array the object isn't really copied but its reference count is increased:

a = [1 2 3];
b = 5;
c = {4, 6};
varargin = {a,b,c};

Here just the reference counts of objects that are pointed to by a, b and c are increased. When you do this:

varargin{1}(2) = 7;

because it wants to write to the object that is pointed to by a, it makes a copy of that array object and sets the second element of the new array to 7. The new array is placed in the first cell of varargin and the reference count of the object that is pointed to by a is decreased. However the MATLAB jit compiler may do more optimizations and it may create the variables in-place and so no cell array is created at all. Another possible optimization may be related to small objects like scalars. They are cheap objects and can be cheaply copied and they possibly don't have a reference count.

As @rahnema1 stated, MATLAB's copy-on-write mechanism (also known as lazy copying) applies to every copy, not only to function arguments.

One way to demonstrate this is using the following MEX-file modified from Yair's Undocumented MATLAB Blog:

#include "mex.h"
#include <cstdint>
void mexFunction( int /*nlhs*/, mxArray* plhs[], int nrhs, mxArray const* prhs[]) {
   if (nrhs < 1) mexErrMsgTxt("One input required.");
   plhs[0] = mxCreateNumericMatrix(1, 1, mxUINT64_CLASS, mxREAL);
   std::uint64_t* out = static_cast<std::uint64_t*>(mxGetData(plhs[0]));
   out[0] = reinterpret_cast<std::uint64_t>(mxGetData(prhs[0]));
}

You could save that as getaddr.cpp and compile with mex getaddr.cpp. You now have a function that shows you the address where data is stored for an array.

For example, if we make a copy of an array, the copy will have the same data address. When we then write to the copy, its data address will change:

>> a=zeros(5);
>> getaddr(a)
ans =
  uint64
   105553130112928
>> a(1)=1;
>> getaddr(a)
ans =
  uint64
   105553130112928
>> b=a;
>> getaddr(b)
ans =
  uint64
   105553130112928
>> b(1)=4;
>> getaddr(b)
ans =
  uint64
   105553130078944

The same can be seen for cell arrays, which is directly relevant to the question because the input arguments are collected in a cell array varargin:

>> a=zeros(5);
>> b=zeros(8);
>> v={a,b};
>> getaddr(a)
ans =
  uint64
   105553130246144
>> getaddr(v{1})
ans =
  uint64
   105553130246144

Note that a cell array is nothing more than an array whose data type is "array", and therefore can contain an array of any type as its elements. Basically it's an array of pointers to other arrays.

Related