(GLSL) Ray Tracing in One Weekend, how to add depth of field to view matrix?

I have my shader setup to generate rays from a view and projection matrix coming from GLM, the GLSL function looks like this:

Ray Camera_getRay(inout uvec4 useed) {
    vec3 rayNDC = vec3(
        ((2.0f * fragCoord.x) / iResolution.x - 1.0f),
        (1.0f - (2.0f * fragCoord.y) / iResolution.y),
        1.0f
    );

    vec4 rayClip = vec4( rayNDC.x, rayNDC.y * -1, -1.0f, 1.0f );

    vec4 rayCamera = inverse(projection) * rayClip;
    rayCamera.z = -1.0f, rayCamera.w = 0.0f;

    vec3 direction = normalize((inverse(view) * rayCamera).xyz);

    return Ray(position, direction);
}

I am trying to add the randomized offset as described in the Defocus Blur chapter.

I have gotten this far:

float aperture = 0.1;
float lens_radius = aperture / 2;
vec3 rd = lens_radius * sampleUnitDisk(useed);

But I am not sure what to do with rd next, do I simply add it to the ray's position? do I also need to add it to the ray direction? Do I alter the view matrix? Any help is appreciated.