PhotoCaputure causes memory leak in Unity

Question

I am using unity's Photocapture Object to take videos with hololens. I wrote the code based on the official sample code, but I changed the process in the update function to take multiple images instead of just one.

https://docs.unity3d.com/2018.4/Documentation/Manual/windowsholographic-photocapture.html

However, when I run this code, it eventually runs out of pagefile.sys, goes out of memory, and aborts. I've searched and found that most of the memory leaks are caused by texture 2d, but in this code, they occur even though I omit them. Furthermore, the unity profiler does not show any gradual increase in memory usage after the code is executed. What could be the cause? I would appreciate it if you could tell me.

enter image description here

using UnityEngine;
using System;
using System.Linq;
using UnityEngine.XR;
using UnityEngine.Windows.WebCam;
using System.Threading.Tasks;
using UnityEngine.Networking;
using System.Text;
using System.IO;
using System.Net;
using System.Collections;
using System.Collections.Generic;

public class photocap : MonoBehaviour
{
    PhotoCapture PhotoCapture = null;
    Texture2D targetTexture = null;
    Resolution cameraResolution;
    Renderer quadRenderer;
    float dt = 0;



    void Start()
    {
        cameraResolution = PhotoCapture.SupportedResolutions.OrderByDescending((res) => res.width * res.height).First();
        GameObject quad = GameObject.CreatePrimitive(PrimitiveType.Quad);
        quad.transform.localScale = new Vector3(0.2f, 0.2f, 0.2f);
        quadRenderer = quad.GetComponent() as Renderer;
        quadRenderer.material = new Material(Shader.Find("Unlit/Texture"));


        quad.transform.parent = this.transform;
        quad.transform.localPosition = new Vector3(0.0f, 0.0f, 0.3f);

    }


    async void StartCapture()
    {


        PhotoCapture.CreateAsync(false, delegate (PhotoCapture captureObject)
        {
            PhotoCapture = captureObject;
            CameraParameters cameraParameters = new CameraParameters();
            cameraParameters.hologramOpacity = 0.0f;
            cameraParameters.cameraResolutionWidth = cameraResolution.width;
            cameraParameters.cameraResolutionHeight = cameraResolution.height;
            cameraParameters.pixelFormat = CapturePixelFormat.BGRA32;


            PhotoCapture.StartPhotoModeAsync(cameraParameters, delegate (PhotoCapture.PhotoCaptureResult result)
            {

                PhotoCapture.TakePhotoAsync(OnCapturedPhotoToMemory);
            });
        });
    }

    void OnCapturedPhotoToMemory(PhotoCapture.PhotoCaptureResult result, PhotoCaptureFrame photoCaptureFrame)
    {



        PhotoCapture.StopPhotoModeAsync(OnStoppedPhotoMode);


    }

    void OnStoppedPhotoMode(PhotoCapture.PhotoCaptureResult result)
    {

        PhotoCapture.Dispose();
        PhotoCapture = null;
    }
    void Update()
    {
        dt += Time.deltaTime;

        if (dt > 3)
        {
            dt = 0.0f;
            StartCapture();
        }

    }



}

user_7111561 · Answer

I'm pretty sure you also want/have to dispose the photoCaptureFrame!

otherwise the there stored texture(s) stay forever in your application memory.

void OnCapturedPhotoToMemory(PhotoCapture.PhotoCaptureResult result, PhotoCaptureFrame photoCaptureFrame)
{
    // TODO: Obviously you first would want to do something with the just captured image ...

    photoCaptureFrame.Dispose();

    PhotoCapture.StopPhotoModeAsync(OnStoppedPhotoMode);
}

Oh and another thing:

You are currently starting new captures based on the time passed.

What happens if taking the picture, processing the data and diposing takes longer than that interval?

You probably rather want to trigger a new capture delay only after the first capture actually was fully completed!

And actually why even create, start, stop and dispose the capture all the time? You could just stick to one and always reuse it

private PhotoCapture _photoCapture = null;
private Texture2D targetTexture = null;
private Resolution cameraResolution;
private Renderer quadRenderer;
private float dt = 0;

private void Start()
{
    cameraResolution = PhotoCapture.SupportedResolutions.OrderByDescending((res) => res.width * res.height).First();
    var quad = GameObject.CreatePrimitive(PrimitiveType.Quad);
    quad.transform.localScale = new Vector3(0.2f, 0.2f, 0.2f);
    quadRenderer = quad.GetComponent();
    quadRenderer.material = new Material(Shader.Find("Unlit/Texture"));

    targetTexture = new Texture2D(cameraResolution.width, cameraResolution.height);

    quadRenderer.material.mainTexture = targetTexture;

    quad.transform.parent = transform;
    quad.transform.localPosition = new Vector3(0.0f, 0.0f, 0.3f);

    StartCapture();
}

private void StartCapture()
{
    PhotoCapture.CreateAsync(false, OnPhotoCaptureCreated);
}

private void OnPhotoCaptureCreated(PhotoCapture captureObject)
{
    _photoCapture = captureObject;
    var cameraParameters = new CameraParameters
    {
        hologramOpacity = 0.0f,
        cameraResolutionWidth = cameraResolution.width,
        cameraResolutionHeight = cameraResolution.height,
        pixelFormat = CapturePixelFormat.BGRA32
    };

    _photoCapture.StartPhotoModeAsync(cameraParameters, OnPhotoCaptureStarted);
}

private void OnPhotoCaptureStarted(PhotoCapture.PhotoCaptureResult result)
{
    // Take the first picture
    TakePhoto();
    // or if you really want more delay
    //Invoke(nameof(TakePhoto, 3f));
}

private void TakePhoto()
{
    _photoCapture.TakePhotoAsync(OnCapturedPhotoToMemory);
}

private void OnCapturedPhotoToMemory(PhotoCapture.PhotoCaptureResult result, PhotoCaptureFrame photoCaptureFrame)
{
    // Don't all the time destroy and create textures
    // simply upload the new data to the already existing image
    photoCaptureFrame.UploadImageDataToTexture(targetTexture);

    Debug.Log("Captured");

    photoCaptureFrame.Dispose();

    // without the overhead of stopping and disposing etc simply take the next image
    TakePhoto();
    // or if you really want more delay
    //Invoke(nameof(TakePhoto, 3f));
}

PhotoCaputure causes memory leak in Unity

1 Answers