Android Camera2 why do the proportions of my preview change after rotation followed by rotation back?

user_15394 0
Viewed 887

In my app I have a camera preview on a SurfaceView and it is working almost exactly as I want it to, with one weirdly annoying problem:

When I enter camera view on my Sony Xperia or my testing Samsung S7 ( the emulator seems fine ) it shows the preview correctly. When I rotate it, the preview rotates correctly, when I rotate it back, the preview loses it's proportions and the view is distorted. After the first rotation, every time it rotates back I have that same distortion.

The approach that works so far is that I use scaling of my SurfaceView:

    FrameLayout.LayoutParams newScale = new FrameLayout.LayoutParams(width, height, Gravity.CENTER );
    mTextureView.setLayoutParams(newScale);
    mTextureView.setScaleX( xScale);
    mTextureView.setScaleY( yScale);

Then I create the captureSession:

  mCaptureSession = cameraCaptureSession;
            try {

                mPreviewRequestBuilder.set(CaptureRequest.CONTROL_AF_MODE,
                        CaptureRequest.CONTROL_AF_MODE_CONTINUOUS_PICTURE);

               mPreviewRequestBuilder.set(CaptureRequest.SCALER_CROP_REGION, cropRectangle);
                mPreviewRequest = mPreviewRequestBuilder.build();
                mCaptureSession.setRepeatingRequest(mPreviewRequest, getCaptureCallback(), mBackgroundHandler);
            } catch (CameraAccessException e) { 
                // and so on.
            }

Obviously by this point I have got pretty frustrated, so I have been logging the heck out of my rotations and this is where I am getting super-confused.

This looks fine:

========================
View Dimensions (720, 1184)
Sensor Dimensions: (3984, 5512)
Final size: 720, 1280
========================
View ratio: 1.7777778 previewRatio: 1.6444445 scale: 1.0
Image preview size is (720, 1184) scale is: (1.0) image size is (720, 1280)
Image scale is (1.3835341, 0.925) Max Image is (3984, 5512) cropRectangle is (441, 0 -> 3543, 5512)

Then I rotate it:

========================
View Dimensions (1184, 720)
Sensor Dimensions: (5512, 3984)
Final size: 1280, 720
========================
View ratio: 0.5625 previewRatio: 0.6081081 scale: 1.0
Image preview size is (1184, 720) scale is: (1.0) view size is (1280, 720)
Image scale is (0.925, 1.3835342) Max Image is (5512, 3984) cropRectangle is (0, 441 -> 5512, 3543)

Now I rotate it back:

========================
View Dimensions (720, 1184)
Sensor Dimensions: (3984, 5512)
Final size: 720, 1280
========================
View ratio: 1.7777778 previewRatio: 1.6444445 scale: 1.0
Image preview size is (720, 1184) scale is: (1.0) view size is (720, 1280)
Image scale is (1.3835341, 0.925) Max Image is (3984, 5512) cropRectangle is (441, 0 -> 3543, 5512)

Those scales and that crop rectangle look absolutely identical to me, but for some reason the last image is distorted way wider, as though the x-scale was closer to 1.8 or similar. I can rotate back to landscape in either direction and it looks fine, then turn back to portrait and everything is fat again.

I'm wondering whether this is a characteristic of the camera trying to auto-correct but I'm getting short on ideas about how to troubleshoot this further. How can I persuade my phone that having been rotated twice it should show the same proportions that it showed me in the first place?

Update: If I use setXScale and set it to 1.0 on the second run instead of the value it had on the first, it looks correct. But when I compare the transformation matrix of the SurfaceView when the image proportions are bad and when they are good, they are identical, so this looks like a compensation for the bad scaling in the preview. I can change the way the scaling behaves so the initial preview is too narrow and the proportions are correct after rotation, but again this doesn't really help because I need to be able to swap between portrait and landscape without the image being warped by it, which seems as though it should be the default behaviour. Opening the view in Landscape behaves in much the same way - distorted when it is first opened, then fine.

The CaptureSession appears to be re-created correctly each time.

Second Update: Calling dumpsys SurfaceFlinger shows a possible cause: 

-------------------------------------------------------------------------------
Layer name
           Z |  Comp Type |   Disp Frame (LTRB) |          Source Crop (LTRB)
-------------------------------------------------------------------------------
SurfaceView - com.myapp.CameraActivity#0   4294967294 |     Device |    0    0  720 1184 |    0.0    0.0  960.0  720.0
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -

So here it looks as though the problem is that the source crop is coming in way too small for the size of the display frame, hence the stretched image. My device believes it is getting an image at 1280x720 but for some reason the Source Crop is indicating a smaller size. Could this be to do with the proportions of the sensor? Do I need to be picking from only the sizes that match the sensor ratio?

android-camera2 android
2 Answers

Use Custom Texture View as compared to FrameLayout and theen access it through Surface it will solve your issues.

 private void createCameraPreviewSession() {
        try {
            SurfaceTexture texture = mTextureView.getSurfaceTexture();
            assert texture != null;
            texture.setDefaultBufferSize(mPreviewSize.getWidth(), mPreviewSize.getHeight());


            Surface surface = new Surface(texture);


            mPreviewRequestBuilder
                    = mCameraDevice.createCaptureRequest(CameraDevice.TEMPLATE_PREVIEW);
            mPreviewRequestBuilder.addTarget(surface);


            mCameraDevice.createCaptureSession(Arrays.asList(surface, mImageReader.getSurface()),
                    new CameraCaptureSession.StateCallback() {

                        @Override
                        public void onConfigured(@NonNull CameraCaptureSession cameraCaptureSession) {
                            // The camera is already closed
                            if (null == mCameraDevice) {
                                return;
                            }

                            mCaptureSession = cameraCaptureSession;
                            try {
                               mPreviewRequestBuilder.set(CaptureRequest.CONTROL_AF_MODE,
                                        CaptureRequest.CONTROL_AF_MODE_CONTINUOUS_PICTURE);

                                setAutoFlash(mPreviewRequestBuilder);
                                mPreviewRequest = mPreviewRequestBuilder.build();
                                mCaptureSession.setRepeatingRequest(mPreviewRequest,
                                        mCaptureCallback, mBackgroundHandler);
                            } catch (CameraAccessException e) {
                                e.printStackTrace();
                            }
                        }

                        @Override
                        public void onConfigureFailed(
                                @NonNull CameraCaptureSession cameraCaptureSession) {
                            showToast("Failed");
                        }
                    }, null
            );
        } catch (CameraAccessException e) {
            e.printStackTrace();
        }
}

And retrieve rotation from specific device base on camera outputs

 /**
     * Retrieves the JPEG orientation from the specified screen rotation.
     *
     * @param rotation The screen rotation.
     * @return The JPEG orientation (one of 0, 90, 270, and 360)
     */
    private int getOrientation(int rotation) {
        return (ORIENTATIONS.get(rotation) + mSensorOrientation + 270) % 360;
}
user_62801560

This is how you can design AutoTextureView and call that in XML files

public class AutoFitTextureView extends TextureView {

    private int mRatioWidth = 0;
    private int mRatioHeight = 0;

    public AutoFitTextureView(Context context) {
        this(context, null);
    }

    public AutoFitTextureView(Context context, AttributeSet attrs) {
        this(context, attrs, 0);
    }

    public AutoFitTextureView(Context context, AttributeSet attrs, int defStyle) {
        super(context, attrs, defStyle);
    }

    public void setAspectRatio(int width, int height) {
        if (width < 0 || height < 0) {
            throw new IllegalArgumentException("Size cannot be negative.");
        }
        mRatioWidth = width;
        mRatioHeight = height;
        requestLayout();
    }

    @Override
    protected void onMeasure(int widthMeasureSpec, int heightMeasureSpec) {
        super.onMeasure(widthMeasureSpec, heightMeasureSpec);
        int width = MeasureSpec.getSize(widthMeasureSpec);
        int height = MeasureSpec.getSize(heightMeasureSpec);
        if (0 == mRatioWidth || 0 == mRatioHeight) {
            setMeasuredDimension(width, height);
        } else {
            if (width < height * mRatioWidth / mRatioHeight) {
                setMeasuredDimension(width, width * mRatioHeight / mRatioWidth);
            } else {
                setMeasuredDimension(height * mRatioWidth / mRatioHeight, height);
            }
        }
    }

}

and then set the listener to track your textureview

private final TextureView.SurfaceTextureListener mSurfaceTextureListener
            = new TextureView.SurfaceTextureListener() {

        @Override
        public void onSurfaceTextureAvailable(SurfaceTexture texture, int width, int height) {
            openCamera(width, height);
        }

        @Override
        public void onSurfaceTextureSizeChanged(SurfaceTexture texture, int width, int height) {
            configureTransform(width, height);
        }

        @Override
        public boolean onSurfaceTextureDestroyed(SurfaceTexture texture) {
            return true;
        }

        @Override
        public void onSurfaceTextureUpdated(SurfaceTexture texture) {
        }

    };
user_62801560
Related