public abstract class

Packet<T>

extends java.lang.Object

 java.lang.Object

↳androidx.camera.core.processing.Packet<T>

Gradle dependencies

compile group: 'androidx.camera', name: 'camera-core', version: '1.5.0-alpha01'

  • groupId: androidx.camera
  • artifactId: camera-core
  • version: 1.5.0-alpha01

Artifact androidx.camera:camera-core:1.5.0-alpha01 it located at Google repository (https://maven.google.com/)

Overview

Represents a single image and its metadata such as Exif and transformation info.

The metadata of this class, e.g. Packet.getSize() or Packet.getFormat() must be consistent with the image data in Packet.getData(). For example, if the is Bitmap, then Packet.getSize() must equal to Bitmapx Bitmap.

On the other hand, the metadata in Packet.getData() should be ignored. For example, when is ImageProxy. The value of Packet.getCropRect() should be used regardless of ImageProxy.getCropRect(). Similarly, if the Packet.getData() is JPEG bytes, the caller should use Packet.getExif() as the source-of-the-truth and ignore the Exif info encoded in the JPEG bytes. This enables us to modify the metadata efficiently without encoding it in the image.

Summary

Constructors
publicPacket()

Methods
public abstract CameraCaptureResultgetCameraCaptureResult()

Gets the CameraCaptureResult associated with this frame.

public abstract RectgetCropRect()

Gets the crop rect.

public abstract java.lang.ObjectgetData()

The image data object.

public abstract ExifgetExif()

The Exif info extracted from JPEG bytes.

public abstract intgetFormat()

Gets the format of the image.

public abstract intgetRotationDegrees()

Gets rotation degrees

public abstract MatrixgetSensorToBufferTransform()

Gets sensor-to-buffer transformation.

public abstract SizegetSize()

Gets the size of the image.

public booleanhasCropping()

Returns true if the Packet needs cropping.

public static Packet<Bitmap>of(Bitmap data, Exif exif, Rect cropRect, int rotationDegrees, Matrix sensorToBufferTransform, CameraCaptureResult cameraCaptureResult)

Creates Bitmap based Packet.

public static Packet<UnknownReference>of(byte[] data[], Exif exif, int format, Size size, Rect cropRect, int rotationDegrees, Matrix sensorToBufferTransform, CameraCaptureResult cameraCaptureResult)

Creates byte array based Packet.

public static Packet<ImageProxy>of(ImageProxy data, Exif exif, Rect cropRect, int rotationDegrees, Matrix sensorToBufferTransform, CameraCaptureResult cameraCaptureResult)

Creates ImageProxy based Packet.

public static Packet<ImageProxy>of(ImageProxy data, Exif exif, Size size, Rect cropRect, int rotationDegrees, Matrix sensorToBufferTransform, CameraCaptureResult cameraCaptureResult)

Creates ImageProxy based Packet with overridden image size.

from java.lang.Objectclone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructors

public Packet()

Methods

public abstract java.lang.Object getData()

The image data object.

Possible values are ImageProxy, java.nio.ByteBuffer, Bitmap etc. The metadata in the data, e.g. ImageProxy.getCropRect(), should be ignored. Instead, the caller should use the metadata of the Packet. e.g. Packet.getCropRect().

public abstract Exif getExif()

The Exif info extracted from JPEG bytes.

Returns:

null if Packet.getData() is a non-JPEG ImageProxy. Exif does not work with non-JPEG format. In that case, the exif data can be obtained via ImageInfo.populateExifData(ExifData.Builder).

public abstract int getFormat()

Gets the format of the image.

This value must match the format of the image in Packet.getData().

For Bitmap type, the value is . If the Bitmap has a gainmap, it can be converted to JPEG_R format on API level 34+

public abstract Size getSize()

Gets the size of the image.

This value must match the dimension of the image in Packet.getData().

public abstract Rect getCropRect()

Gets the crop rect.

This value is based on the coordinate system of the image in Packet.getData().

public abstract int getRotationDegrees()

Gets rotation degrees

This value is based on the coordinate system of the image in Packet.getData().

public abstract Matrix getSensorToBufferTransform()

Gets sensor-to-buffer transformation.

This value represents the transformation from sensor coordinate system to the coordinate system of the image buffer in Packet.getData().

public abstract CameraCaptureResult getCameraCaptureResult()

Gets the CameraCaptureResult associated with this frame.

public boolean hasCropping()

Returns true if the Packet needs cropping.

public static Packet<Bitmap> of(Bitmap data, Exif exif, Rect cropRect, int rotationDegrees, Matrix sensorToBufferTransform, CameraCaptureResult cameraCaptureResult)

Creates Bitmap based Packet.

public static Packet<ImageProxy> of(ImageProxy data, Exif exif, Rect cropRect, int rotationDegrees, Matrix sensorToBufferTransform, CameraCaptureResult cameraCaptureResult)

Creates ImageProxy based Packet.

public static Packet<ImageProxy> of(ImageProxy data, Exif exif, Size size, Rect cropRect, int rotationDegrees, Matrix sensorToBufferTransform, CameraCaptureResult cameraCaptureResult)

Creates ImageProxy based Packet with overridden image size.

When the image is rotated in the HAL, the size of the Image class do not match the image content. We might need to override it with the correct value. The size of the Image class always matches the size.

public static Packet<UnknownReference> of(byte[] data[], Exif exif, int format, Size size, Rect cropRect, int rotationDegrees, Matrix sensorToBufferTransform, CameraCaptureResult cameraCaptureResult)

Creates byte array based Packet.

Source

/*
 * Copyright 2022 The Android Open Source Project
 *
 * Licensed under the Apache License, Version 2.0 (the "License");
 * you may not use this file except in compliance with the License.
 * You may obtain a copy of the License at
 *
 *      http://www.apache.org/licenses/LICENSE-2.0
 *
 * Unless required by applicable law or agreed to in writing, software
 * distributed under the License is distributed on an "AS IS" BASIS,
 * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
 * See the License for the specific language governing permissions and
 * limitations under the License.
 */

package androidx.camera.core.processing;

import static androidx.camera.core.internal.utils.ImageUtil.isJpegFormats;
import static androidx.core.util.Preconditions.checkNotNull;

import android.graphics.Bitmap;
import android.graphics.ImageFormat;
import android.graphics.Matrix;
import android.graphics.Rect;
import android.media.Image;
import android.util.Size;

import androidx.annotation.NonNull;
import androidx.annotation.Nullable;
import androidx.camera.core.ImageInfo;
import androidx.camera.core.ImageProxy;
import androidx.camera.core.impl.CameraCaptureResult;
import androidx.camera.core.impl.utils.Exif;
import androidx.camera.core.impl.utils.TransformUtils;

import com.google.auto.value.AutoValue;

import java.nio.ByteBuffer;

/**
 * Represents a single image and its metadata such as Exif and transformation info.
 *
 * <p>The metadata of this class, e.g. {@link #getSize()} or {@link #getFormat()} must be
 * consistent with the image data in {@link #getData()}. For example, if the {@link T} is
 * {@link Bitmap}, then {@link #getSize()} must equal to {@link Bitmap#getWidth()}x
 * {@link Bitmap#getHeight()}.
 *
 * <p>On the other hand, the metadata in {@link #getData()} should be ignored. For example, when
 * {@link T} is {@link ImageProxy}. The value of {@link Packet#getCropRect()} should be used
 * regardless of {@link ImageProxy#getCropRect()}. Similarly, if the {@link #getData()} is JPEG
 * bytes, the caller should use {@link #getExif()} as the source-of-the-truth and ignore the Exif
 * info encoded in the JPEG bytes. This enables us to modify the metadata efficiently without
 * encoding it in the image.
 *
 * @param <T> image data type. Possible values are {@link ImageProxy}, {@link ByteBuffer},
 *            {@link Bitmap} etc.
 */
@AutoValue
public abstract class Packet<T> {

    /**
     * The image data object.
     *
     * <p>Possible values are {@link ImageProxy}, {@link ByteBuffer}, {@link Bitmap} etc. The
     * metadata in the data, e.g. {@link ImageProxy#getCropRect()}, should be ignored.
     * Instead, the caller should use the metadata of the {@link Packet}. e.g.
     * {@link #getCropRect()}.
     */
    @NonNull
    public abstract T getData();

    /**
     * The Exif info extracted from JPEG bytes.
     *
     * @return null if {@link #getData()} is a non-JPEG {@link ImageProxy}. {@link Exif} does not
     * work with non-JPEG format. In that case, the exif data can be obtained via
     * {@link ImageInfo#populateExifData}.
     */
    @Nullable
    public abstract Exif getExif();

    /**
     * Gets the format of the image.
     *
     * <p>This value must match the format of the image in {@link #getData()}.
     *
     * <p>For {@link Bitmap} type, the value is {@link ImageFormat#FLEX_RGBA_8888}. If the Bitmap
     * has a gainmap, it can be converted to JPEG_R format on API level 34+
     */
    public abstract int getFormat();

    /**
     * Gets the size of the image.
     *
     * <p>This value must match the dimension of the image in {@link #getData()}.
     */
    @NonNull
    public abstract Size getSize();

    /**
     * Gets the crop rect.
     *
     * <p>This value is based on the coordinate system of the image in {@link #getData()}.
     */
    @NonNull
    public abstract Rect getCropRect();

    /**
     * Gets rotation degrees
     *
     * <p>This value is based on the coordinate system of the image in {@link #getData()}.
     */
    public abstract int getRotationDegrees();

    /**
     * Gets sensor-to-buffer transformation.
     *
     * <p>This value represents the transformation from sensor coordinate system to the
     * coordinate system of the image buffer in {@link #getData()}.
     */
    @NonNull
    public abstract Matrix getSensorToBufferTransform();


    /**
     * Gets the {@link CameraCaptureResult} associated with this frame.
     */
    @NonNull
    public abstract CameraCaptureResult getCameraCaptureResult();

    /**
     * Returns true if the {@link Packet} needs cropping.
     */
    public boolean hasCropping() {
        return TransformUtils.hasCropping(getCropRect(), getSize());
    }

    /**
     * Creates {@link Bitmap} based {@link Packet}.
     */
    @NonNull
    public static Packet<Bitmap> of(@NonNull Bitmap data, @NonNull Exif exif,
            @NonNull Rect cropRect, int rotationDegrees, @NonNull Matrix sensorToBufferTransform,
            @NonNull CameraCaptureResult cameraCaptureResult) {
        return new AutoValue_Packet<>(data, exif, ImageFormat.FLEX_RGBA_8888,
                new Size(data.getWidth(), data.getHeight()),
                cropRect, rotationDegrees, sensorToBufferTransform, cameraCaptureResult);
    }

    /**
     * Creates {@link ImageProxy} based {@link Packet}.
     */
    @NonNull
    public static Packet<ImageProxy> of(@NonNull ImageProxy data, @Nullable Exif exif,
            @NonNull Rect cropRect, int rotationDegrees, @NonNull Matrix sensorToBufferTransform,
            @NonNull CameraCaptureResult cameraCaptureResult) {
        return of(data, exif, new Size(data.getWidth(), data.getHeight()), cropRect,
                rotationDegrees, sensorToBufferTransform, cameraCaptureResult);
    }

    /**
     * Creates {@link ImageProxy} based {@link Packet} with overridden image size.
     *
     * <p>When the image is rotated in the HAL, the size of the {@link Image} class do not
     * match the image content. We might need to override it with the correct value. The size of
     * the {@link Image} class always matches the {@link android.view.Surface} size.
     */
    @NonNull
    public static Packet<ImageProxy> of(@NonNull ImageProxy data, @Nullable Exif exif,
            @NonNull Size size, @NonNull Rect cropRect, int rotationDegrees,
            @NonNull Matrix sensorToBufferTransform,
            @NonNull CameraCaptureResult cameraCaptureResult) {
        if (isJpegFormats(data.getFormat())) {
            checkNotNull(exif, "JPEG image must have Exif.");
        }
        return new AutoValue_Packet<>(data, exif, data.getFormat(), size, cropRect, rotationDegrees,
                sensorToBufferTransform, cameraCaptureResult);
    }

    /**
     * Creates byte array based {@link Packet}.
     */
    @NonNull
    public static Packet<byte[]> of(@NonNull byte[] data, @NonNull Exif exif,
            int format, @NonNull Size size, @NonNull Rect cropRect,
            int rotationDegrees, @NonNull Matrix sensorToBufferTransform,
            @NonNull CameraCaptureResult cameraCaptureResult) {
        return new AutoValue_Packet<>(data, exif, format, size, cropRect,
                rotationDegrees, sensorToBufferTransform, cameraCaptureResult);
    }
}