public interface

VideoSink

 androidx.media3.exoplayer.video.VideoSink

Gradle dependencies

compile group: 'androidx.media3', name: 'media3-exoplayer', version: '1.5.0-alpha01'

  • groupId: androidx.media3
  • artifactId: media3-exoplayer
  • version: 1.5.0-alpha01

Artifact androidx.media3:media3-exoplayer:1.5.0-alpha01 it located at Google repository (https://maven.google.com/)

Overview

A sink that consumes decoded video frames and images from video and image renderers.

Multiple renderers can feed the same sink, but not in parallel.

Summary

Fields
public static final intINPUT_TYPE_BITMAP

Input frames come from a Bitmap.

public static final intINPUT_TYPE_SURFACE

Input frames come from a surface.

Methods
public voidclearOutputSurfaceInfo()

Clears the set output surface info.

public voidenableMayRenderStartOfStream()

Enables this video sink to render the start of the stream to its output surface even if the renderer is not started yet.

public voidflush(boolean resetPosition)

Flushes the video sink.

public SurfacegetInputSurface()

Returns the input where the video sink consumes input frames from.

public booleanhandleInputBitmap(Bitmap inputBitmap, TimestampIterator timestampIterator)

Handles an input Bitmap.

public booleanhandleInputFrame(long framePresentationTimeUs, boolean isLastFrame, long positionUs, long elapsedRealtimeUs, VideoSink.VideoFrameHandler videoFrameHandler)

Handles a video input frame.

public voidinitialize(Format sourceFormat)

Initializes the video sink.

public booleanisEnded()

Returns whether all the data has been rendered to the output surface.

public booleanisInitialized()

Returns whether the video sink is initialized.

public booleanisReady(boolean rendererOtherwiseReady)

Returns whether the video sink is able to immediately render media to its output surface from the current position.

public voidjoin(boolean renderNextFrameImmediately)

Joins the video sink to a new stream.

public voidonInputStreamChanged(int inputType, Format format)

Informs the video sink that a new input stream will be queued.

public voidonRendererDisabled()

Called when the Renderer currently feeding this sink is disabled.

public voidonRendererEnabled(boolean mayRenderStartOfStream)

Called when the Renderer currently feeding this sink is enabled.

public voidonRendererStarted()

Called when the Renderer currently feeding this sink is started.

public voidonRendererStopped()

Called when the Renderer currently feeding this sink is stopped.

public voidrelease()

Releases the sink.

public voidrender(long positionUs, long elapsedRealtimeUs)

Incrementally renders processed video frames to the output surface.

public voidsetChangeFrameRateStrategy(int changeFrameRateStrategy)

Changes the used when calling .

public voidsetListener(VideoSink.Listener listener, java.util.concurrent.Executor executor)

Sets a VideoSink.Listener on this sink.

public voidsetOutputSurfaceInfo(Surface outputSurface, Size outputResolution)

Sets the output surface info.

public voidsetPendingVideoEffects(java.util.List<Effect> videoEffects)

Sets video effects to apply after the next stream change.

public voidsetPlaybackSpeed(float speed)

Sets the playback speed.

public voidsetStreamTimestampInfo(long streamStartPositionUs, long streamOffsetUs, long bufferTimestampAdjustmentUs, long lastResetPositionUs)

Sets information about the timestamps of the current input stream.

public voidsetVideoEffects(java.util.List<Effect> videoEffects)

Sets video effects to apply immediately.

public voidsetVideoFrameMetadataListener(VideoFrameMetadataListener videoFrameMetadataListener)

Sets the VideoFrameMetadataListener.

Fields

public static final int INPUT_TYPE_SURFACE

Input frames come from a surface.

public static final int INPUT_TYPE_BITMAP

Input frames come from a Bitmap.

Methods

public void onRendererEnabled(boolean mayRenderStartOfStream)

Called when the Renderer currently feeding this sink is enabled.

public void onRendererDisabled()

Called when the Renderer currently feeding this sink is disabled.

public void onRendererStarted()

Called when the Renderer currently feeding this sink is started.

public void onRendererStopped()

Called when the Renderer currently feeding this sink is stopped.

public void setListener(VideoSink.Listener listener, java.util.concurrent.Executor executor)

Sets a VideoSink.Listener on this sink. Callbacks are triggered on the supplied java.util.concurrent.Executor.

Parameters:

listener: The VideoSink.Listener.
executor: The java.util.concurrent.Executor to dispatch the callbacks.

public void initialize(Format sourceFormat)

Initializes the video sink.

Parameters:

sourceFormat: The format of the first input video or image.

public boolean isInitialized()

Returns whether the video sink is initialized.

public void flush(boolean resetPosition)

Flushes the video sink.

After calling this method, any frames stored inside the video sink are discarded.

Parameters:

resetPosition: Whether to reset the current position.

public boolean isReady(boolean rendererOtherwiseReady)

Returns whether the video sink is able to immediately render media to its output surface from the current position.

The renderer should be ready if and only if the video sink is ready.

Parameters:

rendererOtherwiseReady: Whether the renderer is ready except for the video sink.

public boolean isEnded()

Returns whether all the data has been rendered to the output surface.

public Surface getInputSurface()

Returns the input where the video sink consumes input frames from.

Must be called after the sink is initialized.

public void setVideoFrameMetadataListener(VideoFrameMetadataListener videoFrameMetadataListener)

Sets the VideoFrameMetadataListener.

public void setPlaybackSpeed(float speed)

Sets the playback speed.

public void setVideoEffects(java.util.List<Effect> videoEffects)

Sets video effects to apply immediately.

public void setPendingVideoEffects(java.util.List<Effect> videoEffects)

Sets video effects to apply after the next stream change.

public void setStreamTimestampInfo(long streamStartPositionUs, long streamOffsetUs, long bufferTimestampAdjustmentUs, long lastResetPositionUs)

Sets information about the timestamps of the current input stream.

Parameters:

streamStartPositionUs: The start position of the buffer presentation timestamps of the current stream, in microseconds.
streamOffsetUs: The offset that is added to the buffer presentation timestamps by the player, in microseconds.
bufferTimestampAdjustmentUs: The timestamp adjustment to add to the buffer presentation timestamps to convert them to frame presentation timestamps, in microseconds.
lastResetPositionUs: The renderer last reset position, in microseconds.

public void setOutputSurfaceInfo(Surface outputSurface, Size outputResolution)

Sets the output surface info.

public void clearOutputSurfaceInfo()

Clears the set output surface info.

public void setChangeFrameRateStrategy(int changeFrameRateStrategy)

Changes the used when calling .

The default value is C.VIDEO_CHANGE_FRAME_RATE_STRATEGY_ONLY_IF_SEAMLESS.

public void enableMayRenderStartOfStream()

Enables this video sink to render the start of the stream to its output surface even if the renderer is not started yet.

This is used to update the value of mayRenderStartOfStream passed to VideoSink.onRendererEnabled(boolean).

public void onInputStreamChanged(int inputType, Format format)

Informs the video sink that a new input stream will be queued.

Must be called after the sink is initialized.

Parameters:

inputType: The VideoSink.InputType of the stream.
format: The Format of the stream.

public boolean handleInputFrame(long framePresentationTimeUs, boolean isLastFrame, long positionUs, long elapsedRealtimeUs, VideoSink.VideoFrameHandler videoFrameHandler)

Handles a video input frame.

Must be called after the corresponding stream is signalled.

Parameters:

framePresentationTimeUs: The frame's presentation time, in microseconds.
isLastFrame: Whether this is the last frame of the video stream.
positionUs: The current playback position, in microseconds.
elapsedRealtimeUs: in microseconds, taken approximately at the time the playback position was positionUs.
videoFrameHandler: The VideoSink.VideoFrameHandler used to handle the input frame.

Returns:

Whether the frame was handled successfully. If false, the caller can try again later.

public boolean handleInputBitmap(Bitmap inputBitmap, TimestampIterator timestampIterator)

Handles an input Bitmap.

Must be called after the corresponding stream is signalled.

Parameters:

inputBitmap: The Bitmap to queue to the video sink.
timestampIterator: The times within the current stream that the bitmap should be shown at. The timestamps should be monotonically increasing.

Returns:

Whether the bitmap was queued successfully. If false, the caller can try again later.

public void render(long positionUs, long elapsedRealtimeUs)

Incrementally renders processed video frames to the output surface.

Parameters:

positionUs: The current playback position, in microseconds.
elapsedRealtimeUs: in microseconds, taken approximately at the time the playback position was positionUs.

public void join(boolean renderNextFrameImmediately)

Joins the video sink to a new stream.

The sink will mask VideoSink.isReady(boolean) as true for a short time to avoid interrupting an ongoing playback, even if the first frame hasn't yet been rendered to the output surface.

Parameters:

renderNextFrameImmediately: Whether the next frame should be rendered as soon as possible or only at its preferred scheduled release time.

public void release()

Releases the sink.

Source

/*
 * Copyright 2023 The Android Open Source Project
 *
 * Licensed under the Apache License, Version 2.0 (the "License");
 * you may not use this file except in compliance with the License.
 * You may obtain a copy of the License at
 *
 *      http://www.apache.org/licenses/LICENSE-2.0
 *
 * Unless required by applicable law or agreed to in writing, software
 * distributed under the License is distributed on an "AS IS" BASIS,
 * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
 * See the License for the specific language governing permissions and
 * limitations under the License.
 */
package androidx.media3.exoplayer.video;

import static java.lang.annotation.ElementType.TYPE_USE;

import android.graphics.Bitmap;
import android.os.SystemClock;
import android.view.Surface;
import androidx.annotation.FloatRange;
import androidx.annotation.IntDef;
import androidx.media3.common.C;
import androidx.media3.common.Effect;
import androidx.media3.common.Format;
import androidx.media3.common.VideoSize;
import androidx.media3.common.util.Size;
import androidx.media3.common.util.TimestampIterator;
import androidx.media3.common.util.UnstableApi;
import androidx.media3.exoplayer.Renderer;
import java.lang.annotation.Documented;
import java.lang.annotation.Retention;
import java.lang.annotation.RetentionPolicy;
import java.lang.annotation.Target;
import java.util.List;
import java.util.concurrent.Executor;

/**
 * A sink that consumes decoded video frames and images from video and image {@linkplain
 * androidx.media3.exoplayer.Renderer renderers}.
 *
 * <p>Multiple renderers can feed the same sink, but not in parallel.
 */
@UnstableApi
public interface VideoSink {

  /** Thrown by {@link VideoSink} implementations. */
  final class VideoSinkException extends Exception {
    /**
     * The {@link Format} of the frames set to the {@link VideoSink} when this exception occurred.
     */
    public final Format format;

    /** Creates a new instance. */
    public VideoSinkException(Throwable cause, Format format) {
      super(cause);
      this.format = format;
    }
  }

  /** Listener for {@link VideoSink} events. */
  interface Listener {
    /** Called when the sink renders the first frame on the output surface. */
    void onFirstFrameRendered(VideoSink videoSink);

    /** Called when the sink dropped a frame. */
    void onFrameDropped(VideoSink videoSink);

    /**
     * Called before a frame is rendered for the first time after setting the output surface, and
     * each time there's a change in the size, rotation or pixel aspect ratio of the video being
     * rendered.
     */
    void onVideoSizeChanged(VideoSink videoSink, VideoSize videoSize);

    /** Called when the {@link VideoSink} encountered an error. */
    void onError(VideoSink videoSink, VideoSinkException videoSinkException);

    /** A no-op listener implementation. */
    Listener NO_OP =
        new Listener() {
          @Override
          public void onFirstFrameRendered(VideoSink videoSink) {}

          @Override
          public void onFrameDropped(VideoSink videoSink) {}

          @Override
          public void onVideoSizeChanged(VideoSink videoSink, VideoSize videoSize) {}

          @Override
          public void onError(VideoSink videoSink, VideoSinkException videoSinkException) {}
        };
  }

  /** Handler for a video frame. */
  interface VideoFrameHandler {

    /**
     * Renders the frame on the {@linkplain #getInputSurface() input surface}.
     *
     * @param renderTimestampNs The timestamp to associate with this frame when it is sent to the
     *     surface.
     */
    void render(long renderTimestampNs);

    /** Skips the frame. */
    void skip();
  }

  /**
   * Specifies how the input frames are made available to the video sink. One of {@link
   * #INPUT_TYPE_SURFACE} or {@link #INPUT_TYPE_BITMAP}.
   */
  @Documented
  @Retention(RetentionPolicy.SOURCE)
  @Target(TYPE_USE)
  @IntDef({INPUT_TYPE_SURFACE, INPUT_TYPE_BITMAP})
  @interface InputType {}

  /** Input frames come from a {@link #getInputSurface surface}. */
  int INPUT_TYPE_SURFACE = 1;

  /** Input frames come from a {@link Bitmap}. */
  int INPUT_TYPE_BITMAP = 2;

  /** Called when the {@link Renderer} currently feeding this sink is enabled. */
  void onRendererEnabled(boolean mayRenderStartOfStream);

  /** Called when the {@link Renderer} currently feeding this sink is disabled. */
  void onRendererDisabled();

  /** Called when the {@link Renderer} currently feeding this sink is started. */
  void onRendererStarted();

  /** Called when the {@link Renderer} currently feeding this sink is stopped. */
  void onRendererStopped();

  /**
   * Sets a {@link Listener} on this sink. Callbacks are triggered on the supplied {@link Executor}.
   *
   * @param listener The {@link Listener}.
   * @param executor The {@link Executor} to dispatch the callbacks.
   */
  void setListener(Listener listener, Executor executor);

  /**
   * Initializes the video sink.
   *
   * @param sourceFormat The format of the first input video or image.
   * @throws VideoSink.VideoSinkException If initializing the sink failed.
   */
  void initialize(Format sourceFormat) throws VideoSinkException;

  /** Returns whether the video sink is {@linkplain #initialize(Format) initialized}. */
  boolean isInitialized();

  /**
   * Flushes the video sink.
   *
   * <p>After calling this method, any frames stored inside the video sink are discarded.
   *
   * @param resetPosition Whether to reset the current position.
   */
  void flush(boolean resetPosition);

  /**
   * Returns whether the video sink is able to immediately render media to its output surface from
   * the current position.
   *
   * <p>The renderer should be {@linkplain Renderer#isReady() ready} if and only if the video sink
   * is ready.
   *
   * @param rendererOtherwiseReady Whether the renderer is ready except for the video sink.
   */
  boolean isReady(boolean rendererOtherwiseReady);

  /** Returns whether all the data has been rendered to the output surface. */
  boolean isEnded();

  /**
   * Returns the input {@link Surface} where the video sink consumes input frames from.
   *
   * <p>Must be called after the sink is {@linkplain #initialize(Format) initialized}.
   */
  Surface getInputSurface();

  /** Sets the {@link VideoFrameMetadataListener}. */
  void setVideoFrameMetadataListener(VideoFrameMetadataListener videoFrameMetadataListener);

  /** Sets the playback speed. */
  void setPlaybackSpeed(@FloatRange(from = 0, fromInclusive = false) float speed);

  /** Sets {@linkplain Effect video effects} to apply immediately. */
  void setVideoEffects(List<Effect> videoEffects);

  /**
   * Sets {@linkplain Effect video effects} to apply after the next stream {@linkplain
   * VideoSink#onInputStreamChanged(int, Format) change}.
   */
  void setPendingVideoEffects(List<Effect> videoEffects);

  /**
   * Sets information about the timestamps of the current input stream.
   *
   * @param streamStartPositionUs The start position of the buffer presentation timestamps of the
   *     current stream, in microseconds.
   * @param streamOffsetUs The offset that is added to the buffer presentation timestamps by the
   *     player, in microseconds.
   * @param bufferTimestampAdjustmentUs The timestamp adjustment to add to the buffer presentation
   *     timestamps to convert them to frame presentation timestamps, in microseconds.
   * @param lastResetPositionUs The renderer last reset position, in microseconds.
   */
  void setStreamTimestampInfo(
      long streamStartPositionUs,
      long streamOffsetUs,
      long bufferTimestampAdjustmentUs,
      long lastResetPositionUs);

  /** Sets the output surface info. */
  void setOutputSurfaceInfo(Surface outputSurface, Size outputResolution);

  /** Clears the set output surface info. */
  void clearOutputSurfaceInfo();

  /**
   * Changes the {@link C.VideoChangeFrameRateStrategy} used when calling {@link
   * Surface#setFrameRate}.
   *
   * <p>The default value is {@link C#VIDEO_CHANGE_FRAME_RATE_STRATEGY_ONLY_IF_SEAMLESS}.
   */
  void setChangeFrameRateStrategy(@C.VideoChangeFrameRateStrategy int changeFrameRateStrategy);

  /**
   * Enables this video sink to render the start of the stream to its output surface even if the
   * renderer is not {@linkplain #onRendererStarted() started} yet.
   *
   * <p>This is used to update the value of {@code mayRenderStartOfStream} passed to {@link
   * #onRendererEnabled(boolean)}.
   */
  void enableMayRenderStartOfStream();

  /**
   * Informs the video sink that a new input stream will be queued.
   *
   * <p>Must be called after the sink is {@linkplain #initialize(Format) initialized}.
   *
   * @param inputType The {@link InputType} of the stream.
   * @param format The {@link Format} of the stream.
   */
  void onInputStreamChanged(@InputType int inputType, Format format);

  /**
   * Handles a video input frame.
   *
   * <p>Must be called after the corresponding stream is {@linkplain #onInputStreamChanged(int,
   * Format) signalled}.
   *
   * @param framePresentationTimeUs The frame's presentation time, in microseconds.
   * @param isLastFrame Whether this is the last frame of the video stream.
   * @param positionUs The current playback position, in microseconds.
   * @param elapsedRealtimeUs {@link SystemClock#elapsedRealtime()} in microseconds, taken
   *     approximately at the time the playback position was {@code positionUs}.
   * @param videoFrameHandler The {@link VideoFrameHandler} used to handle the input frame.
   * @return Whether the frame was handled successfully. If {@code false}, the caller can try again
   *     later.
   */
  boolean handleInputFrame(
      long framePresentationTimeUs,
      boolean isLastFrame,
      long positionUs,
      long elapsedRealtimeUs,
      VideoFrameHandler videoFrameHandler)
      throws VideoSinkException;

  /**
   * Handles an input {@link Bitmap}.
   *
   * <p>Must be called after the corresponding stream is {@linkplain #onInputStreamChanged(int,
   * Format) signalled}.
   *
   * @param inputBitmap The {@link Bitmap} to queue to the video sink.
   * @param timestampIterator The times within the current stream that the bitmap should be shown
   *     at. The timestamps should be monotonically increasing.
   * @return Whether the bitmap was queued successfully. If {@code false}, the caller can try again
   *     later.
   */
  boolean handleInputBitmap(Bitmap inputBitmap, TimestampIterator timestampIterator);

  /**
   * Incrementally renders processed video frames to the output surface.
   *
   * @param positionUs The current playback position, in microseconds.
   * @param elapsedRealtimeUs {@link android.os.SystemClock#elapsedRealtime()} in microseconds,
   *     taken approximately at the time the playback position was {@code positionUs}.
   * @throws VideoSinkException If an error occurs during rendering.
   */
  void render(long positionUs, long elapsedRealtimeUs) throws VideoSinkException;

  /**
   * Joins the video sink to a new stream.
   *
   * <p>The sink will mask {@link #isReady} as {@code true} for a short time to avoid interrupting
   * an ongoing playback, even if the first frame hasn't yet been rendered to the output surface.
   *
   * @param renderNextFrameImmediately Whether the next frame should be rendered as soon as possible
   *     or only at its preferred scheduled release time.
   */
  void join(boolean renderNextFrameImmediately);

  /** Releases the sink. */
  void release();
}