Configuration

As of configuration stage, and throughout processing, GstAudioDecoder provides various (context) parameters, e.g. describing the format of output audio data (valid when output caps have been set) or current parsing state. Conversely, subclass can and should configure context to inform base class of its expectation w.r.t. buffer handling.

Data processing

Shutdown phase

Subclass is responsible for providing pad template caps for source and sink pads. The pads need to be named "sink" and "src". It also needs to set the fixed caps on srcpad, when the format is ensured. This is typically when base class calls subclass' set_format function, though it might be delayed until calling gst_audio_decoder_finish_frame .

In summary, above process should have subclass concentrating on codec data processing while leaving other matters to base class, such as most notably timestamp handling. While it may exert more control in this area (see e.g. pre_push ), it is very much not recommended.

In particular, base class will try to arrange for perfect output timestamps as much as possible while tracking upstream timestamps. To this end, if deviation between the next ideal expected perfect timestamp and upstream exceeds “tolerance”, then resync to upstream occurs (which would happen always if the tolerance mechanism is disabled).

In non-live pipelines, baseclass can also (configurably) arrange for output buffer aggregation which may help to redue large(r) numbers of small(er) buffers being pushed and processed downstream.

On the other hand, it should be noted that baseclass only provides limited seeking support (upon explicit subclass request), as full-fledged support should rather be left to upstream demuxer, parser or alike. This simple approach caters for seeking and duration reporting using estimated input bitrates.

Things that subclass need to take care of:

GST_AUDIO_DECODER_ERROR()

#define             GST_AUDIO_DECODER_ERROR(el, weight, domain, code, text, debug, ret)

Utility function that audio decoder elements can use in case they encountered a data processing error that may be fatal for the current "data unit" but need not prevent subsequent decoding. Such errors are counted and if there are too many, as configured in the context's max_errors, the pipeline will post an error message and the application will be requested to stop further media processing. Otherwise, it is considered a "glitch" and only a warning is logged. In either case, ret is set to the proper value to return to upstream/caller (indicating either GST_FLOW_ERROR or GST_FLOW_OK).

GST_AUDIO_DECODER_SINK_PAD()

#define GST_AUDIO_DECODER_SINK_PAD(obj)        (((GstAudioDecoder *) (obj))->sinkpad)

Gives the pointer to the sink GstPad object of the element.

GST_AUDIO_DECODER_SRC_PAD()

#define GST_AUDIO_DECODER_SRC_PAD(obj)         (((GstAudioDecoder *) (obj))->srcpad)

Gives the pointer to the source GstPad object of the element.

GST_AUDIO_DECODER_INPUT_SEGMENT()

#define GST_AUDIO_DECODER_INPUT_SEGMENT(obj)   (GST_AUDIO_DECODER_CAST (obj)->input_segment)

Gives the input segment of the element.

GST_AUDIO_DECODER_OUTPUT_SEGMENT()

#define GST_AUDIO_DECODER_OUTPUT_SEGMENT(obj)   (GST_AUDIO_DECODER_CAST (obj)->output_segment)

Gives the output segment of the element.

GST_AUDIO_DECODER_STREAM_LOCK()

#define GST_AUDIO_DECODER_STREAM_LOCK(dec)   g_rec_mutex_lock (&GST_AUDIO_DECODER (dec)->stream_lock)

GST_AUDIO_DECODER_STREAM_UNLOCK()

#define GST_AUDIO_DECODER_STREAM_UNLOCK(dec) g_rec_mutex_unlock (&GST_AUDIO_DECODER (dec)->stream_lock)

gst_audio_decoder_finish_frame ()

GstFlowReturn
gst_audio_decoder_finish_frame (GstAudioDecoder *dec,
                                GstBuffer *buf,
                                gint frames);

Collects decoded data and pushes it downstream.

buf may be NULL in which case the indicated number of frames are discarded and considered to have produced no output (e.g. lead-in or setup frames). Otherwise, source pad caps must be set when it is called with valid data in buf .

Note that a frame received in gst_audio_decoder_handle_frame() may be invalidated by a call to this function.

gst_audio_decoder_set_output_format ()

gboolean
gst_audio_decoder_set_output_format (GstAudioDecoder *dec,
                                     const GstAudioInfo *info);

Configure output info on the srcpad of dec .

gst_audio_decoder_negotiate ()

gboolean
gst_audio_decoder_negotiate (GstAudioDecoder *dec);

Negotiate with downstream elements to currently configured GstAudioInfo. Unmark GST_PAD_FLAG_NEED_RECONFIGURE in any case. But mark it again if negotiate fails.

gst_audio_decoder_allocate_output_buffer ()

GstBuffer *
gst_audio_decoder_allocate_output_buffer
                               (GstAudioDecoder *dec,
                                gsize size);

Helper function that allocates a buffer to hold an audio frame for dec 's current output format.

gst_audio_decoder_get_allocator ()

void
gst_audio_decoder_get_allocator (GstAudioDecoder *dec,
                                 GstAllocator **allocator,
                                 GstAllocationParams *params);

Lets GstAudioDecoder sub-classes to know the memory allocator used by the base class and its params .

Unref the allocator after use it.

gst_audio_decoder_get_audio_info ()

GstAudioInfo *
gst_audio_decoder_get_audio_info (GstAudioDecoder *dec);

gst_audio_decoder_get_estimate_rate ()

gint
gst_audio_decoder_get_estimate_rate (GstAudioDecoder *dec);

gst_audio_decoder_get_delay ()

gint
gst_audio_decoder_get_delay (GstAudioDecoder *dec);

gst_audio_decoder_get_drainable ()

gboolean
gst_audio_decoder_get_drainable (GstAudioDecoder *dec);

Queries decoder drain handling.

gst_audio_decoder_get_latency ()

void
gst_audio_decoder_get_latency (GstAudioDecoder *dec,
                               GstClockTime *min,
                               GstClockTime *max);

Sets the variables pointed to by min and max to the currently configured latency.

gst_audio_decoder_get_max_errors ()

gint
gst_audio_decoder_get_max_errors (GstAudioDecoder *dec);

gst_audio_decoder_get_min_latency ()

GstClockTime
gst_audio_decoder_get_min_latency (GstAudioDecoder *dec);

Queries decoder's latency aggregation.

gst_audio_decoder_get_needs_format ()

gboolean
gst_audio_decoder_get_needs_format (GstAudioDecoder *dec);

Queries decoder required format handling.

gst_audio_decoder_get_parse_state ()

void
gst_audio_decoder_get_parse_state (GstAudioDecoder *dec,
                                   gboolean *sync,
                                   gboolean *eos);

Return current parsing (sync and eos) state.

gst_audio_decoder_get_plc ()

gboolean
gst_audio_decoder_get_plc (GstAudioDecoder *dec);

Queries decoder packet loss concealment handling.

gst_audio_decoder_get_plc_aware ()

gint
gst_audio_decoder_get_plc_aware (GstAudioDecoder *dec);

gst_audio_decoder_get_tolerance ()

GstClockTime
gst_audio_decoder_get_tolerance (GstAudioDecoder *dec);

Queries current audio jitter tolerance threshold.

gst_audio_decoder_set_estimate_rate ()

void
gst_audio_decoder_set_estimate_rate (GstAudioDecoder *dec,
                                     gboolean enabled);

Allows baseclass to perform byte to time estimated conversion.

gst_audio_decoder_set_drainable ()

void
gst_audio_decoder_set_drainable (GstAudioDecoder *dec,
                                 gboolean enabled);

Configures decoder drain handling. If drainable, subclass might be handed a NULL buffer to have it return any leftover decoded data. Otherwise, it is not considered so capable and will only ever be passed real data.

MT safe.

gst_audio_decoder_set_latency ()

void
gst_audio_decoder_set_latency (GstAudioDecoder *dec,
                               GstClockTime min,
                               GstClockTime max);

Sets decoder latency.

gst_audio_decoder_set_max_errors ()

void
gst_audio_decoder_set_max_errors (GstAudioDecoder *dec,
                                  gint num);

Sets numbers of tolerated decoder errors, where a tolerated one is then only warned about, but more than tolerated will lead to fatal error. You can set -1 for never returning fatal errors. Default is set to GST_AUDIO_DECODER_MAX_ERRORS.

gst_audio_decoder_set_min_latency ()

void
gst_audio_decoder_set_min_latency (GstAudioDecoder *dec,
                                   GstClockTime num);

Sets decoder minimum aggregation latency.

MT safe.

gst_audio_decoder_set_needs_format ()

void
gst_audio_decoder_set_needs_format (GstAudioDecoder *dec,
                                    gboolean enabled);

Configures decoder format needs. If enabled, subclass needs to be negotiated with format caps before it can process any data. It will then never be handed any data before it has been configured. Otherwise, it might be handed data without having been configured and is then expected being able to do so either by default or based on the input data.

MT safe.

gst_audio_decoder_set_plc ()

void
gst_audio_decoder_set_plc (GstAudioDecoder *dec,
                           gboolean enabled);

Enable or disable decoder packet loss concealment, provided subclass and codec are capable and allow handling plc.

MT safe.

gst_audio_decoder_set_plc_aware ()

void
gst_audio_decoder_set_plc_aware (GstAudioDecoder *dec,
                                 gboolean plc);

Indicates whether or not subclass handles packet loss concealment (plc).

gst_audio_decoder_set_tolerance ()

void
gst_audio_decoder_set_tolerance (GstAudioDecoder *dec,
                                 GstClockTime tolerance);

Configures decoder audio jitter tolerance threshold.

MT safe.

gst_audio_decoder_set_allocation_caps ()

void
gst_audio_decoder_set_allocation_caps (GstAudioDecoder *dec,
                                       GstCaps *allocation_caps);

Sets a caps in allocation query which are different from the set pad's caps. Use this function before calling gst_audio_decoder_negotiate(). Setting to NULL the allocation query will use the caps from the pad.

Since: 1.10

gst_audio_decoder_set_use_default_pad_acceptcaps ()

void
gst_audio_decoder_set_use_default_pad_acceptcaps
                               (GstAudioDecoder *decoder,
                                gboolean use);

Lets GstAudioDecoder sub-classes decide if they want the sink pad to use the default pad query handler to reply to accept-caps queries.

By setting this to true it is possible to further customize the default handler with GST_PAD_SET_ACCEPT_INTERSECT and GST_PAD_SET_ACCEPT_TEMPLATE

Since: 1.6

gst_audio_decoder_merge_tags ()

void
gst_audio_decoder_merge_tags (GstAudioDecoder *dec,
                              const GstTagList *tags,
                              GstTagMergeMode mode);

Sets the audio decoder tags and how they should be merged with any upstream stream tags. This will override any tags previously-set with gst_audio_decoder_merge_tags().

Note that this is provided for convenience, and the subclass is not required to use this and can still do tag handling on its own.

gst_audio_decoder_proxy_getcaps ()

GstCaps *
gst_audio_decoder_proxy_getcaps (GstAudioDecoder *decoder,
                                 GstCaps *caps,
                                 GstCaps *filter);

Returns caps that express caps (or sink template caps if caps == NULL) restricted to rate/channels/... combinations supported by downstream elements.

Since: 1.6

struct GstAudioDecoder

struct GstAudioDecoder;

The opaque GstAudioDecoder data structure.

struct GstAudioDecoderClass

struct GstAudioDecoderClass {
  GstElementClass element_class;

  /* virtual methods for subclasses */

  gboolean      (*start)              (GstAudioDecoder *dec);

  gboolean      (*stop)               (GstAudioDecoder *dec);

  gboolean      (*set_format)         (GstAudioDecoder *dec,
                                       GstCaps *caps);

  GstFlowReturn (*parse)              (GstAudioDecoder *dec,
                                       GstAdapter *adapter,
                                       gint *offset, gint *length);

  GstFlowReturn (*handle_frame)       (GstAudioDecoder *dec,
                                       GstBuffer *buffer);

  void          (*flush)              (GstAudioDecoder *dec, gboolean hard);

  GstFlowReturn (*pre_push)           (GstAudioDecoder *dec,
                                       GstBuffer **buffer);

  gboolean      (*sink_event)         (GstAudioDecoder *dec,
                                       GstEvent *event);
  gboolean      (*src_event)          (GstAudioDecoder *dec,
                                       GstEvent *event);

  gboolean      (*open)               (GstAudioDecoder *dec);

  gboolean      (*close)              (GstAudioDecoder *dec);

  gboolean      (*negotiate)          (GstAudioDecoder *dec);

  gboolean      (*decide_allocation)  (GstAudioDecoder *dec, GstQuery *query);

  gboolean      (*propose_allocation) (GstAudioDecoder *dec,
                                       GstQuery * query);

  gboolean      (*sink_query)         (GstAudioDecoder *dec, GstQuery *query);

  gboolean      (*src_query)          (GstAudioDecoder *dec, GstQuery *query);

  GstCaps *     (*getcaps)            (GstAudioDecoder * dec,
                                       GstCaps * filter);

  gboolean      (*transform_meta)     (GstAudioDecoder *enc, GstBuffer *outbuf,
                                       GstMeta *meta, GstBuffer *inbuf);
};

Subclasses can override any of the available virtual methods or not, as needed. At minimum handle_frame (and likely set_format ) needs to be overridden.

GST_AUDIO_DECODER_SINK_NAME

#define GST_AUDIO_DECODER_SINK_NAME    "sink"

The name of the templates for the sink pad.

GST_AUDIO_DECODER_SRC_NAME

#define GST_AUDIO_DECODER_SRC_NAME     "src"

The name of the templates for the source pad.

GST_AUDIO_DECODER_MAX_ERRORS

#define GST_AUDIO_DECODER_MAX_ERRORS     10

Default maximum number of errors tolerated before signaling error.

The “min-latency” property

  “min-latency”              gint64

Aggregate output data to a minimum of latency time (ns).

Flags: Read / Write

Allowed values: >= 0

Default value: 0

The “plc” property

  “plc”                      gboolean

Perform packet loss concealment (if supported).

Flags: Read / Write

Default value: FALSE

The “tolerance” property

  “tolerance”                gint64

Perfect ts while timestamp jitter/imperfection within tolerance (ns).

Flags: Read / Write

Allowed values: >= 0

Default value: 0