[GStreamer] 内存管理

• GstMemory：管理一小块实际的虚拟内存
• GstBuffer：一个对象，是element、plugin、application互相信息交互的最小数据单位，内部包含GstMemory、时间信息和一些其他信息。
• GstMeta：附加在GstBuffer上，用来提供一些描述内存的额外信息。
• GstBufferPool：buffer池，但是只能分配尺寸固定的buffer。

GstMemory：

/**
* GstMemory:
* @mini_object: parent structure
* @allocator: pointer to the #GstAllocator
* @parent: parent memory block
* @maxsize: the maximum size allocated
* @align: the alignment of the memory
* @offset: the offset where valid data starts
* @size: the size of valid data
*
* Base structure for memory implementations. Custom memory will put this structure
* as the first member of their structure.
*/
struct _GstMemory {
GstMiniObject mini_object;

GstAllocator *allocator;

GstMemory *parent;
gsize maxsize;
gsize align;
gsize offset;
gsize size;
};

GstMemory由GstAllocator创建。

GstAllocator：

/**
* GstAllocator:
* @mem_map: the implementation of the GstMemoryMapFunction
* @mem_unmap: the implementation of the GstMemoryUnmapFunction
* @mem_copy: the implementation of the GstMemoryCopyFunction
* @mem_share: the implementation of the GstMemoryShareFunction
* @mem_is_span: the implementation of the GstMemoryIsSpanFunction
* @mem_map_full: the implementation of the GstMemoryMapFullFunction.
* Will be used instead of @mem_map if present. (Since: 1.6)
* @mem_unmap_full: the implementation of the GstMemoryUnmapFullFunction.
* Will be used instead of @mem_unmap if present. (Since: 1.6)
*
* The #GstAllocator is used to create new memory.
*/
struct _GstAllocator
{
GstObject object;

const gchar *mem_type;

/*< public >*/
GstMemoryMapFunction mem_map;
GstMemoryUnmapFunction mem_unmap;

GstMemoryCopyFunction mem_copy;
GstMemoryShareFunction mem_share;
GstMemoryIsSpanFunction mem_is_span;

GstMemoryMapFullFunction mem_map_full;
GstMemoryUnmapFullFunction mem_unmap_full;

/*< private >*/
gpointer _gst_reserved[GST_PADDING - 2];

GstAllocatorPrivate *priv;
};

内存分配器，是一个接口层，如果自行管理内存分配，可以自己实现这个接口。

内存分配和使用：

使用 gst_allocator_alloc 分配内存，此函数返回的是一个 GstMemory 指针，GstMemory内的数据必须通过 gst_memory_map 保护才可以访问，然后通过 gst_memory_unmap 释放保护后别人才可以访问，类似于加锁解锁。

[...]

GstMemory *mem;
GstMapInfo info;
gint i;

/* allocate 100 bytes */
mem = gst_allocator_alloc (NULL, 100, NULL);

/* get access to the memory in write mode */
gst_memory_map (mem, &info, GST_MAP_WRITE);

/* fill with pattern */
for (i = 0; i < info.size; i++)
info.data[i] = i;

/* release memory */
gst_memory_unmap (mem, &info);

[...]

GstBuffer ：

• GstBuffer内部包含了n个GstMemory，同时也包含了一些meta data：
• dts 、pts：用来进行同步。
• duration：buffer内数据总时长，不是整个流的总时长。
• offset ：buffer存视频时，表示当前buffer内第一帧数据在整个流中是第多少个帧；buffer存音频时，???
• offset_end：buffer存视频时，表示当前buffer内最后一帧数据在整个流中是第多少个帧；buffer存音频时，？？？

/**
* GstBuffer:
* @mini_object: the parent structure
* @pool: pointer to the pool owner of the buffer
* @pts: presentation timestamp of the buffer, can be #GST_CLOCK_TIME_NONE when the
* pts is not known or relevant. The pts contains the timestamp when the
* media should be presented to the user.
* @dts: decoding timestamp of the buffer, can be #GST_CLOCK_TIME_NONE when the
* dts is not known or relevant. The dts contains the timestamp when the
* media should be processed.
* @duration: duration in time of the buffer data, can be #GST_CLOCK_TIME_NONE
* when the duration is not known or relevant.
* @offset: a media specific offset for the buffer data.
* For video frames, this is the frame number of this buffer.
* For audio samples, this is the offset of the first sample in this buffer.
* For file data or compressed data this is the byte offset of the first
* byte in this buffer.
* @offset_end: the last offset contained in this buffer. It has the same
* format as @offset.
*
* The structure of a #GstBuffer. Use the associated macros to access the public
* variables.
*/
struct _GstBuffer {
GstMiniObject mini_object;

/*< public >*/ /* with COW */
GstBufferPool *pool;

/* timestamp */
GstClockTime pts;
GstClockTime dts;
GstClockTime duration;

/* media specific offset */
guint64 offset;
guint64 offset_end;
};

默认情况下GstBuffer是不可写的，需要先通过gst_buffer_make_writable函数处理，然后才可以修改里面的内容，比如dts、pts、duration、offset等等。

三种创建GstBuffer的方法:

gst_buffer_new 创建GstBuffer 对象，但是创建的对象不包含 GstMemory，需要手动创建GstMemory再进行绑定；

gst_buffer_new_allocate 同时创建GstBuffer 和 指定数量的GstMemory，然后还会自动完成绑定动作；

gst_buffer_new_wrapped_full 为现有GstMemory创建一个GstBuffer。

[...]
GstBuffer *buffer;
GstMemory *mem;
GstMapInfo info;

/* make empty buffer */
buffer = gst_buffer_new ();

/* make memory holding 100 bytes */
mem = gst_allocator_alloc (NULL, 100, NULL);

/* add the buffer */
gst_buffer_append_memory (buffer, mem);

[...]

/* get WRITE access to the memory and fill with 0xff */
gst_buffer_map (buffer, &info, GST_MAP_WRITE);
memset (info.data, 0xff, info.size);
gst_buffer_unmap (buffer, &info);

[...]

/* free the buffer */
gst_buffer_unref (buffer);

[...]

GstMeta：

/**
* GstMeta:
* @flags: extra flags for the metadata
* @info: pointer to the #GstMetaInfo
*
* Base structure for metadata. Custom metadata will put this structure
* as the first member of their structure.
*/
struct _GstMeta {
GstMetaFlags flags;
const GstMetaInfo *info;
};

GstMeta可以随意地附加到一个GstBuffer中。

#include <gst/video/gstvideometa.h>

[...]
GstVideoCropMeta *meta; //GstMeta的一个子类

/* buffer points to a video frame, add some cropping metadata */
meta = gst_buffer_add_video_crop_meta (buffer);

/* configure the cropping metadata */
meta->x = 8;
meta->y = 8;
meta->width = 120;
meta->height = 80;
[...]#include <gst/video/gstvideometa.h>

[...]
GstVideoCropMeta *meta;

/* buffer points to a video frame, get the cropping metadata */
meta = gst_buffer_get_video_crop_meta (buffer);

if (meta) {
/* render frame with cropping */
_render_frame_cropped (buffer, meta->x, meta->y, meta->width, meta->height);
} else {
/* render frame */
_render_frame (buffer);
}
[...]

 如何实现自定义的 GstMeta  ：Memory allocation

GstBufferPool：

由内存池分配的 GstBuffer 具备完全相同的 meta data，唯一不一样的就是指向的内存区域。

GstBufferPool 以 GstBuffer 为管理单位，当且仅当 GstBufferPool 为 inactive 状态时，可以配置 GstBuffer 的meta data；当且仅当 GstBufferPool 为 active 状态时，可以分配和销毁GstBuffer。

GstStructure *config;

[...]

/* get config structure */
config = gst_buffer_pool_get_config (pool);

/* set caps, size, minimum and maximum buffers in the pool */
gst_buffer_pool_config_set_params (config, caps, size, min, max);

/* configure allocator and parameters */
gst_buffer_pool_config_set_allocator (config, allocator, ¶ms);

/* store the updated configuration again */
gst_buffer_pool_set_config (pool, config);

[...]

The following options can be configured on a ​​GstBufferPool​​:

• The caps of the buffers to allocate.
• The size of the buffers. This is the suggested size of the buffers in the pool. The pool might decide to allocate larger buffers to add padding.
• The minimum and maximum amount of buffers in the pool. When minimum is set to​​\> 0​​, the bufferpool will pre-allocate this amount of buffers. When maximum is not 0, the bufferpool will allocate up to maximum amount of buffers.
• The allocator and parameters to use. Some bufferpools might ignore the allocator and use its internal one.
• Other arbitrary bufferpool options identified with a string. a bufferpool lists the supported options with​​gst_buffer_pool_get_options()​​ and you can ask if an option is supported with ​​gst_buffer_pool_has_option()​​. The option can be enabled by adding it to the configuration structure with ​​gst_buffer_pool_config_add_option ()​​. These options are used to enable things like letting the pool set metadata on the buffers or to add extra configuration options for padding, for example.

gst_buffer_pool_set_active(pool, TRUE)用来激活内存池，之后用gst_buffer_pool_acquire_buffer ()来获取内存池对象。

[...]

GstFlowReturn ret;
GstBuffer *buffer;

ret = gst_buffer_pool_acquire_buffer (pool, &buffer, NULL);
if (G_UNLIKELY (ret != GST_FLOW_OK))
goto pool_failed;

[...]

GST_QUERY_ALLOCATION：