原來Qt從視訊中獲取每一幀資料如此簡單
阿新 • • 發佈：2019-01-02
有時候需要在視訊上畫圖，所以需要能獲取到每一幀視訊資料。
以前從視訊檔案或視訊流中得到幀，一般都是使用qt + ffmpeg或qt + vlc。
qt對顯示處理視訊大體有以下方法：
1. QMediaPlayer + QVideoWidget
這種方法只適合簡單的顯示視訊功能，不適合對視訊進行處理（比如畫圖）
2. QMediaPlayer + QGraphicsVideoItem + QGraphicsScene + QGraphicsView
這種方法功能強大，除了顯示視訊功能，還可以做複雜的圖形處理(具體可以檢視QGraphicsScene的使用)
3. QMediaPlayer + QAbstractVideoSurface
這種方法比較簡單，是我下面要介給的。可以獲取到每一幀視訊資料，基本可以實現與qt + ffmpeg或qt + vlc相同的效果。
自定義VideoSurface：
class VideoSurface : public QAbstractVideoSurface { Q_OBJECT public: VideoSurface(QObject *parent = Q_NULLPTR); ~VideoSurface(); QList<QVideoFrame::PixelFormat> supportedPixelFormats(QAbstractVideoBuffer::HandleType handleType = QAbstractVideoBuffer::NoHandle) const; bool present(const QVideoFrame &frame); signals: void frameAvailable(QVideoFrame &frame); };
實現如下：
VideoSurface::VideoSurface(QObject *parent) : QAbstractVideoSurface(parent) { } VideoSurface::~VideoSurface() { } QList<QVideoFrame::PixelFormat> VideoSurface::supportedPixelFormats(QAbstractVideoBuffer::HandleType handleType) const { QList<QVideoFrame::PixelFormat> listPixelFormats; listPixelFormats << QVideoFrame::Format_ARGB32 << QVideoFrame::Format_ARGB32_Premultiplied << QVideoFrame::Format_RGB32 << QVideoFrame::Format_RGB24 << QVideoFrame::Format_RGB565 << QVideoFrame::Format_RGB555 << QVideoFrame::Format_ARGB8565_Premultiplied << QVideoFrame::Format_BGRA32 << QVideoFrame::Format_BGRA32_Premultiplied << QVideoFrame::Format_BGR32 << QVideoFrame::Format_BGR24 << QVideoFrame::Format_BGR565 << QVideoFrame::Format_BGR555 << QVideoFrame::Format_BGRA5658_Premultiplied << QVideoFrame::Format_AYUV444 << QVideoFrame::Format_AYUV444_Premultiplied << QVideoFrame::Format_YUV444 << QVideoFrame::Format_YUV420P << QVideoFrame::Format_YV12 << QVideoFrame::Format_UYVY << QVideoFrame::Format_YUYV << QVideoFrame::Format_NV12 << QVideoFrame::Format_NV21 << QVideoFrame::Format_IMC1 << QVideoFrame::Format_IMC2 << QVideoFrame::Format_IMC3 << QVideoFrame::Format_IMC4 << QVideoFrame::Format_Y8 << QVideoFrame::Format_Y16 << QVideoFrame::Format_Jpeg << QVideoFrame::Format_CameraRaw << QVideoFrame::Format_AdobeDng; //qDebug() << listPixelFormats; // Return the formats you will support return listPixelFormats; } bool VideoSurface::present(const QVideoFrame &frame) { // Handle the frame and do your processing if (frame.isValid()) { QVideoFrame cloneFrame(frame); emit frameAvailable(cloneFrame); return true; } return false; }
看了上面的程式碼就知道，只需要外部連線frameAvailable訊號就可以獲取到每一幀資料。
具體使用：
QMediaPlayer *mediaPlayer = new QMediaPlayer;
VideoSurface *videoSurface = new VideoSurface;
mediaPlayer->setVideoOutput(videoSurface);
mediaPlayer->setMedia(QUrl("rtsp://admin:[email protected]"));
mediaPlayer->play();把frameAvailable訊號與顯示視窗的槽連線，比如：
connect(videoSurface, SIGNAL(frameAvailable(QVideoFrame &)), this, SLOT(ProcessFrame(QVideoFrame &)));void QtVideoTest::ProcessFrame(QVideoFrame &frame)
{
qDebug() << "=============ProcessFrame===============";
qDebug() << "width : " << frame.width() << " height : " << frame.height();
qDebug() << "start time : " << frame.startTime()/1000 << "ms";
qDebug() << "end time : " << frame.endTime()/1000 << "ms";
qDebug() << "pixelFormat :" << frame.pixelFormat();
frame.map(QAbstractVideoBuffer::ReadOnly);
QImage recvImage(frame.bits(), frame.width(), frame.height(), QVideoFrame::imageFormatFromPixelFormat(frame.pixelFormat()));
qDebug() << "frame data size :" << frame.mappedBytes();
frame.unmap();
}如上，QVideoFrame轉QImage，拿QImage進行畫圖操作就簡單了。