專案記錄：MPEG-DASH整理3
阿新 • • 發佈：2018-12-07
專案記錄：MPEG-DASH整理3
該專案的目標是 基於3DOF的視訊內容 實現一個6DOF的播放器.
觀看者戴著VR頭顯,在位置發生運動之後,播放器能夠根據其位置的不同,下載不同位置的碼流,進行解碼渲染.簡單地實現6DOF.
伺服器端:
-
伺服器端必須能夠有多個視點位置的全景視訊,作為不同的
AdaptationSet儲存起來. -
那麼客戶端如何知道伺服器端的各個
AdaptationSet的位置呢?我通過給每個
AdaptationSet增加一個位置資訊的欄位.- 比如
<Viewpoint schemeIdUri="urn:mpeg:mpegI:omaf:2018:vwpt" value="0,0,0,0"/>
- 比如
-
以下是生成的
MPD表示例:<?xml version="1.0" encoding="utf-8"?> <MPD xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns="urn:mpeg:dash:schema:mpd:2011" xmlns:xlink="http://www.w3.org/1999/xlink" xsi:schemaLocation="urn:mpeg:DASH:schema:MPD:2011 http://standards.iso.org/ittf/PubliclyAvailableStandards/MPEG-DASH_schema_files/DASH-MPD.xsd"profiles="urn:mpeg:dash:profile:isoff-live:2011" type="static" mediaPresentationDuration="PT15.0S" minBufferTime="PT0.0S" maxSegmentDuration="PT1.0S"> <ProgramInformation moreInformationURL="http://127.0.0.1/"> <Title>seg\1.mpd generated by GPAC</Title> </ProgramInformation> <BaseURL>http://127.0.0.1/</BaseURL> <Period duration="PT15.0S"> <AdaptationSet segmentAlignment="true" bitstreamSwitching="true" maxWidth="3840" maxHeight="1920" maxFrameRate="30/1" par="16:9" lang="und"> <Viewpoint schemeIdUri="urn:mpeg:mpegI:omaf:2018:vwpt" value="0,0,0,0"/> <Representation id="1" mimeType="video/mp4" codecs="hev1.010101" width="3840" height="1920" frameRate="30" sar="1:1" startWithSAP="1" bandwidth="1504040"> <SegmentList timescale="1200000" duration="48000"> <Initialization sourceURL="TestStream0_ViewPoint0/init.mp4" /> <SegmentURL media="TestStream0_ViewPoint0/1.m4s" /> <SegmentURL media="TestStream0_ViewPoint0/2.m4s" /> <SegmentURL media="TestStream0_ViewPoint0/3.m4s" /> <SegmentURL media="TestStream0_ViewPoint0/4.m4s" /> <SegmentURL media="TestStream0_ViewPoint0/5.m4s" /> <SegmentURL media="TestStream0_ViewPoint0/6.m4s" /> <SegmentURL media="TestStream0_ViewPoint0/7.m4s" /> <SegmentURL media="TestStream0_ViewPoint0/8.m4s" /> <SegmentURL media="TestStream0_ViewPoint0/9.m4s" /> <SegmentURL media="TestStream0_ViewPoint0/10.m4s" /> </SegmentList> </Representation> </AdaptationSet> <AdaptationSet segmentAlignment="true" bitstreamSwitching="true" maxWidth="3840" maxHeight="1920" maxFrameRate="30/1" par="16:9" lang="und"> <Viewpoint schemeIdUri="urn:mpeg:mpegI:omaf:2018:vwpt" value="1,1,1,1"/> <Representation id="1" mimeType="video/mp4" codecs="hev1.010101" width="3840" height="1920" frameRate="30" sar="1:1" startWithSAP="1" bandwidth="7508880"> <SegmentList timescale="1200000" duration="48000"> <Initialization sourceURL="TestStream1_ViewPoint1/init.mp4" /> <SegmentURL media="TestStream1_ViewPoint1/1.m4s" /> <SegmentURL media="TestStream1_ViewPoint1/2.m4s" /> <SegmentURL media="TestStream1_ViewPoint1/3.m4s" /> <SegmentURL media="TestStream1_ViewPoint1/4.m4s" /> <SegmentURL media="TestStream1_ViewPoint1/5.m4s" /> <SegmentURL media="TestStream1_ViewPoint1/6.m4s" /> <SegmentURL media="TestStream1_ViewPoint1/7.m4s" /> <SegmentURL media="TestStream1_ViewPoint1/8.m4s" /> <SegmentURL media="TestStream1_ViewPoint1/9.m4s" /> <SegmentURL media="TestStream1_ViewPoint1/10.m4s" /> <SegmentURL media="TestStream1_ViewPoint1/11.m4s" /> <SegmentURL media="TestStream1_ViewPoint1/12.m4s" /> <SegmentURL media="TestStream1_ViewPoint1/13.m4s" /> <SegmentURL media="TestStream1_ViewPoint1/14.m4s" /> <SegmentURL media="TestStream1_ViewPoint1/15.m4s" /> <SegmentURL media="TestStream1_ViewPoint1/16.m4s" /> </SegmentList> </Representation> </AdaptationSet> </Period> </MPD>
客戶端:
- 客戶端通過解析
MPD表能夠獲得得到所有AdaptationSet的ViewPoint,進行儲存 - 在檢測到使用者的位置發生變化時,進行與這些參考位置進行比較. 選擇適合的視訊碼流進行下載.
ViewPoint 欄位解析
-
IAdaptationSet類中提供了很多IDescriptor的介面virtual const std::vector<IDescriptor *>& GetViewpoint () const = 0;通過
GetViewpoint這個介面獲得MPD中給每個AdaptationSet新增的 視點位置 -
於是,新建一個類來管理這些
ViewPoint,一個ViewPoint表示當前VR頭盔對應的位置.
dash::mpd::ViewPointManager *viewpointManager; // 管理MPD表所有的ViewPoint
dash::mpd::ViewPoint *currentViewPoint; // 表示當前的ViewPoint
IViewPointManager.h
/**
* @class dash::mpd::IViewPointManager
* @brief the Class to Manage the ViewPoint of every AdaptationSet
* @author JZChen
* Email: [email protected]
* @version 1.0
* @date 2018.11.29
* @copyright Xidian MMC 203, All Rights Reserved \n\n
* This source code and its use and distribution, is subject to the terms
* and conditions of the applicable license agreement.
*/
#ifndef VIEWPOINTMANGER_H_
#define VIEWPOINTMANGER_H_
#include "config.h"
#include "IAdaptationSet.h"
namespace dash {
namespace mpd {
class ViewPoint {
public:
ViewPoint() :viewId(0), centerX(0), centerY(0), centerZ(0) {}
ViewPoint(int id,int x,int y,int z) :viewId(id), centerX(x), centerY(y), centerZ(z) {}
ViewPoint(const ViewPoint& rh) :viewId(rh.viewId), centerX(rh.centerX), centerY(rh.centerY), centerZ(centerZ) {}
void operator=(const ViewPoint& rh) {
this->viewId = rh.viewId;
this->centerX = rh.centerX;
this->centerY = rh.centerY;
this->centerZ = rh.centerZ;
}
void SetViewPoint(int id, int x, int y, int z) {
this->viewId = id;
this->centerX = x;
this->centerY = y;
this->centerZ = z;
}
ViewPoint(IDescriptor *pDes) {
ViewPoint();
std::string tmp = pDes->GetValue();
printf("%s\n", tmp);
// ViewPoint的Value格式如下: ViewID,CenterX,CenterY,CenterZ
std::string::iterator it_begin;
std::string::iterator it_end;
it_begin = tmp.begin();
it_end = find(tmp.begin(), tmp.end(), ',');
viewId = std::stoi(tmp.substr(it_begin - tmp.begin(), it_end - it_begin));
it_begin = ++it_end;
it_end = find(tmp.begin(), tmp.end(), ',');
centerX = std::stoi(tmp.substr(it_begin - tmp.begin(), it_end - it_begin));
it_begin = ++it_end;
it_end = find(tmp.begin(), tmp.end(), ',');
centerY = std::stoi(tmp.substr(it_begin - tmp.begin(), it_end - it_begin));
it_begin = ++it_end;
it_end = find(tmp.begin(), tmp.end(), ',');
centerZ = std::stoi(tmp.substr(it_begin - tmp.begin(), it_end - it_begin));
}
// TODO:精細化這個判斷條件
bool operator==(const ViewPoint& rh) {
if (centerX == rh.centerX&&
centerY == rh.centerY&&
centerZ == rh.centerZ) {
return true;
}
else {
return false;
}
}
int viewId;
int centerX;
int centerY;
int centerZ;
};
}
}
namespace dash {
namespace mpd {
class ViewPointManager {
public:
ViewPointManager():SizeOfAdaptationSet(0), SizeOfViewPoint(0){
}
ViewPointManager(std::vector<IAdaptationSet*> AdaptationSetList) {
ViewPointManager();
SizeOfAdaptationSet = AdaptationSetList.size();
this->AdaptationSetList = AdaptationSetList;
}
std::vector<IAdaptationSet*> AdaptationSetList;
std::vector<ViewPoint*> ViewPointList; // 與AdaptationSetList一一對應
int SizeOfAdaptationSet; // 一共有多少個AdaptationSet
int SizeOfViewPoint; // 一個多少個ViewPoint,
// ================ Get Functions ====================== //
const std::vector<IAdaptationSet*>& GetAdaptationSetList() const {
return AdaptationSetList;
}
const std::vector<ViewPoint*>& GetViewPointList() const {
return ViewPointList;
}
// ================ Set Functions ====================== //
bool SetViewPointList(const std::vector<IAdaptationSet*> &AdaptationSetList) {
this->AdaptationSetList = AdaptationSetList;
return SetViewPointList();
}
bool SetViewPointList() {
// <1> 獲得到所有的GetAdaptationSet
SizeOfAdaptationSet = AdaptationSetList.size();
for (int i = 0; i < SizeOfAdaptationSet; ++i) {
/// 一般一個AdaptationSet只能有一個位置
auto DesViewpointList = AdaptationSetList[i]->GetViewpoint();
// <2> 獲得所有的DescriptionDesViewpointList
auto SizeOfDesViewpointList = DesViewpointList.size();
for (int j = 0; j < SizeOfDesViewpointList; ++j) {
auto DesViewpoint = DesViewpointList[j];
// <3> 儲存所有的ViewPoint
ViewPointList.push_back(new ViewPoint(DesViewpoint));
}
}
if (ViewPointList.size())
SizeOfViewPoint = ViewPointList.size();
if (SizeOfViewPoint <= 0) {
std::cout << "Exception at SetViewPointList" << std::endl;
return false;
}
return true;
}
/// tobeContinued
};
}
}
#endif
- 通過這個類我們就可以管理所有的VIewPoint了.在位置發生變化的時候,取到對應的ViewPoint的Adaptation.
- 現在還沒有寫好.應該改成
Map儲存.
切換下載視訊源
-
在
DoBuffering執行緒中,需要根據當前頭盔的位置,進行選擇判斷.// 根據number,獲得新的媒體段 media = dashreceiver->logic->GetSegment(number); -
可以通過遍歷
ViewPointManager中的視點位置與當前的視點位置進行比較進行下載更新Segment