H264 視訊檔案 幀格式 傳輸封裝等
Standards Track [Page 2] RFC 3984 RTP Payload Format for H.264 Video February 2005 1.
按照RFC3984協議實現H264視訊流媒體
nalu單元 包起始 0x 00 00 00 01
H．264 NAL格式及分析器
http://hi.baidu.com/zsw%5Fdavy/b ... c409cc7cd92ace.html
http://hi.baidu.com/zsw_davy/blo ... 081312c8fc7acc.html
----------------------------------位元流資訊----------------------------------------------
①NALU(Network Abstract Layer Unit)：兩標準中的位元流都是以NAL為單位，每個NAL單元包含一個RBSP，NALU的頭資訊定義了RBSP所屬型別。型別一般包括序列引數集（SPS）、影象引數集（PPS）、增強資訊（SEI）、條帶（Slice）等，其中，SPS和PPS屬於引數集，兩標準採用引數集機制是為了將一些主要的序列、影象引數（解碼影象尺寸、片組數、參考幀數、量化和濾波引數標記等）與其他引數分離，通過解碼器先解碼出來。此外，為了增強影象的清晰度，AVS-M添加了影象頭（Picture head）資訊。讀取NALU流程中，每個NALU前有一個起始碼0x000001，為防止 內部0x000001序列競爭，H.264編碼器在最後一位元組前插入一個新的位元組——0x03，所以解碼器檢測到該序列時，需將0x03刪掉，而AVS-M只需識別出起始碼0x000001。
②讀取巨集塊型別（mb type）和巨集塊編碼模板（cbp）：編解碼影象以巨集塊劃分，一個巨集塊由一個16*16亮度塊和相應的一個8*8cb和一個8*8cr色度塊組成。
(a) 兩標準的幀內、幀間預測時巨集塊的劃分是有區別的。H.264中，I_slice亮度塊有Intra_4*4和Intra_16*16兩種模式，色度塊只有8*8模式；P_slice巨集塊分為16*16、16*8、8*16、8*8、8*4、4*8、4*4共7種模式。而AVS-M中，I_slice亮度塊有I_4*4和I_Direct兩模式，P_slice時巨集塊的劃分和H.264中的劃分一致。
(b) 兩標準的巨集塊cbp值計算也不相同。H.264中，Intra_16*16巨集塊的亮度（色度）cbp直接通過讀mb type得到；非Intra_16*16巨集塊的亮度cbp=coded_block_pattern%16，色度cbp=coded_block_pattern/16 。其中，亮度cbp最低4位有效，每位決定對應巨集塊的殘差係數能不能為0；色度cbp為0時，對應殘差係數為0，cbp為1時，DC殘差係數不為0，AC係數為0，cbp為2時，DC、AC殘差係數都不為0。AVS-M中，當巨集塊型別不是P_skip時，直接從碼流中得到cbp的索引值，並以此索引值查表得到codenum值，再以codenum查表分別得到幀內／幀間cbp。此cbp為6位，每位代表巨集塊按8*8劃分時能不能包含非零係數，當變換系數不為0時，需進一步讀cbp_4*4中每位值來判斷一個8*8塊中4個4*4塊的係數能不能為0。
---------------------------------------------------------------------------------------------
總的來說H264的碼流的打包方式有兩種,一種為annex-b byte stream format的格式，這個是絕大部分編碼器的預設輸出格式，就是每個幀的開頭的3~4個位元組是H264的start_code,0x00000001或者0x000001。
另一種是原始的NAL打包格式，就是開始的若干位元組（1，2，4位元組）是NAL的長度，而不是start_code,此時必須藉助某個全域性的資料來獲得編碼器的profile,level,PPS,SPS等資訊才可以解碼。
----------------------------------------------------------------------------
AVC vs. H.264
AVC and H.264 are synonymous. The standard is known by the full names "ISO/IEC 14496-10" and "ITU-T Recommendation H.264". In addition, a number of alternate names are used (or have been) in reference to this standard. These include:
-
MPEG-4 part 10
-
MPEG-4 AVC
-
AVC
-
MPEG-4 (in the broadcasting world MPEG4 part 2 is ignored)
-
H.264
-
JVT (Joint Video Team, nowadays rarely used referring to actual spec)
-
H.26L (early drafts went by this name)
All of the above (and those I've missed) include the Annex B byte-stream format. Unlike earlier MPEG1/2/4 and H.26x codecs, the H.264 specification proper does not
define a full bit-stream syntax. It describes a number of NAL (Network Abstraction Layer) units, a sequence of which can be decoded into video frames. These NAL units have no boundary markers, and rely on some unspecified format to provide framing.
Annex B of of the document specifies one such format, which wraps NAL units in a format resembling a traditional MPEG video elementary stream, thus making it suitable for use with containers like MPEG PS/TS unable to provide the required framing. Other formats,
such as ISO base media based formats, are able to properly separate the NAL units and do not need the Annex B wrapping.
The H.264 spec suffers from a deficiency. It defines several header-type NAL units (SPS and PPS) without specifying how to pack them into the single codec data field available in most containers. Fortunately, most containers seem to have adopted the packing
used by the ISO format known as MP4.
1. H.264起始碼
在網路傳輸h264資料時，一個UDP包就是一個NALU,解碼器可以很方便的檢測出NAL分界和解碼。但是如果編碼資料儲存為一個檔案，原來的解碼器將無法從資料流中分別出每個NAL的起始位置和終止位置，為此h.264用起始碼來解決這一問題。
H.264編碼時，在每個NAL前新增起始碼 0x000001，解碼器在碼流中檢測到起始碼，當前NAL結束。為了防止NAL內部出現0x000001的資料，h.264又提出'防止競爭 emulation prevention"機制，在編碼完一個NAL時，如果檢測出有連續兩個0x00位元組，就在後面插入一個0x03。當解碼器在NAL內部檢測到0x000003的資料，就把0x03拋棄，恢復原始資料。
0x000000 >>>>>> 0x00000300
0x000001 >>>>>> 0x00000301
0x000002 >>>>>> 0x00000302
0x000003 >>>>>> 0x00000303
附上h.264解碼nalu中檢測起始碼的演算法流程
for(;;)
{
if next 24 bits are 0x000001
{
startCodeFound = true
break;
}
else
{
flush 8 bits
}
}// for(;;)
if(true == startCodeFound)
{
//startcode found
// Flush the start code found
flush 24 bits
//Now navigate up to next start code and put the in between stuff
// in the nal structure.
for(;;)
{
get next 24 bits & check if it equals to 0x000001
if(false == (next 24 bits == 000001))
{
// search for pattern 0x000000
check if next 24 bits are 0x000000
if(false == result)
{
// copy the byte into the buffer
copy one byte to the Nal unit
}
else
{
break;
}
}
else
{
break;
}
}//for(;;)
}
2. MPEG4起始碼
MPEG4的特色是VOP，沒有NALU的概念，仍使用startcode對每幀進行分界。MPEG4的起始碼是0x000001. 另外MPEG4中很多起始碼也很有用，比如video_object_sequence_start_code 0x000001B0 表示一個視訊物件序列的開始,VO_start_code 0x000001B6 表示一個VOP的開始. 0x000001B6之後的兩位，是00表示 I frame， 01 表示 P frame， 10 表示 B frame.
H．264 NAL格式及分析器程式原始碼
email:[email protected]
NAL分析器程式原始碼
/*!
**************************************************************************************
* \file
* nalchek.c
* \brief
* NALU handling common to encoder and decoder
l [email protected]
l Aug.18.2006 ***************************************************************************************
*/
#include "stdio.h"
#include "stdlib.h"
#include "nalucommon.h"
//#####################################
//#本程式檢查一個緩衝區裡面有幾個NALU #
//#輸出每個NALU的型別到輸出檔案 #
//#輸入為一個帶有標記/NALU+標記/的檔案#
//#####################################
void main(void)
{
FILE *pSrc;
FILE *pRep;
if((pSrc=fopen("H264.rec","r+b"))==NULL)
exit(-1);
else printf("open src file succeed\n");
if((pRep=fopen("H264.rep","w+t"))==NULL)
exit(-1);
else printf("open report file succeed\n");
ReportNalu(pSrc,pRep);
fclose(pSrc);
fclose(pRep);
}
/*! **************************************************************************************
* \file
* nalucommon.h.h
* \brief
* NALU handling common to encoder and decoder
* [email protected]
* Aug.18.2006 ***************************************************************************************
*/
#include
#ifndef _NALUCOMMON_H_
#define _NALUCOMMON_H_
typedef struct
{
int startcodeprefix_len; //! 4 for parameter sets and first slice in picture, 3 for everything else
(suggested)
unsigned len; //! Length of the NAL unit (Excluding the start code, which does not belong
to the NALU)
unsigned max_size; //! Nal Unit Buffer size
int nal_unit_type; //! NALU_TYPE_xxxx
int nal_reference_idc; //! NALU_PRIORITY_xxxx
int forbidden_bit; //! should be always FALSE
unsigned char *buf; //! conjtains the first byte followed by the EBSP
} NALU_t;
#define MAXRBSPSIZE 64000
#define NALU_TYPE_SLICE 1
#define NALU_TYPE_DPA 2
#define NALU_TYPE_DPB 3
#define NALU_TYPE_DPC 4
#define NALU_TYPE_IDR 5
#define NALU_TYPE_SEI 6
#define NALU_TYPE_SPS 7
#define NALU_TYPE_PPS 8
#define NALU_TYPE_AUD 9
#define NALU_TYPE_EOSEQ 10
#define NALU_TYPE_EOSTREAM 11
#define NALU_TYPE_FILL 12
#define NALU_PRIORITY_HIGHEST 3
#define NALU_PRIORITY_HIGH 2
#define NALU_PRIRITY_LOW 1
#define NALU_PRIORITY_DISPOSABLE 0
#ifndef TRUE
#define TRUE 1
#endif
#ifndef FALSE
#define FALSE 0
#endif
#ifndef BOOL
#define BOOL unsigned int
#endif
void ReportNalu(FILE* src,FILE* rep);
void ParseRecBuf(unsigned char *p,int bufsize,FILE* pReport);
int ShowBits (unsigned char buffer[],int totbitoffset,int bytecount, int numbits);
BOOL next_bits(unsigned char *p,int bufpos,int bs,int id );
static int FindStartCode (unsigned char *Buf, int bufpos, int bufsize, int next_start_pos, BOOL fIRFD);
BOOL gETNALandPROC(unsigned char *p,int bufpos,int bufsize, BOOL fisrtfd,NALU_t *mNal,int inc_SIZE,FILE
*pReport);
#endif
//############################################################
//# src 檔案格式如下
//# #-#-#-#-#-B-INT32LEN-T264DST - #-#-#-#-B-INT32LEN #
//# 1 1 1 1 1 1 4 INT32LEN #
//############################################################
void ReportNalu(FILE* src,FILE* rep)
{
unsigned char tempbuf[1];
unsigned char *pBuf;
int *dstlen;
int fpos=0;
int fstart=0;
int FileSize=0;
int BufSize;
int dstpacketid=0;
unsigned int SymState=0;
fseek(src,0,SEEK_END);
FileSize=ftell(src);
printf("the src file size is %d \n",FileSize);
fseek(src,0,SEEK_SET);
do {
while (fread(tempbuf,sizeof(unsigned char),1,src))
{
if(tempbuf=="#") SymState++;
else
{
fpos=fstart+1;
SymState=0;
fseek(src,fpos,SEEK_SET);
}
if(SymState==4)
{fpos+=4;break;}
}
fread(tempbuf,sizeof(unsigned char),1,src);
if(tempbuf=="B")
{
SymState++;
fpos+=1;
}
else
{
fpos=fstart+1;
fseek(src,fpos,SEEK_SET);
}
if(SymState==5)
{
dstpacketid++;
fprintf(rep,"the %d dstpacket\n",dstpacketid);
dstlen=(int*)malloc(sizeof(int));
fread(dstlen,sizeof(unsigned char),4,src);
BufSize=(int*)dstlen;
pBuf=(unsigned char*)malloc(sizeof(unsigned char)*BufSize);
fpos+=4;
fread(pBuf,sizeof(unsigned char),BufSize,src);
ParseRecBuf(pBuf,BufSize,rep);
fstart=fpos;
free(dstlen);
free(pBuf);
}
} while(fpos!=FileSize);//
}
//####################################################
//# unsigned char *p --T264 encode的一個dst長度為len#
//# int bufsize --buffer的大小 #
//##FILE* pReport --檔案指標，存放NAL的分析結果 #
//####################################################
void ParseRecBuf(unsigned char *p,int bufsize,FILE* pReport)
{
int bufpos=0;
int bUFSTART=0;
struct NALU_t *mNal;
BOOL isfirst;
BOOL starcode_ex;
BOOL zero_ex;
BOOL trail_ex;
int nALSIZE=0;
int nal_id=0;
BOOL firstfd=FALSE;
while (bufpos<bufsize)
{
if( !next_bits(p,bufpos,bufsize, 24 ) &&
!next_bits(p,bufpos,bufsize,32 ) &&
next_bits( p,bufpos,bufsize,8 ))
{bufpos+=8;isfirst=TRUE;firstfd=TRUE;}// leading_zero_8bits /* equal to 0x00 */ //f(8)
else{ isfirst=FALSE; }
if(!next_bits( p,bufpos,bufsize,24 ) &&
next_bits( p,bufpos,bufsize,8 )
)
{
bufpos+=8;//zero_byte /* equal to 0x00 */ f(8)
zero_ex=TRUE;
if( bufpos<bufsize &&
next_bits(p,bufpos,bufsize, 24 ) )
{
bufpos+=24; // start_code_prefix_one_3bytes /* equal to 0x000001 */ f(24)
starcode_ex=TRUE;
mNal=(struct NALU_t* )malloc(sizeof( NALU_t));
if(gETNALandPROC(p,bufpos,bufsize,firstfd,mNal,nALSIZE,pReport))
{
nal_id++;
fprintf(pReport," id of the nal packet above is %d\n",nal_id);
bufpos+=(nALSIZE>>3);
if(( bufpos<bufsize) &&
(!next_bits(p,bufpos,bufsize, 24 )) &&
(!next_bits(p,bufpos,bufsize, 32 ) ) &&
(next_bits(p,bufpos, bufsize,8 )))
{
bufpos+=8; trail_ex=TRUE ;/* trailing_zero_8bitsequal to 0x00 */
}
else trail_ex=FALSE;
free(mNal);
}
else printf("cannot get any nal units\n");
}
else{starcode_ex=FALSE;}
}
else
{ zero_ex=FALSE; }
if (!zero_ex || !starcode_ex)
bufpos=bUFSTART+1;
else bUFSTART=bufpos;
}
}
//############################################
//#unsigned char *p =--buffer containing NALU#
//#int bufpos--current buffer position #
//#int bs---buffersize #
//#int id #
//# id=24 ----0x000001----FALSE #
//# id=32 ----0x00000001---FALSE #
//# id=8 ----0x00------TRUE #
//############################################
BOOL next_bits(unsigned char *p,int bufpos,int bs,int id )
{
BOOL rERULT;
switch(id) {
case 8:
if(ShowBits(p,bufpos,bs,8)==0x00) rERULT=TRUE ;
else rERULT=FALSE;
break;
case 24:
if(ShowBits(p,bufpos,bs,24)!=0x000001) rERULT=FALSE;
else rERULT=TRUE;
break;
case 32 :
if(ShowBits(p,bufpos,bs,32)!=0x00000001) rERULT=FALSE;
else rERULT=TRUE;
break;
default:
break;
}
return rERULT;
}
//##############################################
// # \brief #
// # Reads bits from the bitstream buffer #
// # \param buffer #
// # buffer containing VLC-coded data bits #
// # \param totbitoffset #
// # bit offset from start of partition #
// # \param bytecount #
// # total bytes in bitstream #
// # \param numbits #
// # number of bits to read #
// #############################################
int ShowBits (unsigned char buffer[],int totbitoffset,int bytecount, int numbits)
{
register int inf;
long byteoffset; // byte from start of buffer
int bitoffset; // bit from start of byte
byteoffset= totbitoffset/8;
bitoffset= 7-(totbitoffset%8);
inf=0;
while (numbits)
{
inf <<=1;
inf |= (buffer[byteoffset] & (0x01<>bitoffset;
numbits--;
bitoffset--;
if (bitoffset < 0)
{
byteoffset++;
bitoffset += 8;
if (byteoffset > bytecount)
{
return -1;
}
}
}
return inf; // return absolute offset in bit from start of frame
}
//#########################################################################
// # \brief
#
// # returns if new start code is found at byte aligned position buf. #
// # new-startcode is of form N 0x00 bytes, followed by a 0x01 byte. #
// # \return
#
// # 1 if start-code is found or \n #
// # 0, indicating that there is no start code #
// # #
// # \param Buf #
// # pointer to byte-stream #
// # \param bufpos #
// # indicates current bufpos.
//# \bufsize
// # indicates total buffer size #
// # \param next_star_pos #
// # indicates the next_start_code pos #
// # \param fIRFD
// # is firs nal already found
// ########################################################################
static int FindStartCode (unsigned char *Buf, int bufpos, int bufsize, int next_start_pos, BOOL fIRFD)
{
int info;
int tPOS;
int tSTART=bufpos;
BOOL sTARFOUND=FALSE;
info = 1;
while (!sTARFOUND && tSTART<bufsize) {
for (tPOS = 0; tPOS < 3; tPOS++)
if(Buf[tSTART+tPOS] != 0)
info = 0;
if(Buf[tSTART+tPOS] != 1)
info = 0;
if (info==0) tSTART++;
else sTARFOUND=TRUE;
}
if (fIRFD && sTARFOUND)
{
if(Buf[tSTART-1]==0 &&
Buf[tSTART-2]==0)
{ info=2;//trailing zero found
next_start_pos=tSTART-2;}
else {info=1;
next_start_pos=tSTART;}
}
return info;
}
//#####################################
//# unsigned char *p #
//# int bufpos #
//# NALU_t *mNal #
//# int n_SIZE #
//# #
//# #
//#####################################
BOOL gETNALandPROC(unsigned char *p,
int bufpos,
int bufsize,
BOOL fisrtfd,
NALU_t *mNal,
int inc_SIZE,
FILE *pReport)
{
BOOL getNfailed=FALSE;
int nal_BUFSIZE=0;
int next_start_point=0;
int mNal_size=0;
int b_bufpos=bufpos/8;
unsigned int finresult;
BOOL info=TRUE;
finresult=FindStartCode (p,b_bufpos,bufsize,next_start_point,fisrtfd);
if (finresult==1 || finresult==2) {
nal_BUFSIZE=next_start_point-b_bufpos-1;
inc_SIZE=nal_BUFSIZE;
mNal->buf=(unsigned char *)malloc(sizeof(unsigned char)*nal_BUFSIZE);
mNal->len=nal_BUFSIZE;
memcpy (mNal->buf, &p[b_bufpos+1], mNal->len);
mNal->forbidden_bit = (mNal->buf[0]>>7) & 1;
mNal->nal_reference_idc = (mNal->buf[0]>>5) & 3;
mNal->nal_unit_type = (mNal->buf[0]) & 0x1f;
fprintf(pReport,"#########################################################################\n");
fprintf(pReport,"nal len is %d\n",mNal->len);
fprintf(pReport,"fobidden bit is % d\n",mNal->forbidden_bit);
fprintf(pReport,"nal_reference_idc is d%\n",mNal->nal_reference_idc);
switch(mNal->nal_unit_type) {
case NALU_TYPE_SLICE :
fprintf(pReport," nal type is * NALU_TYPE_SLICE\n");
break;
case NALU_TYPE_DPA:
fprintf(pReport,"nal type is * NALU_TYPE_DPA\n");
break;
case NALU_TYPE_DPB :
fprintf(pReport,"nal type is * NALU_TYPE_DPB\n");
break;
case NALU_TYPE_DPC:
fprintf(pReport,"nal type is * NALU_TYPE_DPC\n");
break;
case NALU_TYPE_IDR:
fprintf(pReport,"nal type is * NALU_TYPE_IDR\n");
break;
case NALU_TYPE_SEI:
fprintf(pReport,"nal type is * NALU_TYPE_SEI\n");
break;
case NALU_TYPE_SPS:
fprintf(pReport,"nal type is * NALU_TYPE_SPS\n");
break;
case NALU_TYPE_PPS:
fprintf(pReport,"nal type is * NALU_TYPE_PPS\n");
break;
case NALU_TYPE_AUD:
fprintf(pReport,"nal type is * NALU_TYPE_AUD\n");
break;
case NALU_TYPE_EOSEQ:
fprintf(pReport,"nal type is * NALU_TYPE_EOSEQ\n");
break;
case NALU_TYPE_EOSTREAM:
fprintf(pReport,"nal type is * NALU_TYPE_EOSTREAM\n");
break;
case NALU_TYPE_FILL :
fprintf(pReport,"nal type is * NALU_TYPE_FILL\n");
break;
default:
break;
}
fprintf(pReport,"nal start code len is %d\n",mNal->startcodeprefix_len);
fprintf(pReport,"#########################################################################\n");
free(mNal->buf);
mNal->buf=NULL;
}
else info=FALSE;
return info;
}