Lesson 7 : Decoding MPEG I Frames | Lesson 9 : Audio Buffers | Contents
In the previous lesson we how to decode I Frames. Now we will extend the program we wrote to decode P and B Frames as well.
Decoding P and B frames are much more complicated, since they depend on other frames. So we have to buffer a previous frame and a future frame to decode P and B frames. The name "future frame" indicates that the frame should be displayed after the current frame. However, it is actually encoded before the current frame. So the display order of frames in a MPEG sequence is different from the decoding order. In this example, we are concerned only with decoding order of the frames.
The reference frame for a P frame is the previous I or P frame. The reference frames for a B frame is the previous two I or P frames (in decoding order). We keep these reference frames in ByteImages prev{y,u,v} and future{y,u,v}, and use swap to swap the "pointer" to these ByteImages as we decode new I and P frames.
P and B frames are encoded using motion vectors that indicate how much each macroblock (16 x 16 pixels) of the reference frame(s) moved (this is sort of how it's done; we will not go into all of the details here). Dalì provides VectorImages to store these motion vector. Each macroblock in the frame has a motion vector, so the size of the VectorImage is w/16 x h/16, where w x h is the dimension of the frame.
The VectorImages, along with ScImages, are initialized when P and B frames are parsed from a BitStream. With VectorImages and reference ByteImages the ScImages can be decoded into YUV ByteImages.
The code is shown below with changes highlighted in blue.
cd c:/Dali/doc/tutorial/ package require DvmBasic package require DvmMpeg package require DvmPnm package require DvmColor
proc swap {a b} {
upvar $a aa
upvar $b bb
set temp $aa
set aa $bb
set bb $temp
}
set bp [bitparser_new] set bs [bitstream_mmap_read_new tennis.mpg] bitparser_wrap $bp $bs set sh [mpeg_seq_hdr_new] mpeg_seq_hdr_find $bp mpeg_seq_hdr_parse $bp $sh set w [mpeg_seq_hdr_get_width $sh] set h [mpeg_seq_hdr_get_height $sh] set y [byte_new $w $h]
set prevy [byte_new $w $h] set futurey [byte_new $w $h] set prevu [byte_new [expr $w/2] [expr $h/2]] set futureu [byte_new [expr $w/2] [expr $h/2]] set prevv [byte_new [expr $w/2] [expr $h/2]] set futurev [byte_new [expr $w/2] [expr $h/2]] set fwdmv [vector_new [expr $w/16] [expr $h/16]] set bwdmv [vector_new [expr $w/16] [expr $h/16]]
set r [byte_new $w $h]
set g [byte_new $w $h]
set b [byte_new $w $h]
set u [byte_new [expr $w/2] [expr $h/2]]
set v [byte_new [expr $w/2] [expr $h/2]]
set scy [sc_new [expr $w/8] [expr $h/8]]
set scu [sc_new [expr $w/16] [expr $h/16]]
set scv [sc_new [expr $w/16] [expr $h/16]]
set fh [mpeg_pic_hdr_new]
set len [mpeg_pic_hdr_find $bp]
set counter 0
while {1} {
mpeg_pic_hdr_parse $bp $fh
set type [mpeg_pic_hdr_get_type $fh]
if {$type == "i"} {
swap futurey prevy
swap futureu prevu
swap futurev prevv
mpeg_pic_i_parse $bp $sh $fh $scy $scu $scv
sc_i_to_byte $scy $y
sc_i_to_byte $scu $u
sc_i_to_byte $scv $v
yuv_to_rgb_420 $y $u $v $r $g $b
swap y futurey
swap u futureu
swap v futurev
} elseif { $type == "p"} {
swap futurey prevy
swap futureu prevu
swap futurev prevv
mpeg_pic_p_parse $bp $sh $fh $scy $scu $scv $fwdmv
sc_p_to_y $scy $fwdmv $prevy $y
sc_p_to_uv $scu $fwdmv $prevu $u
sc_p_to_uv $scv $fwdmv $prevv $v
yuv_to_rgb_420 $y $u $v $r $g $b
swap y futurey
swap u futureu
swap v futurev
} else {
mpeg_pic_b_parse $bp $sh $fh $scy $scu $scv $fwdmv $bwdmv
sc_b_to_y $scy $fwdmv $bwdmv $prevy $futurey $y
sc_b_to_uv $scu $fwdmv $bwdmv $prevu $futureu $u
sc_b_to_uv $scv $fwdmv $bwdmv $prevv $futurev $v
yuv_to_rgb_420 $y $u $v $r $g $b
}
set len [mpeg_pic_hdr_find $bp]
if {$len == -1} {
break
}
incr counter
puts $counter }
Try to modify the above program so that it output the frames in display order. There is a field in the picture header that indicates the frame number in display order. It can be retrieved by mpeg_pic_hdr_get_temporal_ref.
In the next three lessons, we will say good bye to MPEG temporarily and look at Dalì support for audio.
As usual, deallocate all structures.
mpeg_pic_hdr_free $fh mpeg_seq_hdr_free $sh bitstream_mmap_read_free $bs bitparser_free $bp byte_free $r byte_free $g byte_free $b byte_free $y byte_free $u byte_free $v byte_free $prevy byte_free $prevu byte_free $prevv byte_free $futurey byte_free $futureu byte_free $futurev sc_free $scy sc_free $scu sc_free $scv vector_free $fwdmv vector_free $bwdmv
Code for this lesson: l8.tcl.
Lesson 7 : Decoding MPEG I Frames | Lesson 9 : Audio Buffers | Contents
Last Updated : 05/01/2025 09:31:46