第二章基本信息论5_信源冗余度.ppt

,单击此处编辑母版标题样式,单击此处编辑母版文本样式,第二级,第三级,第四级,第五级,*,2.5,信源冗余度,非平稳信源熵：假设平稳，极限熵,平稳信源熵：,m,阶马尔可夫信源：,一阶马尔可夫信源：,无记忆信源熵：,等概率无记忆信源熵：,例,二元信源，若信源符号0和1等概率分布，且符号间无相关性，则其信源熵达到最大值：,若发送12个符号，则12个符号含有的信息量为：,若信源符号间有相关性，则信源熵达不到最大熵。若实际上为0.8比特/符号，则发送12个符号只能传递12*0.8=9.6比特的信息量。,若要传递12比特的信息量，则需要,n,=12/0.8=15,个符号，有3个符号的冗余。,一、冗余度的定义,冗余度（剩余度）,相对熵,信息变差,H,0,：,符号等概率且无相关性的理想离散信源熵,例,二元信源，若信源符号0和1等概率分布，且符号间无相关性，则其信源熵达到最大值：,当符号间有相关性时，实际熵：,信源相对熵：,信源冗余度：,信息变差：,例,信源为26个英文字母和1个空格，分析其信源熵。,英语字母,出现概率,英语字母,出现概率,英语字母,出现概率,A,0.064,J,0.001,S,0.051,B,0.013,K,0.005,T,0.08,C,0.022,L,0.032,U,0.023,D,0.032,M,0.020,V,0.008,E,0.103,N,0.057,W,0.018,F,0.021,O,0.063,X,0.001,G,0.015,P,0.015,Y,0.016,H,0.047,Q,0.001,Z,0.001,I,0.058,R,0.048,空格,0.189,1）若信源等概率分布，且无相关性，则信源熵：,2）按实际概率分布，且无相关性，则信源熵：,3）看成一阶马尔可夫信源，则信源熵：,4）看成二阶马尔可夫信源，则信源熵：,5）看成无穷阶马尔可夫信源，则信源熵：,消息的冗余为提高通信效率、压缩信号容量提供了基础。,二、冗余的利用,信源编码：通过减少冗余来提高通信效率,适当的冗余可提高抗干扰能力,信道编码：通过增加冗余来提高通信的抗干扰能力,