第4章n-多元回归估计与假设检验.ppt

单击此处编辑母版标题样式,单击此处编辑母版文本样式,第二级,第三级,第四级,第五级,*,单击此处编辑母版标题样式,单击此处编辑母版文本样式,第二级,第三级,第四级,第五级,*,单击此处编辑母版标题样式,单击此处编辑母版文本样式,第二级,第三级,第四级,第五级,*,单击此处编辑母版标题样式,单击此处编辑母版文本样式,第二级,第三级,第四级,第五级,*,单击此处编辑母版标题样式,单击此处编辑母版文本样式,第二级,第三级,第四级,第五级,*,单击此处编辑母版标题样式,单击此处编辑母版文本样式,第二级,第三级,第四级,第五级,*,单击此处编辑母版标题样式,单击此处编辑母版文本样式,第二级,第三级,第四级,第五级,*,单击此处编辑母版标题样式,单击此处编辑母版文本样式,第二级,第三级,第四级,第五级,*,1,第,4,章多元回归分析,:,估计与假设检验,Multiple Regression Analysis,y=,b,0,+,b,1,x,1,+,b,2,x,2,+.,b,k,x,k,+u,Estimation and Inference,2,Parallels with Simple Regression,Y,i,=,b,0,+,b,1,X,i1,+,b,2,X,i2,+.,b,k,X,ik,+u,i,b,0,is still the,intercept,b,1,to,b,k,all called,slope parameters,also called,partial regression coefficients,and any coefficient,b,j,denote the change of,Y,with the changes of,X,j,as,all the other independent variables fixed,.,u,is still the,error term,(or,disturbance,),Still minimizing the sum of squared residuals,so have k+1 first order conditions,3,Obtaining OLS Estimates,4,Obtaining OLS Estimates,cont.,The above estimated equation is called the,OLS regression line,or the,sample regression function(SRF),the above equation is the estimated equation,is not the really equation.The really equation is population regression line which we don,t know.We only estimate it.So,using a different sample,we can get another different estimated equation line.The,population regression line,is,5,Interpreting Multiple Regression,8,A,“,Partialling Out,”,Interpretation,9,“,Partialling Out,”,continued,Previous equation implies that regressing,Y,on,X,1,and,X,2,gives same effect of,X,1,as regressing,Y,on residuals from a regression of,X,1,on,X,2,This means only the part of,X,i1,that is uncorrelated with,X,i2,are being related to,Y,i,so we,re estimating the effect of,X,1,on,Y,after,X,2,has been,“,partialled out,”,10,The wage determinations,The estimated equation as below:,wage=,-,2.873+0.599,educ+,0.022,exper+,0.169,tenure,Now,we first regress,educ,on,exper,and,tenure,to patial out the,exper,and,tenure,s effects.Then we regress,wage,on the,residuals,of,educ,on,exper,and,tenure,.Whether we get the same result.?,educ=,13.575-0.0738,exper+,0.048,tenure,denote residuals,resid,wage,=5.896+0.599,resid,We can see that the coefficient of,resid,is the same of the coefficien of the variable,educ,in the first estimated equation.So is,in the second equation.,11,Goodness-of-Fit:,R,2,12,Goodness-of-Fit,13,Goodness-of-Fit(continued),How do we think about how well our sample regression line fits our sample data?,Can compute the fraction of the total sum of squares(SST)that is explained by the model,call this the R-squared of regression,R,2,=ESS/TSS=1 RSS/TSS,14,Goodness-of-Fit(continued),15,More about,R,-squared,R,2,can never decrease when another independent variable is added to a regression,and usually will increase,Because,R,2,will usually increase with the number of independent variables,it is not a good way to compare models,16,An Example,Using wage determination model to show that when we add another new independent variable will increase the value of R,2.,17,Adjusted,R,-Squared,R,2,is simply an estimate of how much variation in,y,is explained by,X,1,X,2,X,k,.,That is,Recall that the,R,2,will always increase as more variables are added to the model,The adjusted,R,2,takes into account the number of variables in a model,and may decrease,18,Adjusted,R,-Squared(cont),Most packages will give you both,R,2,and adj-,R,2,You can compare the fit of 2 models(with the same,Y,)by comparing the adj-,R,2,wge=-,3.391,+,0.644,educ+,0.070,exper,adj,-R,2,=0.2222,wge=-,2.222+0.569,educ+,0.190,tenure,adj,-R,2,=0.2992,You cannot use the adj-,R,2,to compare models with different,y,s(e.g.,y,vs.ln(,Y,),wge=-,3.391,+,0.644,educ+,0.070,exper,adj,-R,2,=0.2222,log,(,wge,),=,0.404,+,0.087,educ+,0.026,exper,adj,-R,2,=0.3059,Because the variance of the dependent variables is different,the comparation btw them make no sense.,19,Assumptions for Unbiasedness,20,Assumptions for Unbiasedness,Population model is,linear,in parameters:,Y=,b,0,+,b,1,X,1,+,b,2,X,2,+,b,k,X,k,+,u,We can use a,sample,of size,n,(,X,i1,X,i2,X,ik,Y,i,):,i,=1,2,n,from the population model,so that the sample model is,Y,i,=,b,0,+,b,1,X,i1,+,b,2,X,i2,+,b,k,X,ik,+,u,i,Cov(,uX,i,)=0,E(,uX,i,)=0,i,=1,2,n.,E(,u|X1,X2,Xk,)=0,implying that all of the explanatory variables are,exogenous.,E(,u|,X,)=0,where,X,=(,X1,X2,Xk,),which will reduce to E(,u,)=0 if independent variables,X,are not random variables.,None of the,X,s is constant,and there are no,exact linear,relationships among them.,The new additional assumption.,21,About multicollinearity,It does allow the independent variables to be correlated;they just cannot be,perfectly linear correlated,.,Student performance:,colGPA=,b,0,+b,1,hsGPA+,b,2,ACT+,b,3,skipped+u,Consumption function:,consum=,b,0,+b,1,inc,+,b,2,inc,2,+u,But,the following is invalid:,log(consum)=,b,0,+b,1,log,(,inc,),+,b,2,log,(,inc,2,),+u,In this case,we can not estimate the regression coefficients,b,1,b,2,.,22,Unbiasedness of OLS estimation,Under the three assumptions above,we can get,23,Too Many or Too Few Variables,24,Too Many or Too Few Variables,What happens if we include variables in our specification that don,t belong?,There is no effect on our parameter estimate,and OLS remains unbiased,What if we exclude a variable from our specification that does belong?,OLS will usually be biased,25,Omitted Variable Bias,26,Omitted Variable Bias(cont),27,Omitted Variable Bias(cont),28,Omitted Variable Bias(cont),There are two cases where the estimated parameter is unbiased:,If,b,2,=0,so that,X,2,does not appear in the true model,If tilde of,d,1,=0,the tilde,b,1,is unbiased for,b,1,29,Summary of Direction of Bias,Corr(X,1,X,2,)0,Corr(,X,1,X,2,)0,Positive bias,Negative bias,b,2,0 and H,1,:,b,j,0,One-Sided Alternatives(cont),0,c,a,(1-a),Fail to reject,reject,58,An Example:Hourly Wage Equation,Wage determination:(wooldridge,p123),log(,w,ge,)=0.284+0.092,educ+,0.0041,exper+,0.022,tenure,(0.104)(0.007)(0.0017)(0.003),n,=526 R,2,=0.316,Whether the return to,exper,controlling for,educ,and,tenure,is zero in the population,against the alternative that it is positive.,H,0,:,b,exper,=,0 vs.H,1,:,b,exper,0,The,t,statistic is,t,=0.0041/0.0017,2.41,The degree of freedom:,df=n-k-1=526-3-1=522,The critical value of 5%is 1.645,And the t statistic is larger than the critical value,ie.,2.411.645,That is,we will reject the null hypothesis and,b,exper,is really positive.,0,1.645,(1-a),Fail to reject,5%,reject,59,Another example:Student Performance and School Size,Whether the school size has effect on student performance?,math10,math test scores,reveal the student performance,totcomp,average annual teacher compensation,staff,the number of staff per one thousand students,enroll,student enrollment,reveal the school size.,The Model Equation,math10,=,b,0,+,b,1,totcomp,+,b,2,staff,+,b,3,enroll+u,H,0,:,b,3,=0,H,1,:,b,3,-1.645,so we can,t reject the null hypothesis.,-1.645,reject,-091,60,One-sided vs Two-sided,Because the,t,distribution is symmetric,testing H,1,:,b,j,0 is straightforward.The critical value is just the negative of before,We can reject the null if the,t,statistic,c,then we fail to reject the null,For a two-sided test,we set the critical value based on,a,/2 and reject H,0,:,b,j,=0 if the,absolute value,of the,t,statistic,c,61,y,i,=,b,0,+,b,1,X,i1,+,+,b,k,X,ik,+u,i,H,0,:,b,j,=0 H,1,:,b,j,0,c,0,a/2,(1-a),-c,a/2,Two-Sided Alternatives,reject,reject,fail to reject,62,Summary for H,0,:,b,j,=0,Unless otherwise stated,the alternative is assumed to be,two-sided,If we reject the null,we typically say,“,X,j,is,statistically significant at the 100,a,%level,”,If we fail to reject the null,we typically say,“,X,j,is,statistically insignificant at the 100,a,%level,”,63,An Example:Determinants of College GPA(wooldridge,p128),Variables:,colGPA,college GPA,skipped,the average number of lectures missed per week,ACT,achievement test score,hsGPA,high school GPA,The estimated model,olGPA=1.39+0.412 hsGPA+0.015 ACT 0.083 skipped,(0.33)(0.094)(0.011)(0.026),n=141,R,2,=0.234,H,0,:,b,skipped,=0,H,1,:,b,skipped,0,fd:n-k-1=137,the critical value,t,137,=1.96,The t statistic is|-0.083/0.026|=3.19,t,137,=1.96,so we will reject the null hypothesis and,the,b,skipped,is signanificantly beyond zero.,-1.96,reject,-3.19,1.96,reject,64,Testing other hypotheses,A more general form of the,t,statistic recognizes that we may want to test something like H,0,:,b,j,=,a,j,In this case,the appropriate,t,statistic is,65,An Example:Campus Crime and Enrollment(wooldridge,p129),Variables,crime,the annual number of crimes on college campuses,enroll,student enrollment,reveal the size of college.,The regression model,log(,crime,)=,b,0,+,b,1,log(,enroll,)+,u,Whether,b,1,=1,that is H,0,:,b,1,=1,H,1,:,b,1,1,log(,crime,)=,-6.63,+1.27 log(,enroll,),(1.03)(0.11),n=97 R,2,=0.585,df:n-k-1=95,the critical value at 5%is t,95,=1.645,The t-statistic is(1.27-1)/0.11,2.45,t,95,=1.645,So we reject the null hypothesis and the evidence prove that,b,1,1.,66,Confidence Intervals,Another way to use classical statistical testing is to construct a confidence interval using the same critical value as was used for a two-sided test,A 100(1-,a,)%confidence interval is defined as,67,Computing,p,-values for,t,tests,An alternative to the classical approach is to ask,“,what is the smallest significance level at which the null would be rejected?,”,So,compute the,t,statistic,and then look up what percentile it is in the appropriate,t,distribution,this is the,p,-value,p,-value is the probability we would observe the,t,statistic we did,if the null were true,68,Stata and,p,-values,t,tests,etc.,Most computer packages will compute the,p,-value for you,assuming a two-sided test,If you really want a one-sided alternative,just divide the two-sided,p,-value by 2,Stata provides the,t,statistic,p,-value,and 95%confidence interval for H,0,:,b,j,=0 for you,in columns labeled,“,t,”,“,P|t|,”,and,“,95%Conf.Interval,”,respectively,69,Testing a Linear Combination,Suppose instead of testing whether,b,1,is equal to a constant,you want to test if it is equal to another parameter,that is H,0,:,b,1,=,b,2,or,b,1,-,b,2,=0,Use same basic procedure for forming a t statistic,70,Testing Linear Combination(cont),71,Testing a Linear Combo(cont),So,to use formula,need,s,12,which standard output does not have,Many packages will have an option to get it,or will just perform the test for you,In Stata,after reg,Y X,1,X,2,X,k,you would type,test,X,1,=X,2,to get a,p,-value for the test,More generally,you can always restate the problem to get the test you want,72,Example:,Suppose you are interested in the effect of campaign expenditures on outcomes,Model is,voteA,=,b,0,+,b,1,log(,expendA,)+,b,2,log(,expendB,)+,b,3,prtystrA,+,u,H,0,:,b,1,=-,b,2,or H,0,:,q,1,=,b,1,+,b,2,=0,b,1,=,q,1,b,2,so substitute in and rearrange,voteA,=,b,0,+,q,1,log(,expendA,)+,b,2,log(,expendB),log(,expendA,)+,b,3,prtystrA,+,u,73,Example(cont):,This is the same model as originally,but now you get a standard error for,b,1,b,2,=,q,1,directly from the basic regression,Any linear combination of parameters could be tested in a similar manner,Other examples of hypotheses about a single linear combination of parameters:,b,1,=1+,b,2,;,b,1,=5,b,2,;,b,1,=-1/2,b,2,;etc,74,Multiple Linear Restrictions,Everything we,ve done so far has involved testing a single linear restriction,(e.g.,b,1,=,0,or,b,1,=,b,2,),However,we may want to jointly test multiple hypotheses about our parameters,A typical example is testing,“,exclusion restrictions,”,we want to know if a group of parameters are all equal to zero,75,Testing Exclusion Restrictions,Now the null hypothesis might be something like H,0,:,b,k-q+1,=0,.,b,k,=0,The alternative is just H,1,:H,0,is not true,Can,t just check each,t,statistic separately,because we want to know if the,q,parameters are,jointly,significant at a given level it is possible for none to be individually significant at that level,76,Exclusion Restrictions(cont),To do the test we need to estimate the,“,restricted model,”,without,X,k-q+1,X,k,included,as well as the,“,unrestricted model,”,with all,X,s included,Intuitively,we want to know if the change in RSS is big enough to warrant inclusion of,X,k-q+1,X,k,77,The,F,statistic,The,F,statistic is always positive,since the RSS from the restricted model can,t be less than the RSS from the unrestricted.,Essentially the,F,statistic is measuring the relative increase in RSS when moving from the unrestricted to restricted model,q,=number of restrictions,or,df,r,df,ur,n k,1=,df,ur,78,The,F,statistic(cont),To decide if the increase in RSS when we move to a restricted model is,“,big enough,”,to reject the exclusions,we need to know about the sampling distribution of our,F,stat,Not surprisingly,F,F,q,n-k-1,where,q,is referred to as the numerator degrees of freedom and,n k,1 as the denominator degrees of freedom,79,0,c,(1-a),f(,F,),F,The,F,statistic(cont),reject,fail to reject,Reject H,0,at,a,significance level,if,F,c,a,80,Example:the determinations of league baseball player,s salaries(wooldridge,p143),The regression model,log(,salary,)=,b,0,+,b,1,year+,b,2,gamesyr+,b,3,bavg+,b,4,hrunsyr+,b,5,rbisyr+u,salary,the 1993 total salary,years,years in the league,gamesyr,average games played per year,bavg,career batting average,hrunsyr,home runs per year,rbisyr,runs battled in per year,The null hypothesis is H,0,:,b,3,=0,b,4,=0,b,5,=0,which is called,multiple hypotheses test,or,joint hypotheses test.,The alternative hypothesis is H,1,:H,0,is not true.The unrestricted model:,log(,salary,)=,11.19,+0.0689,year+,0.0126,gamesyr+,0.00098,bavg+,0.0144,hrunsyr+,0.0108,rbisyr,(0.29)(0.0689)(0.0026)(0.00110)(0.0161)(0.0072),n=353,SSR=183.186,R,2,=0.6278,The restricted model,log(,salary,)=,11.22,+0.0713,year+,0.0202,gamesyr,(0.11)(0.0125)(0.0013),n=353,SSR=198.311,R,2,=0.5971,81,Example:the determinations of league baseball player,s salaries,cont.,The restricted number and the degree of the freedom of restricted model is q=3;,The degree of freedom of unrestricted model is 353-5-1=347;,Then the F statistic is,82,The,R,2,form of the,F,statistic,Because the RSS,s may be large and unwieldy,an alternative form of the formula is useful,We use the fact that RSS=TSS(1 R,2,)for any regression,so can substitute in for RSS,u,and RSS,ur,83,Example:Parent,s Education in a Birth Weight Equation(wooldridge,p150),Variables,bwght,birth weight in pounds;,cigs,average number of cigarettes the mother smoked per day during pregnancy;,parity,the birth order of this child;,faminc,annual family income;,motheduc,years of schooling for the mother;,fatheduc,years of schooling for the father.,Model:,bwght=,b,0,+,b,1,cigs+,b,2,parity+,b,3,faminc+,b,4,motheduc+,b,5,fatheduc+u,Whether the parents,education has any effect on birth weight?This is stated as H,0,:,b,4,=0,b,5,=0,so,q,=2.,bwght=,114.524-0.596,cigs+,1.788,parity+,0.056,faminc-,0.370,motheduc+,0.472,fatheduc,(3.728)(0.110)(0.659)(0.037)(0.320)(0.283),n=1191 R,2,=0.0387,bwght=,115.47