####Math 502 Statistical Inference.####\\ ###Spring 2015### ~~META:title=(Archive) Math 502 Statistical Inference.~~ * **Instructor:** [[people:qiao:]] * **Email:** * **Phone number:** (607) 777-2593 * **Office:** WH 134 * **Meeting time & location: ** MWF 8:30 - 9:30 at WH 100E. * **Office hours: ** MW 3:00 - 5:00 ===== Prerequisite ===== Math 501. ===== Learning Objectives ===== - Coverage: Chapters 6 through 10 in Casella & Berger. - Sufficiency, completeness, likelihood, estimation, testing, decision theory, Bayesian inference, sequential procedures, multivariate distributions and inference, nonparametric inference, asymptotic theory. ===== Recommended Texts ===== The required text is **Casella & Berger** (see below). Some reference texts are listed below as well. * **Casella, G., & Berger, R. L. (2002)**. //Statistical inference.// Australia: Thomson Learning. * Lehmann, E. L. (1999). Elements of large-sample theory. New York: Springer. * Lehmann, E. L., & Casella, G. (1998). //Theory of point estimation.// New York: Springer. * Shao, J. (1999). //Mathematical statistics.// New York: Springer. * Shao, J. (2005). //Mathematical Statistics: Exercises and Solutions.// New York, NY: Springer. * Hogg, R. V. and Craig, A. (1995). //Introduction to Mathematical Statistics.// Prentice Hall, Englewood Cliffs, NJ ===== Grading ===== * Homework (40%): there will be weekly homework assignments, due at the beginning of each Wednesday class. * Midterm exams (20%+20%): there will be two midterm exams. * Midterm exam 1: Friday, February 27, 2015 * Midterm exam 2: Friday, April 3, 2015 * Final exam (20%): Wednesday, May 13, 2015 from 8:00 AM to 10:00 AM. ===== Homework assignments ===== ==== Week 1 ==== * 01/28: Notes, Example (2), (a)-(e); Textbook, Exercises 6.1, 6.3. * 01/30: Notes, Example (4), b and c. Use both methods and for each method try to use different representations (so that your answers are not unique). Textbook, Exercises 6.2, 6.5, 6.8, 6.9. Due on 02/04 ---- ==== Week 2 ==== * 02/04 & 06: Textbook, Exercises 6.10, 6.11, 6.13, 6.14, 6.15, 6.18, 6.19, 6.20, 6.23, 6.25. * 02/09: - Let $X_{1}\cdots X_{n}$ be i.i.d with density function defined as $f(x|\theta)=e^{-\lambda(x-\mu)},~x>\mu,~\lambda>0,~\mu\in \mathbb{R}$. Prove that $(X_{(1)},W)$ is the sufficient statistic of $\theta=(\mu,\lambda)$, where $W=\sum^{n}_{i=2}(X_{(i)}-X_{(1)})$. - Textbook: Exercises 7.1, 7.2, 7.6. Due on 02/11 ---- ==== Week 3 ==== * 02/11: Textbook, Exercises 7.7, 7.8, 7.10, 7.11. * 02/13: - Notes: Show that a Bayes estimator depends on the data through a sufficient statistic. - Notes: if $X_i$'s are iid given $\theta$, are they iid marginally? Why? - Textbook, Exercises 7.14, 7.22, 7.23 (read "conjugate prior" as "prior"), 7.24, 7.25. Solutions to some of the questions may appear clearer after you read the lecture notes for Monday's class. * 2/16: Textbook: Exercises 7.9, 7.12, 7.50. Due on 02/18 ---- ==== Week 4 ==== * 02/18: Textbook, Exercises 7.19, 7.20, 7.21. * 02/20: Textbook, Exercises 7.37, 7.46, 7.49, 7.51, 7.52. * 02/23: Textbook, Exercises 7.53, 7.57, 7.58. Due on 02/25 ---- ==== Week 5 ==== * 02/27: Textbook, Exercises 7.59, 7.44, 7.48, 7.60, 7.63. * 03/02: Textbook, Exercises 7.40, 7.65, 7.66. Due on 03/06 ---- ==== Week 6 ==== * 03/06: Textbook, Exercises 8.1, 8.2, 8.3, 8.5, 8.6. * 03/09: Textbook, Exercises 8.7, 8.8, 8.9. Due on 03/11 ---- ==== Week 7 ==== * 03/11: Textbook, Exercises 8.12, 8.13, 8.15, 8.17, 8.20. * 03/13: Textbook, Exercises 8.19, 8.21. * 03/16: * Textbook, Exercises 8.25, 8.27. * In the class, I showed that if we remove $k>0$ from the necessity condition of the NPL, then when $k=0$, we must have $\beta_\phi(\theta_1)=1$ for the UMP level $\alpha$ test $\phi$. Complete my work by arguing why in this case $\phi$ still must satisfy equation (2) in the NPL, that is, why it must be the case that $\phi=1$ when $f(x|\theta_1)>0$ wp1. [Hint: $\phi\le 1$. This proof should not last more than 2 lines.] Due on 03/18 ---- ==== Week 8 ==== * 03/18: Textbook, Exercises 8.28, 8.29, 8.30, 8.33. * 03/20: Textbook, Exercises 8.37, 8.38, 8.47. * 03/23: Textbook, Exercises 9.1, 9.2, 9.3. * Additional: Carry out the following simulation project. Submit the R code and report the result properly. - Use R to generate 10 observations from $N(1,4)$. - Now pretend that you only known that the data were from $N(\mu,4)$ without knowing $\mu$ and construct a 80% confidence interval for $\mu$. - Repeat Steps 1 and 2 100 times. * Count the proportion among the 100 trials where the C.I. contains the true mean? * What is the relation between the proportion and the confidence coefficient? * Repeat Steps 1, 2 and 3, but pretend that you know neither the mean $\mu$ nor the variance $\sigma^2$. Then compare the lengths of the confidence intervals between the current and the previous settings. Make comments on the lengths and discuss why there is a difference. Due on 03/25 ---- ==== Week 9 ==== * 03/25: Submit all your code and output, preferably using LaTex. In a numerical problem, unless stated otherwise, $1-\alpha=0.95$. Textbook, Exercises - 9.4; In addition, assume that $n=10$ and $m=15$ and that $\sigma_X^2=1$ and $\sigma_Y^2=3$, generate some $X_i$'s and $Y_j$'s. Then use a numerical method to provide a CI based on the generated (observed) data. Then repeat the whole process for 1000 times. Report the number of time that the true $\lambda=3$ is covered by the CIs. - 9.6; here assume that $X\sim bin(n,p)$ is observed and $n$ is known. Next, let $n=50$ and generate $X$ with $p=0.3$. Numerically provide the CI for the observed $X$. Repeat for 1000 times and report the number of times that the true $p=0.3$ is covered by the CI. - 9.12. - 9.13(b). * 03/27: Textbook, Exercises: 9.16, 9.17 and 9.23. In 9.17, you need to find the shortest confidence interval using the pivotal method (and prove it using a result in class). Moreover, find in addition a second CI using pivotal method with equal left and right probabilities. Assume that $\alpha=0.05$ and verify that your shortest confidence interval is indeed shorter than the second one. * 03/30: - Textbook, Exercise: 9.37 - Assume that $X_1,\dots,X_n$ are iid from Cauchy, where $f(x)=[\pi (1+x^2)]^{-1}$. - Calculate $\int_{-\infty}^\infty |x|f(x)dx$. - What is the mean of $X_1$? - Can we apply the SLLN to prove that $\overline X_n\rightarrow \mu_X$ a.s.? - Let $n=100$, simulate the sample and calculate $\overline X_n$. Then repeat this for 500 times. Collect all the $\overline X_n$'s and sort them (from the smallest to the greatest) and plot the sorted $\overline X_n$ values. Due on 04/1 ---- ==== Week 10/11/12 ==== * 03/27: Textbook, Exercises: 10.1, 10.2 * Spring break * 04/17: -Let $W_n$ be a random variable with mean $\mu$ and variance $C/n^\nu$ with $\nu>0$. Prove that $W_n$ is consistent with $\mu$. -Let $Y_n$ be the $n$th order statistic of a random sample of size $n$ from uniform$(0,\theta)$. Prove that $\sqrt{Y_n}$ is consistent with $\sqrt{\theta}$. Can you use Theorem 1 on page 51 of the lecture notes? -Let $Y_n$ be the $n$th order statistic of a random sample of size $n$ with continuous CDF $F(\cdot)$. Define $Z_n=n[1-F(Y_n)]$. Find the limiting distribution of $Z_n$. That is, is $Z_n$ convergent to some random variable, in what mode? -In the question above, let $F$ be the CDF for standard normal. Let $n$ be a large number. Then numerically verify you claim of the limiting distribution above by comparing $P(Z_n\le t)$ with $P(Z\le t)$ for arbitrary $t$ where $Z$ is the limiting random variable of $Z_n$. -In general, $X_n\Rightarrow X$ and $Y_n\Rightarrow Y$ cannot imply $X_n+Y_n\Rightarrow X+Y$. Please give a counterexample to illustrate this. The symbol $\Rightarrow$ means convergence in distribution. * 04/20: Textbook, - Exercises: 10.4, 10.5, 10.6. - For $X\sim bin(n,p)$, let $\tau(p)=1/(1-p)$. What can we say about $\hat{\tau}$ for $p\ne 1$? - For $X_1,\dots,X_n\sim Unif(0,\theta)$, find the MLE of $\theta$. Find an unbiased estimator based which is a function of the MLE. Calculate the variance of this unbiased estimator. Calculate the theoretical optimal variance due to the CRLB. Compare them. Due on 04/22 ---- ==== Week 13 ==== * 4/22: Textbook. Exercises: 10.8, 10.19(a), 10.35. * 4/24: Textbook. Exercises: 10.31, 10.32, 10.33, 10.34, 10.36, 10.37 * 4/27: In exercise 10.36, you were asked to derive two Wald statistics to run approximate large sample test. Now let $n=25$, $\alpha=1$, $H_0:\beta=\beta_0=2$. Please numerically compare the power of these two test when the true value of $\beta$ is 3, by running the test on the data for 10,000 times, and see which one rejects the null hypothesis more often. Try to interpret the result. I am not satisfied with some of your answers to 9.23 in the homework returned today. I am giving a second chance for those who lost points for 9.23. You may submit your new answers (especially the numerical answers) along with this homework. I will consider adding back some points to that homework assignment. Please indicate that how many points you lost for 9.23. For the numerical answer, I have provided a Monte Carlo method to calculate the p value in the solution. You should use some other approach. For example, you can calculate the p value by taking the sum of the probabilities of $x$ which satisfies $LR(x)$p_3=0.3$, $n=100$, using simulation. Note that this is the probability of making type II error. Present the program. - Compute $P(H_o|H_1)$ when $p_1=p_2=p_5$, $p_3=0.3$, $n=1$. - Remark 1: in computing, sometimes it is better to use $\log(0^0)$ instead of $0*\log(0)$ as the latter can cause numerical trouble. - Remark 2: What is the difference in degrees of freedom? Think how many additional constraints are imposed. - Remark 3: You can try several combinations of $p_k$'s that satisfy $p_1=p_2=p_5$, $p_3=0.3$. myfun=function(n){ m=1000 x=rgamma(m,n,1)/n # m X’s y=-2*(n*log(x)+n*(1-x)) # m λ’s u=rchisq(m,1) qqplot(y,u,main=paste("QQ plot, n=",n)) lines(y,y) sy=sort(y) plot(sy,ppoints(sy), xlim=c(0.5,2), ylim=c(0.4,0.9), type="l", lty=1, main=paste("CDF, n=",n)) lines(sy,pchisq(sy,1), xlim=c(0.5,2), ylim=c(0.4,0.9), type="l", lty=2) } pdf("fig10.pdf",height=9.0, width=6.5) par(mfrow=c(2,2)) n=1 myfun(n) n=100 myfun(n) dev.off() Due on 05/06