Minimaxity and Efficiency in Exponential Family Regression: From Star-Shaped to Convex Constraints

Authors

Yikun Li
Affiliation: Department of Statistics and Data Science, Northwestern University
Email: YikunLi2028@u.northwestern.edu

Guanghong Yi
Affiliation: Department of Statistics, George Mason University
Email: GuanghongYi2025@u.northwestern.edu

Matey Neykov
Affiliation: Department of Statistics and Data Science, Northwestern University
Email: mneykov@northwestern.edu

Abstract

This paper establishes the minimax estimation rate for nonparametric exponential family regression under star-shaped constraints. We consider a parameter space $K$ that is a star-shaped subset of the hypercube $[-M, M]^n$ for a known constant $M > 0$. We operate under the assumption that the underlying exponential family is nonsingular with a twice continuously differentiable log-partition function. Our main result demonstrates that the minimax rate of the $\ell _{2}$ error of such estimation problem is $\epsilon^{* 2} \wedge \operatorname{diam}(K)^2$ up to constants exclusively depending on $M$. Here, the critical radius $\epsilon^{* }$ is defined as \[ \epsilon^* = \sup \{\epsilon \left\lvert\right. \epsilon^2 \kappa(M) \le \log N^{\text{loc}}(\epsilon,c)\}, \] where $N^{\text{loc}}(\epsilon,c)$ denotes the local metric entropy of $K$, and $\kappa(M) > 0, c>0$ are constants depending only on $M$. Such minimax rate is established by a match between an information-theoretic lower bound and an upper bound implied by a theoretical algorithm.

Furthermore, we investigate the computational aspects of this estimation problem. Under mildly stronger assumptions on the constraint set $K$, we propose a computationally efficient, polynomial-time algorithm. We prove that the resulting estimator achieves the minimax optimal rate up to poly-logarithmic factors in the dimension $n$ and the geometric parameters of $K$.

Finally, to illustrate the efficacy of our framework, we derive the minimax optimal rates for some concrete examples.

The image illustrating the main concepts of the work.

Availability

The preprint of this work is available here, or you can find the up-to-date PDF file here.