Note to Self: Norm of a Gaussian Random Vector

Let g be a standard Gaussian vector—that is, a vector populated by independent standard normal random variables. What is the expected length \mathbb{E} \|g\| of g? (Here, and throughout, \|\cdot\| denotes the Euclidean norm of a vector.) The length of g is the square root of the sum of n independent standard normal random variables

    \[\|g\| = \sqrt{g_1^2 + \cdots + g_n^2},\]

which is known as a \chi random variable with n degrees of freedom. (Not to be confused with a \chi^\mathbf{2} random variable!) Its mean value is given by the rather unpleasant formula

    \[\mathbb{E} \|g\| = \sqrt{2} \frac{\Gamma((n+1)/2)}{\Gamma(n/2)},\]

where \Gamma(\cdot) is the gamma function. If you are familiar with the definition of the gamma function, the derivation of this formula is not too hard—it follows from a change of variables to n-dimensional spherical coordinates.

This formula can be difficult to interpret and use. Fortunately, we have the rather nice bounds

(1)   \[\sqrt{n-1} < \frac{n}{\sqrt{n+1}} < \mathbb{E} \|g\| < \sqrt{n}. \]

This result appears, for example, page 11 of this paper. These bounds show that, for large n, \mathbb{E} \|g\| is quite close to \sqrt{n}. The authors of the paper remark that this inequality can be probed by induction. I had difficulty reproducing the inductive argument for myself. Fortunately, I found a different proof which I thought was very nice, so I thought I would share it here.

Our core tool will be Wendel’s inequality (see (7) in Wendel’s original paper): For x > 0 and 0 < s < 1, we have

(2)   \[\frac{x}{(x+s)^{1-s}} < \frac{\Gamma(x+s)}{\Gamma(x)} < x^s. \]

Let us first use Wendel’s inequality to prove (1). Indeed, invoke Wendel’s inequality with x = n/2 and s = 1/2 and multiply by \sqrt{2} to obtain

    \[\frac{\sqrt{2} \cdot n/2}{(n/2+1/2)^{1/2}} < \sqrt{2}\frac{\Gamma((n+1)/2)}{\Gamma(n/2)} = \mathbb{E}\|g\| < \sqrt{2}\cdot \sqrt{n/2},\]

which simplifies directly to (1).

Now, let’s prove Wendel’s inequality (2). The key property for us will be the strict log-convexity of the gamma function: For real numbers x,y > 0 and 0 < s < 1,

(3)   \[\Gamma((1-s)x + sy) < \Gamma(x)^{1-s} \Gamma(y)^s. \]

We take this property as established and use it to prove Wendel’s inequality. First, use the log-convexity property (3) with y = x+1 to obtain

    \[\Gamma(x+s) = \Gamma((1-s)x + s(x+1)) < \Gamma(x)^{1-s} \Gamma(x+1)^s.\]

Divide by \Gamma(x) and use the property that \Gamma(x+1)/\Gamma(x) = x to conclude

(4)   \[\frac{\Gamma(x+s)}{\Gamma(x)} < \left( \frac{\Gamma(x+1)}{\Gamma(x)} \right)^s = x^s. \]

This proves the upper bound in Wendel’s inequality (2). To prove the lower bound, invoke the upper bound (4) with x+s in place of x and 1-s in place of s to obtain

    \[\frac{\Gamma(x+1)}{\Gamma(x+s)} < (x+s)^{1-s}.\]

Multiplying by \Gamma(x+s), dividing by (x+s)^{1-s}\Gamma(x), and using \Gamma(x+1)/\Gamma(x) = x again yields

    \[\frac{\Gamma(x+s)}{\Gamma(x)} > \frac{\Gamma(x+1)}{\Gamma(x)} \cdot \frac{1}{(x+s)^{1-s}} = \frac{x}{(x+s)^{1-s}},\]

finishing the proof of Wendel’s inequality.

Notes. The upper bound in (1) can be proven directly by Lyapunov’s inequality: \mathbb{E} \|g\| \le (\mathbb{E} \|g\|^2)^{1/2} = n^{1/2}, where we use the fact that \|g\|^2 = g_1^2 + \cdots + g_n^2 is the sum of n random variables with mean one. The weaker lower bound \mathbb{E} \|g\| \ge \sqrt{n-1} follows from a weaker version of Wendel’s inequality, Gautschi’s inequality.

After the initial publication of this post, Sam Buchanan mentioned another proof of the lower bound \mathbb{E} \|g\| \ge \sqrt{n-1} using the Gaussian Poincaré inequality. This inequality states that, for a function f : \real^n \to \real,

    \[\Var(f(g)) \le \mathbb{E} \| \nabla f(g)\|^2.\]

To prove the lower bound, set f(g) := \|g\| which has gradient \nabla f(g) = g/\|g\|. Thus,

    \[\mathbb{E} \| \nabla f(g)\|^2 = 1 \ge \Var(f(g)) = \mathbb{E} \|g\|^2 - (\mathbb{E} \|g\|)^2 = n -  (\mathbb{E} \|g\|)^2.\]

Rearrange to obtain \mathbb{E} \|g\| \ge \sqrt{n-1}.

Leave a Reply

Your email address will not be published.