{"id":1438,"date":"2023-02-21T17:28:11","date_gmt":"2023-02-21T17:28:11","guid":{"rendered":"https:\/\/www.ethanepperly.com\/?p=1438"},"modified":"2025-05-21T21:30:27","modified_gmt":"2025-05-21T21:30:27","slug":"note-to-self-norm-of-a-gaussian-random-vector","status":"publish","type":"post","link":"https:\/\/www.ethanepperly.com\/index.php\/2023\/02\/21\/note-to-self-norm-of-a-gaussian-random-vector\/","title":{"rendered":"Note to Self: Norm of a Gaussian Random Vector"},"content":{"rendered":"\n

Let \"g\" be a standard Gaussian vector\u2014that is, a vector populated by independent standard normal random variables<\/a>. What is the expected length \"\mathbb{E} \|g\|\" of \"g\"? (Here, and throughout, \"\|\cdot\|\" denotes the Euclidean norm<\/a> of a vector.) The length of \"g\" is the square root of the sum of \"n\" independent standard normal random variables<\/p>\n\n\n\n

  <\/span>   <\/span>\"\[\|g\| = \sqrt{g_1^2 + \cdots + g_n^2},\]\"<\/p><\/p>\n\n\n\n

which is known as a \"\chi\" random variable<\/a> with \"n\" degrees of freedom. (Not to be confused with a \"\chi^\mathbf{2}\" random variable<\/a>!) Its mean value is given by the rather unpleasant formula<\/p>\n\n\n\n

  <\/span>   <\/span>\"\[\mathbb{E} \|g\| = \sqrt{2} \frac{\Gamma((n+1)/2)}{\Gamma(n/2)},\]\"<\/p><\/p>\n\n\n\n

where \"\Gamma(\cdot)\" is the gamma function<\/a>. If you are familiar with the definition of the gamma function, the derivation of this formula is not too hard\u2014it follows<\/a> from a change of variables to \"n\"-dimensional spherical coordinates.<\/p>\n\n\n\n

This formula can be difficult to interpret and use. Fortunately, we have the rather nice bounds<\/p>\n\n\n\n

(1) <\/span>   <\/span>\"\[\sqrt{n-1} < \frac{n}{\sqrt{n+1}} < \mathbb{E} \|g\| < \sqrt{n}. \]\"<\/p><\/p>\n\n\n\n

This result appears, for example, page 11 of this paper<\/a>. These bounds show that, for large \"n\", \"\mathbb{E} \|g\|\" is quite close to \"\sqrt{n}\". The authors of the paper<\/a> remark that this inequality can be proved by induction. I had difficulty reproducing the inductive argument for myself. Fortunately, I found a different proof which I thought was very nice, so I thought I would share it here.<\/p>\n\n\n\n

Our core tool will be Wendel’s inequality<\/a> (see (7) in Wendel’s original paper<\/a>): For \"x > 0\" and \"0 < s < 1\", we have

(2) <\/span>   <\/span>\"\[\frac{x}{(x+s)^{1-s}} < \frac{\Gamma(x+s)}{\Gamma(x)} < x^s. \]\"<\/p><\/p>\n\n\n\n

Let us first use Wendel’s inequality to prove (1). Indeed, invoke Wendel’s inequality with \"x = n/2\" and \"s = 1/2\" and multiply by \"\sqrt{2}\" to obtain<\/p>\n\n\n\n

  <\/span>   <\/span>\"\[\frac{\sqrt{2} \cdot n/2}{(n/2+1/2)^{1/2}} < \sqrt{2}\frac{\Gamma((n+1)/2)}{\Gamma(n/2)} = \mathbb{E}\|g\| < \sqrt{2}\cdot \sqrt{n/2},\]\"<\/p><\/p>\n\n\n\n

which simplifies directly to (1).<\/p>\n\n\n\n

Now, let’s prove Wendel’s inequality (2). The key property for us will be the strict log-convexity of the gamma function<\/a>: For real numbers \"x,y > 0\" and \"0 < s < 1\",<\/p>\n\n\n\n

(3) <\/span>   <\/span>\"\[\Gamma((1-s)x + sy) < \Gamma(x)^{1-s} \Gamma(y)^s. \]\"<\/p><\/p>\n\n\n\n

We take this property as established and use it to prove Wendel’s inequality. First, use the log-convexity property (3) with \"y = x+1\" to obtain<\/p>\n\n\n\n

  <\/span>   <\/span>\"\[\Gamma(x+s) = \Gamma((1-s)x + s(x+1)) < \Gamma(x)^{1-s} \Gamma(x+1)^s.\]\"<\/p><\/p>\n\n\n\n

Divide by \"\Gamma(x)\" and use the property<\/a> that \"\Gamma(x+1)/\Gamma(x) = x\" to conclude<\/p>\n\n\n\n

(4) <\/span>   <\/span>\"\[\frac{\Gamma(x+s)}{\Gamma(x)} < \left( \frac{\Gamma(x+1)}{\Gamma(x)} \right)^s = x^s. \]\"<\/p><\/p>\n\n\n\n

This proves the upper bound in Wendel’s inequality (2). To prove the lower bound, invoke the upper bound (4) with \"x+s\" in place of \"x\" and \"1-s\" in place of \"s\" to obtain<\/p>\n\n\n\n

  <\/span>   <\/span>\"\[\frac{\Gamma(x+1)}{\Gamma(x+s)} < (x+s)^{1-s}.\]\"<\/p><\/p>\n\n\n\n

Multiplying by \"\Gamma(x+s)\", dividing by \"(x+s)^{1-s}\Gamma(x)\", and using \"\Gamma(x+1)/\Gamma(x) = x\" again yields<\/p>\n\n\n\n

  <\/span>   <\/span>\"\[\frac{\Gamma(x+s)}{\Gamma(x)} > \frac{\Gamma(x+1)}{\Gamma(x)} \cdot \frac{1}{(x+s)^{1-s}} = \frac{x}{(x+s)^{1-s}},\]\"<\/p><\/p>\n\n\n\n

finishing the proof of Wendel’s inequality.<\/p>\n\n\n\n

Notes.<\/strong> The upper bound in (1) can be proven directly by Lyapunov’s inequality: \"\mathbb{E} \|g\| \le (\mathbb{E} \|g\|^2)^{1/2} = n^{1/2}\", where we use the fact that \"\|g\|^2 = g_1^2 + \cdots + g_n^2\" is the sum of \"n\" random variables with mean one. The weaker lower bound \"\mathbb{E} \|g\| \ge \sqrt{n-1}\" follows from a weaker version of Wendel’s inequality, Gautschi’s inequality<\/a>.<\/p>\n\n\n\n

After the initial publication of this post, Sam Buchanan<\/a> mentioned<\/a> another proof of the lower bound \"\mathbb{E} \|g\| \ge \sqrt{n-1}\" using the Gaussian Poincar\u00e9 inequality<\/a>. This inequality states that, for a function \"f : \real^n \to \real\",<\/p>\n\n\n\n

  <\/span>   <\/span>\"\[\Var(f(g)) \le \mathbb{E} \| \nabla f(g)\|^2.\]\"<\/p><\/p>\n\n\n\n

To prove the lower bound, set \"f(g) := \|g\|\" which has gradient \"\nabla f(g) = g/\|g\|\". Thus,<\/p>\n\n\n\n

  <\/span>   <\/span>\"\[\mathbb{E} \| \nabla f(g)\|^2 = 1 \ge \Var(f(g)) = \mathbb{E} \|g\|^2 - (\mathbb{E} \|g\|)^2 = n - (\mathbb{E} \|g\|)^2.\]\"<\/p><\/p>\n\n\n\n

Rearrange to obtain \"\mathbb{E} \|g\| \ge \sqrt{n-1}\".<\/p>\n","protected":false},"excerpt":{"rendered":"

Let be a standard Gaussian vector\u2014that is, a vector populated by independent standard normal random variables. What is the expected length of ? (Here, and throughout, denotes the Euclidean norm of a vector.) The length of is the square root of the sum of independent standard normal random variables     which is known asRead more<\/a><\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[9],"tags":[],"class_list":["post-1438","post","type-post","status-publish","format-standard","hentry","category-note-to-self"],"_links":{"self":[{"href":"https:\/\/www.ethanepperly.com\/index.php\/wp-json\/wp\/v2\/posts\/1438","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.ethanepperly.com\/index.php\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.ethanepperly.com\/index.php\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.ethanepperly.com\/index.php\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/www.ethanepperly.com\/index.php\/wp-json\/wp\/v2\/comments?post=1438"}],"version-history":[{"count":13,"href":"https:\/\/www.ethanepperly.com\/index.php\/wp-json\/wp\/v2\/posts\/1438\/revisions"}],"predecessor-version":[{"id":2089,"href":"https:\/\/www.ethanepperly.com\/index.php\/wp-json\/wp\/v2\/posts\/1438\/revisions\/2089"}],"wp:attachment":[{"href":"https:\/\/www.ethanepperly.com\/index.php\/wp-json\/wp\/v2\/media?parent=1438"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.ethanepperly.com\/index.php\/wp-json\/wp\/v2\/categories?post=1438"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.ethanepperly.com\/index.php\/wp-json\/wp\/v2\/tags?post=1438"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}