The purpose of this note is to describe the Gaussian hypercontractivity inequality. As an application, we’ll obtain a weaker version of the Hanson–Wright inequality.
The Noise Operator
We begin our discussion with the following question:
Let
be a function. What happens to
, on average, if we perturb its inputs by a small amount of Gaussian noise?
Let’s be more specific about our noise model. Let be an input to the function
and fix a parameter
(think of
as close to 1). We’ll define the noise corruption of
to be
(1)
Here, is the standard multivariate Gaussian distribution. In our definition of
, we both add Gaussian noise
and shrink the vector
by a factor
. In particular, we highlight two extreme cases:
- No noise. If
, then there is no noise and
.
- All noise. If
, then there is all noise and
. The influence of the original vector
has been washed away completely.
The noise corruption (1) immediately gives rise to the noise operator1The noise operator is often called the Hermite operator. The noise operator is related to the Ornstein–Uhlenbeck semigroup operator by a change of variables,
.
. Let
be a function. The noise operator
is defined to be:
(2)
The noise operator computes the average value of when evaluated at the noisy input
. Observe that the noise operator maps a function
to another function
. Going forward, we will write
to denote
.
To understand how the noise operator acts on a function , we can write the expectation in the definition (2) as an integral:






See below for an illustration. The red solid curve is a function , and the blue dashed curve is
.
As we decrease from
to
, the function
is smoothed more and more. When we finally reach
,
has been smoothed all the way into a constant.
Random Inputs
The noise operator converts a function to another function
. We can evaluate these two functions at a Gaussian random vector
, resulting in two random variables
and
.
We can think of as a modification of the random variable
where “a
fraction of the variance of
has been averaged out”. We again highlight the two extreme cases:
- No noise. If
,
. None of the variance of
has been averaged out.
- All noise. If
,
is a constant random variable. All of the variance of
has been averaged out.
Just as decreasing smoothes the function
until it reaches a constant function at
, decreasing
makes the random variable
more and more “well-behaved” until it becomes a constant random variable at
. This “well-behavingness” property of the noise operator is made precise by the Gaussian hypercontractivity theorem.
Moments and Tails
In order to describe the “well-behavingness” properties of the noise operator, we must answer the question:
How can we measure how well-behaved a random variable is?
There are many answers to this question. For this post, we will quantify the well-behavedness of a random variable by using the norm.2Using norms is a common way of measuring the niceness of a function or random variable in applied math. For instance, we can use Sobolev norms or reproducing kernel Hilbert space norms to measure the smoothness of a function in approximation theory, as I’ve discussed before on this blog.
The norm of a (
-valued) random variable
is defined to be
(3)





The norms of random variables control the tails of a random variable—that is, the probability that a random variable is large in magnitude. A random variables with small tails is typically thought of as a “nice” or “well-behaved” random variable. Random quantities with small tails are usually desirable in applications, as they are more predictable—unlikely to take large values.
The connection between tails and norms can be derived as follows. First, write the tail probability
for
using
th powers:
(4)



Gaussian Contractivity
Before we introduce the Gaussian hypercontractivity theorem, let’s establish a weaker property of the noise operator, contractivity.
Proposition 1 (Gaussian contractivity). Choose a noise level
and a power
, and let
be a Gaussian random vector. Then
contracts the
norm of
:
This result shows that the noise operator makes the random variable no less nice than
was.
Gaussian contractivity is easy to prove. Begin using the definition of the noise operator (2) and norm (3):






Gaussian Hypercontractivity
The Gaussian contractivity theorem shows that is no less well-behaved than
is. In fact,
is more well-behaved than
is. This is the content of the Gaussian hypercontractivity theorem:
Theorem 2 (Gaussian hypercontractivity): Choose a noise level
and a power
, and let
be a Gaussian random vector. Then
In particular, for
,
We have highlighted the case because it is the most useful in practice.
This result shows that as we take smaller, the random variable
becomes more and more well-behaved, with tails decreasing at a rate

We will prove the Gaussian hypercontractivity at the bottom of this post. For now, we will focus on applying this result.
Multilinear Polynomials
A multilinear polynomial is a multivariate polynomial in the variables
in which none of the variables
is raised to a power higher than one. So,
(5)

For multilinear polynomials, we have the following very powerful corollary of Gaussian hypercontractivity:
Corollary 3 (Absolute moments of a multilinear polynomial of Gaussians). Let
be a multilinear polynomial of degree
. (That is, at most
variables
occur in any monomial of
.) Then, for a Gaussian random vector
and for all
,
Let’s prove this corollary. The first observation is that the noise operator has a particularly convenient form when applied to a multilinear polynomial. Let’s test it out on our example (5) from above. For
We see that the expectation applies to each variable separately, resulting in each replaced by
. This trend holds in general:
Proposition 4 (noise operator on multilinear polynomials). For any multilinear polynomial
,
.
We can use Proposition 4 to obtain bounds on the norms of multilinear polynomials of a Gaussian random variable. Indeed, observe that
The final step of our argument will be to compute . Write
as










Thus, putting all of the ingredients together, we have


Hanson–Wright Inequality
To see the power of the machinery we have developed, let’s prove a version of the Hanson–Wright inequality.
Theorem 5 (suboptimal Hanson–Wright). Let be a symmetric matrix with zero on its diagonal and
be a Gaussian random vector. Then
Hanson–Wright has all sorts of applications in computational mathematics and data science. One direct application is to obtain probabilistic error bounds for the error incurred by a stochastic trace estimation formulas.
This version of Hanson–Wright is not perfect. In particular, it does not capture the Bernstein-type tail behavior of the classical Hanson–Wright inequality
Let’s prove our suboptimal Hanson–Wright inequality. Set . Since
has zero on its diagonal,
is a multilinear polynomial of degree two in the entries of
. The random variable
is mean-zero, and a short calculation shows its
norm is
Thus, by Corollary 3,
(6)



(7)
Now, we must optimize the value of to obtain the sharpest possible bound. To make this optimization more convenient, introduce a parameter


Proof of Gaussian Hypercontractivity
Let’s prove the Gaussian hypercontractivity theorem. For simplicity, we will stick with the case, but the higher-dimensional generalizations follow along similar lines. The key ingredient will be the Gaussian Jensen inequality, which made a prominent appearance in a previous blog post of mine. Here, we will only need the following version:
Theorem 6 (Gaussian Jensen). Let
be a twice differentiable function and let
be jointly Gaussian random variables with covariance matrix
. Then
(8)
holds for all test functionsif, and only if,
(9)
Here, denotes the entrywise product of matrices and
is the Hessian matrix of the function
.
To me, this proof of Gaussian hypercontractivity using Gaussian Jensen (adapted from Paata Ivanishvili‘s excellent post) is amazing. First, we reformulate the Gaussian hypercontractivity property a couple of times using some functional analysis tricks. Then we do a short calculation, invoke Gaussian Jensen, and the theorem is proved, almost as if by magic.
Part 1: Tricks
Let’s begin with “tricks” part of the argument.
Trick 1. To prove Gaussian hypercontractivity holds for all functions
, it is sufficient to prove for all nonnegative functions
.
Indeed, suppose Gaussian hypercontractivity holds for all nonnegative functions . Then, for any function
, apply Jensen’s inequality to conclude
Thus, assuming hypercontractivity holds for the nonnegative function , we have

Trick 2. To prove Gaussian hypercontractivity for all
, it is sufficient to prove the following “bilinearized” Gaussian hypercontractivity result:
holds for all
with
. Here,
is the Hölder conjugate to
.
Indeed, this follows3This argument may be more clear to parse if we view and
as functions on
equipped with the standard Gaussian measure
. This result is just duality for the
norm. from the dual characterization of the norm of
:
Trick 3. Let
be a pair of standard Gaussian random variables with correlation
. Then the bilinearized Gaussian hypercontractivity statement is equivalent to
Indeed, define for the random variable in the definition of the noise operator
. The random variable
is standard Gaussian and has correlation
with
, concluding the proof of Trick 3.
Finally, we apply a change of variables as our last trick:
Trick 4. Make the change of variables
and
, yielding the final equivalent version of Gaussian hypercontractivity:
for all functions
and
(in the appropriate spaces).
Part 2: Calculation
We recognize this fourth equivalent version of Gaussian hypercontractivity as the conclusion (8) to Gaussian Jensen with
We now enter the calculation part of the proof. First, we compute the Hessian of :



















