Misplaced Pages

In mathematics, the Bussgang theorem is a theorem of stochastic analysis. The theorem states that the cross-correlation between a Gaussian signal before and after it has passed through a nonlinear operation are equal to the signals auto-correlation up to a constant. It was first published by Julian J. Bussgang in 1952 while he was at the Massachusetts Institute of Technology.

Statement

Let ${\displaystyle \left\{X(t)\right\}}$ be a zero-mean stationary Gaussian random process and ${\displaystyle \left\{Y(t)\right\}=g(X(t))}$ where ${\displaystyle g(\cdot )}$ is a nonlinear amplitude distortion.

If ${\displaystyle R_{X}(\tau )}$ is the autocorrelation function of ${\displaystyle \left\{X(t)\right\}}$, then the cross-correlation function of ${\displaystyle \left\{X(t)\right\}}$ and ${\displaystyle \left\{Y(t)\right\}}$ is

${\displaystyle R_{XY}(\tau )=CR_{X}(\tau ),}$

where ${\displaystyle C}$ is a constant that depends only on ${\displaystyle g(\cdot )}$.

It can be further shown that

${\displaystyle C={\frac {1}{\sigma ^{3}{\sqrt {2\pi }}}}\int _{-\infty }^{\infty }ug(u)e^{-{\frac {u^{2}}{2\sigma ^{2}}}}\,du.}$

Derivation for One-bit Quantization

It is a property of the two-dimensional normal distribution that the joint density of ${\displaystyle y_{1}}$ and ${\displaystyle y_{2}}$ depends only on their covariance and is given explicitly by the expression

${\displaystyle p(y_{1},y_{2})={\frac {1}{2\pi {\sqrt {1-\rho ^{2}}}}}e^{-{\frac {y_{1}^{2}+y_{2}^{2}-2\rho y_{1}y_{2}}{2(1-\rho ^{2})}}}}$

where ${\displaystyle y_{1}}$ and ${\displaystyle y_{2}}$ are standard Gaussian random variables with correlation ${\displaystyle \phi _{y_{1}y_{2}}=\rho }$.

Assume that ${\displaystyle r_{2}=Q(y_{2})}$, the correlation between ${\displaystyle y_{1}}$ and ${\displaystyle r_{2}}$ is,

${\displaystyle \phi _{y_{1}r_{2}}={\frac {1}{2\pi {\sqrt {1-\rho ^{2}}}}}\int _{-\infty }^{\infty }\int _{-\infty }^{\infty }y_{1}Q(y_{2})e^{-{\frac {y_{1}^{2}+y_{2}^{2}-2\rho y_{1}y_{2}}{2(1-\rho ^{2})}}}\,dy_{1}dy_{2}}$.

Since

${\displaystyle \int _{-\infty }^{\infty }y_{1}e^{-{\frac {1}{2(1-\rho ^{2})}}y_{1}^{2}+{\frac {\rho y_{2}}{1-\rho ^{2}}}y_{1}}\,dy_{1}=\rho {\sqrt {2\pi (1-\rho ^{2})}}y_{2}e^{\frac {\rho ^{2}y_{2}^{2}}{2(1-\rho ^{2})}}}$,

the correlation ${\displaystyle \phi _{y_{1}r_{2}}}$ may be simplified as

${\displaystyle \phi _{y_{1}r_{2}}={\frac {\rho }{\sqrt {2\pi }}}\int _{-\infty }^{\infty }y_{2}Q(y_{2})e^{-{\frac {y_{2}^{2}}{2}}}\,dy_{2}}$.

The integral above is seen to depend only on the distortion characteristic ${\displaystyle Q()}$ and is independent of ${\displaystyle \rho }$.

Remembering that ${\displaystyle \rho =\phi _{y_{1}y_{2}}}$, we observe that for a given distortion characteristic ${\displaystyle Q()}$, the ratio ${\displaystyle {\frac {\phi _{y_{1}r_{2}}}{\phi _{y_{1}y_{2}}}}}$ is ${\displaystyle K_{Q}={\frac {1}{\sqrt {2\pi }}}\int _{-\infty }^{\infty }y_{2}Q(y_{2})e^{-{\frac {y_{2}^{2}}{2}}}\,dy_{2}}$.

Therefore, the correlation can be rewritten in the form

${\displaystyle \phi _{y_{1}r_{2}}=K_{Q}\phi _{y_{1}y_{2}}}$.

The above equation is the mathematical expression of the stated "Bussgang‘s theorem".

If ${\displaystyle Q(x)={\text{sign}}(x)}$, or called one-bit quantization, then ${\displaystyle K_{Q}={\frac {2}{\sqrt {2\pi }}}\int _{0}^{\infty }y_{2}e^{-{\frac {y_{2}^{2}}{2}}}\,dy_{2}={\sqrt {\frac {2}{\pi }}}}$.

Arcsine law

If the two random variables are both distorted, i.e., ${\displaystyle r_{1}=Q(y_{1}),r_{2}=Q(y_{2})}$, the correlation of ${\displaystyle r_{1}}$ and ${\displaystyle r_{2}}$ is

${\displaystyle \phi _{r_{1}r_{2}}=\int _{-\infty }^{\infty }\int _{-\infty }^{\infty }Q(y_{1})Q(y_{2})p(y_{1},y_{2})\,dy_{1}dy_{2}}$.

When ${\displaystyle Q(x)={\text{sign}}(x)}$, the expression becomes,

${\displaystyle \phi _{r_{1}r_{2}}={\frac {1}{2\pi {\sqrt {1-\rho ^{2}}}}}\left}$

where ${\displaystyle \alpha ={\frac {y_{1}^{2}+y_{2}^{2}-2\rho y_{1}y_{2}}{2(1-\rho ^{2})}}}$.

Noticing that

${\displaystyle \int _{-\infty }^{\infty }\int _{-\infty }^{\infty }p(y_{1},y_{2})\,dy_{1}dy_{2}={\frac {1}{2\pi {\sqrt {1-\rho ^{2}}}}}\left=1}$,

and ${\displaystyle \int _{0}^{\infty }\int _{0}^{\infty }e^{-\alpha }\,dy_{1}dy_{2}=\int _{-\infty }^{0}\int _{-\infty }^{0}e^{-\alpha }\,dy_{1}dy_{2}}$, ${\displaystyle \int _{0}^{\infty }\int _{-\infty }^{0}e^{-\alpha }\,dy_{1}dy_{2}=\int _{-\infty }^{0}\int _{0}^{\infty }e^{-\alpha }\,dy_{1}dy_{2}}$,

we can simplify the expression of ${\displaystyle \phi _{r_{1}r_{2}}}$ as

${\displaystyle \phi _{r_{1}r_{2}}={\frac {4}{2\pi {\sqrt {1-\rho ^{2}}}}}\int _{0}^{\infty }\int _{0}^{\infty }e^{-\alpha }\,dy_{1}dy_{2}-1}$

Also, it is convenient to introduce the polar coordinate ${\displaystyle y_{1}=R\cos \theta ,y_{2}=R\sin \theta }$. It is thus found that

${\displaystyle \phi _{r_{1}r_{2}}={\frac {4}{2\pi {\sqrt {1-\rho ^{2}}}}}\int _{0}^{\pi /2}\int _{0}^{\infty }e^{-{\frac {R^{2}-2R^{2}\rho \cos \theta \sin \theta \ }{2(1-\rho ^{2})}}}R\,dRd\theta -1={\frac {4}{2\pi {\sqrt {1-\rho ^{2}}}}}\int _{0}^{\pi /2}\int _{0}^{\infty }e^{-{\frac {R^{2}(1-\rho \sin 2\theta )}{2(1-\rho ^{2})}}}R\,dRd\theta -1}$.

Integration gives

${\displaystyle \phi _{r_{1}r_{2}}={\frac {2{\sqrt {1-\rho ^{2}}}}{\pi }}\int _{0}^{\pi /2}{\frac {d\theta }{1-\rho \sin 2\theta }}-1=-{\frac {2}{\pi }}\arctan \left({\frac {\rho -\tan \theta }{\sqrt {1-\rho ^{2}}}}\right){\Bigg |}_{0}^{\pi /2}-1={\frac {2}{\pi }}\arcsin(\rho )}$，

This is called "Arcsine law", which was first found by J. H. Van Vleck in 1943 and republished in 1966. The "Arcsine law" can also be proved in a simpler way by applying Price's Theorem.

The function ${\displaystyle f(x)={\frac {2}{\pi }}\arcsin x}$ can be approximated as ${\displaystyle f(x)\approx {\frac {2}{\pi }}x}$ when ${\displaystyle x}$ is small.

Price's Theorem

Given two jointly normal random variables ${\displaystyle y_{1}}$ and ${\displaystyle y_{2}}$ with joint probability function

${\displaystyle {\displaystyle p(y_{1},y_{2})={\frac {1}{2\pi {\sqrt {1-\rho ^{2}}}}}e^{-{\frac {y_{1}^{2}+y_{2}^{2}-2\rho y_{1}y_{2}}{2(1-\rho ^{2})}}}}}$,

we form the mean

${\displaystyle I(\rho )=E(g(y_{1},y_{2}))=\int _{-\infty }^{+\infty }\int _{-\infty }^{+\infty }g(y_{1},y_{2})p(y_{1},y_{2})\,dy_{1}dy_{2}}$

of some function ${\displaystyle g(y_{1},y_{2})}$ of ${\displaystyle (y_{1},y_{2})}$. If ${\displaystyle g(y_{1},y_{2})p(y_{1},y_{2})\rightarrow 0}$ as ${\displaystyle (y_{1},y_{2})\rightarrow 0}$, then

${\displaystyle {\frac {\partial ^{n}I(\rho )}{\partial \rho ^{n}}}=\int _{-\infty }^{\infty }\int _{-\infty }^{\infty }{\frac {\partial ^{2n}g(y_{1},y_{2})}{\partial y_{1}^{n}\partial y_{2}^{n}}}p(y_{1},y_{2})\,dy_{1}dy_{2}=E\left({\frac {\partial ^{2n}g(y_{1},y_{2})}{\partial y_{1}^{n}\partial y_{2}^{n}}}\right)}$.

Proof. The joint characteristic function of the random variables ${\displaystyle y_{1}}$ and ${\displaystyle y_{2}}$ is by definition the integral

${\displaystyle \Phi (\omega _{1},\omega _{2})=\int _{-\infty }^{\infty }\int _{-\infty }^{\infty }p(y_{1},y_{2})e^{j(\omega _{1}y_{1}+\omega _{2}y_{2})}\,dy_{1}dy_{2}=\exp \left\{-{\frac {\omega _{1}^{2}+\omega _{2}^{2}+2\rho \omega _{1}\omega _{2}}{2}}\right\}}$.

From the two-dimensional inversion formula of Fourier transform, it follows that

${\displaystyle p(y_{1},y_{2})={\frac {1}{4\pi ^{2}}}\int _{-\infty }^{\infty }\int _{-\infty }^{\infty }\Phi (\omega _{1},\omega _{2})e^{-j(\omega _{1}y_{1}+\omega _{2}y_{2})}\,d\omega _{1}d\omega _{2}={\frac {1}{4\pi ^{2}}}\int _{-\infty }^{\infty }\int _{-\infty }^{\infty }\exp \left\{-{\frac {\omega _{1}^{2}+\omega _{2}^{2}+2\rho \omega _{1}\omega _{2}}{2}}\right\}e^{-j(\omega _{1}y_{1}+\omega _{2}y_{2})}\,d\omega _{1}d\omega _{2}}$.

Therefore, plugging the expression of ${\displaystyle p(y_{1},y_{2})}$ into ${\displaystyle I(\rho )}$, and differentiating with respect to ${\displaystyle \rho }$, we obtain

${\displaystyle {\begin{aligned}{\frac {\partial ^{n}I(\rho )}{\partial \rho ^{n}}}&=\int _{-\infty }^{\infty }\int _{-\infty }^{\infty }g(y_{1},y_{2})p(y_{1},y_{2})\,dy_{1}dy_{2}\\&=\int _{-\infty }^{\infty }\int _{-\infty }^{\infty }g(y_{1},y_{2})\left({\frac {1}{4\pi ^{2}}}\int _{-\infty }^{\infty }\int _{-\infty }^{\infty }{\frac {\partial ^{n}\Phi (\omega _{1},\omega _{2})}{\partial \rho ^{n}}}e^{-j(\omega _{1}y_{1}+\omega _{2}y_{2})}\,d\omega _{1}d\omega _{2}\right)\,dy_{1}dy_{2}\\&=\int _{-\infty }^{\infty }\int _{-\infty }^{\infty }g(y_{1},y_{2})\left({\frac {(-1)^{n}}{4\pi ^{2}}}\int _{-\infty }^{\infty }\int _{-\infty }^{\infty }\omega _{1}^{n}\omega _{2}^{n}\Phi (\omega _{1},\omega _{2})e^{-j(\omega _{1}y_{1}+\omega _{2}y_{2})}\,d\omega _{1}d\omega _{2}\right)\,dy_{1}dy_{2}\\&=\int _{-\infty }^{\infty }\int _{-\infty }^{\infty }g(y_{1},y_{2})\left({\frac {1}{4\pi ^{2}}}\int _{-\infty }^{\infty }\int _{-\infty }^{\infty }\Phi (\omega _{1},\omega _{2}){\frac {\partial ^{2n}e^{-j(\omega _{1}y_{1}+\omega _{2}y_{2})}}{\partial y_{1}^{n}\partial y_{2}^{n}}}\,d\omega _{1}d\omega _{2}\right)\,dy_{1}dy_{2}\\&=\int _{-\infty }^{\infty }\int _{-\infty }^{\infty }g(y_{1},y_{2}){\frac {\partial ^{2n}p(y_{1},y_{2})}{\partial y_{1}^{n}\partial y_{2}^{n}}}\,dy_{1}dy_{2}\\\end{aligned}}}$

After repeated integration by parts and using the condition at ${\displaystyle \infty }$, we obtain the Price's theorem.

${\displaystyle {\begin{aligned}{\frac {\partial ^{n}I(\rho )}{\partial \rho ^{n}}}&=\int _{-\infty }^{\infty }\int _{-\infty }^{\infty }g(y_{1},y_{2}){\frac {\partial ^{2n}p(y_{1},y_{2})}{\partial y_{1}^{n}\partial y_{2}^{n}}}\,dy_{1}dy_{2}\\&=\int _{-\infty }^{\infty }\int _{-\infty }^{\infty }{\frac {\partial ^{2}g(y_{1},y_{2})}{\partial y_{1}\partial y_{2}}}{\frac {\partial ^{2n-2}p(y_{1},y_{2})}{\partial y_{1}^{n-1}\partial y_{2}^{n-1}}}\,dy_{1}dy_{2}\\&=\cdots \\&=\int _{-\infty }^{\infty }\int _{-\infty }^{\infty }{\frac {\partial ^{2n}g(y_{1},y_{2})}{\partial y_{1}^{n}\partial y_{2}^{n}}}p(y_{1},y_{2})\,dy_{1}dy_{2}\end{aligned}}}$

Proof of Arcsine law by Price's Theorem

If ${\displaystyle g(y_{1},y_{2})={\text{sign}}(y_{1}){\text{sign}}(y_{2})}$, then ${\displaystyle {\frac {\partial ^{2}g(y_{1},y_{2})}{\partial y_{1}\partial y_{2}}}=4\delta (y_{1})\delta (y_{2})}$ where ${\displaystyle \delta ()}$ is the Dirac delta function.

Substituting into Price's Theorem, we obtain,

${\displaystyle {\frac {\partial E({\text{sign}}(y_{1}){\text{sign}}(y_{2}))}{\partial \rho }}={\frac {\partial I(\rho )}{\partial \rho }}=\int _{-\infty }^{\infty }\int _{-\infty }^{\infty }4\delta (y_{1})\delta (y_{2})p(y_{1},y_{2})\,dy_{1}dy_{2}={\frac {2}{\pi {\sqrt {1-\rho ^{2}}}}}}$.

When ${\displaystyle \rho =0}$, ${\displaystyle I(\rho )=0}$. Thus

${\displaystyle E\left({\text{sign}}(y_{1}){\text{sign}}(y_{2})\right)=I(\rho )={\frac {2}{\pi }}\int _{0}^{\rho }{\frac {1}{\sqrt {1-\rho ^{2}}}}\,d\rho ={\frac {2}{\pi }}\arcsin(\rho )}$,

which is Van Vleck's well-known result of "Arcsine law".

Application

This theorem implies that a simplified correlator can be designed. Instead of having to multiply two signals, the cross-correlation problem reduces to the gating of one signal with another.

References

1. ^ J.J. Bussgang,"Cross-correlation function of amplitude-distorted Gaussian signals", Res. Lab. Elec., Mas. Inst. Technol., Cambridge MA, Tech. Rep. 216, March 1952.
2. ^ Vleck, J. H. Van. "The Spectrum of Clipped Noise". Radio Research Laboratory Report of Harvard University (51).
3. ^ Vleck, J. H. Van; Middleton, D. (January 1966). "The spectrum of clipped noise". Proceedings of the IEEE. 54 (1): 2–19. doi:10.1109/PROC.1966.4567. ISSN 1558-2256.
4. ^ Price, R. (June 1958). "A useful theorem for nonlinear devices having Gaussian inputs". IRE Transactions on Information Theory. 4 (2): 69–72. doi:10.1109/TIT.1958.1057444. ISSN 2168-2712.
5. ^ Papoulis, Athanasios (2002). Probability, Random Variables, and Stochastic Processes. McGraw-Hill. p. 396. ISBN 0-07-366011-6.

Further reading

• E.W. Bai; V. Cerone; D. Regruto (2007) "Separable inputs for the identification of block-oriented nonlinear systems", Proceedings of the 2007 American Control Conference (New York City, July 11–13, 2007) 1548–1553

• Theorems regarding stochastic processes