In other words, a line drawn through the middle of a stationary process — i.e. the trend line — is flat. It may have 'seasonal' cycles around this trend line, but overall it does not trend up nor down.
Since stationarity is an assumption underlying many statistical procedures used in
time series analysis, non-stationary data are often transformed to become stationary. The most common cause of violation of stationarity is a trend in the mean, which can be due either to the presence of a
unit root or of a deterministic trend. In the former case of a unit root, stochastic shocks have permanent effects, and the process is not
mean-reverting. In the latter case of a deterministic trend, the process is called a
trend-stationary process, and stochastic shocks have only transitory effects after which the variable tends toward a deterministically evolving (non-constant) mean.
A trend stationary process is not strictly stationary, but can easily be transformed into a stationary process by removing the underlying trend, which is solely a function of time. Similarly, processes with one or more unit roots can be made stationary through differencing. An important type of non-stationary process that does not include a trend-like behavior is a
cyclostationary process, which is a stochastic process that varies cyclically with time.
For many applications strict-sense stationarity is too restrictive. Other forms of stationarity such as wide-sense stationarity or N-th-order stationarity are then employed. The definitions for different kinds of stationarity are not consistent among different authors (see
Other terminology).
White noise is the simplest example of a stationary process.
An example of a
discrete-time stationary process where the sample space is also discrete (so that the random variable may take one of N possible values) is a
Bernoulli scheme. Other examples of a discrete-time stationary process with continuous sample space include some
autoregressive and
moving average processes which are both subsets of the
autoregressive moving average model. Models with a non-trivial autoregressive component may be either stationary or non-stationary, depending on the parameter values, and important non-stationary special cases are where
unit roots exist in the model.
Example 1
Let be any scalar
random variable, and define a time-series , by
Then is a stationary time series, for which realisations consist of a series of constant values, with a different constant value for each realisation. A
law of large numbers does not apply on this case, as the limiting value of an average from a single realisation takes the random value determined by , rather than taking the
expected value of .
The time average of does not converge since the process is not
ergodic.
Example 2
As a further example of a stationary process for which any single realisation has an apparently noise-free structure, let have a
uniform distribution on and define the time series by
Then is strictly stationary since ( modulo ) follows the same uniform distribution as for any .
Example 3
Keep in mind that a
weakly white noise is not necessarily strictly stationary. Let be a random variable uniformly distributed in the interval and define the time series
Then
So is a white noise in the weak sense (the mean and cross-covariances are zero, and the variances are all the same), however it is not strictly stationary.
Nth-order stationarity
In Eq.1, the distribution of samples of the stochastic process must be equal to the distribution of the samples shifted in time for all. N-th-order stationarity is a weaker form of stationarity where this is only requested for all up to a certain order . A random process is said to be N-th-order stationary if:[2]: p. 152
(Eq.2)
Weak or wide-sense stationarity
Definition
A weaker form of stationarity commonly employed in
signal processing is known as weak-sense stationarity, wide-sense stationarity (WSS), or covariance stationarity. WSS random processes only require that 1st
moment (i.e. the mean) and
autocovariance do not vary with respect to time and that the 2nd moment is finite for all times. Any strictly stationary process which has a finite
mean and
covariance is also WSS.[3]: p. 299
The first property implies that the mean function must be constant. The second property implies that the autocovariance function depends only on the difference between and and only needs to be indexed by one variable rather than two variables.[2]: p. 159 Thus, instead of writing,
the notation is often abbreviated by the substitution :
This also implies that the
autocorrelation depends only on , that is
The third property says that the second moments must be finite for any time .
Motivation
The main advantage of wide-sense stationarity is that it places the time-series in the context of
Hilbert spaces. Let H be the Hilbert space generated by {x(t)} (that is, the closure of the set of all linear combinations of these random variables in the Hilbert space of all square-integrable random variables on the given probability space). By the positive definiteness of the autocovariance function, it follows from
Bochner's theorem that there exists a positive measure on the real line such that H is isomorphic to the Hilbert subspace of L2(μ) generated by {e−2πiξ⋅t}. This then gives the following Fourier-type decomposition for a continuous time stationary stochastic process: there exists a stochastic process with
orthogonal increments such that, for all
where the integral on the right-hand side is interpreted in a suitable (Riemann) sense. The same result holds for a discrete-time stationary process, with the spectral measure now defined on the unit circle.
When processing WSS random signals with
linear,
time-invariant (
LTI)
filters, it is helpful to think of the correlation function as a
linear operator. Since it is a
circulant operator (depends only on the difference between the two arguments), its eigenfunctions are the
Fourier complex exponentials. Additionally, since the
eigenfunctions of LTI operators are also
complex exponentials, LTI processing of WSS random signals is highly tractable—all computations can be performed in the
frequency domain. Thus, the WSS assumption is widely employed in signal processing
algorithms.
Definition for complex stochastic process
In the case where is a complex stochastic process the
autocovariance function is defined as and, in addition to the requirements in Eq.3, it is required that the pseudo-autocovariance function depends only on the time lag. In formulas, is WSS, if
(Eq.4)
Joint stationarity
The concept of stationarity may be extended to two stochastic processes.
Joint strict-sense stationarity
Two stochastic processes and are called jointly strict-sense stationary if their joint cumulative distribution remains unchanged under time shifts, i.e. if
(Eq.5)
Joint (M + N)th-order stationarity
Two random processes and is said to be jointly (M + N)-th-order stationary if:[2]: p. 159
(Eq.6)
Joint weak or wide-sense stationarity
Two stochastic processes and are called jointly wide-sense stationary if they are both wide-sense stationary and their cross-covariance function depends only on the time difference . This may be summarized as follows:
(Eq.7)
Relation between types of stationarity
If a stochastic process is N-th-order stationary, then it is also M-th-order stationary for all .
If a stochastic process is second order stationary () and has finite second moments, then it is also wide-sense stationary.[2]: p. 159
If a stochastic process is wide-sense stationary, it is not necessarily second-order stationary.[2]: p. 159
If a stochastic process is strict-sense stationary and has finite second moments, it is wide-sense stationary.[3]: p. 299
If two stochastic processes are jointly (M + N)-th-order stationary, this does not guarantee that the individual processes are M-th- respectively N-th-order stationary.[2]: p. 159
Other terminology
The terminology used for types of stationarity other than strict stationarity can be rather mixed. Some examples follow.
Priestley uses stationary up to orderm if conditions similar to those given here for wide sense stationarity apply relating to moments up to order m.[4][5] Thus wide sense stationarity would be equivalent to "stationary to order 2", which is different from the definition of second-order stationarity given here.
Honarkhah and
Caers also use the assumption of stationarity in the context of multiple-point geostatistics, where higher n-point statistics are assumed to be stationary in the spatial domain.[6]
Tahmasebi and
Sahimi have presented an adaptive Shannon-based methodology that can be used for modeling of any non-stationary systems.[7]
Differencing
One way to make some time series stationary is to compute the differences between consecutive observations. This is known as
differencing. Differencing can help stabilize the mean of a time series by removing changes in the level of a time series, and so eliminating trends. This can also remove seasonality, if differences are taken appropriately (e.g. differencing observations 1 year apart to remove year-lo).
Transformations such as logarithms can help to stabilize the variance of a time series.
One of the ways for identifying non-stationary times series is the
ACF plot. Sometimes, patterns will be more visible in the ACF plot than in the original time series; however, this is not always the case.[8]
Another approach to identifying non-stationarity is to look at the
Laplace transform of a series, which will identify both exponential trends and sinusoidal seasonality (complex exponential trends). Related techniques from
signal analysis such as the
wavelet transform and
Fourier transform may also be helpful.
^Gagniuc, Paul A. (2017). Markov Chains: From Theory to Implementation and Experimentation. USA, NJ: John Wiley & Sons. pp. 1–256.
ISBN978-1-119-38755-8.
^
abcdefgPark,Kun Il (2018). Fundamentals of Probability and Stochastic Processes with Applications to Communications. Springer.
ISBN978-3-319-68074-3.
^
abIonut Florescu (7 November 2014). Probability and Stochastic Processes. John Wiley & Sons.
ISBN978-1-118-59320-2.
^Priestley, M. B. (1981). Spectral Analysis and Time Series. Academic Press.
ISBN0-12-564922-3.
^Honarkhah, M.; Caers, J. (2010). "Stochastic Simulation of Patterns Using Distance-Based Pattern Modeling". Mathematical Geosciences. 42 (5): 487–517.
doi:
10.1007/s11004-010-9276-7.