Autocorrelation test for independenceDHIRAJ D SBlockedUnblockFollowFollowingMay 11Autocorrelation is a characteristic of data in which the correlation between the values of the same variables is based on related objects.
It violates the assumption of instance independence, which underlies most of the conventional models.
It generally exists in those types of data-sets in which the data, instead of being randomly selected, is from the same source.
What does autocorrelation look like?Here is a list of random integers generated.
See if you can identify a pattern by looking at the values in the sequence.
If you look carefully at the following table of random integers generated, you’ll notice that every number in the 5th, 10th, 15th, and 20th position is a larger value.
Important variables to remember?m — is the lag, the space between the numbers being tested.
i — is the index, or the number in the sequence that you start withN — the number of numbers generated in a sequenceM — is the largest integer such thatThe test described below requires the computation of the autocorrelation between every m numbers (m is the lag) starting with the i th number.
The autocorrelationbetween the following numbers.
The value M is is the largest integer such thatwhere N is the total numbers in the sequence.
Since a nonzero autocorrelation implies a lack of independence, the following two-tailed test is appropriate:For large values of M, the distribution of the estimator ofdenoted with a hat, is approximately normal if the valuesare uncorrelated.
Then the test statistic can be formed as follows:which is distributed normally with a mean of zero and a variance of 1, under the assumption of independence, for large M.
The formula for(with a hat), in a slightly different form, and the standard deviation of the estimator,are as follows (Scmidt and Taylor 1970):andAfter computing,do not reject the null hypothesis of independence if, whereis the level of significance.
If> 0, the subsequence is said to exhibit positive autocorrelation.
On the other hand, if< 0, the subsequence is exhibiting negative autocorrelation, which means the m values are followed by greater values.
Figuring out the value of M?To figure out the value of M, you must use some simple algebra.
For example, the equation:must be solved using the given values i the index, N the number of elements in the sequence, and m the lag.
For example:Since the value M must be an integer, the 0.
8 is truncated and the 4 is saved as the value of M.
Thus, M is equal to 4.
Python ImplementationDrawbacks when using autocorrelationAutocorrelation is not very sensitive to small values of M, when the values being tested are on the low side.
For example, if all the values were equal to zero, then the resulting value would be -1.
95, which is not enough to reject the hypothesis.
Many sequences can be formed in a set of date (the sequence of all numbers, the sequence from the first, fifth, …, numbers and so forth.
If the alpha is equal to 0.
05, then there is a possibility of rejecting a true hypothesis.
For example, if these are 10 independent events, the possibility of finding of finding no autocorrelation alone is (0.
95)10 or 0.
60.
Thus, 40% of the time significant autocorrelation would be detected when it does not exist.
.. More details