Nonparametric method for testing the hypothesis of independence of random variables and its application in the analysis of remote sensing data

全文:

Introduction

The universal and generally accepted criterion for testing hypotheses about the distributions of random variables, including their independence, is the Pearson criterion [1]. When using it, it is necessary to solve the problem of partitioning the area of values of random variables into multivariate intervals and to establish the law of distribution of the criterion that determines the dependences between the probabilistic characteristics of random variables. In [2$–$4] a new approach is proposed that allows simplifying the test of the hypothesis of independence of random variables using a nonparametric algorithm of nuclear-type pattern recognition corresponding to the maximum likelihood criterion. The idea of the approach is to solve a two-alternative problem of pattern recognition. The classes being considered are defined by assumptions about dependence and independence of random variables. On this basis, a training sample is formed from the initial statistical data on observations of random variables and the problem of pattern recognition is solved. The ratio between the estimates of recognition error probabilities of the introduced classes confirms or refutes the hypothesis being considered.

The purpose of this paper is to generalise and develop a nonparametric method of testing the hypothesis of independence of random variables for conditions of the large volume of statistical data and its application in the analysis of information on remote sensing of anthropogenic territories.

Methodology for testing the hypothesis of independence of random variables

Let there be a sample $V=\left(x^i,i=\overline{1,n}\right)$ of the $n$ volume, composed of independent observations of a two-dimensional random variable $x=\left(x_1,x_2\right)$. Let us suppose that the $V$ sample is drawn from the general population characterised by the densities of the probabilities $p\left(x_1\right)p\left(x_2\right)$ or $p\left(x_1,x_2\right)$. On the basis of statistical data of $V$ it is necessary to test the hypothesis

$H_0:p\left(x_1,x_2\right)\equiv p\left(x_1\right)p\left(x_2\right)$

of independence of random variables $x_1$, $x_2$.

To test the $H_0$ hypothesis let us solve the two-alternative problem of pattern recognition. By classes ${\Omega }_1$, ${\Omega }_2$ areas of definition for probability densities $p\left(x_1\right)p\left(x_2\right)$, $p\left(x_1,x_2\right)$ are meant. Under these conditions, the Bayesian decision rule corresponding to the maximum likelihood criterion has the following form

$m\left(x\right):\left\{\begin{array}{l}x\in {\Omega }_1,\text{if }p\left(x_1,x_2\right)<p\left(x_1\right)p\left(x_2\right),\\ x\in {\Omega }_2,\text{if }p\left(x_1,x_2\right)>p\left(x_1\right)p\left(x_2\right).\end{array}\right.$

In contrast to the traditional formulation of the pattern recognition problem, while synthesizing a decisive rule $m\left(x\right)$ there is no a priori training sample containing information about the belonging of the $V$ sample elements to one or another class. This information must be discovered in the process of implementation of the $H_0$ hypothesis testing methodology, which is based on the following actions.

From the $V$ sample recover probability densities $p\left(x_1,x_2\right)$, $p\left(x_1\right)p\left(x_2\right)$, using their non-parametric Rosenblatt $–$ Parzen type estimates [5; 6],

$\overline{p}\left(x_1,x_2\right)=\frac{1}{n{c}_1{c}_2}{\displaystyle \sum _{i=1}^{n}F\left(\frac{x_1-x_1^i}{c_1}\right)F\left(\frac{x_2-x_2^i}{c_2}\right)}$,

$\overline{p}\left(x_1\right)\overline{p}\left(x_2\right)=\frac{1}{n^2{c}_1{c}_2}{\displaystyle \sum _{i=1}^n{\displaystyle \sum _{j=1}^{n}F\left(\frac{x_1-x_1^i}{c_1}\right)F\left(\frac{x_2-x_2^j}{c_2}\right)}}$.

In the statistics $\overline{p}\left(x_1,x_2\right)$, $\overline{p}\left(x_1\right)\overline{p}\left(x_2\right)$ nuclear functions $F\left(u_v\right)$ satisfy the conditions of positivity, symmetry and normalization.

The values of blurring coefficients $c_v$, $v$ = 1, 2 of the nuclear functions decrease as the $n$ volume of the $V$ sample of statistical data increases. Then the nonparametric decision rule for classification of random variables $x=\left(x_1,x_2\right)$ is written as follows

$\overline{m}\left(x\right):\left\{\begin{array}{l}x\in {\Omega }_1,\text{if }\overline{p}\left(x_1,x_2\right)<\overline{p}\left(x_1\right)\overline{p}\left(x_2\right),\\ x\in {\Omega }_2,\text{if }\overline{p}\left(x_1,x_2\right)>\overline{p}\left(x_1\right)\overline{p}\left(x_2\right).\end{array}\right.$

The optimal blurring coefficients of the nuclear functions of the $\overline{m}\left(x\right)$ decision rule are chosen on the basis of the analysis of approximation properties of nonparametric estimates of probability densities $\overline{p}\left(x_1,x_2\right)$, $\overline{p}\left(x_1\right)$, $\overline{p}\left(x_2\right)$ from the minimum condition, their corresponding estimates of standard deviations from the $p\left(x_1,x_2\right)$, $p\left(x_1\right)$, $p\left(x_2\right)$. For example, for $\overline{p}\left(x_1\right)$ such a criterion is [7$–$11]

${\displaystyle \underset{-\infty }{\overset{\infty }{\int }}{\overline{p}}^2\left(x_1\right)d{x}_1}-\frac{2}{n}{\displaystyle \sum _{j=\text{'}}^n\overline{p}\left(x_1^j\right)}$.

Let us define the estimates of probabilities of pattern recognition errors ${\overline{\rho }}_1\left(\overline{c}\left(1\right)\right)$, ${\overline{\rho }}_2\left(\overline{c}\left(2\right)\right)$ using the $\overline{m}\left(x\right)$ dsision rule on the basis of raw statistical data of $V$ at optimal blurring coefficients $\overline{c}\left(1\right)=\left({\overline{c}}_1\left(1\right),{\overline{c}}_2\left(1\right)\right)$, $\overline{c}\left(2\right)=\left({\overline{c}}_1\left(2\right),{\overline{c}}_2\left(2\right)\right)$ of the nuclear functions of statistics $\overline{p}\left(x_1\right)\overline{p}\left(x_2\right)$, $\overline{p}\left(x_1,x_2\right)$ respectively.

The values ${\overline{\rho }}_t\left(\overline{c}\left(1\right),\overline{c}\left(2\right)\right)$ are calculated in the ‘rolling examination’ mode on the $V$ sample assuming that its elements belong to the ${\Omega }_t$ class.

${\overline{\rho }}_t\left(\overline{c}\left(1\right),\overline{c}\left(2\right)\right)=\frac{1}{n}{\displaystyle \sum _{j=1}^{n}1\left(\delta \left(j\right),\overline{\delta }\left(j\right)\right),t=1,2}$,

where $\delta \left(j\right)=t$ are designations of the type of $x^t=\left(x_1^t,x_2^t\right)\in {\Omega }_t$;

$\overline{\delta }\left(j\right)=\left\{\begin{array}{l}t,\text{if}{x}^j\in {\Omega }_t\\ \mathrm{0,}\text{if}{x}^j\notin {\Omega }_t\end{array}\right.$

$–$ «solving» the algorithm of $\overline{m}\left(x\right)$ about the belonging of the $x^j$ situation to on of the classes ${\Omega }_t$, $t=1,2$.

While calculating ${\overline{\rho }}_t\left(\overline{c}\left(1\right),\overline{c}\left(2\right)\right)$ in accordance with the ‘rolling examination’ methodology the situation $x^j=\left(x_1^j,x_2^j\right)$ from the $V$ sample, which is fed into the algorithm of $\overline{m}\left(x\right)$ for control, is excluded from the process of producing statistics $\overline{p}\left(x_1,x_2\right)$, $\overline{p}\left(x_1\right)\overline{p}\left(x_2\right)$.

The indicator function is defined by the expression

$1\left(\delta \left(j\right),\overline{\delta }\left(j\right)\right)=\left\{\begin{array}{l}0,if\delta \left(j\right)=\overline{\delta }\left(j\right),\\ 1,if\delta \left(j\right)\ne \overline{\delta }\left(j\right).\end{array}\right.$

Let us denote by ${\overline{\overline{\rho }}}_t$ the value of the estimation of the probability of pattern recognition error assuming that the $V$ sample elements belong to the class ${\Omega }_t$, $t=1,2$. Let us compare the values ${\overline{\overline{\rho }}}_1$, ${\overline{\overline{\rho }}}_2$.

Then the $H_0$ hypothesis is valid if ${\overline{\overline{\rho }}}_1$ < ${\overline{\overline{\rho }}}_2$. Otherwise, at ${\overline{\overline{\rho }}}_2$ < ${\overline{\overline{\rho }}}_1$ the random variables $x_1$ and $x_2$ are independent.

When the $n$ volume of the $V$ smple is limited, the problem of confidence estimation of probabilities of pattern recognition errors arises. For its solution, the traditional methodology of confidence estimation of probabilities or Kolmogorov $–$ Smirnov criterion is used.

For example, when using the Kolmogorov $–$ Smirnov criterion, the deviation ${\overline{D}}_{12}=\left|{\overline{\overline{\rho }}}_1-{\overline{\overline{\rho }}}_2\right|$ is compared to the threshold value [12]

$D_{\beta }=\sqrt{-\ln \left(\frac{\beta }{2}\right)/n}$.

Here $\beta$ is a probability (risk) of rejecting the hypothesis ${\overline{H}}_0$: ${\rho }_1={\rho }_2$. If the ratio ${\overline{D}}_{12}$ < $D_{\beta }$ is satisfied, then the ${\overline{H}}_0$ hypothesis is valid and the risk of rejecting it does not exceed the value ofβ. At ${\overline{D}}_{12}$ > $D_{\beta }$ the ${\overline{H}}_0$ hypothesis rejected.

Formation of sets of independent random variables

There is a sample of observations $V=\left(x_v^i,v=\overline{1,k},i=\overline{1,n}\right)$ of the $n$ volume composed of statistically independent observations of the components of the multivariate random variable $x=\left(x_v,v=\overline{1,k}\right)$. The type of the $p\left(x\right)$ probability density function is unknown a priori. It is necessary according to the statistics of $V$, using the hypothesis testing criterion proposed above [13$–$16]

$H_{vj}:p\left(x_v,x_j\right)\equiv p\left(x_v\right)p\left(x_j\right)$

For the components $x_v$, $v=\overline{1,k}$, $x_j$, $j=\overline{1,k}$, $v>j$, to form the sets of the independent random variables $x\left(t\right)=\left(x_v,v\in I_t\right)$, $t=\overline{1,m}$. The $m$ number of sets of components of the random variable x is unknown, and $I_t$ is a set of component numbers that make up the set $x\left(t\right)$.

The proposed methodology is based on performing the following steps:

1. In accordance with the above recommendations, to test the $H_{vj}$ hypotheses for each pair of the components $\left(x_v,x_j\right)$ of the multivariate random variable $x=\left(x_v,v=\overline{1,k}\right)$. The number of such pairs corresponds to the value $k\left(k-1\right)/2$.
2. Based on the results of step 1, construct an information graph $G\left(X,A\right)$, where $X$ is a set of its vertices corresponding to the components of the random variable $x$, and $A$ is a set of edges. Between the two vertices $x_v$, $x_j$ there is an edge if the $H_{vj}$ hypothesis is satisfied, i.e. the components $x_v$, $x_j$ are independent.
3. Analyse the information graph $G\left(X,A\right)$ and determine its complete subgraphs $G\left(X_t,A_t\right)$, $t=\overline{1,m}$. Each vertice of the subgraph $G\left(X_t,A_t\right)$ has an edge if the components of the random variable x are independent. Detect complete subgraphs using algorithms for cutting the original graph, which are based on analysing its adjacency matrix. The components $x_v$, $v\in I_t$ correwsponding to the vertices of the complete subgraph $G\left(X_t,A_t\right)$ form a set of independent random variables.

Modification of the method of testing the hypothesis of independence of random variables in conditions of large volumes of statistical data

With large $n$ volumes of the statistical data $V=\left(x_1^i,x_2^i,i=\overline{1,n}\right)$ regression estimates of probability densities $\overline{p}\left(x_1,x_2\right)$, $\overline{p}\left(x_1\right)$, $\overline{p}\left(x_2\right)$ are used in the proposed methodology. These estimates are based on the compression of the original information, e.g., $V_1=\left(x_1^i,i=\overline{1,n}\right)$ into the data array ${\overline{V}}_1=\left({\overline{p}}_1^j,z^j,j=\overline{1,N}\right)$ by decomposing the area of values $x_1$ into $N$ intervals. Here $z^j$ are centres of sampling intervals of values $x_1$, and ${\overline{p}}_1^j={\overline{P}}_1^j/\Delta$ is probability density estimation in the $j$ ^th interval; $\Delta$ is a sampling interval length; ${\overline{P}}_1^j$ is frequency of occurrence of the $x_1^i$ values from the $V_1$ sample in the interval numbered $j$. Then the regression estimate of the probability density function $p\left(x_1\right)$ according to ${\overline{V}}_1$ has the form [17; 18]

$\overline{p}\left(x_1\right)=\frac{1}{{с}_1}{\displaystyle \sum _{j=1}^N{\overline{P}}_1^j\Phi \left(\frac{x_1-z^j}{c_1}\right)}$.

The proposed approach allows reducing by orders of magnitude the $n$ volume of initial statistical information when estimating probability densities. The peculiarity of the statistics of the $\overline{p}\left(x_1\right)$ type allows simplifying considerably the choice of coefficients of $с$ blurring of nuclear functions in the $\overline{p}\left(x_1\right)$ statistics from the condition of minimum criterion

$\frac{1}{N}{\displaystyle \sum _{i=1}^N\left({\overline{p}}_1^i-\overline{p}\left(x_1^i\right)\right)}$.

By analogy the estimation of the probability densities $p\left(x_2\right)$, $p\left(x_1,x_2\right)$ is carried out. Regression estimates of probability densities are used in testing the hypothesis of independence of random variables according to the proposed methodology.

Analysing the results of the computational experiment

The effectiveness of the proposed method of testing the hypothesis of independence of two-dimensional random variables and Pearson's criterion in the conditions of ambiguous dependences at different volumes of statistical data has been compared [19$–$21]. The sensors of random variables $x_1$, $x_2$ were formed on the basis of the uniform distribution law $x_1$, which was used in the calculation of the values of $x_2$ in the form of nonlinear transformations $x_1$. At the same time the values of $x_2$ were superimposed with disturbances with the normal distribution law, which has zero mathematical expectation and standard deviation $\sigma$. An example of the values of random variables $x_1$ and $x_2$ is shown in Fig. 1.

Рис. 1. Значения случайных величин x₁, x₂ из выборки исходных статистических данных V при n = 500 и σ = 0,5 (темные точки), а при σ = 2 (серые точки) при использовании зависимостей различной сложности

Fig. 1. Values x₁, x₂ of random variables from a sample of initial statistical data V at n = 500 and σ = 0.5 (dark dots), and at σ = 2 (grey dots) when using dependencies of varying complexity

When testing the independence hypothesis of a two-dimensional random component based on the Pearson criterion, the results of the optimal selection of the number of sampling intervals are used [22$–$24]

$N^{*}={\left(\frac{3}{4}{\Delta }_1{\Delta }_2{‖p\left(x_1,x_2\right)‖}^{2}n\right)}^{1/2}$.

The value ${‖p\left(x_1,x_2\right)‖}^2={\displaystyle \underset{-\infty }{\overset{\infty }{\int }}{\displaystyle \underset{-\infty }{\overset{\infty }{\int }}{p}^2\left(x_1,x_2\right)d{x}_{1}d{x}_2}}$, and ${\Delta }_v$ is the length of the interval between values of the random value $x_v$, $v$ = 1.2. The works [25$–$27] are devoted to the traditional formulas of discretization of the range of values of random quantities.

By the results of computational experiment the offered methodology and Pearson's criterion at the analysis of ambiguous dependences between random variables in conditions of relatively small volumes of statistical data and mean square deviations σ of interferences are comparable and unmistakably determine dependence of random variables. This conclusion does not hold for the dependence between random variables (Fig. 1, a), when the Pearson criterion does not establish dependence under the conditions n = 100 and σ $\in$ [0.5; 2]. As σ increases, the efficiency of the criteria being compared decreases. This fact is explained by the peculiarities of ambiguous dependences and large values of σ, when the area of definition of random variables hides the desired dependence. With the increase in the n volume of initial data the efficiency of the compared criteria for testing the hypothesis of independence of random variables increases. This conclusion is expected, since asymptotic properties of nonparametric estimates of probability densities and frequencies of occurrence of random variables in their two-dimensional intervals rise as n increases. The advantage of the proposed methodology for testing the hypothesis of independence of random variables is observed at small values of σ, limited and large n. At large n and σ, the advantage of Pearson's criterion is often revealed if the procedure of optimal discretisation of the area of values of a two-dimensional random variable is used [22].

Application of the proposed methodology in analysing remote sensing data

The developed methodology was tested when analysing the remote sensing data [2; 28]. The object of the study is anthropogenic territories (quarry, suburban development) in the vicinity of the city of Krasnoyarsk. The initial information was formed on the fragments of Sentinel-2 satellite imagery on 26.08.2021 (Fig. 2). The spectral channels $x_j$, $j=\overline{1,9}$ were used. These channels are characterised by wavelengths (nanometres): $x_1$ $–$ (458$–$523), $x_2$ $–$ (543$–$578), $x_3$ $–$ (650$–$680), $x_4$ $–$ (698$–$713), $x_5$ $–$ (733$–$748), $x_6$ $–$ (773$–$793), $x_7$ $–$ (785$–$899), $x_8$ $–$ (1565$–$1655), $x_9$ $–$ (2100$–$2280).

Рис. 2. Фрагменты спутниковой съемки Sentinel-2. Антропогенные территории:

a $–$ карьер; b $–$ пригородная застройка

Fig. 2. Fragments of Sentinel-2 satellite imagery. Anthropogenic territories:

a $–$ quarry; b $–$ suburban development

The proposed methodology allows forming pairs of independent and dependent random variables by changing the ratio between their parameters. The application of the methodology allowed us to detect 31 and 29 pairs of spectral features with strong linear dependence for the objects ‘quarry’ and ‘suburban development’, respectively. The obtained results are presented in Fig. 3.

Рис. 3. Иллюстрация сильной линейной зависимости между парами спектральных признаков (x_i, x_j), характеризующихся оценками коэффициентов корреляции больше 0,9:

a $–$ карьер; b $–$ пригородная застройка

Fig. 3. Illustration of a strong linear relationship between pairs of spectral features (x_i, x_j) characterized by correlation coefficient estimates greater than 0.9:

a $–$ quarry; b $–$ suburban development

Additionally, non-linear dependences between spectral features were found for the object ‘quarry’

$\left(x_1,x_9\right)$, $\left(x_1,x_8\right)$, $\left(x_1,x_7\right)$, $\left(x_1,x_5\right)$, $\left(x_1,x_4\right)$

and the object ‘suburban development’

$\left(x_7,x_9\right),\left(x_4,x_9\right),\left(x_3,x_9\right),\left(x_2,x_9\right),\left(x_1,x_9\right),\left(x_1,x_8\right),\left(x_1,x_7\right)$.

The obtained results are reliable for all pairs of spectral features, since the condition $\left|{\overline{\rho }}_1-{\overline{\rho }}_2\right|$ > $D_{\beta }$ is met at $D_{\beta }$ = 0.029 and the risk $\beta$ = 0.025 reject the $H_0$ hypothesis of equality of values ${\rho }_1$, ${\rho }_2$.

The problem of detecting anthropogenic areas from spectral data is considered. The error of their recognition in the space of spectral features $x=\left(x_j,j=\overline{1,9}\right)$ based on the training sample $V=\left(x^i,\sigma \left(i\right),i=\overline{1,n}\right)$ is equal to 0.012, where $n=n_1+n_2$, $n_1$ = 3377 (‘quarry’, $\sigma \left(i\right)$ = 1), $n_2$ = 5049 (‘Suburban Development’ $\sigma \left(i\right)$ = 2). When excluding from the training sample, for example, the spectral features $\left(x_4,x_5\right)$, $\left(x_5,x_6\right)$, $\left(x_4,x_5,x_6\right)$ the estimates of pattern recognition errors correspond to the values 0.011; 0.01; 0.008. The obtained reduction in pattern recognition errors is not reliable compared to the error estimate in feature space $x_j$, $j=\overline{1,9}$. Nevertheless, the obtained result justifies the possibility of reducing spectral features in the synthesis of decision-making algorithms and simplifying their optimisation.

Conclusion

The methodology of testing the hypothesis of independence of pairs of random variables, based on the use of nonparametric algorithm of pattern recognition, allows bypassing the problem of discretisation of the area of the values of random variables into multidimensional intervals. This problem is inherent in the generally recognised Pearson criterion. The conditions of competence of the proposed method and Pearson's criterion in the analysis of unambiguous and ambiguous dependences between random variables are determined. Using the apparatus of graph theory, the proposed method is developed in the formation of sets of independent random variables. The obtained results are generalised in testing the hypothesis of independence of random variables for large volumes of statistical data on the basis of compression of initial information, which allows increasing by orders of magnitude the computational efficiency of the problems being solved. The effectiveness of the proposed methodology is confirmed when analysing remote sensing data of anthropogenic territories and assessing their states. In the presence of a set of spectral features characterised by a strong linear dependence between its pairs, it is possible to reduce the number of spectral features in the recognition of anthropogenic territories with a decrease in the estimate of the probability of error in their recognition.

作者简介

Anna Sharueva

编辑信件的主要联系方式.
Email: anna-denisyuk@yandex.ru
ORCID iD: 0009-0003-4255-4554

head of the remote sensing laboratory, assistant

参考

补充文件

附件文件

动作

1. JATS XML

下载

2. Fig. 1. Values x1, x2 of random variables from a sample of initial statistical data V at n = 500 and σ = 0.5 (dark dots), and at σ = 2 (grey dots) when using dependencies of varying complexity

下载 (86KB)

索引源数据

3. Fig. 2. Fragments of Sentinel-2 satellite imagery. Anthropogenic territories: a  quarry; b  suburban development

下载 (62KB)

索引源数据

4. Fig. 3. Illustration of a strong linear relationship between pairs of spectral features (xi, xj) characterized by correlation coefficient estimates greater than 0.9: a  quarry; b  suburban development

下载 (45KB)

索引源数据