TO THE PROBLEM OF NONPARAMETRIC ROBUST ESTIMATION OF THE REGRESSION FUNCTION ON OBSERVATIONS

Full Text

Introduction. The problem of restitution of regression function on observations with outliers is considered [1-5]. When studying the task [6; 7], we use the suggested ro- bust estimation procedure, which is a correction of the training sample free of outliers [8-11]. Thus, we obtain cs > 0 , lims®¥ s(cs )k =¥ , lims®¥ cs = 0 . (3) In case of multi-dimensional data (k-dimensional) it is: å x j - x s k æ i ö ç y ÕФç j ÷ Подпись: i the function value and its restitution without outliers. = ø , i=1 Y (x) j =1 è cs ÷ (4) In the last decades of the last century, the intensive s s k æ x - xi ö development and application of nonparametric and robust methods of data processing began [12-17]. The reason is that on the one hand there is the need to control complex economic and social structures without parametric descriptions, as well as technical objects, for which, for example, the applied methods stability is important to failures and noise in the operation of recording equip- ment, on the other hand, the development of computing technology, which makes it possible to implement labori- ous algorithms [18]. Nonparametric regression function estimatimation on observations. Nonparametric estimation of regression function on observations for a one-dimensional case is the åÕФ ç j j ÷ i=1 j =1 ç cs ÷ è ø xi, yi, i = 1, s , - sample of observations; Ф(v) - bell-shaped function; v - random variable, cs - blur coefficient. Robust nonparametric regression function estimation on observations. Step-by-step experiment scheme is as followed: 1. The initial sample on an actual object is obtained. 2. We set up the blur coefficient and choose the bell- shaped function . 3. We check each sample point for estimation quality. If the estimation quality is sufficient and inequalfollowing [19; 20]: ity “more 2s2 ” is not satisfied, then the initial sample i ç c ÷ ås y Ф æ x - xi ö Y (x) = i=1 è s ø , (1) becomes the working sample. If the estimation quality is not sufficient and inequality “more 2s2 ” is satisfied, then outliers are exs s æ x - x ö åФç i ÷ i=1 è cs ø Ф(v) - is the kernel. The kernel is a finite bell-shaped square integrable function satisfying conditions [19; 20]: cluded from the initial sample and less points will become the working sample. 4. We restitute the regression function by means of nonparametric estimation. Computing experiment. y = sin (x)2 is a function 0 < Ф(v) < ¥ "v Î Ç(v) , 1 Фæ x - xi ö dx = 1 , c ò ç ÷ s è cs ø chosen for the computing experiment. When forming the training sample, outliers were artificially added. The triangular kernel is used as a bell-shaped function lim 1 Фæ x - xi ö = d( x - x ) , (2) Ф(v): c n®¥ ç ÷ i s è cs ø Ф(v) = ìï1- v , v £ 1, (5) í cs - blur coefficient which satisfies the following conditions: ïî0, v > 1. Further we perform the work with the entire sample constructing the function and its restitution, we find the For illustrative purposes, we will add the perturbation action to some observations: criterion of accuracy. As the criterion of nonparametric estimation accuracy we use the quadratic criterion: hi = lyix, where xÎ[-1,1] , noise level is l = 5 %. (8) s s2 = 2 (6) There are the elements of the sample, its approxima- å( yi - ys (xi )) , i =1 yi - a true sample received on the formulas given above; ys(xi) - is a nonparametric estimate. After checking the accuracy criterion, we pay atten- tion to the points at which the restitution error is big and they satisfy criterion (7). Elements of the training sample tion and two outliers on fig. 1. The restitution accuracy is 0.36. It is the same on fig. 2, except that 5 % noise level is added. The restitution accuracy is 0.40. It should be noted that restitution accuracy depends on whether there is the noise in the function. There are five outliers on fig. 3. The restitution accu- racy has obviously changed and is equal to 0.54. that satisfy the requirement: ri > 2s2, (7) Using sample. ri > 2s2 , we exclude outliers from the initial where ri = ( yi - ys (xi )), i = 1, s, are allocated and ex- cluded from the initial sample. We consider in fig. 1 - is a training sample, 2 - is nonparametric estimation. The triangular kernel was used as a bell-shaped finite function, We present the results of the numerical experiment il- lustrating the effectiveness of an algorithm. We consider restitution of regression function on observations, which has several outliers at a sample size 100. Fig. 4 displays algorithm work with regard to robast estimation. In this case the sample size decreased because the program excluded outliers enterfering good restitu- tion. In fig. 5 the 5 % noise level is added to the restituted function, accuracy of restitution decreased - 0.11, that is more than in fig. 4. Note that restitution accuracy signifi- cantly increased, not 0.36 and 0.54, but 0.06. It means that the given function was basically completely restituted. As an experiment, the same function with the same outliers but for a smaller sample size - 60 was considered. Fig. 1. Two-outlier restituted function Рис. 1. Восстановленная функция с учетом двух выбросов Fig. 2. Two-outlier restituted function with 5 % noise level Рис. 2. Восстановленная функция с учетом двух выбросов и помехой 5 % Fig. 3. Five-outlier restituted function Рис. 3. Восстановленная функция с учетом пяти выбросов Fig. 4. None-outlier restituted function Рис. 4. Восстановленная функция без учета выбросов Fig. 5. None-outlier restituted function with 5 % noise level Рис. 5. Восстановленная функция без учета выбросов, но с помехой 5 % Sample units, its approximation and two outliers are also given in fig. 6. Restitution accuracy decreased to 0.45. Fig. 7 displays sample units with five outliers, the restitution error is 0.69. In fig. 8 the restituted function without outliers is presented, the accuracy of restitution is 0.14. It is worth noticing that the sample size considerably influences restitution accuracy. For example, the accuracy of 100 elements sample size with regard to two outliers was 0.36, in the same case of 60 elements sample size it was 0.45. For descriptive reasons we will consider one more similar function: y = cos(x)2 · sin(x) with 100 elements sample size. Sample units with one and three outliers respectively are given in fig. 9-11. Restitution accuracy at one outlier is 0.31, and at three - 0.41. In fig. 10 the 5 % noise level was added to one-outlier res- tituted function. The restitution accuracy - 0.33. In this case, accuracy of restitution was not strongly affected by the noise. Fig. 12 shows the restitution of function without outliers, the accuracy of restitution is 0.04. And in fig. 13 there is already 5 % noise level, restitution accuracy is 0.12. In this case, accuracy significantly decreased. Fig. 2. Two-outlier function estimation Рис. 6. Восстановленная функция с учетом двух выбросов Fig. 7. Five-outlier restituted function Рис. 7. Восстановленная функция с учетом пяти выбросов Fig. 8. None-outlier restituted function Рис. 8. Восстановленная функция без учета выбросов Fig. 9. One-outlier restituted function Рис. 9. Восстановленная функция с учетом одного выброса Fig. 10. One-outlier restituted function with 5 % noise level Рис. 103. Восстановленная функция с учетом одного выброса и помехой 5 % Fig. 11. Three-outlier restituted function 830 Рис. 11. Восстановленная функция с учетом трех выбросов Fig. 12. None-outlier restituted function Рис. 12. Восстановленная функция без учета выбросов Fig. 13. None-outlier restituted function with 5 % noise level Рис. 13. Восстановленная функция без учета выбросов, но с помехой 5 % Conclusion. The main result of the article is that by means of the robast estimation approach it is possible to obtain significantly better function restitution quality on observations. It is worth noticing that restitution accu- racy considerably increased after we excluded outliers. For descriptive reasons of the experiment several func- tions for restitution were considered. For the first function two sample sizes 100 and 60 were considered, we were visually convinced that the sample size has not small value for restitution. The restitution accuracy is signifi- cantly higher if the sample size is equal to 100 rather than if it is equal to 60.

About the authors

L. N. Sopova

Reshetnev Siberian State University of Science and Technology

31, Krasnoyarsky Rabochy Av., Krasnoyarsk, 660037, Russian Federation

S. S. Chernova

Siberian Federal University, Institute of space and Information Technologies

Email: chsvetlanas@gmail.com
26b, Academica Kirenskogo Str., Krasnoyarsk, 660074, Russian Federation

References

Supplementary files