MULTILEVEL EVALUATION OF QUALITY INTERACTIVE ELECTRONIC TECHNICAL MANUALS FOR AVIATION TECHNOLOGY

Full Text

Introduction. The aim of assessing the quality of IETM for aircraft is to obtain a consolidated opinion on the achieved degree of satisfaction of the needs for the creation of interactive electronic technical manuals, in the conditions of the fuzziness of the initial expert data used for evaluation. The assessment of the quality of the IETM on operation and repair of aviation equipment is the de- termination of their compliance with the requirements of a predetermined network of indicators of this quality. Mathematical basis for setting quality indicators for multi-level assessment. Assessment of the quality of interactive electronic technical manuals (IETM) on opera- tion and repair of aviation equipment is the determination of their compliance with the requirements of a predeter- mined network of indicators of this quality [1]. If the in- dicator is a complex nesting of other, simpler indicators (aggregated indicator), then the one-time act of analysis and evaluation of quality develops into a multi-step se- quence of qualimetric evaluation. The aggregated indica- tors in this qualimetric evaluation will be compositionally formed from more simple indicators, resulting in the for- mation of hierarchical network of indicators. In such a hierarchical network at the terminal level, as a rule, there are indicators that can be directly measured or expertly evaluated, quantitatively or qualitatively. Such indicators of the terminal level of the hierarchical network of indica- tors of assessment of quality of IETM on operation and repair of aircraft are called elementary. Their set is de- noted by {qi}. At higher levels of the hierarchical network of quality assessment of IETM on operation and repair of aviation equipment compositionally complex, called ag- gregated, indicators {qij} are established, aggregating weighted composition of indicators, from the composition of elementary indicators and(or) other aggregated indica- tors. Root vertex of the specified hierarchical network results is an integral index Q0, that is the quality of the IETM on operation and repair of aircraft. As part of this hierarchical network, the compositional or aggregate importance of any quality indicator qi for the assessment of the integral indicator Q0 will be different. Accordingly, for the numerical fixing of composite or aggregate indicator of the importance qi in the convolu- tion of the nearest aggregated indicator according to the structure of the hierarchical network of indicators, the weight coefficient wm, n is entered - the local rank of the m-th indicator in the n-th convolution, wm, n Î(0,1); wm, n Î R. In addition, for each aggregated indicator the normalization condition must be satisfied: å wm,n = 1. m Accordingly, the numerical value of the rank of each elementary or intermediate aggregated index qi in the convolution of the integral index Q0, according to the hierarchical network of indicators, is determined by the weighting factor bm* - the global rank of the m-th index in the convolution of the integral Q0. Respectively b = Õ w Q0 , * m m,n qm where bm* Î (0,1); bm* Î R. The specific features of the definition of input infor- mation from the expert group for measuring or evaluating local and global ranks determines the classification basis for identifying of the corresponding particular methods within the framework of the developed method of quali- metric assessment of IETM on operation and repair of aviation equipment. The basic feature of the separation of a particular sub-method is a specific feature of obtaining expert information at the input [2; 3]. As a rule, the role of this feature includes the dimension and character of the scale for instrumental measurement or qualitative and quantitative assessment of elementary indicators, a form for taking into account the fuzziness of the input expert data, differences in the mathematical convolution of in- termediate aggregated and elementary indicators into the assessment of an integrated indicator. Algorithmic- qualimetric essence of quality assessment is initially re- duced to the definition of a set of properties (individual qualities) of the object being evaluated - IETM on opera- tion and repair of aviation equipment. The values of the estimates for these properties are evaluated on a scale. Assessment of the quality of IETM on operation and repair of aircraft is associated with non-numeric or “soft” measurements. “Measurement” should be understood as a manipulation in which the properties of the IETM on op- eration and repair of aircraft equipment are correlated in accordance with some strictly ordered gradations of qual- ity. With this approach, not only real numbers are consid- ered in the role of quantitative measurement results, but also other algebraic groups, necessarily having an order relation between their elements, a number of specialized scales of measurement and evaluation of the quality properties. in accordance with (1). Accordingly, together with the strict ordering relationship by preference, it is necessary to introduce a non-strict order relation: Objectively predetermined use of soft, non-numeric (q(r ) > q( s) ) Û ((q(r ) > q( s) )Ú ("i q(r ) = q( s ) )). (2) i i measurements in assessing the quality of IETM on opera- tion and repair of aircraft specifies the fact that particular and aggregated indicators from a single hierarchical net- In this case, it is possible to determine the relation of strict order through the relation of non-strict order: work of quality assessment indicators can be assessed or measured on scales (nominal, ordinal, ratios, differences), (q(r ) > q(s ) ) Û ((q(r ) > q( s) ) Ù (q(r ) ¹ q( s) )). (3) accordingly, the numerical values of these measurements and estimates will be set on different algebraic groups. According to [4-8], it was established that for the categorical features peculiar to the IETM on operation When carrying out the ordering of IETM on operation and repair of aircraft using the dominance ratio, a signifi- cant complexity arises - the presence of a large number of objects of assessment q (r), q (s), which are not comparable i i and maintenance of aviation technology quality assess- ment the rational form of convolution of the integral evaluation criterion would be the additive linear criterion of the normalized form: in relation to the non-strict order (2). The probability of comparability with respect to (2) of these random vectors is determined by the expression 1 Подпись: n Q = å wiqi . i=1 P{(q%(r) > q%(s) ) Ú (q%(s) > q%(r) )} = 2m-1 . (4) The above-shown application of heterogeneous scales of soft measurements or evaluation of elementary indica- tors leads to such a nature of the data that is input to the method, that this character is understood as insufficiency or deficiency of the input information [9]. This affects the algorithm for calculating the local ranks {wi}, which play the role of accounting for the different values of the input data in the developed method of assessing the quality of IETM on operation and repair of aircraft. Based on the foregoing, the mathematical apparatus of randomized aggregated indicators [10-14] was adopted for the mathematical basis of the developed qualimetric method. Aggregated and integral indicators of quality of IETM on operation and repair of aircraft are calculated through From (4) it can be concluded that the ability to find comparable multi-criteria quality assessments of IETM on operation and repair of aviation technology decreases rapidly with an increase in the number of analyzed indica- tors. So, if IETM is estimated by m = 11 indicators then the probability that a pair of randomly selected electronic manuals is comparable in all indicators at once is less than one thousandth. To ensure the comparability of multi- parameter quality assessments aggregated indicators are used the essence of which is to form elementary indica- tors q = (q1,…, qm) along a vector of some aggregated indicator Qk, which is a function Q = Q(q) = Q(q1,…, qm) of a vector of elementary exponents q satisfying the con- dition of monotonicity in the form the normalized additive convolution of the elementary indicators included in the corresponding network of indi- " q( j) , q(l) Î{q : q = (q ,..., q ), q Î[0, 1]} (5) 1 m i cators [15]. Basing on this, it is possible to evaluate k alternative options for the implementation or projects of IETM on operation and repair of aircraft. It is assumed that their sets of estimates for elementary indicators are represented by numerical vectors q(j) = (q (j),…, q (j)), {q( j) > q(l)} Þ {Q(q( j) ) ³ Q(q(l) )}. In turn, the most powerful aggregated indicator is an integral indicator of Q. Synthesis and ranking of a multi-level tree of as- 1 m sessment indicators. In the most general form the funci = 1,…, m, j = 1,…, k. Each such numerical vector is a multiparametric assessment of the corresponding alterna- tive implementation of IETM on operation and repair of aircraft in the form of a family of values of elementary indicators q = (q ,…, q ). Further, it is assumed that on tion of synthesis or convolution of elementary indicators in the aggregated ones for IETM on the operation and repair of aviation equipment takes the form -1 æ m ö 1 m Qf (q; w) = Qf (q1,.., qm ; w1,..., wm ) = f ç å wi f(qi ) ÷. the community of all evaluated realizations of IETM, quantitatively described by the indicated numerical vectors of evaluations by elementary indicators, the relation è i=1 ø Conducting the substitution of stretched / compressed of strict dominance is given: values a q(0) , Подпись: i i i = 1,..., m in the convolution formula al- (q(r ) > q( s ) )Û (("i q(r ) ³ q( s ) )Ù($j q(r ) > q( s ) )). (1) lows synthesizing the following expression i i j j Q (a q(0) ,..., a q(0); w) = The expression describing condition (1) should be ´ æ m w ö æ 1 1 m w ö m m (0) (0) considered as follows: IETM q(r) is superior IETM q(s) in the considered indicator when it is not less preferable = çÕai i ÷´çÕqi i ÷ = A Q´ (q1 è i=1 ø è i=1 ø ,..., qm ; w). (6) by each component of the elementary index (q (r) ≥ q (s)) Basing on expression (6) the following conclusions i i and there is an elementary or more particular aggregated indicator for which the first IETM is preferable to the second ((q (r) > q (s))). The ordering of the analyzed IETM can be made: if elementary indicators are measured or assessed on the scale of relations, then the multiplicative aggregated indicator is measured or estimated on the scale j j on operation and repair of aircraft will be a strict ordering of relations with stretching / compression A. Defining and marking the so-called “difference scale shift”, setting the beginning of reference as qi = 0 and the end, as qi = 1, is much easier than choosing the coefficient of stretching / compression of the ratio scale. This is the main reason for choosing the additive normalized synthe- sizing function in the proposed method for estimating IETM on the operation and repair of aircraft. In the mathematical apparatus of the method of quality assess- ment of IETM on operation and repair of aviation equip- ment, it is precisely the additive normalized convolutions Q+(q; w) of elementary indicators that are used to calcu- late the aggregated and integral estimates of these elec- tronic manuals. An example of the synthesis of a multi-level tree of indicators for the assessment of IETM on operation and repair of aircraft is presented in fig. 1. We consider: some list of five indicators, and initial values of quality charac- teristics, compositionally included in the aggregated and integral quality indicators of IETM on operation and re- pair of aviation equipment, as well as a set of vectors of values of elementary quality indicators of nine realiza- tions of a specific IETM from various competing devel- opment organizations for which a noticeable difference in technological approaches, development methods, software implementation, etc. is stated. It is implied that the vectors of values of the elementary indicators of the quality of IETM was conducted according to the originally specified five basic indicators z1,…, zn. From the initial two tables in fig. 1 it follows that all elementary indicators z1,…, z5 are estimated (i. e. ”softly measured”) on the scale of names, except for the indicator z5, which is estimated on the ordinal scale. In the framework of the example, it is obvious that in order to ensure further integral convolu- tion, it is necessary to evaluate all the considered indica- tors on the ordinal scale: <Satisfies; rather satisfies than does not satisfy; Does not satisfy>. Fig. 1. An example of setting IETM quality indicators when synthesizing a multi-level tree of assessment indicators Рис. 1. Пример задания показателей качества ИЭТР при синтезе многоуровневого дерева показателей оценки Let us assume that, in the framework of the considered example, the experience of experts allowed the scaling of 1 N (m,n) (t ) N (m,n) å w (t ) i the values shown in fig. 1 to the indicated ordinal scale. The obtained values of the elementary indicators wi = M w%i = N (m, n) å wi t =1 = t =1 , m (7) y1,…, y5 of the quality of the IETM have exactly the ordi- nal character. This means that they are not real numbers and arithmetic operations cannot be performed on them. To carry out such operations, the indicated values of the and the corresponding value of the variance of a random variable or standard quadratic deviation as a measure of the accuracy of this significance rank value elementary exponents y1,…, y5 must be given a numerical form. In other words, it is necessary to specify the map- ping x = φ(y) of the gradations of the index y to the set of real numbers R1. This mapping should preserve the si = = = m -1 + 1 m -1 = . (8) order of the gradations. From the infinite family of admism2 (m +1) n m(m +1) sible assignments of ordinal scales, on which the parame- ters y1,…, y5 are measured, in the framework of the exam- ple a simple transformation was initially chosen, resulting in the above mentioned ordinal scale to a decreasing number of natural numbers, respectively: <3, 2, 1 >. On the basis of this conversion, the values x1,…, x5 of the indicators are obtained, directly estimated on the scale of real numbers R1 and rolled up into quality aggre- gated indicators of IETM on the operation and repair of aircraft. Traditionally, both in the framework of the given ex- ample of estimating elementary indicators x1,…, x5, and in the established practice of evaluating other combinations of elementary indicators of IETM on operation and repair of aircraft, the use of linear normalization is accepted and practically justified. This allows getting the values of elementary indicators q1,…, q5 in the form reduced to the (0, 1) interval, as shown in the table in the lower part of fig. 1. for the observed example. Each of the rows in the table at the bottom of fig. 1 is a multiparametric estimate of q = (q1,…, q5) vector characteristics of IETM on opera- tion and repair of aircraft of specific performance from a particular contractor. Determining the values of the ranks that determine the significance of the IETM quality indicators in their com- positions from the fuzzy and (or) insufficient initial data The proposed mathematical and algorithmic tools for taking into account the compositional importance of sim- pler indicators for assessing the quality of IETM in the composition of more complex ones using ranks of signifi- cance determined by the methods of “soft computing”, make it possible to take into account any fragmentary and fuzzy expert data on the comparative weight of individual indicators in the aggregated ones. This provides the de- veloped method of evaluation with the necessary flexibil- ity when working with the initial heterogeneous informa- tion of qualimetric evaluation. The procedure of ranking indicators in the composi- tions, as a fundamental component of the method for as- sessing the quality of IETM on operation and repair of aviation equipment, can be illustrated with an example for three simpler indicators (m = 3) that are combined into a composition of a more complex indicator. The ranking results are presented in the right table in fig. 2. + Analyzing the presented example of the implementa- tion of the procedure of ranking indicators in the compo- sitions of more complex quality indicators of IETM on operation and repair of aviation equipment, it is necessary to pay attention that the mathematical consideration of the fuzziness and noise of the input qualimetric data I makes it possible to increase the accuracy of estimates of the ranks of the importance of elementary indicators wi (I ) , obtained during the examination. It consists in building a family of all possible numerical vectors of significance i = 1,…, m, in aggregated and integral indicators, Q( j ) , ranks, taking into account fuzzy (expert) data on the prior- ity of the relevant indicators, and calculating a aggregated numerical vector of ranks of significance. This device provides for the construction of a numerical grid for m IETM quality indicators with discrete step n. Further, this grid is thinned out according to fuzzy data I from experts on the priority of certain indicators. Formally, such initial data are presented in the form of a system of inequalities. As a result of exceptions from the initial digital grid of numerical vectors that do not correspond to fuzzy (expert) data I, n numerical vectors remain, after normalization of which it becomes possible to calculate the ranks of com- positional significance for m IETM quality indicators. In other words, ranks of significance are defined as the alge- braic average over each of the m indicators on the set of n thinned vectors of N initial number vectors. Each rank of significance, taking into account the fuzziness of the data set for the thinning of the original digital grid, is calcu- lated as a random variable: that is, the mathematical ex- pectation of the value of the rank of significance is deter- mined j = 1,…, k. In practice, this is reflected in a decrease in the standard deviation of si(I) and S(J)(I), as well as an in- crease in the reliability of the ranking of elementary and aggregated indicators in the compositions (direct to 1 the probabilities of dominance of p(r, s; I), r, s = 1,…, m and P(j, l; I), j, l = 1,…, k). The procedure for calculating the values of indica- tors of quality assessment IETM. The calculation of the values of indicators for assessing the quality of IETM on operation and repair of aircraft in the conditions of the primary application of the procedure for ranking indica- tors in compositions of more complex quality indicators has a number of specific features: - when determining the ranks of significance on the basis of direct, estimated input data on the priorities of simpler indicators of evaluation; - in determining the ranks of significance based on expert sampling of data consisting of examples of previ- ously evaluated implementations of IETM; - when determining the ranks of significance based on fuzzy initial data on the priority of elementary and aggre- gate indicators in the composition of the integral; - when determining the ranks of significance based on a multi-level hierarchical network of indicators. The structure of the procedure for calculating the val- ues of the quality assessment indicators of IETM on op- eration and repair of aviation equipment is shown in gen- eralized form in fig. 3 The considered procedure for assessing IETМ on the operation and repair of aircraft, depending on the methods of obtaining input expert data, is as follows: a) quality assessment in determining ranks of signifi- cance based on direct, estimated input data on the priori- ties of simpler assessment indicators; b) assessment of quality in determining the ranks of significance based on an expert sample of data consisting of examples of previously evaluated implementations of the IETM; c) quality assessment in determining the ranks of sig- nificance based on fuzzy initial data on the priority of elementary and aggregated indicators in the composition of the integral; d) quality assessment in determining the ranks of sig- nificance based on a multi-level hierarchical network of indicators. The generalized scheme for calculating the values of the IETM quality assessment indicators, for evaluating specific implementations of these manuals, includes three main steps: 1. Construction of numerical vectors of values of elementary quality indicators. 2. Calculation of ranks that determine the signifi- cance of quality indicators in their compositions accord- ing to fuzzy and (or) insufficient initial data on the com- positional significance of more particular indicators in the composition of more aggregated indicators. 3. Calculation of the integral quality indicator, as a weighted arithmetic average of mathematically analogous more particular indicators on the basis of numerical vec- tors of values of elementary indicators and ratings of sig- nificance for the analyzed performance of IETM on op- eration and repair of aviation equipment values. Fig. 2. An example of the procedure of ranking indicators implementation in the compositions of more complex indicators of quality Рис. 2. Пример реализации процедуры ранжирования показателей в композициях более сложных показателей качества Fig. 3. A generalized presentation of the structure of the procedure for calculating the values of IETМ quality assessment indicators Рис. 3. Обобщенное представление структуры процедуры расчета значений показателей оценки качества ИЭТР The described generalized scheme for calculating the values of quality assessment indicators is the essence of the developed quality assessment method of IETM on operation and repair of aircraft equipment. It allows you to generalize the previously described and disclosed pro- cedures for scaling and randomizing ranks of the signifi- cance of quality assessment indicators for the specified IETM logically. Conclusion. The proposed method of multilevel qual- ity assessment of IETM on aviation technology, based on the use of a mathematical soft computing apparatus for processing primary qualimetric data obtained from ex- perts and making it possible to draw conclusions about the achieved quality level of created IETM on the opera- tion and repair of aviation equipment.

About the authors

E. A. Frolova

Saint Petersburg State University of Aerospace Instrumentation

Email: frolovaelena@mail.ru
67, Bolshaya Morskaya Str., Saint Petersburg, 190000, Russian Federation

References

Supplementary files