Development and psychometric assessment of the test “Predictive style”

Full Text

Introduction

Forecasting is defined as the process of making assumptions about the onset, development, and outcome of future events. Event forecasting is a necessary part of effective adaptation [1]. Affective forecasting processes guide behavioral activity [2], decision making [3], and emotional responses [4]. Incompetence in affective forecasting has been shown to play a role in many mental disorders [1, 5–8].

We assume that the presence of a psychological construct, namely, forecasting style, is an indicator of a person’s attitude toward their process of making forecasts. We consider forecasting style as a personality trait that reflects the pattern of forecasting through two dimensions: the subjective importance of the forecasting process and the subjective evaluation of the favorability of forecasted events. Accordingly, the importance of forecasting can manifest itself in extreme forms: as indifference to forecasting (IF; evaluating the forecasting process as unimportant) or excessive forecasting (EF; giving excessive importance to the forecasting process). The second dimension, the assessment of the favorability of forecasts, manifests itself as excessive optimism (EO; the tendency to expect exceptionally favorable events) or excessive pessimism (EP; the tendency to expect exceptionally unfavorable events). In our opinion, the extreme forms of both constructs are maladaptive, unlike their moderate forms, which allows the formation of realistic and effective forecasts.

We believe that forecasting style is an independent psychological construct related to, but different from, the phenomena linked to the forecasting process itself: forecasting abilities, anticipation, dispositional optimism, and coping strategies [9, 10].

As a personality trait, forecasting style should be sufficiently stable over time. Because of the importance of forecasting in the adaptive capability of an individual, we assume that the features of forecasting style differentiate people with neurotic disorders from those without a mental pathology. If our assumptions are correct, then choosing forecasting style as a target will allow us to develop methods of psychocorrection that will increase personal stability and adaptation. To verify these assumptions, we developed a “forecasting style” test and assessed its psychometric parameters [11].

The aim of this study was to develop and conduct a psychometric assessment of a Forecasting Style test.

Materials and methods

The development of the Forecasting Style test was conducted in the following stages.

First stage. A bank of statements was compiled for the initial version of the test, before an assessment of the content of the items and exploratory factor analysis were performed. Based on the results of this stage, a working version of the test was created.

Second stage. Confirmatory factor analysis of the working version was performed to study the factor structure and measures of compliance of a four-scale test model. Next, the reproducibility of the model was checked on a new sample, and the reliability and internal consistency of the methodology and retest reliability were evaluated.

Third stage. Studies that focus on examining measures of external validity were performed, namely, convergent, divergent, and criterion validity.

The following quality benchmarks were used for the confirmatory factor analysis of the model: comparative fit index (CFI) >0.9, Tucker-Lewis index (TLI) >0.9, standardized root-mean-square residual (SRMR)=0.08, and root mean-square-error of approximation (RMSEA)=0.08. To study the reliability of the technique, Cronbach’s α and MacDonald’s ω coefficients of above 0.7 were used as threshold. Spearman correlation coefficient was used to test for correlation between the results of the working version of the methodology and the external criteria. To test for differences between the groups with and without mental disorders, Welch’s t-test was used. Jamovi (Version 2.3) software was used to analyze the data.

The study sample included the following groups.

1. The first group comprised 348 students of Kazan Federal University and Kazan State Medical University (275 women and 52 men), who, according to self-reports, had not sought psychiatric or psychotherapeutic help and did not exhibit symptoms of mental disorders at the time of the study. The participants’ age ranged from 18 to 34 years (Mean=19 years). The test data from this group were used to develop a working version of the test.
2. The second group comprised 196 students of Kazan Federal University (51 men and 145 women), who, according to self-reports, had not sought psychiatric or psychotherapeutic help and did not exhibit symptoms of mental disorders at the time of the study. The participants’ ages ranged from 18 to 28 years (Mean=20 years). This sample group passed the working version of the test, and the data thus obtained were used as a control to assess criterion validity.
3. The third group comprised 103 patients with neurotic mental disorders who were being treated in Bekhterev Republican Clinical Psychiatric Hospital and Municipal Clinical Hospital No. 18 (27 men and 75 women). The patients were diagnosed based on the criteria of the 10th revision of the International Classification of Diseases (F32, F41.1, F41.2). The patients’ ages ranged from 18 to 65 years (Mean=36 years).

Results and discussion

Based on the theoretical understanding of “Forecasting Style”, a bank of 88 statements was constructed at first. The order of the statements in the initial test version was randomized using a random number generator. The participants were asked to rate the degree of their agreement with the given statements on a 4-point Likert scale (1=“Completely disagree”, 2=“Somewhat disagree”, 3=“Somewhat agree”, and 4=“Completely agree”). The testing was performed during the participants’ free time between 10:00 hrs and 17:00 hrs.

The results of scree test showed that a four-factor structure would be the most suitable in this study. Therefore, in the exploratory factor analysis, we selected statements with a factor loading of more than 0.3 for all four scales of the test. Additionally, substantive assessment of the scales was performed in parallel.

After combining the 88 statements and randomizing their order, we compiled a working version of the test consisting of 21 statements. Six statements were used for factor 1 (the EF scale), five for factor 2 (the EP scale), six for factor 3 (the EO scale), and four for factor 4 (the IF scale). The measures of compliance for this model obtained from the exploratory factor analysis were as follows: TLI=0.965; RMSEA=0.0311; 95% confidence interval (CI) [0.0146, 0.0443].

Table 1 shows the factor loadings for each of the 21 statements obtained from the exploratory factor analysis.

 

Table 1. Factor loading of statements based on exploratory factor analysis

№	Factors				Uniqueness
	1	2	3	4
1	–0.16113	–0.0726	0.4672	0.11985	0.701
21	–0.19186	–0.2210	0.6616	0.14983	0.486
9	–0.02747	0.0856	0.6508	0.04503	0.500
12	0.07119	0.3005	0.5119	0.12876	0.511
6	–0.01662	0.1103	0.5052	–0.25688	0.700
17	0.15775	0.2140	0.5240	–0.21827	0.685
15	0.15382	0.1624	0.2197	0.51086	0.573
10	0.14206	0.2232	–0.1099	0.48573	0.696
4	–0.01476	–0.1209	–0.0154	0.46824	0.800
19	0.05884	0.0304	0.0495	0.61289	0.601
18	0.53847	–0.1542	–0.0543	0.04032	0.590
14	0.68248	0.0312	–0.0751	–0.00448	0.518
3	0.56716	–0.0320	–0.0260	0.08947	0.653
16	0.44336	–0.1221	0.1234	0.02351	0.767
11	0.51904	–0.1936	–0.1070	0.02643	0.548
7	0.57298	–0.1652	0.0204	0.10656	0.572
5	–0.23324	0.3298	0.1365	0.27552	0.544
8	–0.21963	0.5402	0.0390	0.14951	0.454
13	–0.26408	0.6394	–0.0293	0.11190	0.326
20	–0.00222	0.7591	0.0672	–0.05846	0.406
2	–0.18701	0.5128	0.0723	0.21473	0.454

Note. 1) Maximum likelihood extraction was used in conjunction with oblimin rotation. 2) The approval number is obtained from the working version of the test. 3) Factor loadings above 0.3 are shown in bold.

 

The reliability analysis results showed that the working version of the test had sufficient reliability; the entire test (Cronbach’s α=0.851, McDonald’s ω=0.862), the IF scale (Cronbach’s α=0.63, McDonald’s ω=0.639), the EF scale (Cronbach’s α=0.748, McDonald’s ω=0.756), the EO scale (Cronbach’s α=0.778, McDonald’s ω=0.784), and the EP scale (Cronbach’s α=0.845, McDonald’s ω=0.850). We attributed the relatively low reliability values on the IF scale to the low number of questions used. Including more questions would have increased the coefficient values but reduced the model quality; hence, we settled for a balanced version.

We conducted confirmatory factor analysis to verify the effectiveness of the resulting model. The results supported the four-factor structure of the working version of the test. The measures of compliance of the theoretical model with empirical data were within the thresholds (CFI=0.927; TLI=0.917; SRMR=0.0538; RMSEA=0.0497; 95% CI [0.0403, 0.0589]).

Table 2 shows the test form for Forecasting Style. The participants were given the following instructions. “Please rate each of the statements below from 1 to 4 to indicate how you agree with it. There are no right or wrong answers, you are just expressing your opinion. Answer according to how you feel and not on how most people would respond”. The scores were graded as follows.

 

Table 2. The test form of the test «Prognostic style»

No	Statement	Score
1	I often can't fall sleep for a long time, as I keep thinking about various scenarios that may unfold the next day.
2	If success awaits me in the future, it is most likely accidental.
3	There are no problems that cannot be solved.
4	I am a fatalist in the sense that it is not worth making plans; things will happen as they happen.
5	When I am offered something new, I often refuse it because I do not believe in success.
6	If I don’t have a few plans, I cannot calm down.
7	In case of uncertainty, I usually expect everything to turn out fine.
8	I’m sure that I’m going to fail.
9	If I am not absolutely sure that there will be no trouble, then my anxiety won’t go away.
10	I tend to go with the flow without thinking ahead.
11	I can handle any problem.
12	I often get lost in my forecasts.
13	It is unlikely that I will succeed in the future.
14	I always try to be optimistic about the obstacles that may arise.
15	I often rely on chance.
16	Even if others can’t handle a problem, I can definitely handle it.
17	If you are not in control, then everything will go wrong.
18	All my dreams can come true.
19	When an unpleasant event occurs, it is easier to give up because everything will resolve itself.
20	“And then they lived happily ever after” is not about me.
21	In many situations, the number of possible options just blows my brain.

 

1=“Completely disagree”, 2=“Somewhat disagree”, 3=“Somewhat agree”, and 4=“Completely agree”.

The results were processed by adding up the points on the scales.

• The EF scale — Statements 1, 6, 9, 12, 17, 21
• The EP scale — Statements 2, 5, 8, 13, 20
• The EO scale — Statements 3, 7, 11, 14, 16, 18
• The IF scale — Statements 4, 10, 15, 19

Furthermore, we tested the model on a new sample using the working version (n=198) to assess its reproducibility. The confirmatory factor analysis results showed that the model yielded similar compliance measures (CFI=0.873; TLI=0.854; SRMR=0.0675; RMSEA=0.0643; 95% CI [0.0535, 0.0751]).

Pearson’s correlation analysis was performed to study the internal consistency of the scales. All of the statements exhibited significant correlation with their scales (p <0.05). Additionally, the test scales were correlated with each other (p <0.01); IF and EF (r=0.219; p <0.001), EP and EF (r=0.569; p <0.001), EP and IF (r=0.385; p <0.001), EO and EF (r=-0.363; p <0.001), and EO and EP (r=-0.592; p <0.001). No correlations were found, except for the IF and EO scales.

The working version was re-tested (n=86) after 2 months to check for retest reliability. There were significant correlations among all scales (Pearson coefficient, p <0.05) [12], and the correlation coefficients were moderate and strong, indicating the retest reliability of the technique.

Figure 1 shows the results of the correlation analysis among the scales of the Forecasting style test before and after 2 months.

 

Fig. 1. Correlations of “Predictive Style” Test scales 2 months apart; БП — indifference to forecasts; Ипр — over forecasting; ИП — excessive pessimism; ИО — excessive optimism.

 

In the third stage, convergent, divergent, and criterion validity were examined. Conceptually, optimism and pessimism are close to our methodology in terms of making positive and negative expectations of future events. To evaluate positive and negative expectations of future events, we used the test of dispositional optimism, or the Life Orientation Test (LOT; n=39) (Carver and Scheier, 1985; Russian adaptation: Gordeeva, Sychev, and Osin, 2010). Unrealistic optimism [12], which is defined as the expectation of a better personal future than what is reasonably likely, can also be considered a close concept. Differences in forecasting style are expressed as excessively favorable or unfavorable expectations in overly generalized and uncontested formulations. For example, the use of words, such as everything, always, anyone, accurately, and confident. Therefore, we expect the EO and EP scales to be correlated with the scales of positive and negative expectations but the strength of the correlation to be moderate.

Forecasting style scales reflect attitudes toward the forecasting process and its success. Therefore, we expect the scales to be related to the level of forecasting ability and competence. To verify this assumption, we used the test of anticipation consistency (Mendelevich, 2003) (n=104) and the test of predictive ability (Regush, 2003) (n=62).

The forecasting process serves as an important adaptation mechanism. Some stress management strategies are directly focused on making forecasts. To test the relationship between coping strategies and the forecasting process, we used the Melbourne decision making questionnaire (Mann et al., 1997; Russian adaptation: Kornilova, 2013) (n=62) and the Lazarus coping test (Lazarus and Folkman, 1988; Russian adaptation: Kryukova, Kuftyak, Zamyshlyaeva, 2004) (n=46) [13].

Figure 2 shows the correlation between the scales of the Forecasting Style test and the scales of the LOT. The EO scale was significantly correlated with the positive expectations subscale; however, the strength of the correlation was moderate. The EP scale was significantly correlated with the negative expectations subscale (the strength of the correlation was noticeable) and inversely correlated with the positive expectations subscale (the strength of the correlation was moderate). Several conclusions can be drawn from this. These findings confirm the conceptual proximity of EO and EP to positive and negative expectations of the future (dispositional optimism). The moderate strength of the correlations indicates that the constructs are not identical. This justifies the separation of EO and EP as independent concepts.

 

Fig. 2. Correlation of the “Predictive Style” test with the dispositional optimism test. Shows statistically significant correlations. The dotted line shows inverse correlations. ИО — excessive optimism; ИП — excessive pessimism.

 

Figure 3 shows the correlation between the scales of the Forecasting Style test and the scales of the tests of anticipation ability and forecasting ability. The IF scale had a weak inverse correlation with all of the scales of anticipatory consistency, except the spatial anticipation consistency scale. The IF scale indicates that the forecasting process is assessed as unimportant, whereas the anticipatory consistency scale indicates the evaluation of the success of forecasting. Indeed, giving little importance to forecasts leads to less successful forecasting.

 

Fig. 3. Correlation of the scales of the “Predictive Style” test with the scales of the TAS/PC anticipatory consistency test and the predictive ability test. The figure shows statistically significant correlations. The dotted line shows inverse correlations. БП — indifference to forecasts; ПАС — spatial anticipatory consistency; ИПр — over forecasting; ОАС — general anticipatory competence; ИО — excessive optimism; ЛСАС — personal-situational anticipatory competence; ИП — excessive pessimism; ВАС — temporary anticipatory consistency.

 

The EF scale had a weak inverse correlation with the general and spatial anticipation consistency scales. This relationship highlights the difference between the overuse of forecasting and its success. In anxiety disorders, patients are prone to ruminating about possible future scenarios with an exaggerated focus on unlikely but dangerous outcomes. Such forecasting cannot be considered effective. In our opinion, the inverse correlation that we obtained justifies this interpretation.

The EF scale had a weak and negative correlation with the scales of general and spatial anticipation consistency. This may indicate that excessively pessimistic expectations of the future reduce the success of anticipation, leading to an overestimation of the probability of negative events occurring.

The EO scale was moderately correlated with the predictive ability scale, whereas the EP scale was negatively correlated with it. The test scales describe the ability to predict a set of thinking qualities, such as analyticity, depth, awareness, flexibility, perspective, and evidence [12]. It is possible that people who are more inclined to optimism than to pessimism make more successful forecasts. There is a connection between optimism and resilience and other factors of psychological adaptation and stability [14], which may depend, among other things, on the ability to make forecasts.

However, the moderate correlation we found highlights that EO is not fully associated with predictive ability. This is also evidenced by the absence of a link with the anticipation of consistency. However, this explanation needs to be clarified by future studies.

Figure 4 shows the correlation between the scales of the Predictive Style test and the scales of the Melbourne decision making questionnaire and the Lazarus coping test. The IF scale had a moderate and positive correlation with the avoidance and procrastination scales and a weak correlation with the over-vigilance scale.

 

Fig. 4. Correlation of the “Predictive Style” test scales with the Melbourne Decision Making Questionnaire and “Lazarus Coping Test” scales. The figure shows statistically significant correlations. The dotted line shows inverse correlations. БП — indifference to forecasts; Ипр — over forecasting; ИП — excessive pessimism; ИО — excessive optimism.

 

The EF scale had a significant and moderate correlation with the avoidance scale and a moderate correlation with the scales of over-vigilance, escape–avoidance, and procrastination; all of the observed correlations were direct. Getting stuck in the forecasting process can help to move away from stress-related activity.

The EO scale was inversely and moderately correlated with the scales of acceptance of responsibility, procrastination, avoidance, and over-vigilance and had a weak and positive relationship with the scale of vigilance, which indicates that people who tend to have more optimistic expectations of the future are less likely to avoid activity and over-forecast but also to take less responsibility for events, relying instead on a positive course of events unfolding.

The EP scale was significantly correlated with the scales of avoidance, over-vigilance, escape–avoidance, and procrastination; all of the observed correlations were direct. These findings indicated that people with more pessimistic expectations of the future are more focused on moving away from stressors and overly focused on anticipating future events. The EP scale was also inversely correlated with the vigilance and positive overestimation scales. Vigilance is defined as productive coping and reveals only one aspect of coping with uncertainty, namely, thinking about alternatives. Pessimistic expectations of the future make it difficult to consider the overall picture for the future development of events. The inverse relationship with positive overestimation may be due to a general tendency to focus on the negative aspects of events.

The significant correlations obtained allow us to conclude that the methodology had convergent validity. Additionally, these correlations reveal nuances in the related concepts. Most of our assumptions regarding the correlation of forecasting style with other psychological constructs were supported.

To substantiate divergent validity (n=39), we used Marlowe-Crowne’s scale for self-assessment of the need for approval (Marlowe and Crowne, 1960; Russian adaptation: Khanin, 1974) [15]. All of the scales, except the EF scale (Spearman’s r=-0.321; p=0.046), were found to not be significantly correlated with the scale of need for approval. Accordingly, we concluded that social desirability had a low influence on the participants’ responses.

As shown above, the attitude toward the forecasting process is correlated with coping strategies and anticipation consistency. These psychological phenomena play a role in the formation and maintenance of neurotic disorders [16, 17]. Forecasting style can be associated with the symptoms of psychopathology and determine its specifics. EF can support ruminative processes, and IF reflects the degree of hopelessness. More pessimistic expectations of the future are associated with depressive or anxious traits.

Accordingly, we concluded that the scales of the working version of the test were related to the symptoms of depression and anxiety. We expected to find statistically significant differences between subjects with and without mental disorders. These parameters were selected as a criterion for checking the criterion validity of the test.

To evaluate the severity of anxiety symptoms, we used the Sheehan anxiety self-assessment scale (Sheehan (1983); the author of the Russian adaptation is not known) [18]. The severity of depression symptoms was assessed using the Zung depression self-assessment scale (Zung, 1965; Russian adaptation: T.I. Balashova, 1988) [19]. These methods were selected from the list of methods recommended by the Russian Society of Psychiatrists for the psychodiagnostics of anxiety-phobic and depressive disorders [18].

Table 3 shows that the Forecasting Style test scales were significantly correlated with both the control and experimental groups after combining the samples, except the IF scale and the experimental group. The EF and EP scales were directly and moderately correlated, whereas the EO scale was inversely correlated.

 

Table 3. Correlation of the “Predictive Style” test scales with the Sheehan Patient-Rated Anxiety Scale. and Zung Self-Rating Depression Scale

Self-assessment	Group	EF	IF	EP	EO
Self-assessment of anxiety	Control	R=0.573 р <0.001	R=0.184 р=0.009	R=0.357 р <0.001	R=–0.295 р <0.001
	Experimental	R=0.504 р <0.001	R=0.133 р=0.493	R=0.413 р <0.001	R=–0.224 р=0.021
	General	R=0.566 р <0.001	R=0.222 р <0.001	R=0.485 р <0.001	R=–0.372 р <0.001
Self-assessment of depression	Control	R=0.614 р <0.001	R=0.164 р=0.022	R=0.548 р <0.001	R=–0.452 р <0.001
	Experimental	R=0.548 р <0.001	R=0.174 р=0.068	R=0.653 р <0.001	R=–0.257 р <0.001
	General	R=0.602 р <0.001	R=0.241 р <0.001	R=0.671 р <0.001	R=–0.578 р <0.001

Note. Statistically significant correlations are in shown in bold.

 

We used multiple linear regression to assess the role of forecasting style as a predictor of symptoms. The results were statistically significant: anxiety (R²=0.379, F=44.8, p <0.001) and depression (R²=0.571, F=97.4, p <0.001). Using stepwise regression, the addition of a group membership factor to the model (control or experimental) led to a significant increase: anxiety (ΔR²=0.034) and depression (ΔR²=0.043). Additionally, we investigated the effects of age and gender, but the models including these factors were not significantly different from the models excluding them.

To study the differences between the control and experimental groups, Welch’s t-test was used. Statistically significant differences were found in all of the scales (Table 4). The group with mental disorders showed higher values on the EF, IF, EP and EO scales.

 

Table 4. Differences between the control group (C) and patients (P)

Scale	Group	Average value	SD	t	p	d-Cohen
EF	P	15.63	3.93	3.46	<001	0.432
	K	14.08	3.22
IF	P	9.51	2.57	3.79	<001	0.468
	K	8.38	2.26
EO	P	15.74	4.11	–6.41	<001	–0.821
	K	18.60	2.72
EP	P	10.89	3.55	6.87	<001	0.872
	K	8.21	2.53

 

The criterion validity of the technique was justified by the differences found between the experimental and control groups, the correlation between the scales of the forecasting style test and depression and anxiety, and sufficient indicators of the regression model.

Conclusions

1. The findings of this study substantiate the application of the psychological construct of forecasting style as an independent psychological phenomenon. The findings demonstrate that the resulting test is internally consistent and has criterion, divergent, and convergent validity and retest reliability.
2. For further research, we recommend the standardization and development of test scales incorporating socio-demographic factors and examining the applicability of the test on samples of patients with other nosological forms.

Additional information

Funding source. The work was conducted and the article was published using the personal funds of the author's team.

Competing interests. The authors declare that there is no potential conflict of interest requiring disclosure in this article.

Authors’ contribution. A.S. Granitsa — literature review, experimental design, collecting and preparation of samples, writing the main part of the text, making final edits; A.R. Lotfullina — literature review, collecting and preparation of samples, writing the main part of the text; V.V. Perishivkina — literature review, collecting and preparation of samples, writing the main part of the text.

Acknowledgments. The authors express their gratitude to the chief physician of the RCPB named after Academician V.M. Bekhtereva to I.I. Akhmetzyanov and the head 2 by the women's department to F.G. Kalimullin, to the chief physician of the State Clinical Hospital No. 18 of Kazan R.S. Bakirov and the head by the Psychotherapy department to B.I. Akberov, for administrative support of the study. The authors are grateful to I.R. Kalimullina, Associate Professor of the Department of Neurology with courses in Psychiatry, Clinical Psychology and Medical Genetics at Kazan Federal University, as well as M.M. Galiev, a student of the Faculty of Medicine at IFMIB K(P)Thank you for your help in collecting the research material.

About the authors

Alexander S. Granitsa

Kazan Federal University

Author for correspondence.
Email: hebechblu@yandex.ru
ORCID iD: 0000-0002-0498-7397
SPIN-code: 4775-7844

MD, Cand. Sci. (Med.), Assoc. Prof., Depart. of Neurology with courses in Psychiatry, Clinical Psychology and Medical Genetics

Russian Federation, Kazan

Adelina R. Lotfullina

Kazan Federal University

Email: lotfullinaadelina2001@mail.ru
ORCID iD: 0009-0001-0510-9625

Student, Faculty of Medicine

Russian Federation, Kazan

Vlada V. Perishivkina

Kazan Federal University

Email: vlada.pereshivkina@gmail.com
ORCID iD: 0009-0008-4366-7290

Student, Faculty of Medicine

Russian Federation, Kazan

References

Supplementary files