Validation of an Instrument That Measures Factors Affecting Saudi Parents’ Tolerance of Risky Play: A Rasch Analysis

Within occupational therapy, play is viewed as a primary occupation of childhood through which children achieve important developmental skills (Lynch & Moore, 2016). Play is often thought of as being fun, spontaneous, safe, and uncertain (Bundy, 1997). Risky play is a form of play that is defined as “thrilling and exciting forms of physical play that involves uncertainty and a risk of physical injury” (Sandseter, 2010, p. 22). Risky outdoor play has been linked to both physical and cognitive development (Barker et al., 2014; Brussoni et al., 2012; Gray et al., 2015; Sandseter, 2007; Sandseter & Kennair, 2011), being cited as helping children of all abilities develop confidence, resilience, and self-regulation, among other beneficial life skills (Bundy et al., 2015).

Despite the widely recognized benefits of risky play, children often experience limited opportunities to take reasonable risks in play (Brussoni et al., 2015; Bundy et al., 2008; Gray et al., 2015; Little et al., 2011; Tremblay et al., 2015). Beetham et al. (2019) noted that children with disabilities, in particular, experience disparities and fewer opportunities to engage in risky play than their typically developing peers.

Researchers both in and outside of the field of occupational therapy (Wyver, Bundy, et al., 2010; Wyver, Tranter, et al., 2010) have questioned the excessive focus on safety that results in “surplus safety” practices in modern societies and limits children’s access to active play experiences. In the literature on risky play, there has been a paradigm shift toward balancing the duty of care and dignity of risk (Brussoni et al., 2012, 2015; Sandseter & Kennair, 2011). Duty of care is a legal term that requires people to take reasonable care and avoid causing harm to others (Merriam-Webster, n.d.). Ibrahim and Davis (2013) defined dignity of risk as “the principle of allowing an individual the dignity afforded by risk-taking, with subsequent enhancement of personal growth and quality of life” (p. 189). In the context of risky play, parents’ duty of care and responsibility toward protecting the well-being of their children should not overshadow or lead to restricting children’s choices to take risks out of fear of failure or negative outcomes (Brussoni et al., 2012; Niehues et al., 2016). Because balancing the two concepts can be a complex task that requires consideration of children’s needs and abilities while maintaining their safety and autonomy, there has been a strong interest in parents’ perceptions of their children’s engagement in risky activities.

The literature includes several internal and external factors that influence how parents react to their children’s participation in such play. For instance, Little (2010) and Niehues et al. (2015) interviewed Australian parents, mostly mothers, to understand their views on the benefits of risky play and their attitudes toward children’s risk taking. Although the parents acknowledged the potential benefits of children engaging in age-appropriate risk, their tolerance for risk varied depending on the situation and context (Little, 2010, 2015; Niehues et al., 2013, 2015). Creighton et al. (2017), on the other hand, explored how Canadian fathers of children ages 2 to 7 yr perceived risk. Fathers’ responses ranged from very risk tolerant to accepting some risk taking, to overly protective. These varied perspectives demonstrate that the concept of risk taking in play is multilayered and complex (Creighton et al., 2017). Creighton et al.’s (2017) study builds on the qualitative findings of Brussoni and colleagues (Brussoni et al., 2013; Brussoni & Olsen, 2013), who found that, compared with mothers, fathers tended to be more tolerant of physical risk taking and viewed risky play as opportunity for risk management and emotional bonding with their children. These studies showed that mothers and fathers may have different orientations to child’s risky play (Brussoni et al., 2013; Creighton et al., 2017; Little, 2010, 2015; Niehues et al., 2013, 2015).

To address the need to promote parents’ tolerance of risky play, Bundy and colleagues (Bundy et al., 2011) developed a “risk reframing” workshop as a component of the Sydney Playground Project (SPP). Through a series of reflective activities, these intervention workshops aimed to promote risk as a multidimensional concept that presents risky play as an avenue for adventure, opportunity, and learning (Niehues et al., 2013) rather than only a source of danger or injury. In light of the promising findings of the in-person workshops, Brussoni et al. (2021) carried out a randomized control trial titled “Go Play Outside!” to determine the efficacy of web-based and in-person versions of the risk reframing intervention with 451 mothers. Compared with those in the control group, mothers who participated in the in-person workshop expressed significantly greater risk tolerance 1 wk after the intervention (β = 0.22, 95% CI = [0.03, 0.40], p = .02). In contrast, mothers who participated in the web-based intervention expressed even greater risk tolerance than mothers in the control group 1 wk after the intervention (β = 0.25, 95% CI = [0.08, 0.42], p = .004) and 3 mo postintervention (β = 0.24, 95% CI = [0.06, 0.42], p = .007; Brussoni et al., 2021).

In a large, cross-sectional study, Jelleyman et al. (2019) examined 1,366 New Zealand parents’ perceptions and practices pertaining to risky play and independent mobility using a set of validated measures. Parents in the study expressed concerns over the restricted safety regulations but still stressed the importance of closely supervising their child’s play activities. This study revealed a discrepancy between parents’ beliefs about the advantages of risky play and their children’s actual engagement in such activities, confirming qualitative findings about parents’ conflicting feelings when it comes to promoting risk taking and balancing safety (Little, 2010, 2015; Niehues et al., 2013, 2015). Like risk-aversive Western parents, authoritative parents in Turkey had supportive thoughts about the benefits of engaging in risky play; however, these beliefs did not translate into practice, showing again a reluctant stance toward risk taking in play (Cevher-Kalburan & Ivrendi, 2016).

In these quantitative studies, the primary outcome measure was the Tolerance for Risk in Play Scale (TRiPS), a group of instruments developed by Bundy and colleagues as part of the SPP (Hill & Bundy, 2014) that includes caregiver and teacher versions. The scale was developed on the basis of the Norwegian model of risky play (Sandseter, 2007). The caregiver version (TRiPS–R) consists of 30 items measuring adults’ tolerance, or change in tolerance, for risk in play. The TRiPS–R was later revised and validated using Rasch modeling (Grady-Dominguez et al., 2020). Rasch analysis revealed excellent evidence for validity and internal reliability of the scale, with a person reliability index of .88. Data from 96.7% of the items conformed to Rasch model assumptions, suggesting strong construct validity (Grady-Dominguez et al., 2020). Grady-Dominguez et al. (2020) made several improvements to enhance the sensitivity of the TRiPS by adding new items to capture the responses of adults with a low tolerance for risk. They also revised items that were originally specific to Australian culture to improve the applicability and usability of the instrument across different cultures.

The body of literature on risky outdoor play includes qualitative and quantitative studies that examined parents’ perceptions of risky play and the variables hindering children’s engagement in such play (Brussoni, Olsen, et al., 2012; Creighton et al., 2017; Jelleyman et al., 2019; Little et al., 2011; Niehues et al., 2013), and these studies originated mostly in Western cultures. Although research in Eastern cultures has begun to emerge (Cevher-Kalburan & Ivrendi, 2016), there is currently no research that focuses specifically on outdoor play in Saudi Arabia. Furthermore, although the TRiPS–R has been useful in measuring adults’ tolerance for risky play, there is no objective measure to assess personal, situational, and cultural factors that influence their risk tolerance. Therefore, for this study, we developed the Factors Affecting Tolerance for Risk in Play Scale (FAC–TRiPS). The purpose of this study was to establish evidence for the construct validity and internal reliability of data gathered with the FAC–TRiPS.

Our study addresses two research questions:

1. What is the evidence for construct validity of the data gathered with the FAC–TRiPS?

2. What is the evidence for internal reliability of the data gathered with the FAC–TRiPS?

This study is a cross-sectional quantitative study for the development of an instrument to measure personal and environmental factors that influence parents’ risk appraisal and tolerance. We obtained approval (Protocol 3276) from the Colorado State University Institutional Review Board before the commencement of data collection.

Inclusion criteria were as follows: Participants were parents who are Saudi, had a child between the ages of 7 and 10, and were able to read and speak English. The sample consisted of 90 parents: 45 men and 45 women from different households. Participants ranged in age from 27 to 51 yr (M = 35.5). Further details of sample characteristics are provided in Table 1. We excluded participants who did not meet the inclusion criteria. To determine the sample size, we followed guidelines established by Wright and Stone (1979) for Rasch studies. These guidelines suggest that, to maintain stability in person measures within a range of ±0.5 logits at a 95% confidence interval, it is advisable to have a sample size that ranges from 64 to 144 for polytomies (rating scale models; Linacre, 1994).

The FAC–TRiPS includes 17 items (Table 2) that each address the question “How likely is this to influence your decision to allow your child to engage in risky play?” The FAC–TRiPS captures a range of personal and environmental factors that influence how parents define, think about, and encourage risk-taking in play. Each item uses a 5-point Likert scale on which 1 = never a factor, 2 = rarely a factor, 3 = neutral, 4 = sometimes makes me say no, and 5 = definitely makes me say no. To develop these items, we conducted a focus group with parents of children ages 3 yr and older. We also reviewed the current literature on risky play and consulted two experts who have specialized knowledge in the subject of outdoor play and who have focused their research on parent and caregiver perceptions of risky play. We piloted the questionnaire with 49 US parents of children ages 7 to 10 yr to examine the meaningfulness of questions, ensure clarity of wording, and uncover practical challenges of using the survey.

We recruited participants through snowball sampling. The first author (Rana Alarawi) began recruiting participants purposively through personal communications. Then, we asked these participants to help us recruit future participants by sending the survey link to their acquaintances. Participants completed the FAC–TRiPS online using Qualtrics Core XM survey software.

We used Rasch software (Winsteps, Version 4.4.4; Linacre, 2020) to analyze the data. Rasch analysis is a psychometric technique used to evaluate the strength and quality of assessment data. Rasch modeling uses principles associated with item response theory to assess performance on tests and questionnaires. It assumes that an item response is a result of an interaction between the level of difficulty presented by the item and the extent to which a respondent possesses the latent trait being measured—in this case, factors affecting tolerance of risky play (Bond et al., 2021). In Rasch, both item difficulty and person tolerance are on the same scale and expressed in log odds probability units (logits). We used the Rasch model and tested the degree to which participants’ responses met two assumptions; Easy items (in this case, factors that influence parents’ risk tolerance) are easy for all parents to endorse, and parents who have greater risk tolerance would be more likely to endorse difficult items.

To measure construct validity, we examined various sources, including point-measure correlations, goodness-of-fit statistics, item spread, and principal-component analysis (PCA). Here, we outline the criteria that we followed for each source on the basis of Linacre’s (2002) recommendations.

To test whether FAC–TRiPS items reflect a logical hierarchy of item difficulty measured, we examined the extent to which the difficulty of items reflected theory or logic (Linacre, 2002).

To determine the degree to which the range of FAC– TRiPS item difficulty matched the range of parent tolerance, we examined the Wright map, a Winsteps-generated map of person–item hierarchies. The Rasch analysis sets the mean item difficulty at 0.0. logits. A match between mean item measure and mean person measure indicates a match between person ability and item difficulty (Bond et al., 2021). Large gaps in the item hierarchy suggest a need for more items to capture the range of factors that influence risk tolerance in the sample. Items that are at the same level of difficulty (i.e., fall on the same line) may be redundant (Bond et al., 2021).

To assess whether responses to the FAC–TRiPS items correlate positively with increased total measure, we examined point-measure correlation coefficients between individual scores and overall item measure scores to ensure that items aligned with the overall construct—in this case, factors influencing risk tolerance in play. Positive point-measure correlations suggest that items correspond with, and contribute positively to, the construct (Bond et al., 2021).

We examined two pairs of fit statistics: infit and outfit, both expressed as mean square (MS) and Z standardized (Zstd) values. These provide evidence of how well the data conform to Rasch model assumptions. Ideally, MS values would be 1 and Zstd values would be 0, but the researchers determine the acceptable range (Bond et al., 2021). Because of the preliminary nature of this study, for both items and persons, we accepted MS values between 0.5 and 1.5 and Zstd values between −2 and +2 (Bond et al., 2021; Wright & Linacre, 1994).

Although Rasch modeling can be used to examine the extent to which an assessment measures a unidimensional construct, a PCA of residuals, provided by Winsteps, can be used to examine the extent to which the construct is not unidimensional. Thus, the PCA can contribute to understanding items that fail to conform to Rasch expectations (Bond et al., 2021). We examined multiple sources to assess the unidimensionality of the FAC–TRiPS. Unexplained variance of the first contrast less than 2.5 eigenvalue units, observed variance closely matching expected variance, and disattenuated correlation coefficients greater than .3 all suggest unidimensionality (i.e., they provide evidence that any additional dimensions of the data are not strong; Bond et al., 2021; Linacre, 2018).

We used differential item functioning (DIF) to test the extent to which items function in systematically different ways for different groups. A Rasch-Welch probability of p ≤ .05 and DIF contrast greater than .43 reveals significant differences on items (Zwick et al., 1999). In this study, we compared DIF on the basis of the parents’ gender.

We examined evidence for internal reliability. We calculated a person reliability index and strata (i.e., the number of levels of the latent trait that an assessment can distinguish). We sought a person reliability index of ≥.80 and a strata value of ≥2.0. We calculated strata using the following formula: strata = (4G + 1)/3, where G = person separation index (Bond et al., 2021).

Rasch modeling is an iterative process (Bond et al., 2021). Iterative analyses revealed that 2 of the 17 items failed to conform to Rasch expectations. After careful consideration, we eliminated nine individual item responses of the 1,530 individual responses (90 × 17) with MS and/or Zstd values outside the accepted range. Although removing these misfitting responses significantly improved person fit, it did not change item fit. What follows is a report of the results of the final iteration in which we continued to include all 17 items.

Visual inspection of the Wright map (Figure 1) showed that the items formed a logical hierarchy (Bond et al., 2021). The 17 items were presented from easiest (internal fears) to most difficult (external factors). That is, items such as “fear of negative judgement from others” and “cultural views that play activity is not appropriate or unsafe” were relatively difficult to endorse, compared with items such as “fear that my child could be injured” and “fear that I could not rescue my child if something happened.” Overall, the hierarchy of the FAC–TRiPS items was logical.

The mean person measure was approximately 0.25 (Figure 1), close to the item mean of 0.0 (in Figure 1, S = 1 SD and T = 2 SD). Eleven participants obtained raw scores above the range of item difficulty, whereas 12 participants obtained scores below the range of item difficulty. There were multiple gaps along the hierarchy, and several items appeared at similar levels of difficulty (Bond et al., 2021).

All items demonstrated positive point-measure correlations, indicating that each item of the FAC–TRiPS aligned with the latent trait.

Data from 15 of the 17 items (88%) conformed to Rasch model expectations. Specifically, the items “fear of negative judgment from others” and “fear of repercussions from an authority (e.g., police)” failed to fit, because the mean and SE values for either infit or outfit statistics were outside of the acceptable ranges (Table 2).

The PCA of residuals demonstrated that observed variance (49.6%) closely matched expected variance (49.7%). The unexplained variance for the first contrast in the PCA had an eigenvalue of 2.7. All items had high disattenuated correlations (>.3). The unexplained variance of the first contrast had an eigenvalue slightly greater than Linacre’s (2018) critical value of 2.5, suggesting possible multidimensionality; however, the raw variance explained by the first contrast was only 8.1%.

To further assess the unidimensionality of the FAC–TRiPS items, we performed an exploratory factor analysis (EFA; Brown, 2015). After comparing the eigenvalues of the actual data to those from the factor analysis, we found that the first factor had a larger eigenvalue than the corresponding eigenvalue from the random data (eigenvalue = 6.73). A second factor had a slightly larger eigenvalue than the corresponding eigenvalue from the parallel analysis (eigenvalue = 1.33). This indicates that EFA results support a two-factor solution. However, the first factor is clearly dominating, which suggests that further analysis of the instrument is needed.

Three items were different between fathers and mothers. “Fear of negative judgment from others” and “fear of repercussion from an authority (e.g., police)” had significant DIFs, t(84) = −2.98, p = .004, and t(82) = −2.65, p = .001, respectively; revealing that these items were easier for mothers than for fathers to endorse as factors that influenced their decisions. However, significant DIF for “influence of media reports of children getting hurt/injured/kidnapped” indicated that this item was easier for male parents than for female parents to endorse, t(83) = 2.08, p = .04.

The person reliability index was .90, providing excellent evidence for internal reliability.

We found an item separation value of 3.86, corresponding to a strata value of 4.35, suggesting that the FAC–TRiPS differentiated more than four levels of parent tolerance.

The purpose of this study was to examine evidence for construct validity and internal reliability of data collected from Saudi Arabian parents using the FAC– TRiPS. Evaluation of multiple sources of construct validity and internal reliability revealed that the FAC– TRiPS produced adequate evidence of validity and reliability. To our knowledge, this is the only study that included an equal representation of mothers’ and fathers’ perspectives of risk taking among children.

Consistent with literature on factors influencing parents’ definitions of risk and decisions around risky play (Brussoni et al., 2012; Cevher-Kalburan & Ivrendi, 2016; Hill & Bundy, 2014; Sandseter, 2007; Sandseter & Kennair, 2011), items on the FAC–TRiPS formed a logical hierarchy. The order of items along the logit measure and pattern of difficulty aligned with the theoretical understanding of the complexity of risk, risk taking, and risky play (Creighton et al., 2017; Little, 2010; Niehues et al., 2013, 2015, 2016). Factors can vary in their impact and interact in complex ways, creating a spectrum of influences, and these factors (items) together form a construct (Tavakol & Wetzel, 2020). In the context of risk tolerance in play, various factors may interplay and contribute to an adult’s overall risk tolerance (construct), making it more of a continuous and nuanced concept. Specifically, the 17 items were presented from easiest (fear of injury) to most difficult to endorse (external and cultural factors). Prior studies indicated that fear of injury is one of the commonly reported factors that prevents parents from promoting risky play (Brussoni & Olsen, 2013; Little et al., 2011; Wyver, Tranter, et al., 2010), which could explain why most parents in our sample agreed with the item. On the other hand, qualitative research on risky play underlined that parents’ risk decisions are multifaceted, complex, and shaped by different social and cultural processes (Creighton et al., 2017; Little, 2010; Niehues et al., 2013, 2015, 2016).

The person mean closely matched the item mean, indicating effective person–item targeting. The Wright map provided additional evidence of how effectively the 17 items are distributed to capture a range of a person’s “ability.” Additionally, there were no items to measure the most risk-tolerant participants. This may not be problematic, because the FAC–TRiPS appeared to distinguish adequately between factors that most and least influenced parents’ risk appraisal and tolerance. Having the ability to distinguish less tolerant parents from those with a high to moderate degree of risk tolerance will aid in accurately determining which parents will most benefit from interventions that target factors that affect their decisions to increase opportunities for risk taking in play. Additionally, several items provided redundant information about the construct (Figure 1; Bond et al., 2021). Although some of the items assessed similar levels of risk tolerance, they reflected a different set of factors. For example, the items “fear that the play or play situation is too difficult for my child [factor related to environment or context]” and “feelings of guilt that something unexpected/bad could happen to my child or other children [factor related to internal fears or feelings]” captured different types of influences; therefore, we decided to retain them.

Although all of the FAC–TRiPS items showed positive point-measure correlations, 2 of the 17 (Items 1 and 2) did not meet the criteria for fit in the final iteration. After examination of item content, we retained the two items, because they are essential to the construct. The two items capture different levels of influence (e.g., the influence of others, including friends, other parents, or authorities) on how parents make decisions about promoting risk taking in play. Previous researchers highlighted the undesirable impact of framing risk negatively (i.e., as danger) as well as the societal judgment placed on parents’ acceptance of risky play (Wyver, Bundy, et al., 2010; Wyver, Tranter, et al., 2010). Thus, we believe that the parents in our sample who did find this factor to be an issue might not be conscious of the influence of others on their decision making or might find it hard to acknowledge that influence. We hope that, by taking the FAC–TRiPS, parents will reflect on their beliefs, and the consequences of those beliefs, and that this will lead them to a better understanding of themselves and their children. This reflective process may encourage the exploration of different approaches to manage contradicting emotions and concerns about risk taking, ultimately supporting children’s participation in risky play.

Additionally, we examined DIF as a potential way of understanding item failure to fit. We found that the failure of two of the items to fit is likely to be explained by differences between mothers and fathers. Women found it easier than men to identify “fear of negative judgment from others” and “fear of repercussions from an authority (e.g., police)” as an influence on their decisions about whether to encourage risky play. In contrast, male parents found that the “influence of media reports of children getting hurt/injured/kidnapped” had a bigger influence on their decisions and actions toward risk taking. Because the questionnaire is in its early development and because of the small sample size, we retained these items, but we acknowledge that it is important to consider these gender differences and monitor them when using the FAC–TRiPS and interpreting findings in future studies. Finding gender differences in our sample may not be surprising, given that researchers have previously documented variations between mothers’ and fathers’ views of and orientations toward children’s engagement in risky play (Brussoni, Creighton, et al., 2013; Brussoni & Olsen, 2013; Creighton et al., 2017). However, we recommend that, if DIF persists in future investigations, different scoring should be used for fathers and mothers when administering the questionnaire.

In Rasch modeling, the PCA is useful for examining the strength of additional dimensions in the data (Linacre, 2018). Our results suggest that the FAC–TRiPS may be multidimensional. Nonetheless, it is essential to recognize that multidimensionality is always present in data (Linacre, 1998). What is of paramount importance is the strength of a second dimension. Linacre (2018) indicated that, for a second dimension to be noticeable, it must be two to three items strong; otherwise, its strength is likely to be due to accidental correlations (Linacre, 2018). Rasch analysis showed that the strength of the second dimension was 2.03, which is far lower than that explained by the Rasch dimension (eigenvalue = 16.7). Both Rasch modeling and EFA revealed that the first dimension is robust and dominant. In summary, although these results suggest a possible secondary dimension that needs to be monitored and examined in future research, the strength of that dimension seems relatively minimal.

We acknowledge that our sample was entirely from Saudi Arabia, which may limit the generalizability to other ethnic groups. Similarly, our participants were relatively highly educated and spoke English as well as their native language(s). Their opinions might not represent those of parents with different educational or language backgrounds. Translating the questionnaire to Arabic would provide additional reach and access to a bigger sample size. We also recommend testing the psychometric properties of the FAC–TRiPS with a more diverse sample.

The FAC–TRiPS yields valid and reliable data measuring factors that influence parents’ tolerance of risk in play. The FAC–TRiPS is short, is easy to use, and takes 4 to 5 min to complete. With further research on the FAC–TRiPS, the findings have the following implications for occupational therapy practice:

• ▪ The FAC–TRiPS, in conjunction with other measures such as the TRiPS, can be a useful instrument for practitioners who work with parents, caregivers and their children in promoting risky outdoor play.

• ▪ The FAC–TRiPS may be used in designing interventions to reframe parents’ and teachers’ perceptions of the benefits of risky play and to create more opportunities for risk taking in the home, community, and school environments.

• ▪ The study contributes knowledge of risky play within an Eastern context and fosters an understanding of the multifaceted nature of parenting when making decisions about children’s participation in risky-play activities.

• ▪ The study highlights the need to create a culture in which the duty of care is balanced with the dignity of risk to enable children of all abilities to participate in playful and age-appropriate risk-taking opportunities.

• ▪ Future studies may aim to determine the feasibility and usability of the FAC–TRiPS among parents of typically and atypically developing children as well as in other cultures.

Our findings offer preliminary evidence for construct validity and internal reliability of data collected with the FAC–TRiPS. Thus, the FAC–TRiPS can be used to assess and understand contextual factors that affect parents’ tolerance for risk in play. Overall, this tool can be a useful foundation for understanding the impact of context on parents’ behaviors and abilities to cope with risk. Our findings also may contribute to expanding the focus of risky-play research toward a deeper look into the societal and cultural influences on parental perspectives of risk taking in play. Given that the instrument in its early stages of development, more research with a larger sample is needed to refine items and enhance the instrument’s functionality and precision.

This research was completed in partial fulfillment of the requirements for the degree of doctor of philosophy in occupation and rehabilitation science at Colorado State University. Rana Alarawi expresses gratitude for King Saudi bin Abdulaziz University for Health Sciences, Jeddah, Saudi Arabia, for awarding her a scholarship that supported her doctoral degree. Without their support, this work would not have been accomplished. Last, we thank the study participants for their time and valuable contributions to this research.