A pilot evaluation of a training programme on understanding and responding to sexual behaviours in children and young people

Participants and procedure

Thirty postgraduate psychology masters and/or doctoral students aged 20 to 45 years (M = 28.1SD = 5.8, 87% women) participated in the study. Participants were of Oceanian (53%), North-West European (20%), 10% South East Asian (10%) and Southern and Eastern European (6.7%) heritage. They had not previously been exposed to the TL framework One participant had however undertaken a two-day course on child protection. Most participants (86.7%) had previously worked with children and young people. See Table 1 for demographics.

Recruitment took place at a metropolitan university in Australia. Eligibility criteria included being currently enrolled in a postgraduate psychology programme and having an interest in or working with children and young people. This purposive sampling allowed us to test the efficacy of the TL framework among members its potential target population (i.e. professionals working with/interested in working with children and young people) both in terms of performance and programme engagement.

Prospective participants received an information sheet before training commenced stipulating that TL training completion was not contingent on research participation. On the training day, those wishing to participate in the study read the information sheet and gave consent to non-identifiable, aggregated reporting and evaluation of their performance data. Training was delivered over two three-hour blocks on one day, and participants completed 20-minute pre- and post-training surveys (see online supplemental material).

Ethics approval

Ethical approval was received from the University of Queensland Health and Behavioural Sciences, Low & Negligible Risk Ethics Sub-Committee (Approval number 2,020,002,047).

Consent to participate

Informed consent was obtained from all participants in the study.

Training programme

The TL programme is a one-day course using scenarios and case studies to train practitioners to Identify, Understand and Respond to both appropriate sexual behaviours among children and young people (e.g. learning about privacy and exploring sexual identity) and cases of sexual abuse and neglect (including risk factors and early signs). Other key topics addressed by the programme include prevention and prioritising safety; responses and barriers to information disclosure; the informed and timely reporting of child sexual endangerment; ethical decision-making; and the effects of vicarious trauma. Presentations, workbook activities, small group discussions and individual reflections are used to deliver the TL programme.

The framework aims to be simple but comprehensive. Practitioners must first learn to Identify the behaviour in line with its context. The next step Understand highlights that all behaviour fundamentally communicates meaning. Understand entails a description of the behaviour and empathetic acknowledgement of the reasons behind it. This is particularly important because many children and young people may lack the appropriate knowledge and linguistic repertoires necessary to effectively communicate their needs and emotions. For example, a 5-year-old child playing ‘doctor’ may indicate curiosity, while forcing other children to play sexual games may suggest experiences of sexual abuse. Using the information from this first step, practitioners continue to the third step: Respond. This involves formulating a plan depending on case severity (i.e. Green, Orange or Red situation), which effectively meets the short- and long-term needs of the young person. Green light behaviours are developmentally appropriate and provide the practitioner with an opportunity to educate and provide support for the children and young people (True Relationships and Reproductive Health Citation2022). Orange light behaviours are cause for concern and should be carefully monitored with provisional targeted support for the young person. Finally, Red light behaviours indicate harm and child endangerment, signalling the need for immediate protection and follow-up support. This easy-to-follow framework allows the practitioner to identify where the behaviour sits on the spectrum of green, orange, or red; understand the primary need the behaviour is communicating; and provide appropriate support and education to ensure that these needs are met.

Measures

The before and after surveys (see online supplemental material, Appendices A and B) were designed specifically for the study to test how participants would understand and respond to various children and young people sexual behaviours before and after undertaking the training programme. The survey measures included scenarios for child sexual behaviours (green light, orange light, and red light) in which participants were asked to list three ideas (‘things’) in response to the following questions: ‘What do you think is being communicated by this behaviour?’ (understand) and ‘How could you respond to meet the child or young person’s needs?’ (respond). The “list 3 things’‘approach helped assess participants’ application of TL knowledge by eliciting reasonably detailed responses. This form of assessment seeks to builds clinical competency by testing reasoning ability and the interpretation of theory and context to solve real-world problems (Thampy, Willert, and Ramani Citation2019). For example, one scenario reads as follows: ‘Maya, aged 8, is observed on more than one occasion not to be wearing underwear when at school. This is noticed by her teacher when they have “listening time” on the carpet’. Each of the scenarios was written by TL experts and together they were utilised to check participants’ understanding of the programme content. Participants were also asked to rate the confidence of their answers to the understand and respond questions on a nine-point Likert scale (1 = not at all to 9 = very confident).

Analysis

At a first step, raw scores were categorised by scenario (1–6). Next, participants’ responses were coded according to criteria developed by TL experts (see online supplemental material). For each scenario to which participants were asked to understand and respond, participant responses were graded against standard scores. Each criterion (e.g. ‘make sense of the world/gathering information/curiosity’ for understand, and ‘shame/embarrassed’ for respond) was assigned standard scores to assist in coding participants’ answers (i.e. 1 = present and correct/should be present; 0 = absent; −1 = present but incorrect/should not be present).

Coders were blind to the training condition and the total scores were averaged to identify mean scenario scores. These scores were used to run paired-sample t-tests using SPSS. A Power analysis was conducted using G power (Faul et al. Citation2007). A priori (two-tailed mean differences) power analysis recommended a sample size of 27 to test the expected medium effect size (Cohen’s d = 0.5) with a power of 0.8.

Interrater reliability

Cohen’s kappa is a robust statistical measure of interrater reliability that corrects for rater agreement due to chance. The kappa coefficient for understand in this study was 0.68. Hence, after accounting for agreement due to chance, there was substantial reliability between two raters for understand. Similarly, the kappa coefficient for response was 0.95. Therefore, after accounting for agreement due to chance, both understand and respond displayed substantial reliability between two raters.

Main findings

As can be seen in Table 2, overall, participants gave more correct responses post-training for Green and Orange scenarios but not Red. The following sections will evaluate results and discuss study outcomes.

Understand

Figure 1 shows the mean accuracy scores for all scenarios for understand according to the standard scoring of understand for each colour. For Green scenarios, people scored significantly better before training (‘Scenario 6’) (M = 1.33SD = 0.55) than after training (‘Scenario 4’) (M = 1.03SD = 0.62), t(29) = 2.07, p = .048. For Orange scenarios, mean accuracy scores before training (‘Scenario 3’) (M = 1.67SD = 0.88) and after training (‘Scenario 2’) (M = 1.93SD = 0.79) were not significantly different, t(29) = −1.31, p = .199. For Red scenarios, participants performed significantly better before training (‘Scenario 1’) (M = 1.60SD = 0.97) than after training (‘Scenario 5’) (M = 0.33SD = 0.88), t(29) = 4.54, p < .001.

To further explore effects, the understand criteria was split into adaptive (i.e. make sense of the world/gathering information/curiosity and age/developmentally appropriate) and maladaptive (i.e. abuse or emotional neglect, coercion/manipulation, lack of supervision, lack of role models). The average scores for adaptive criteria in post scenarios were significantly lower than pre scenarios for Green scenarios (mean difference [MD] = 0.22) and Red (MD = 0.15). There was no difference in scores for adaptive criteria for Orange scenarios. On the other hand, the average scores for maladaptive criterion in post scenarios were not significantly higher than pre scenarios for Green (MD = −0.03) and Orange scenarios (MD = −0.07). For Red scenarios, the pre scores for maladaptive criterion was non-significantly higher than post scores (MD = 0.24). All split effect results for understand had small-to-moderate effect sizes and were not meaningful.

Respond

Figure 2 shows the mean accuracy scores for respond according to the standard scoring of respond for each colour. For Green scenarios, there was no difference between participants’ pre (M = 2.10SD = 1.84) and post scores (M = 2.69SD = 1.39), t(28) = −1.86 p > .05. For Orange scenarios, participants’ difference in pre scores (M = 2.62SD = 1.35) and post scores (M = 2.52 SD = 1.02) was also insignificant t(28) = 0.46 p > .05. Finally for Red scores there was no difference between pre (M = 1.93SD = 2.08) and post (M = 1.93SD = 1.46) scores t(29) = 0 p > .05.

To explore split effects, respond criteria was split into those that were adaptive (i.e. that everything is okay/explains in a sex positive way/in a strengths-based way/wants more information/provides education/child focused), and those that were maladaptive (i.e. shame/embarrassed/like they have done something wrong/explains in a punitive way/responds only with consequences/does nothing). The average difference scores for adaptive and maladaptive criteria were not statistically significant for all types of scenarios.

Post training confidence

On average, participants felt well equipped (76%) to understand and respond to developmentally appropriate and inappropriate behaviours among children and young people. They reported being more informed about the signs of sexual abuse and the significance of behaviour and context. Many felt that the TL framework helped them be empathetic but objective in dealing with self-biases and personal emotions (e.g. ‘I feel more informed and understand to a deeper level age-appropriate sexual behaviours’). Some raised concerns regarding lack of clarity in the criteria (e.g. ‘I have an improved understanding of what behaviours are developmentally appropriate, however I am not confident my answers are correct as I was confused about the criteria for different behaviours/contexts’). Despite this, participants felt that the TL training was useful, easy-to-follow, well-structured, and offered a practical guide for practitioners seeking to navigate the complex waters of sexual behaviours in children and young people.

Post training criticism

Common criticisms reported by participants were that scheduling and time management could be improved; however, opinions varied. Some believed that the content could have been taught in half the time, whereas others thought there was not enough time to cover all the content. Participants also reported that a lot of the content was general knowledge and that they would have benefitted from a clearer focus on TL.

Areas of strength

Despite these disappointing findings, it is important to recognise that this study is the first of its kind. There is limited research examining training in children and young people sexual behaviours. Additionally, previous studies have not evaluated improved understanding and responses to sexual behaviour, instead they have only evaluated confidence. Professional education and training in children and young people sexual behaviours remains rare in both the UK and the Australian context, and there is inconsistent in terminology and approaches concerning children and young people sexual behaviours (King‐Hill Citation2021; O’Brien Citation2010; Wright, Swain, and McPhillips Citation2017). The TL programme aims to provide agencies with shared terminology and understanding to increase professionals’ accuracy and consist-ency in responding to children and young people sexual behaviours. In principle, this should help reduce subjective responses and the likelihood of children and young people experiencing shame regarding their sexual development. Because of this and the reported findings, our study is best viewed as a stepping stone towards empirically assessing the efficacy of TL training in equipping professionals with the tools to understand and respond to children and young people’s sexual behaviours, thereby adding to the knowledge base of child sexual education and protection.

Limitations

Like all research, this study has its limitations. Firstly, in terms of study design, the scenarios given to participants for the pre- and post- assessments were not counterbalanced. Counterbalancing is key to ensuring that pre- and post- assessments have the same level of difficulty. Additionally, the brevity of scenarios limited the specificity of participants’ responses. This included the limited number of scenarios per TL colour. Moreover, the lack of re-tests throughout the year limited our understanding of medium- and long-term effects of TL education and training in real-world scenarios. Furthermore, our study had a small sample size, and participants were predominantly female postgraduate psychology students with limited experience working with children (less than 5 years). This hinders the ecological validity of our study as they are not the target audience.

Due to the open-endedness of the questions in the pre- and post- assessments, some participants provided broad answers that were meaningful, but not amenable to capture by the criteria (e.g. Scenario 2 question ‘What do you think is being communicated by this behaviour’ was answered by a participant as ‘learning about privacy, learning about social rules and developing independence’). Scenario 2 was Orange and most of the participants answers did not fit the criteria/grading due to lack of contextual details within the scenario. For example, Participant 23 wrote, ‘attention seeking, discomfort with underwear, lack of knowledge about privacy/appropriate behaviour’. Some of these responses were difficult to grade against the criteria. As a result of this limitation, to evaluate participants’ responses effectively, the marking criteria need to both ensure that certain key factors are addressed (e.g. understanding of what is developmentally appropriate or inappropriate), and allow for variability between scenarios. Additionally, future study would benefit from qualitative analysis to provide clarity to the gaps in the quantitative analysis.

Enhancing the development of future traffic lights education and training

Throughout the literature, the length of training undertaken is considered a prominent barrier to applying evidence-based practice from training (Gibbert et al. Citation2013; Dreisinger et al. Citation2008). In this study, participants believed that they did not have enough time to process all the information received. Perhaps the training was too short to demonstrate an immediate effect on participants’ understanding of and response to sexual behaviours in children and young people. Although the framework and structure of the programme were comprehensive, increasing training time would allow practitioners to integrate what was learnt in the training with prior knowledge and practice – especially in relation to understanding and responding.

Future research should also consider practitioners’ engagement with content and find ways to ensure that the delivery of content promotes active learning. One way of doing this is to increase the trialability of the content by, for example, allowing practitioners the opportunity to trial and practice. This allows participants to adjust the practices according to their needs, increasing the compatibility of training, and thus increasing the likelihood that practitioners will implement the TL Framework. This could occur through three training sessions throughout the year whereby participants trial the strategies learned about in between training sessions and report back on the success and limitations of their implementation. Furthermore, a peer learning session could be beneficial for practitioners to discuss various situations throughout the year where they have applied TL (Anderson Citation2022). This would allow practitioners to collectively create solutions to the barriers faced and then evaluate their implementation in the next peer session. Additionally, implementing the training in intervals would allow practitioners to revise and look over the resources in their own time to consolidate their knowledge. Considering the varying demands of professionals, future studies could also investigate different formats of the training to allow greater accessibility to people working with children and young people, such as social welfare professionals carrying heavy workloads or busy teachers.

An advantage of The TL Programme is that it reduces subjectivity interferences. It does so by targeting participants’ beliefs as well as their self-efficacy towards implementing TL. The effective implementation of TL practices by practitioners is dependent on the compatibility of the programme with the beliefs and values of the practitioner (Greenhalgh et al. Citation2004; Rogers Citation2003). Considering our non-significant results, future training sessions should focus on measuring participant beliefs both quantitatively and qualitatively throughout the training. This will ensure that practitioner’s beliefs are effectively targeted.