Volume 24 Number 1

The importance of collecting pre- and post-intervention practice data when conducting a randomised controlled trial

Nelly Newall, Gill Lewin and Duncan Boldy

Keywords Pre–post intervention data, randomised controlled trial, skin tears, skin moisturising

PDF

Author(s)

References

Abstract

Both practice and outcomes data sets are integral to any randomised controlled trial (RCT) design and should be collected both before and after the intervention has been implemented to understand if a change has occurred as a result of the intervention. However, it is not unusual for only outcomes data to be collected. In this paper a mixed methods descriptive study, which was a discrete part of a moisturising RCT, is used to demonstrate the importance of collecting pre- and post-intervention practice data. This paper demonstrates how the descriptive study aimed to enhance the researchers’ understanding of the outcomes of the moisturising RCT by determining how practice changed within the facilities in each arm of the study. In summary, this study was designed to ensure the results from the moisturising RCT could be accurately interpreted and it was a critical, but independent component of the moisturising study.

Introduction

Goodman and Gilchrest1 define a randomised controlled trial (RCT) as “a full experimental test of a treatment or intervention that involves random allocation to treatment/intervention or control groups (or to treatment in different orders), ideally using methods that ‘blind’ the allocation to those involved in the study. RCTs usually involve a large and heterogeneous sample of participants, recruited from multiple, geographically scattered sites to ensure that the results are not specific to a single setting” (p. 251). Although often considered the “gold standard” of research methodologies to ascertain if an intervention is effective and safe or not2, the design and reporting of RCTs has not always been optimal3.

There are two main types of RCT: “explanatory” RCTs, which focus on internal validity, ensuring interventions work in a controlled environment; and “pragmatic” RCTs, which have more emphasis on a treatment working in the real clinical world4. It would seem more advantageous to study an intervention in a more realistic environment5 mainly because if it is applicable to the real world, it negates the need for further validation, thus reducing both costs and resources.

An important component of any RCT design is the randomisation of subjects and this can occur in a number of ways. As this study was conducted across many sites and involved multiple staff, a cluster randomisation design was chosen. Cluster randomisation is when groups of individuals are randomly allocated to the intervention or control arm of a study rather than individuals, thus the group or cluster is the unit of allocation6.

Cluster randomisation is designed to reduce contamination bias4 and in this particular study what this meant was that each of the 12 residential sites participating in the RCT was designated as either a treatment or control site7. The intervention consisted of a twice-daily application of a commercially available, standardised pH-neutral, perfume-free moisturiser on residents’ extremities, whereas the control group had an ad hoc or non-standardised skin-moisturising regimen. To ensure consistency in the skin-moisturising regime, entire sites were allocated to the intervention rather than individual residents.

The need to describe practice at baseline and follow-up

Although no literature was found around this topic, baseline data collection is essential to help understand if a change has occurred as a result of the intervention and therefore collecting this data (pre-intervention) as well as outcome data (post-intervention) is an integral part of any RCT design. Also one needs to know if the intervention had been delivered as intended, to ensure trial outcomes can be correctly attributed. In the case of the moisturising RCT, the outcome measure was the incidence in skin tears during the intervention period. If a difference was found in the incidence between the control and intervention groups but no change in moisturising practice occurred, it would not be possible to attribute any positive outcomes to the moisturising intervention. More specifically, if the practice of moisturising was already occurring, results could not be attributed to a change in practice. So, this study was designed to determine what “usual” skin moisturising practice was within the facilities in each arm of the RCT, both before and after the intervention, to enhance the researcher’s understanding of the outcomes of the study.

Skin tears

Payne and Martin8 describe a skin tear as “a traumatic wound occurring principally on the extremities of older adults as a result of friction alone or shearing and friction forces which separate the epidermis from the dermis (partial thickness wound) or which separates both the epidermis and the dermis from underlying structures (full thickness wounds)”8(p.20). Although predominantly found in older adults, skin tears can also develop in newborn infants due to physical characteristics associated with immature skin9 and at the end of life as a consequence of chronic illness or multiple co-morbidities, regardless of age10. However, it is generally agreed that because of the degenerative effects of ageing on the skin and prolonged contact to harmful external elements, skin tears are much more of a risk for older people, particularly those over 80 years of age11.

The prevalence of skin tears in residential aged care facilities in Australia ranges between 10% and 20%12,13 and although initially appearing as a small, simple wound, skin tears can develop into complex, chronic wounds14. Morbidity and mortality risks can increase if complications such as infection or compromised vascular status occur15. For all these reasons there is an emotional and physical cost to the individual and their family as well as a financial cost for the facility8,11,15-17. There is little evidence identifying cost-effective prevention or treatment strategies for skin tears17 but anecdotally moisturising is thought to be a simple method of assisting the skin to rehydrate and retain water, making it less likely to split18. To assess the effectiveness of moisturising as a preventative strategy, a pragmatic cluster RCT of twice-daily moisturising was undertaken among the residents of 12 residential aged care facilities in Western Australia7. This paper explores moisturising practices before and after the intervention.

Method

Study aim

To determine whether a skin moisturising intervention in a pragmatic cluster RCT in 12 residential aged care facilities was implemented according to the protocol and if there was evidence of contamination in the control sites.

Study design

A mixed methods descriptive study designed to identify “usual” moisturising practice pre- and post-intervention in both the control and intervention facilities.

Carers (non-nursing staff employed by the residential aged care facility) were surveyed and site managers were interviewed. The site managers also provided electronic documentary evidence of skin moisturising practice.

Hotopf remarked that “usual care is a difficult term to define because it will depend heavily on the knowledge, skills and resources of the health care professionals delivering it”5(p.329). The perspective of both the carers and site managers was, therefore, important to consider as carers were primarily responsible for implementing any skin care regime, whereas site managers were responsible for defining and managing the implementation of any skin moisturising protocol.

Study sample

The survey sample size was decided in consultation with the aged care organisation and with respect to time and budget constraints. It was expected that the sample would make up approximately 20% of the total number of carers employed at that time. A convenience sample of one manager and a minimum of five carers from each of the 12 residential facilities in the RCT — six intervention facilities and six control facilities — was selected. In the RCT7, 14 sites were described rather than 12. However, in this study the same site manager oversaw both areas, resulting in only 12 interviews at 12 sites.

Sample recruitment

Recruitment occurred during site visits pre-intervention from May to June 2011, the intervention was then introduced in October 2011 until March 2012 and subsequent post-intervention recruitment followed from March to April 2012. Site visits were planned in consultation with the aged care organisation and consideration was given to minimising disruption by avoiding visits during other planned events, such as internal audit or accreditation as well as not during busy times of day such as meal times or during residents’ personal care.

After introducing herself to the site manager, the researcher would wait in the staff tea room and approach carers individually and after confirming their roles, explain the requirements of the survey and give them the information sheet to read. Once they had been given time to ask questions about the study, they were asked to complete the anonymous questionnaire at that time. Completion and return of the questionnaire was taken to indicate the carer had consented to take part in the study.

Site manager interviews were arranged to coincide with the researcher’s scheduled site visits. Like the carers, site managers were given an information sheet to read and time to ask questions. Because the interview was being digitally recorded, a consent form was also signed before the interview commenced.

Data collection

Site visits were randomly assigned to either mornings or afternoons to ensure the sample recruited to complete the questionnaire included carers from both shifts. Pre- and post-intervention data collection was conducted by just one researcher who followed the same process each time. The researcher was blind as regarding the allocation of sites to intervention or control as this is considered best practice in an RCT1,5,19.

Carer survey

Once recruited, carers were asked to complete the questionnaire immediately and independently; and to only ask the researcher (that is, not the other carers) if any clarification was required. Carers were also asked to put the completed questionnaire in the collection box as soon as they had completed it, to maximise the response rate. The box being closed ensured that carers could not read other people’s responses and to encourage full disclosure.

Site manager interviews

Each interview was held in a suitable room (usually the site manager’s office) and consisted of semi-structured questions about current moisturising practice at their site. After the interview was completed, the site manager was asked for documentary evidence of the skin moisturising practice that they had described as happening within their institution, such as electronic skin care plans and treatment sheets dated and signed.

Carer survey tool

The questionnaire was based upon tools previously used in another wound study20, its purpose being to elicit data about: the carer, their training related to skin care and the “usual” skin moisturising practice at their site. Designed to be succinct, the questionnaire mainly comprised closed questions with predetermined response categories. Discussions with the clinical nurse consultant at the residential organisation during the development of the questionnaire ensured content validity. Minor refinements were made to the carer survey tool pre- and post-intervention, but none affected its essential content.

Site manager interview guide

The interview guide was designed by the authors and had previously been found to elicit detailed information (in an unpublished study) about whether pressure injury protocols had been adhered to in a residential setting. Open-ended questions were used to clarify and encourage further disclosure during the interview around site managers’ expectations of staff with regard to carers’ moisturising practice of residents’ skin, communication of these expectations, monitoring and management of moisturising practice, if any related training was occurring, and whether they believed these expectations were being met by the carers. Post-intervention there was no change to the interview guide other than asking the site managers if they had been previously interviewed.

Statistical analysis

The survey responses were data entered and analysed descriptively using the Statistical Package for Social Sciences (SPSS version 18). Pre- and post-intervention data were analysed separately and then compared for both intervention and control groups. As all data were categorical, only non-parametric statistics were used to assess the differences between groups. Statistical significance was assessed using a Chi-square or Fisher’s exact test, as appropriate, with statistical significance being determined at p<0.05.

The audio-taped interviews were transcribed by the researcher and open-ended responses summarised for each of the different aspects of practice. The data from the interviews and surveys were then considered together to construct a complete picture of “usual” skin moisturising practice.

Ethics approval

The research plan and associated documentation were submitted to, and approved by, the Human Research Ethics Committee (HREC) at the university that had auspiced the RCT. Additionally, the aged care organisation granted ethics and research approval to undertake the study across their 12 sites.

Results

The pre- and post-intervention carer survey results for the control and intervention groups were supplemented by findings from the site manager interviews and any electronic documentation observed during the site manager interview.

Informants (carers and site managers)

Eighty-five carers completed the survey pre-intervention: 44 from control facilities, and 41 from intervention facilities. This constituted 18% (n=85/474) of the total number of carers employed at that time. Post-intervention there were 104 respondents, 59 carers from control and 45 from intervention sites. This constituted 20% (n=104/515) of the total number of carers employed at that time.

Of the 104 carers that completed surveys post-intervention, 18% (n=19) had also completed the survey pre-intervention. By group, the number of carers completing the survey twice was very even, 10 control and nine intervention. All 12 site managers were interviewed both pre- and post-intervention, six each from the control and intervention sites. During the interviews it was ascertained that 67% (n=8/12) of site managers were interviewed pre-intervention as well as post-intervention. Of these, 37.5% (n=3/8) were control, and 62.5% (n=5/8) were intervention.

Pre-intervention

Frequency of moisturising

Carer survey responses to this question are provided in Table 1 and indicate there was no “usual” practice. The most common response in both intervention and control sites was “once a day”, followed by “twice a day” and “when the skin looks dry”. The largest difference between the groups was in the proportions of carers who responded “when resident asks”, 16% (n=7) in control sites and 29% (n=12) in intervention sites, but overall the pattern between sites was not statistically significant.

Table 1: Moisturising frequency by group (multi-response answers)

newall table 1.jpg

Other: per shift x 1, on ADL sheet x 1, if resident allows x 1, stated on profile x 1, when needed x 1.
Note: Chi square test calculated combining values for “weekly” and “once a day“ and omitting “other”, did not indicate a statistically significant difference between groups (x2(2, n=99) = 2.13, P=0.35).

 

The majority of site managers, 75% (n=9), also indicated that there was no normative moisturising practice at their site. This was the case for both groups of facilities: 67% (4/6) of managers in control sites and 83% (n=5/6) in intervention sites. Three of the site managers did say that was “usual” practice in their site to moisturise “twice a day”. The survey responses did not, however, support this as only three of the 12 control carers and five of the eight intervention carers responded this way.

Brand of moisturiser used

Survey responses from carers also showed no “usual” moisturising practice across sites in relation to which moisturiser was used. Of the 72 carers who responded, similar proportions in each group used Sorbolene, with the second most commonly identified moisturiser being Abena (Table 2). Site manager interview responses also indicated that various brands of moisturiser were being used.

Table 2: Brand of moisturiser used by group (free text response)

newall table 2.jpg

Note: Chi-square test calculated comparing Sorbolene with Abena did not indicate a statistically significant difference
(x2 (1, n=62) = 1.44, P=0.23).

 

Post-intervention

Frequency of moisturising

Differences between control and intervention sites in the frequency with which residents’ skin was moisturised were found post-intervention (Table 3). Overall, the carers’ most common response was that moisturising was occurring “twice a day”. The proportion of carers giving this response in the intervention group was substantially larger than in the control group and the reverse was true for the response “once a day”.

Table 3: Moisturising frequency by group (multi-response answers)

newall table 3.jpg

a Other: don’t know
Note: Chi-square test indicated a statistically significantly difference between groups when calculated omitting missing values and “other” (x2 (3, n=209) =15.36, p=0.00).

 

Site managers’ comments also indicated that moisturising practice now varied between sites, with 58% (n=7) saying that moisturising occurred “twice a day”, six who managed intervention sites and one a control site. Managers at two other control sites said their practice was to moisturise residents after an assessment of their skin had shown it is required.

The remaining three control site managers indicated that either there was no “usual” practice at their respective sites or they were not aware of it.

Table 4 shows that in control groups, the proportion of carer responses “once a day” 56% (n=33) was smaller post-intervention and that of “twice a day” 46% (n=27) was larger. The size of these changes was, however, not statistically significant.

Table 4: Moisturising frequency pre- and post-intervention in control group (multi-response answers)

newall table 4.jpg

a Other: per shift x 1, on ADL sheet x 1, if resident allows x 1, stated on profile x 1.
Note: Chi-square test calculated combining “weekly” and “once a day” and omitting missing values indicated no statistically significant difference between groups (x2 (3, n=175) = 6.71, P=0.08).

 

Among the intervention sites the changes in proportions of carer responses for all categories were larger than the changes for the control sites and these changes were significantly different (x2(3, n=159) =22.54, P=0.00). The proportion of carers responding “once a day” 18% (n=8) reduced more than twofold and the proportion answering “twice a day” 78% (n=35) more than doubled. These differences were reflected by the site manager responses.

Table 5: Moisturising frequency pre- and post-intervention in intervention group (multi-response answers)

newall table 5.jpg

a Other: when needed x 1 and don’t know x 1.
Note: Chi-square test calculated omitting missing values and “other”, indicated a statistically significant difference between groups (x2 (3, n=159) =22.54, P=0.00).

 

Brand of moisturiser used

Table 6 shows that the majority of carers said they used Abena, with the proportion being significantly larger in intervention sites 91% (n=41) than control sites 34% (n=20). Sorbolene was the second most commonly used moisturiser 34% (n=40) in control sites and 18% (n=8) in intervention sites. When comparing Sorbolene with Abena there was a statistically significant difference (x2 (1, n=109) =27.74, P=0.00).

Table 6: Brand of moisturiser used by group (free text response)

newall table 6.jpg

Note: Chi-square test when calculated comparing Sorbolene with Abena only indicated a statistically significant difference (x2 (1, n=109) =27.74, P=0.00).

 

Consistent with carer responses, all six group intervention site managers said Abena was the moisturiser most commonly used. All the control site managers, on the other hand, said residents’ preferences guided the brand of moisturiser used.

The remaining control site manager did not know which moisturiser was used.

Discussion

This study aimed to determine whether a skin moisturising intervention using an RCT in residential aged care facilities was implemented according to protocol. It also aimed to identify whether there was any contamination, that is, did skin care practice in the control site also change in any way to reflect the new regime in the intervention sites. The intervention regimen consisted of twice-daily moisturising of all participating residents at all six intervention sites with a standardised pH-neutral, perfume-free, moisturising lotion, Abena7.

To achieve these aims, “usual” skin moisturising practice was determined pre- and post-intervention and any differences in moisturising practice between these two time points identified.

The pre-intervention results indicated that there was no “usual” moisturising practice in the study sites prior to the RCT, while the post-intervention results showed the most common practice in the intervention sites was twice-daily moisturising with Abena cream. This indicated that the intervention had been implemented as per protocol and a change in practice had occurred. The results also showed that contamination had not occurred between the intervention and control groups because there were no marked changes in moisturising practice in the control sites. Importantly, this meant that the reduced incidence of skin tears of almost 50%, found in the intervention arm of the RCT7 could with reasonable confidence be attributed to twice-daily skin moisturising.

Although the “cluster” randomisation in the moisturising RCT was a strategy to reduce contamination, contamination was still possible4. It was, therefore, important to determine whether contamination had occurred.

As indicated by both manager and carers there was one intervention site where even before the intervention it was “usual” practice to moisturise “twice a day”. Should the RCT have compared pre- and post-intervention skin tear incidence, that one intervention site was already moisturising would have been an issue, as any reduction in skin tear incidence at that site could not have been attributed to the introduction of the intervention. However, as the RCT compared the incidence rates of the control and intervention groups during the intervention period the fact that one intervention site was following the intervention protocol before the trial was not an issue.

It is the reality of “pragmatic” RCTs that idiosyncrasies occur and as long as they are identified and accounted for results are not confounded4. Research conducted in a more realistic environment is preferable to studies conducted in an artificial context as the latter fail to identify if the treatment under review can be applied in real clinical practice5. In the present study one site already moisturising “twice a day” at baseline, whilst not ideal, was not an issue as described above, but if the study design had been different this awareness would have allowed the researchers to make methodological adjustments.

Study limitations

Using a questionnaire and interviews rather than direct observation could have been considered a limitation as both carers and managers may have reported what they thought they should be doing rather than what was actually being done. To reduce the likelihood of this, documentation was also consulted.

Further research and implications for practice

Some carers and family members reported that they felt applying moisturiser to residents’ skin was an emotional interaction as “touch” appeared to comfort and support residents who they could not always communicate with in other ways. Further research is needed to investigate whether moisturising has psychosocial benefits in addition to its effectiveness in skin tear prevention.

Conclusion

Pragmatic RCTs are considered the “gold standard” for evaluating the effectiveness of new treatments. Within these trials, to ensure results are attributed correctly, it is essential to define baseline or “usual” practice in order to be able to identify whether practice has changed, in the way intended, during the trial.

The aim of the study was to describe “usual” moisturising practice for skin tear prevention both before and after a moisturising RCT had been implemented across multiple sites of a residential aged care organisation in Western Australia. This study, therefore, formed a critical but independent part of the moisturising study.

Little evidence of there being “usual” skin moisturising practice was found pre-intervention in either intervention or control sites. Post-intervention, practice remained ad hoc in the control sites, whereas a significant change in practice, which matched the intervention protocol, had occurred in the intervention sites.

Thus, the intervention could be seen to have been successfully implemented and contamination of the control sites had not occurred. This finding is important as it provides some certainty that the lower skin tear incidence found in the intervention group in the RCT of almost 50%7 could be directly attributed to implementation of the moisturising practice protocol.

Conflicts of interest

None of the authors had any conflicts of interest in relation to this study.

Acknowledgements

Residents and staff of the residential aged care facilities, especially Jessica Dwyer and David Lyle.

Funding

The authors would like to acknowledge the support of the Australian Government’s Cooperative Research Centres Program and the Wound Management Innovation Co-operative Research Centre (WMICRC) as well as in-kind contribution from Silver Chain and Curtin University, Western Australia.

 

Author(s)

Nelly Newall*
RN, MPhil(Nsg)
Clinical Research Coordinator, Silver Chain Group, Adjunct Research Associate, School of Nursing, Midwifery and Paramedicine, Curtin University,
WA, Australia; and Wound Management
Innovation CRC
Email: nelly.newall@gmail.com
Tel: +61 439 976 296

Gill Lewin
MPH, PhD
Director of Research, Silver Chain Group and Professor of Ageing, School of Nursing, Midwifery and Paramedicine, Curtin University, WA, Australia

Duncan Boldy
BSc(Hons), MSc, PhD
Research Advisor, School of Nursing, Midwifery and Paramedicine, Curtin University, WA, Australia

*Corresponding author

References

  1. Goodman M, Gilchrist M. Reading and critiquing randomised controlled trials. Br J Card Nurs 2013;8(5):251–6.
  2. Sackett D, Rosenberg W, Gray J, Haynes R, Richardson W. Evidence-based medicine: What it is and what it isn’t. BMJ 1996;312:71–2.
  3. Weller C, McNeil J, Evans S, Reid C. Improving venous ulcer healing: Designing and reporting randomised controlled trials. Int Wound J 2010;7(1):41–7.
  4. Godwin M, Ruhland L, Casson I, MacDonald S, Delva D, Birtwhistle R et al. Pragmatic controlled clinical trials in primary care: The struggle between external and internal validity. BMC Med Res Methodol 2003;3:1–7.
  5. Hotopf M. The pragmatic randomised controlled trial. Adv in Psych Treat 2002;8:326–33.
  6. McKenzie J, Ryan R, Di Tanna G. Cochrane Consumers and Communication Review Group: cluster randomised controlled trials; 2014. Accessed 2 September 2015. Available from: http://cccrg.cochrane.org.
  7. Carville K, Leslie G, Osseiran-Moisson R, Newall N, Lewin G. The effectiveness of a twice-daily skin-moisturising regimen for reducing the incidence of skin tears. Int Wound J 2014;11:446–53.
  8. Payne R, Martin M. Defining and classifying skin tears: Need for a common language. A critique and revision of the Payne-Martin classification system for skin tears. Ostomy Wound Manag 1993;39(5):16–26.
  9. Bianchi J. Preventing, assessing and managing skin tears. Nurs Times 2012;108(13):12–6.
  10. Sibbald G, Krasner D, Lutz J. SCALE: Skin changes at life’s end. Final consensus statement: 1 October 2009. Wound Care J 2010;23:225–38.
  11. Holmes R, Davidson M, Thompson B, Kelechi T. Skin tears: Care and management of the older adult at home. Home Health Nurse 2013;31(2):90–101.
  12. Edwards H, Chang A, Finlayson K. Creating champions for skin integrity: Final report. Brisbane, Queensland: Queensland University of Technology; 2010.
  13. WoundsWest. WoundsWest wound prevalence survey report. Perth, Western Australia: WoundsWest; 2010.
  14. LeBlanc K, Christensen D, Cook J, Culhane B, Gutierrez O. Prevalence of skin tears in a long-term care facility. J Wound Ostomy Cont Nurs 2013;40(6):1–5.
  15. Stephen-Haynes J, Carville K. Skin tears made easy. Wounds International [Internet]. 2011 3 October 2013; 2(4):4. Available from: http://www.woundsinternational.com.
  16. Carville K, Lewin G, Newall N, Haslehurst P, Michael R, Santamaria N et al. STAR: A consensus for skin tear classification. Primary Intention 2007;15(1):18–28.
  17. LeBlanc K, Baranoski S. Skin tears: State of the science. Consensus statements for the prevention, prediction, assessment, and treatment of skin tears. Adv Skin Wound Care 2011;24(9):2–15.
  18. Ratcliffe C, Fletcher K. Skin tears: A review of the evidence to support prevention and treatment. Ostomy Wound Manag 2007;53(3):32–42.
  19. Moher D, Schulz K, Altman D. The CONSORT statement: Revised recommendations for improving the quality of reports of parallel-group randomized trials. Lancet 2001;357:1191–4.
  20. Lewin G, Carville K, Newall N, Phillipson M, Smith J, Prentice J. Determining the effectiveness of implementing the AWMA guidelines for the prediction and prevention of pressure ulcers in Silver Chain, a large home care agency. Stage 1: Baseline measurement. Primary Intention 2003;11(2):57–72.