Development, validity and reliability testing of the East Midlands Evaluation Tool (EMET) for measuring impacts on trainees’ confidence and competence following end of life care training

Journal article


Whittaker, Becky, Parry, Ruth, Bird, Lydia, Watson, Sharan and Faull, Christina 2017. Development, validity and reliability testing of the East Midlands Evaluation Tool (EMET) for measuring impacts on trainees’ confidence and competence following end of life care training. BMJ Supportive & Palliative Care. https://doi.org/10.1136/bmjspcare-2016-001100
AuthorsWhittaker, Becky, Parry, Ruth, Bird, Lydia, Watson, Sharan and Faull, Christina
Abstract

Abstract Objectives To develop, test and validate a versatile questionnaire, the East Midlands Evaluation Tool (EMET), for measuring effects of end of life care training events on trainees’ self-reported confidence and competence. Methods A paper-based questionnaire was designed on the basis of the English Department of Health's core competences for end of life care, with sections for completion pretraining, immediately post-training and also for longer term follow-up. Preliminary versions were field tested at 55 training events delivered by 13 organisations to 1793 trainees working in diverse health and social care backgrounds. Iterative rounds of development aimed to maximise relevance to events and trainees. Internal consistency was assessed by calculating interitem correlations on questionnaire responses during field testing. Content validity was assessed via qualitative content analysis of (1) responses to questionnaires completed by field tester trainers and (2) field notes from a workshop with a separate cohort of experienced trainers. Test–retest reliability was assessed via repeat administration to a cohort of student nurses. Results The EMET comprises 27 items with Likert-scaled responses supplemented with questions seeking free-text responses. It measures changes in self-assessed confidence and competence on 5 subscales: communication skills; assessment and care planning; symptom management; advance care planning; overarching values and knowledge. Test–retest reliability was found to be good, as was internal consistency: the questions successfully assess different aspects of the same underlying concept. Conclusions The EMET provides a time-efficient, reliable and flexible means of evaluating effects of training on self-reported confidence and competence in the key elements of end of life care

Abstract


Objectives To develop, test and validate a versatile questionnaire, the East Midlands Evaluation Tool (EMET), for measuring effects of end of life care training events on trainees’ self-reported confidence and competence.


Methods A paper-based questionnaire was designed on the basis of the English Department of Health's core competences for end of life care, with sections for completion pretraining, immediately post-training and also for longer term follow-up. Preliminary versions were field tested at 55 training events delivered by 13 organisations to 1793 trainees working in diverse health and social care backgrounds. Iterative rounds of development aimed to maximise relevance to events and trainees. Internal consistency was assessed by calculating interitem correlations on questionnaire responses during field testing. Content validity was assessed via qualitative content analysis of (1) responses to questionnaires completed by field tester trainers and (2) field notes from a workshop with a separate cohort of experienced trainers. Test–retest reliability was assessed via repeat administration to a cohort of student nurses.


Results The EMET comprises 27 items with Likert-scaled responses supplemented with questions seeking free-text responses. It measures changes in self-assessed confidence and competence on 5 subscales: communication skills; assessment and care planning; symptom management; advance care planning; overarching values and knowledge. Test–retest reliability was found to be good, as was internal consistency: the questions successfully assess different aspects of the same underlying concept.


Conclusions The EMET provides a time-efficient, reliable and flexible means of evaluating effects of training on self-reported confidence and competence in the key elements of end of life care

KeywordsEnd of life care training; Competence; Evaluation; Questionnaire; Self-assessment
Year2017
JournalBMJ Supportive & Palliative Care
PublisherBMJ Publishing Group Ltd.
ISSN2045435X
20454368
Digital Object Identifier (DOI)https://doi.org/10.1136/bmjspcare-2016-001100
Web address (URL)http://hdl.handle.net/10545/621396
hdl:10545/621396
Publication dates02 Feb 2017
Publication process dates
Deposited16 Feb 2017, 13:03
Accepted13 Jan 2017
Rights

Archived with thanks to BMJ Supportive & Palliative Care

ContributorsUniversity of Derby and University of Nottingham
File
File Access Level
Open
File
File Access Level
Open
Permalink -

https://repository.derby.ac.uk/item/94877/development-validity-and-reliability-testing-of-the-east-midlands-evaluation-tool-emet-for-measuring-impacts-on-trainees-confidence-and-competence-following-end-of-life-care-training

Download files

  • 10
    total views
  • 2
    total downloads
  • 1
    views this month
  • 1
    downloads this month

Export as

Related outputs

Real talk facilitator manual: Engaging patients with end of life talk
Parry, Ruth, Whittaker, Becky, Pino, Marco, Land, Vicky, Jenkins, Laura, Faull, Victoria, Feathers, Luke and Watson, Sharan 2020. Real talk facilitator manual: Engaging patients with end of life talk.
Implementing real talk: interprofessional education intervention enabling clinicians to develop confidence in open and honest conversations about dying
Watson, Sharan and Whittaker, Becky 2020. Implementing real talk: interprofessional education intervention enabling clinicians to develop confidence in open and honest conversations about dying. BMJ Supportive & Palliative Care. https://doi.org/10.1136/spcare-2020-PCC.66
Why it’s important healthcare professionals talk about dying
Watson, Sharan 2020. Why it’s important healthcare professionals talk about dying. University of Derby.
Testing real talk: an adaptable evidence-based communication skills intervention in end of life talk
Watson, Sharan, Whittaker, Becky and Parry, Ruth 2019. Testing real talk: an adaptable evidence-based communication skills intervention in end of life talk. BMJ Supportive & Palliative Care. https://doi.org/10.1136/bmjspcare-2019-HUKNC.34
20 Real talk - beyond advanced communication skills: outcomes of a residential workshop for palliative care doctors
Whittaker, Becky and Watson, Sharan 2019. 20 Real talk - beyond advanced communication skills: outcomes of a residential workshop for palliative care doctors. BMJ. https://doi.org/10.1136/bmjspcare-2019-ASP.43
Real talk facilitator manual: Engaging patients with end of life talk
Parry, Ruth, Whittaker, Becky, Pino, Marco, Land, Vicky, Faull, Christina, Feathers, Luke and Watson, Sharan 2019. Real talk facilitator manual: Engaging patients with end of life talk. Real Talk.
Exploration of volunteers and support workers initiation of quality of life conversations in hospice palliative day care
Watson, Sharan and Hembrow, Alison 2019. Exploration of volunteers and support workers initiation of quality of life conversations in hospice palliative day care. BMJ. https://doi.org/10.1136/bmjspcare-2019-ASP.152
Dying from liver disease: the importance of end-of-life discussions
Watson, Sharan and Mortimore, Gerri 2018. Dying from liver disease: the importance of end-of-life discussions. Gastrointestinal Nursing. https://doi.org/10.12968/gasn.2018.16.Sup10.S37
Exploration of support workers and volunteers initiation of advance care planning and quality of life conversations in palliative care.
Watson, Sharan and Hembrow, Alison 2018. Exploration of support workers and volunteers initiation of advance care planning and quality of life conversations in palliative care.
Dying matters - What can we do in our communities?
Watson, Sharan, Virk, Navjot Kaur, Brock, Martin, Dickens, Adam, Swanwick, Maelie and Home, Dave 2018. Dying matters - What can we do in our communities?
Death cafés serve up a welcome taste of reality.
Watson, Sharan 2017. Death cafés serve up a welcome taste of reality. Final Choices Publishing Ltd..
Dying matters - Why we need to talk about death.
Watson, Sharan 2018. Dying matters - Why we need to talk about death. University of Derby.
21 Real talk – a novel evidence-based, video-based communication skills training resource.
Parry, Ruth, Whittaker, Becky, Pino, Marco, Watson, Sharan, Hamlyn, Sarah and Faull, Christina 2018. 21 Real talk – a novel evidence-based, video-based communication skills training resource. BMJ Publishing Group Ltd.. https://doi.org/10.1136/bmjspcare-2018-ASPabstracts.21
"Penny's plan" - Dying matters what can we do?
Watson, Sharan, O'Reilly, Chris and Mortimore, Gerri 2018. "Penny's plan" - Dying matters what can we do?
Development, validity and reliability testing of the East Midlands Evaluation Tool (EMET) for measuring impacts on trainees’ confidence and competence following end of life care training
Whittaker, Becky, Parry, Ruth, Bird, Lydia, Watson, Sharan and Faull, Christina 2017. Development, validity and reliability testing of the East Midlands Evaluation Tool (EMET) for measuring impacts on trainees’ confidence and competence following end of life care training.