medigraphic.com
ISSN 2007-5057 (Print)
Investigación en Educación Médica

2018, Number 27

<< Back Next >>

Inv Ed Med 2018; 7 (27)

Validation of a Spanish questionnaire implementing the Stanford Educational Framework for Evaluation of Clinical Teachers

Bitran M, Torres-Sahli M, Padilla O

Language: English
References: 42
Page: 14-24
PDF size: 669.31 Kb.


ABSTRACT

Introduction: Although there are instruments in Spanish to evaluate teacher performance during the initial basic science training years or during medical specialization; there are few instruments for the clinical training years, in which the main role of the teacher is to facilitate experiential learning. The MEDUC30 questionnaire is a Spanish instrument developed by the Pontificia Universidad Católica School of Medicine. It was built using the Stanford Faculty Development Program (SFDP) educational framework for evaluation of clinical teachers’ effectiveness by students. MEDUC30 has been used since 2004 at Pontificia Universidad Católica de Chile and was previously studied with exploratory methods.
Objective: To provide satisfying evidence of validity and reliability to support MEDUC30’s usefulness in Spanish- speaking contexts, using confirmatory analytical methods.
Method: This is an analytical, longitudinal and retrospective study, in which 24,681 MEDUC30 questionnaires evaluating 579 clinical teachers were analysed. They were completed by medical students from 3rd to 7th year of study, from 2004 throughout 2015. The questionnaire’s structure was studied by exploratory (EFA) and confirmatory factor analysis (CFA). Measurement invariance was evaluated with multi-group CFA.
Results: Four different models were compared; a bi-factor model was the best alternative to explain the da ta’s structure. It was composed of one general and six domain-specific factors: [I] Patient-Based Teaching, [II] Communication of Goals, [III] Evaluation and Feedback, [IV] Promotion of Understanding, Retention, and Self-directed Learning, [V] Control of the Session, and [VI] Learning Climate. The overall reliability of MEDUC30 scores was excellent (Cronbach’s α = .98, McDonald’s ω = .98) and that of the six specific factors was very good (Cronbach’s α =.88-.95, McDonald’s ω = .78-.94). Measurement invariance extended over teacher gender, date, semester, year of study, clinical teaching setting, and length of clinical rotation; all of these variables were sources of population heterogeneity.
Conclusions: MEDUC30 is a valid and reliable Spanish instrument to evaluate clinical teachers. It can be used to provide formative feedback to clinical teachers and to provide accurate information to department heads and program directors for resource allocation and promotion purposes.


REFERENCES

  1. Snell L, Tallett S, Haist S, Hays R, Norcini J, Prince K, et al. A review of the evaluation of clinical teaching: new perspectives and challenges. Med Educ [Internet]. 2000 Oct;34(10):862–70. Available from: http://www.ncbi.nlm.nih. gov/pubmed/11012937

  2. Fluit C, Bolhuis S, Grol R, Laan R, Wensing M. Assessing the quality of clinical teachers: a systematic review of content and quality of questionnaires for assessing clinical teachers. J Gen Intern Med [Internet]. 2010 Dec;25(12):1337–45. Available from: http://dx.doi.org/10.1007/s11606-010-1458-y

  3. Litzelman DK, Stratos GA, Marriott DJ, Skeff KM. Factorial validation of a widely disseminated educational framework for evaluating clinical teachers. Acad Med [Internet]. 1998 Jun;73(6):688-95. Available from: http://content.wkhealth. com/linkback/openurl?sid=WKPTLP:landingpage& an=00001888-199806000-00016

  4. Bitran M, Mena B, Riquelme A, Padilla O, Sánchez I, Moreno R. An instrument in Spanish to evaluate the performance of clinical teachers by students. Revista Médica de Chile [Internet]. 2010;138(6):685–93. Available from: http://www.scielo.cl/scielo. php?script=sci_arttext&pid=S0034-988720100006000045.

  5. Valle R, Alaminos I, Contreras E, Salas L, Tomasini P, Varela M. Student Questionnaire to evaluate basic medical science teaching (METEBQ-B). Rev Med IMSS 2004;42(5):405-411. Available from: http://www.facmed.unam.mx/sem/pdf/ articulosrocio/StudentQuestionnaire.pdf

  6. Mazón J, Martínez J, Martínez A. La evaluación de la función docente mediante la opinión del estudiante. Un nuevo instrumento para nuevas dimensiones: COED. RESU. 2009;38:113-140.

  7. Flores F, Gatica F, Sánchez-Mandiola M, Martínez A. Evolución de la evaluación del desempeño docente en la Facultad de Medicina; evidencia de validez y confiabilidad. 2017 Inv Ed Med 6(22)96-103.

  8. Martínez A, Lifshitz A, Ponce R, Aguilar V. Evaluación del desempeño docente en cursos de especialización médica; validación de un cuestionario. 2008 Rev Med IMSS;46:375- 382.

  9. Pizarro M, Solis N , Rojas V, Diaz L, Padilla O, Letelier L, Aizman A, Sarfatis A, Olivos T, Soza A, Delfino A, Latorre G, Ivanovic D , Hoyl T, Bitran M, Arab J, Riquelme A. Development of MEDUC-PG14 survey to assess postgraduate teaching in medical specialties. 2015 Revista Medica De Chile;143(8):1005-1014.

  10. Huete A, Julio R, Rojas V, Herrera C, Padilla O, Solís N, Pizarro M, Etcheberry L, Sarfatis A, Pérez G, Delfino A, Muñoz E, Rivera H, Bitrán M, Riquelme A. Development and validation of the MEDUC-RX32 questionnaire, to evaluate teachers of postgraduate radiology programs. 2014 Revista Chilena De Radiología;20(2):75-80.

  11. Rubin DB. Multiple Imputation after 18+ Years. J Am Stat Assoc [Internet]. 1996;91(434):473–89. Available from: http://www. tandfonline.com/doi/abs/10.1080/01621459.1996.10476908

  12. Rhemtulla M, Brosseau-Liard PÉ, Savalei V. When can categorical variables be treated as continuous? A comparison of robust continuous and categorical SEM estimation methods under suboptimal conditions. Psychol Methods [Internet]. 2012 Sep;17(3):354–73. Available from: http:// dx.doi.org/10.1037/a0029315

  13. Rubin DB. Multiple Imputation after 18+ Years. J Am Stat Assoc [Internet]. 1996;91(434):473–89. Available from: http://www. tandfonline.com/doi/abs/10.1080/01621459.1996.10476908

  14. Bentler PM. Comparative fit indexes in structural models. Psychol Bull [Internet]. 1990 Mar;107(2):238–46. Available from: http://www.ncbi.nlm.nih.gov/pubmed/2320703

  15. Browne MW, Cudeck R. Alternative Ways of Assessing Model Fit. Sociol Methods Res [Internet]. 1992 Nov 1;21(2):230–58. Available from: http://smr.sagepub.com/content/21/2/230. abstract

  16. Hu L, Bentler PM. Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives. Struct Equ Modeling [Internet]. 1999;6(1):1–55. Available from: http://dx.doi.org/10.1080/10705519909540118

  17. Yu C-Y. Evaluating cutoff criteria of model fit indices for latent variable models with binary and continuous outcomes [Inter- net]. University of California Los Angeles; 2002. Available from: http://ww.statmodel2.com/download/Yudissertation.pdf

  18. Brown TA. Confirmatory factor analysis for applied research. Second. New York: The Guilford Press; 2015.

  19. Muthén LK, Muthén BO. Mplus User’s Guide. Seventh. Los Angeles, CA; 2012.

  20. McDonald RP. Test Theory: A Unified Treatment [Internet]. Psychology Press; 2013. 498 p. Available from: http://books.google. cl/books/about/Test_Theory.html?hl=&id=2-V5tOsa_DoC

  21. Zinbarg RE, Revelle W, Yovel I, Li W. Cronbach’s α, Revelle’s β, and McDonald’s ω H: their relations with each other and two alternative conceptualizations of reliability. Psychometrika [Internet]. 2005 Apr 2 [cited 2016 Apr 27];70(1):123–33. Available from: http://link.springer.com/article/10.1007/ s11336-003-0974-7

  22. Satorra A. Scaled and Adjusted Restricted Tests in Multi-Sample Analysis of Moment Structures. In: Heijmans RDH, Pollock DSG, Satorra A, editors. Innovations in Multivariate Statistical Analysis [Internet]. Boston, MA: Springer US; 2000 [cited 2016 May 22]. p. 233–47. (Advanced Studies in Theoretical and Applied Econometrics; vol. 36). Available from: http:// link.springer.com/chapter/10.1007/978-1-4615-4603-0_17

  23. R Core Team. R: A language and environment for statistical computing [Internet]. Vienna, Austria.: R Foundation for Statistical Computing; 2016. Available from: https:// www.r-project.org/

  24. van Buuren S, Groothuis-Oudshoorn K. mice: Multivariate Imputation by Chained Equations in R. J Stat Softw [Internet]. 2011;45(3):1–67. Available from: http://www.jstatsoft. org/v45/i03/

  25. Revelle W. psych: Procedures for Psychological, Psychometric, and Personality Research [Internet]. Evanston, Illinois: Northwestern University; 2016. Available from: http:// CRAN.R-project.org/package=psych

  26. Rosseel Y. lavaan: An R Package for Structural Equation Modeling. J Stat Softw [Internet]. 2012;48(1):1–36. Available from: https://www.jstatsoft.org/index.php/jss/article/ view/v048i02

  27. Venables WN, Ripley BD. Modern Applied Statistics with S [Internet]. Fourth. New York: Springer; 2002. 495 p. (Statistics and Computing ). Available from: http://books.google. cl/books/about/Modern_Applied_Statistics_with_S. html?hl=&id=GeX9CYd_JTkC

  28. Jöreskog KG. Censored variables and censored regression [Internet]. 2002 [cited 2016 May 10]. Available from: http:// www.ssicentral.com/lisrel/techdocs/censor.pdf

  29. Kline RB. Principles and practice of structural equation modeling. Fourth. New York: The Guilford Press; 2015.

  30. Kaiser HF, Rice J. Little Jiffy, Mark IV. Educ Psychol Meas [Internet]. 1974 [cited 2016 May 11]; Available from: http:// dx.doi.org/10.1177/001316447403400115

  31. Reise SP. Invited Paper: The Rediscovery of Bifactor Measurement Models. Multivariate Behav Res [Internet]. 2012 Sep 1;47(5):667–96. Available from: http://dx.doi.org/10.1080 /00273171.2012.715555

  32. Triviño X, Ximena T, Marisol S, Philippa M, Luz M. Impacto de un programa de formación en docencia en una escuela de medicina [Impact of a diploma on medical education in a medical school in Chile]. Revista médica de Chile [Internet]. 2011;139(11):1508–15. Available from: http://dx.doi. org/10.4067/s0034-98872011001100019

  33. Bitran M, Zúñiga D, Pedrals N, Padilla O, Mena B. Medical students’ change in learning styles during the course of the undergraduate program: from ‘thinking and watching’ to ‘thinking and doing’. 2012 Canadian Medical Education Journal 2012, 3(2)e86:e97

  34. Haws J, Rannelli L, Schaefer JP, Zarnke K, Coderre S, Ravani P, McLaughlin K. The attributes of an effective teacher differ between the classroom and the clinic setting. 2016. Adv in Health Sci Educ (2016) 21:833–840 DOI 10.1007/ s10459-016-9669-6

  35. Litzelman DK, Westmoreland GR, Skeff KM, Stratos GA. Factorial validation of an educational framework using residents' evaluations of clinician-educators. Acad Med [Internet]. 1999 Oct;74(10):S25–7. Available from: http://content. wkhealth.com/linkback/openurl?sid=WKPTLP:landingpage &an=00001888-199910000-00030

  36. Mintz M, Southern DA, Ghali WA, Ma IWY. Validation of the 25-Item Stanford Faculty Development Program Tool on Clinical Teaching Effectiveness. Teach Learn Med [Internet]. 2015;27(2):174-81. Available from: http://dx.doi.org/10.1080/ 10401334.2015.1011645

  37. Al Ansari, A., Strachan, K., Hashim, S., & Otoom, S. Analysis of psychometric properties of the modified SETQ tool in undergraduate medical education. 2017 BMC Medical Education. 17, 56. http://doi.org/10.1186/s12909-017-0893-4

  38. AMEE Guide No 20: The good teacher is more than a lecturer: the twelve roles of the teacher R.M. HARDEN & JOY CROSBY Centre for Medical Education, University of Dundee, UK. 2000 Medical Teacher 22(4):334-347

  39. Boor K, Teunissen PW, Scherpbier AJJA, van der Vleuten CPM, van de Lande J, Scheele F. Residents’ perceptions of the ideal clinical teacher—A qualitative study. Eur J Obstet Gynecol Reprod Biol [Internet]. 2008 Oct;140(2):152–7. Available from: http://linkinghub.elsevier.com/retrieve/pii/ S0301211508001395

  40. Paukert JL, Richards BF. How medical students and residents describe the roles and characteristics of their influential clinical teachers. Acad Med [Internet]. 2000 Aug;75(8):843-5. Available from: http://www.ncbi.nlm.nih.gov/pubmed/10965865

  41. Beckman TJ, Cook DA, Mandrekar JN. What is the validity evidence for assessments of clinical teaching? J Gen Intern Med [Internet]. 2005 Dec;20(12):1159–64. Available from: http://dx.doi.org/10.1111/j.1525-1497.2005.0258.x

  42. Downing SM. Reliability: on the reproducibility of assessment data. Med Educ [Internet]. 2004 Sep;38(9):1006–12. Available from: http://dx.doi.org/10.1111/j.1365-2929.2004.01932.x




2020     |     www.medigraphic.com