10.57647/jals.2025.0502.05

The Effects of Writing Prompt Properties on Pre-service Teachers` Essay Writing Performance based on Coh-metrix Analysis

PDF
  • Nader Afshounpour1,
  • Peyman Rajabi*1,
  • Hossein Ahmadi1,
  1. Department of English Language, Mal. C., Islamic Azad University, Malayer, Iran

Received: 2025-06-10

Revised: 2025-07-30

Accepted: 2025-10-20

Published in Issue 2025-12-30

Copyright (c) 2025 Nader Afshounpour, Peyman Rajabi, Hossein Ahmadi (Author)

Creative Commons License

This work is licensed under a Creative Commons Attribution 4.0 International License.

How to Cite

Afshounpour, N., Rajabi, P., & Ahmadi, H. (2025). The Effects of Writing Prompt Properties on Pre-service Teachers` Essay Writing Performance based on Coh-metrix Analysis. Journal of Applied Linguistics Studies, 5(2). https://doi.org/10.57647/jals.2025.0502.05
Crossref
Scopus
Google Scholar
Europe PMC

PDF views: 0

Abstract

Writing prompts are widely used in second language writing assessment and instruction; however, the characteristics of these prompts are often treated as neutral elicitation tools. The properties of writing prompts may systematically influence how writers generate and organize their texts. This study investigated the effect of the writing prompt characteristics on Iranian pre-service English language teachers' essay writing.  For this purpose, the research emphasized the evaluation of essays with six writing prompts with different properties. Sixty   participants out of ninety randomly selected from the male population of Iranian EFL preservice teachers majoring in English at Farhangian University. They were at the age range of 20 to 24. All of them had prior experience in writing academic English paragraphs. They were taken writing courses to enhance their English writing ability, and they had formal training in English writing skills. This research was conducted both quantitatively and qualitatively. The quantitative analysis was incorporated to provide evidence as to whether the writing prompts elicit written essays that is stable and consistent across various prompts.  Pre-service teachers' essays were analyzed using Coh-metrix to provide information about essay properties at different levels. For qualitative data, the interview recordings were transcribed and analyzed. Then common themes were extracted concerning essay composing. The findings suggest that writing prompts are not neutral instruments but important task conditions that shape the linguistic characteristics of l2 written texts.  The results of the study showed that the properties of the prompt have an important effect on some textual characteristics of the response. For example, syntactic complexity, lexical complexity, and connectives. The basic conclusion of this research was that a range of prompt properties should be controlled to keep the prompts equal. Otherwise, the essays textual features changes because of the prompt. Integrating computational text indices with qualitative process data provides a richer understanding of how task conditions influence second language writing performance. 

Keywords

  • Coh-Metrix,
  • EFL Writing,
  • Textual Features,
  • Writing Prompt
PDF

References

  1. Baba, K., & Nitta, R. (2010, May). Dynamic Effects of Task Type Practice on the Japanese EFL University Student's Writing: Text Analysis with Coh-Metrix. In Twenty-Third International FLAIRS Conference.
  2. Brossell, G., & Ash, B. H. (1984). An Experiment with the Wording of Essay Topics. College Composition and Communication, 35(4), 423-425.
  3. Cho,Y. (2003). Assessing Writing: Are we Bound by only One Method? Assessing Writing, 8, 165-191.
  4. Crossley, S. A., & McNamara, D. S. (2011). Understanding Expert Ratings of Essay Quality: Coh-Metrix Analyses of First and Second Language Writing. International Journal of Continuing Engineering Education and Life Long Learning, 21(2-3), 170-191.
  5. Crossley, S. A., Louwerse, M. M., McCarthy, P. M., & McNamara, D. S. (2007). A Linguistic Analysis of Simplified and Authentic Texts. The Modern Language Journal, 91(1), 15-30.
  6. Crowhurst, M. (1990). Teaching and Learning the Writing of Persuasive/Argumentative Discourse. Canadian Journal of Education/Revue Canadienne de L'éducation, 348-359. https://doi.org/10.2307/1495109.
  7. .
  8. Davies, M. (2008). The Corpus of Contemporary American English: 450 million words, 1990-present.
  9. Ferretti, R. P., & Lewis, W. E. (2019). Best Practices in Teaching Argumentative Writing. In S. Graham, C. A. MacArthur, & J. Fitzgerald (Eds.), Best Practices in Writing Instruction, (3rd ed.,135–161). New York: Guilford Press..
  10. Geiser, S., & Studley, R. (2001). UC and SAT: Predictive Validity and Differential Impact of the SAT I and SAT II at the University of California. Oakland: University of California.
  11. Geiser, S., & Studley, W. R. (2002). UC and the SAT: Predictive Validity and Differential Impact of the SAT I and SAT II at the University of California. Educational Assessment, 8(1), 1-26. https://doi.org/10.1207/S15326977EA0801-01
  12. Ghaith, G. (2002). Using Cooperative Learning to Facilitate Alternative Assessment. In Forum (Vol. 40, No. 3, 26-31). http://exchanges. state. gov/forum/.
  13. Graesser, A. C., Jeon, M., McDaniel, B., & Graesser, A. (2008). Survey Interviews with New Communication Technologies: Synthesis and Future Opportunities. Envisioning the Survey Interview of the Future, 267-284.
  14. Graesser, A. C., McNamara, D. S., Cai, Z., Conley, M., Li, H., & Pennebaker, J. (2014). Coh-Metrix Measures Text Characteristics at Multiple Levels of Language and Discourse. The Elementary School Journal, 115(2), 210-229.
  15. Graesser, A. C., McNamara, D. S., Louwerse, M. M., & Cai, Z. (2004). Coh-Metrix: Analysis of Text on Cohesion and Language. Behavior Research Methods, Instruments, & Computers, 36(2), 193-202. https://doi.org/10.3758/BF03195564
  16. Graesser, Arthur, C., McNamara, D.S., & M. Kullikowich., J. (2011). Coh-Metrix: Providing Multilevel Analyses of Text Characteristics, Educational Researcher, Vol. 40 (5), 223-234.
  17. Greenberg, K. (1981). The Effects of Variations in Essay Questions on the Writing Performance of CUNY Freshmen. New York: The City University of New York Instructional Resource Center.
  18. Hall, C., McCarthy, P. M., Lewis, G. A., Lee, D. S., & McNamara, D. S. (2007). Using Coh-Metrix to Assess Differences Between English Language Varieties.
  19. Hall, C., McCarthy, P. M., Lewis, G. A., Lee, D. S., & McNamara, D. S. (2007). Using Coh-Metrix to Assess Differences Between English Language Varieties.
  20. Halliday, M.A.K. and Hasan, R. (1976) Cohesion in English. Longman, London, England
  21. Hoetker, J. (1982). Essay Examination Topics and Students' Writing. College Composition and Communication, 33(4), 377-392. https://doi.org/10.2307/357950
  22. Hoetker, J., & Brossell, G. (1989). The Effects of Systematic Variations in Essay Topics on the Writing Performance of College Freshmen. College Composition and Communication, 40(4), 414-421. https://doi.org/10.2307/358240
  23. Just, M.A. & Carpenter, P.A. (1980). A Theory of Reading: From Eye Fixations to Comprehension. Psychological Review, 87, 329-354.
  24. Kuiken, F., & Vedder, I. (2008). Cognitive Task Complexity and Written Output in Italian and French as a Foreign Language. Journal of Second Language Writing, 17(1), 48-60. https://doi.org/10.1016/j.jslw.2007.08.003
  25. Leu, D. J., Keech, K. L., Murphy, S., & Kinzer, C. (1982). Effects of Two Versions of a Writing Prompt upon Holistic Score and Writing Processes. Gray, JR and Ruth, LP, Properties of Writing tasks: A Study of Alternative Procedures for Holistic Writing Assessment, Berkeley: University of California, Graduate School of Education, Bay Area Writing Project, 215-219.
  26. Lightman, E. J., McCarthy, P. M., Dufty, D. F., & McNamara, D. S. (2007). Using Computational Text Analysis Tools to Compare the Lyrics of Suicidal and Non-suicidal Songwriters. In Proceedings of the Annual Meeting of the Cognitive Science Society (Vol. 29, No. 29).
  27. Lim, G. S. (2010). Investigating Prompt Effects in Writing Performance Assessment. Spaan fellow Working Papers in Second or Foreign Language Assessment .8, 95-115.
  28. Louwerse, M. M., McCarthy, P. M., McNamara, D. S., & Graesser, A. C. (2004). Variation in Language and Cohesion Across Written and Spoken Registers. In Proceedings of the Annual Meeting of the Cognitive Science Society (Vol. 26, No. 26).
  29. McCarthy, P. M., Briner, S. W., Rus, V., & McNamara, D. S. (2007). Textual Signatures: Identifying Text-Types Using Latent Semantic Analysis to Measure the Cohesion of Text Structures (in Natural Language Processing and Text Mining).
  30. McCarthy, P. M., Lewis, G. A., Dufty, D. F., & McNamara, D. S. (2006, May). Analyzing Writing Styles with Coh-Metrix. In Flairs Conference (764-769).
  31. McNamara, D. S., Graesser, A. C., McCarthy, P. M., & Cai, Z. (2014). Automated Evaluation of Text and Discourse with Coh-Metrix. Cambridge University Press.
  32. McNamara, D. S., Louwerse, M. M., & Graesser, A. C. (2002). Coh-Metrix: Automated Cohesion and Coherence scores to Predict Text Readability and Facilitate Comprehension. Technical report, Institute for Intelligent Systems, University of Memphis, Memphis, TN.
  33. McNamara, D. S., Louwerse, M. M., McCarthy, P. M., & Graesser, A. C. (2010). Coh-Metrix: Capturing Linguistic Features of Cohesion. Discourse Processes, 47(4), 292-330.
  34. McNamara, D., Louwerse, M., & Graesser, A. (2002). Coh-Metrix: Automated Cohesion and Coherence Scores to Predict Text Readability and Facilitate Comprehension (Grant Proposal).
  35. McNamara, D.S., Crossley, A.S., & McCarthy., P.M. (2010). Linguistic Features of Writing Quality, Written Communication, Vol. 27 (1), 57-86.
  36. McNamara, D.S., Graesser, A.C., Cai, Zh, & Kulikowich., J.M. (2011). Coh-Metrix Easability Component: Aligning Text Difficulty with Theories of Text Comprehension. AERA 2011. Retrieved from the Internet 23 May 2012.
  37. McNamara, T. (2010). The Use of Language Tests in the Service of Policy: Issues of Validity. Revue Française de Linguistique Appliquée, 15(1), 007-023.
  38. McNamara, T. F. (1996). 1996: Measuring Second Language Performance. London and New York: Addison Wesley Longman.
  39. Ong, J., & Zhang, L. J. (2010). Effects of Task Complexity on the Fluency and Lexical Complexity in EFL Students’ Argumentative Writing. Journal of Second Language Writing, 19(4), 218-233. https://doi.org/10.1016/j.jslw.2010.10.003
  40. Spaan, M. (1993). The Effect of Prompt in Essay Examinations. In D. Douglas & C. Chapelle (Eds.), A New Decade of Language Testing Research (98-122). Alexandria, VA: TESOL.
  41. Ullmann, T. D. (2019). Automated Analysis of Reflection in Writing: Validating Machine Learning Approaches. International Journal of Artificial Intelligence in Education, 29, 217–257. https://doi.org/10.1007/s40593-019-00174-2.
  42. Way, D. P., Joiner, E. G., & Seaman, M. A. (2000). Writing in the Secondary Foreign Language Classroom: The Effects of Prompts and Tasks on Novice Learners of French. The Modern Language Journal, 84(2), 171-184. https://doi.org/10.1111/0026-7902.00060
  43. Wolfe-Quintero, Kate, Inagaki, Sh., & Kim., H. (1998). Second Language Development in Writing: Measures of Fluency, Accuracy, and Complexity (Technical Report). Manoa, Hawaii: Second Language Teaching and Curriculum Center, University of Hawaii.
  44. Yu, G. (2007). Lexical Diversity in MELAB Writing and Speaking Task Performances. Spaan Fellow Working Papers in Second or Foreign Language Assessment, 5, 79-116.
  45. Zhang, S. (1987). Cognitive Complexity and Written Production in English as a Second Language. Language Learning, 37(4), 469-481.