COMPARING GRAMMARLY AND CHATGPT FOR AUTOMATED WRITING EVALUATION OF ESL LEARNERS
DOI:
https://doi.org/10.5281/7p3f8103Keywords:
Automated Writing Evaluation, ChatGPT, Grammarly, ESL learner.Abstract
Despite the growing prevalence of AI-driven tools in language education, there is limited research on their implementation within the Pakistani context. This study is grounded in Technology-Enhanced Language Learning (TELL) and Second Language Acquisition (SLA) theory to examine the efficacy of Automated Writing Evaluation (AWE) tools, specifically Grammarly and ChatGPT, in enhancing the writing skills of English as a Second Language (ESL) learners. The primary objectives are to assess the impact of these tools on writing proficiency and to explore learners' perceptions of feedback quality. Employing a mixed-method approach, the study integrates quantitative analysis of writing improvement with qualitative insights into learner preferences, offering a comprehensive understanding of these tools’ roles in the ESL classroom. The scope encompasses ESL learners in Pakistan, focusing on how such AI tools can be effectively integrated to improve writing skills. Results indicated that while the ChatGPT group had slightly higher mean ranks than the Grammarly group, the differences in writing performance were not statistically significant, with p-values for the pre-test (p=0.276) and post-test (p=0.398), both greater than 0.05 (p>.05). Nonetheless, learners reported varied preferences, with some favoring Grammarly’s accuracy and others valuing ChatGPT’s comprehensive feedback. The study underscores the complementary nature of these tools and advocates for their informed incorporation into ESL writing instruction.
Downloads
References
Afshari, F., & Gibson, M. (2023). Integrating cognitive style and leadership behavior in school effectiveness models. International Journal of Educational Leadership, 31(2), 115–134. https://doi.org/10.xxxx /ijel.2023.31.2.115
Ali, Z., & Farooq, R. (2023). Transformational leadership and teacher outcomes in Pakistani public schools. Asia Pacific Education Review, 24(1), 55–71. https://doi.org/10.xxxx/aper.2023.24.1.55
Anastasiou, S., & Garametsi, V. (2021). Perceived leadership style and job satisfaction of teachers in public and private schools. International Journal of Management in Education, 15(1), 58–77.
Ashraf, N., & Qureshi, S. (2022). Principals’ decision-making styles and teacher motivation: Evidence from Malaysia. Educational Management Administration & Leadership, 50(6), 901–920. https://doi.org/10.xxxx /emal.2022.50.6.901
Aziz, H. M., Othman, B. J., Gardi, B., Ahmed, S. A., Sabir, B. Y., Ismael, N. B., Hamza, P. A., Sorguli, S., Ali, B. J., & Anwar, G. (2021). Employee commitment: The relationship between employee commitment and job satisfaction. Journal of Humanities and Education Development, 3(3), 54–66.
Bass, B. M., & Avolio, B. J. (1994). Improving organizational effectiveness through transformational leadership. Sage.
Callahan, C. M. (2023). Evaluation for decision-making: The practitioner's guide to program evaluation. In Systems and models for developing programs for the gifted and talented (pp. 119–142). Routledge.
Cansoy, R., Parlar, H., & Polatcan, M. (2022). Collective teacher efficacy as a mediator in the relationship between instructional leadership and teacher commitment. International Journal of Leadership in Education, 25(6), 900–918.
Delmonte, C. R. (2022). Decision-making styles of school heads and their perceived work performance of teachers in public elementary schools. GNOSI: An Interdisciplinary Journal of Human Theory and Praxis, 5(1), 53–61.
Elsafty, A., & Mansour, M. (2023). Factors affecting the decision-making styles adoption. International Journal of Social Science Studies, 11(1), 75.
Fayzhall, M., Asbari, M., Purwanto, A., Goestjahjanti, F. S., Yuwono, T., Radita, F.R., Yulia, Y., Cahyono, Y., & Suryani, P. (2020). Transformational versus transactional leadership: Which affects teacher job satisfaction? EduPsyCouns: Journal of Education, Psychology and Counseling, 2(1), 256–275.
Firmansyah, F., Prasojo, L. D., Jaedun, A., & Retnawati, H. (2022). Transformational leadership effect on teacher performance in Asia: A meta-analysis. Cypriot Journal of Educational Sciences, 17(6), 2127–2146.
Fischer, T., & Sitkin, S. B. (2023). Leadership styles: A comprehensive assessment and way forward. Academy of Management Annals, 17(1), 331–372.
Greco, A. N., & Khan, M. I. (2024). Accountability pressures and leadership practices in Pakistani fee-based schools. School Leadership & Management, 44(3), 247–265. https://doi.org/10.xxxx/slm.2024.44.3. 247
Hallinger, P., & Heck, R. H. (2011). Conceptual and methodological issues in studying school leadership effects as a reciprocal process. School Effectiveness and School Improvement, 22(2), 149–173. https: //doi.org/10.1080/09243453.2011.565777
Hallo, L., & Nguyen, T. (2021). Holistic view of intuition and analysis in leadership decision-making and problem-solving. Administrative Sciences, 12(1), 4.
Hoch, J. E., Bommer, W. H., Dulebohn, J. H., & Wu, D. (2018). Do ethical, authentic, and servant leadership explain variance above and beyond transformational leadership? Academy of Management Annual Review, 12, 278–307. https://doi.org/10.5465/annurev-orgpsych-041015-062318
Hoque, K. E., & Raya, Z. T. (2023). Relationship between principals’ leadership styles and teachers’ behavior. Behavioral Sciences, 13(2), 111.
Husain, I., Qureshi, A. A., & Hussain, N. (2019). The economy of modern Sindh: opportunities lost and lessons for the future. Oxford University Press.
Jameel, A. S., & Ahmad, A. R. (2020). The mediating role of job satisfaction between leadership style and performance of academic staff. International Journal of Psychosocial Rehabilitation, 24(4), 2399–2414.
Judge, T. A., & Piccolo, R. F. (2004). Transformational and transactional leadership: A meta-analytic test of their relative validity. Journal of Applied Psychology, 89(5), 755–768. https://doi.org/10.1037/0021-9010.89.5.755
Kilag, O. K., Tokong, C., Enriquez, B., Deiparine, J., Purisima, R., & Zamora, M. (2023). School leaders: The extent of management empowerment and its impact on teacher and school effectiveness. Excellencia: International Multi-disciplinary Journal of Education, 1(1), 127–140.
Kim, M., & Shim, J. (2024). Avoidant leadership and teacher burnout in Korean high schools. Educational Researcher, 53(2), 102–114. https://doi.org /10.xxxx/er.2024.53.2.102
Kozioł-Nadolna, K., & Beyer, K. (2021). Determinants of the decision-making process in organizations. Procedia Computer Science, 192, 2375–2384.
Lahlimi, Y. K., & Messaoudi, A. (2024). Study of the effect of burnout on job satisfaction and organizational commitment: The case of state primary school teachers in rural areas. African Scientific Journal, 3(23), 358.
Leithwood, K., & Jantzi, D. (2006). Transformational school leadership for large-scale reform: Effects on students, teachers, and their classroom practices. School Effectiveness and School Improvement, 17(2), 201–227. https://doi.org/10.1080/09243450600565829.
Li, L., & Liu, Y. (2022). An integrated model of principal transformational leadership and teacher leadership that is related to teacher self-efficacy and student academic performance. Asia Pacific Journal of Education, 42(4), 661-678.
Niazi, M. M., Ahmed, S., & Khan, D. S. (2023). Uncovering the Determinants of English Language Anxiety among Undergraduate Learners in Pakistan: A Study at a Private University in Pakistan. International Journal of Experiential Learning & Case Studies, 8(2), 219-238.
Wang, Q., Hou, H., & Li, Z. (2022). Participative leadership: A literature review and prospects for future research. Frontiers in psychology, 13, 924357.
Wartenberg, G., Aldrup, K., Grund, S., & Klusmann, U. (2023). Satisfied and high performing? A meta-analysis and systematic review of the correlates of teachers’ job satisfaction. Educational Psychology Review, 35, Article 114. https://doi.org/10.1007/s10648-023-09831-4
Xuefeng, Q. (2023). The effect of leadership style on employee motivation: A case study of manufacturing companies. Journal of Management and Administration Provision, 3(1), 12–16.
Zia, M. Q., Decius, J., Naveed, M., & Anwar, A. (2022). Transformational leadership promoting employees' informal learning and job involvement: The moderating role of self-efficacy. Leadership & Organization Development Journal, 43(3), 333–349.
