Artificial Intelligence In Language Education: Exploring Prompting Strategies To Foster Argumentative Writing Skills
Keywords:
ChatGPT, human-AI mediation, language education, Large Language Models, prompt engineeringAbstract
The rapid advancement of Artificial Intelligence (AI), particularly Large Language Models (LLMs), calls for a thorough examination of not only of the opportunities for innovation but also the conditions necessary to foster a productive and informed human-machine relationship in education. This article explores the integration of prompt engineering as a critical transversal skill for effectively implementing AI-based technologies in language education while promoting the development of digital competencies among educators and learners. The study observes variations in interactions between learners and ChatGPT during educational activities designed to enhance argumentative writing skills. Specifically, it examines the reliability and feasibility of ChatGPT in providing meaningful and relevant feedback on argumentative writing through the analysis of task-specific interactions between secondary school students and the language model. Additionally, it explores how the iterative process of prompt construction and refinement adopted by participants shapes ChatGPT’s responses when evaluating learners’ argumentative texts. By analysing the impact of different prompt strategies on the chatbot’s outputs, the study offers practical guidelines for leveraging AI to foster language acquisition, AI literacy, and critical thinking through the evaluation, validation, and optimization of learners’ interactions with ChatGPT.
References
Anderson, N., McGowan, A., Galway, L., Hanna, P., Collins, M., &Cutting, D. (2023). Implementing generative AI and large language models in education. In Proceedings of the 7th International Symposium on Innovative Approaches in Smart Technologies(ISAS 2023) (pp. 1-6). IEEE. https://doi.org/10.1109/isas60782.2023.10391517Balboni, P. E. (2011). Conoscenza, verità, etica nell’ educazione linguistica. Guerra.
Bender, E. M., & Koller, A. (2020). Climbing towards NLU: On meaning, form, and understanding in the age of data. In D. Jurafsky, J. Chai, N. Schluter &J. Tetreault (Eds.), Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics (pp. 5185-5198). Association for Computational Linguistics. 10.18653/v1/2020.acl-main.463
Bozkurt, A. (2023). Generative artificial intelligence (AI) powered conversational educational agents: The inevitable paradigm shift. Asian Journal of Distance Education, 18(1), 198–204. https://www.asianjde.com/ojs/index.php/AsianJDE/article/view/718
Cain, W. (2024). Prompting change: Exploring prompt engineering in large language model AI and its potential to transform education. TechTrends, 68, 47–57. https://doi.org/10.1007/s11528-023-00896-0
Chen, B., Zhang, Z., Langrené, N., & Zhu, S. (2024). Unleashing the potential of prompt engineering in large language models: A comprehensive review. ArXiv, abs/2310.14735v5. https://doi.org/10.48550/arXiv.2310.14735
Chini, M., & Bosisio, C. (2014). Fondamenti di glottodidattica. Carocci.
Council of Europe. (2001). Common European framework of reference for languages: Learning, teaching, assessment. Cambridge University Press.
Council of Europe. (2020). Common European framework of reference for languages: Learning, teaching, assessment – Companion volume. Council of Europe Publishing.
Dang, H., Mecke, L., Lehmann, F., Goller, S., & Busheck, D. (2022). How to prompt? Opportunities and challenges of zero- and few-shot learning for human-AI interactionin creative applications of generative models. In GenAICHI: Generative AI and Computer Human Interaction, Workshop (CHI’22) (pp. 1-7). Association for Computing Machinery.
De Mauro, T., & Ferreri, S. (2005). Glottodidattica come linguistica educativa. In Voghera, M., Basile, G., & Guerriero, A. R. (Eds.), E.LI.C.A. Educazione linguistica econoscenze per l’accesso (pp. 17–28). Guerra. Eager, B., & Brunton, R. (2023). Prompting higher education towards AI-augmented teaching and learning practice. Journal of University Teaching and Learning Practice, 20(5), 1–19. https://doi.org/10.53761/1.20.5.02
Gao, T., Fisch, A., & Chen, D. (2021). Making pre-trained language models better few-shot learners. In C. Zong, F. Xia, W. Li & R. Navigli (Eds.), Proceedings of the59thAnnual Meeting of the Association for Computational Linguistics and the11th International Joint Conference on Natural Language Processing (Volume 1: Longpapers) (pp. 3816–3830). Association for Computational Linguistics. 10.18653/v1/2021.acl-long.295
Kazemitabaar, M., Hou, X., Henley, A., Ericson, B. J., Weintrop, D., & Grossman, T. (2023). How novices use LLM-based code generators to solve CS1 coding tasks inaself-paced learning environment. In A. Mühling & I. Jormanainen (Eds.), Proceedings of the 23rd Koli Calling International Conference on Computing Education Research (pp. 1-12). Association for Computing Machinery. https://doi.org/10.1145/3631802.3631806
Knoth, N., Tolzin, A., Janson, A., & Leimeister, J. M. (2024). AI literacy and its implications for prompt engineering strategies. Computers and Education, 6, 1–14. https://doi.org/10.1016/j.caeai.2024.100225
Korzynski, P., Mazurek, G., Krzypkowska, P., & Kurasinski, A. (2023). Artificial intelligence prompt engineering as a new digital competence: Analysis of generativeAI technologies such as ChatGPT. Entrepreneurial Business and Economics Review, 11(3), 25–38. https://doi.org/10.15678/EBER.2023.110302
Lee, U., Jung, H., Jeon, Y., Sohn, Y., Hwang, W., Moon, J., & Kim, H. (2023). Few-shot is enough: Exploring ChatGPT prompt engineering method for automatic question generation in English education. Education and Information Technologies, 29(9), 11483–11515. https://doi.org/10.1007/s10639-023-12249-8
Li, H., Leung, J., & Shen, Z. (2024). Towards goal-oriented prompt engineering for large language models: A survey. ArXiv, abs/2401.14043v3. https://doi.org/10.48550/arXiv.2401.14043
Linardatos, P., Papastefanopoulos, V., & Kotsiantis, S. (2021). Explainable AI: A review of machine learning interpretability methods. Entropy, 23, 1–18. https://doi.org/10.3390/e23010018
Liu, L. (2023). Analyzing the text contents produced by ChatGPT: Prompts, feature-components in responses, and a predictive model. Journal of Educational Technology Development and Exchange, 16(1), 49–70. https://doi.org/10.18785/jetde.1601.03
Liu, N. F., Lin, K., Hewitt, J., Paranjape, A., Bevilacqua, M., Petroni, F., &Liang, P. (2024). Lost in the middle: How language models use long contexts. In Transactions of theAssociation for Computational Linguistics (pp. 157–173). MITPress. 10.1162/tacl_a_00638
Liu, P., Yuan, W., Fu, J., Jiang, Z., Hayashi, H., & Neubig, G. (2023). Pre-train, prompt, andpredict: A systematic survey of prompting methods in natural language processing. ACM Computer Surveys, 55(9), 1–35. https://doi.org/10.1145/3560815
Lo, L. S. (2023). The CLEAR path: A framework for enhancing information literacy through prompt engineering. The Journal of Academic Librarianship, 49(4), 1–3. https://doi.org/10.1016/j.acalib.2023.102720
Lu, Y., Bartolo, M., Moore, A., Riedel, S., & Stenetorp, P. (2022). Fantastically ordered prompts and where to find them: Overcoming few-shot prompt order sensitivity. InS. Muresan, P. Nakov & A. Villavicencio (Eds.), Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: LongPapers)(pp. 8086–8098). Association for Computational Linguistics. 10.18653/v1/2022.acllong.556
Mondal, S., Bappon, S. D., & Roy, C. K. (2024). Enhancing user interaction in ChatGPT: Characterizing and consolidating multiple prompts for issue resolution. In D. Spinellis, A. Bacchelli & E. Constantinou (Eds.), Proceedings of the 21st International Conference on Mining Software Repositories (MSR ’24) (pp. 222-226). Associationfor Computing Machinery. https://doi.org/10.1145/3643991.3645085
Nurminen, M., & Papula, N. (2018). Gist MT users: A snapshot of the use and users of one online MT tool. In J. A. Pérez-Ortiz, F. Sánchez-Martínez, M. Esplà-Gomis, M. Popovic, C. Rico, A. Martins, J. Van den Bogaert & M. L. Forcada (Eds.), Proceedings of the 21st Annual Conference of the European Association for Machine Translation (pp. 199–208). European Association for Machine Translation.
O’Connor, J., & Andreas, J. (2021). What context features can transformlanguage models use?In C. Zong, F. Xia, W. Li & R. Navigli (Eds.), Proceedings of the 59thAnnual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers) (pp. 851–864). Association for Computational Linguistics. 10.18653/v1/2021.acl-long.70
Ranieri, M., Cuomo, S., & Biagini, G. (2023). Scuola e intelligenza artificiale: Percorsi di alfabetizzazione critica. Carrocci.
Sawalha, G., Taj, I., & Shoufan, A. (2024). Analyzing student prompt and their effect onChatGPT’s performance. Cogent Education, 11(1), 1–20. https://doi.org/10.1080/2331186X.2024.2397200
Sheese, B., Liffiton, M., Savelka, J., & Denny, P. (2024). Patterns of student help-seeking when using a large language model-powered programming assistant. In N. Herbert &C. Seton (EDS.), Proceedings of the 26th Australasian Computing Education Conference (ACE ‘24) (pp. 49-57). Association for Computing Machinery. https://doi.org/10.1145/3636243.3636249
Su, Y., Lin, Y., & Lai, C. (2023). Collaborating with ChatGPT in argumentative writing classrooms. Assessing Writing, 57, 1–13. https://doi.org/10.1016/j.asw.2023.100752
Tan, B., Yang, Z., Al-Shedivat, M., Xing, E. P., Hu, Z. (2021). Progressive generation of longtext with pretrained language models. In K. Toutanova, A. Rumshisky, L. Zettlemoyer, D. Hakkani-Tur, S. Bethard, R. Cotterell, T. Chakraborty &Y. Zhou(Eds.), Proceedings of the 2021 Conference of the North American Chapter of the Associationfor Computational Linguistics: Human Language Technologies (pp. 4313-4324). Association for Computational Linguistics. 10.18653/v1/2021.naacl-main.341
Theophilou, E., Koyutürk, C., Yavari, M., Bursic, S., Donabauer, G., Telari, A., Testa, A., Boiano, R., Hernandez-Leo, D., Ruskov, M., Taibi, D., Gabbiadini, A., &Ognibene, D. (2023). Learning to prompt in the classroom to understand AI limits: Apilot study. InR. Basili, D. Lembo, C. Limongelli & A. Orlandini (Eds.), Proceedings of the 22nd International Conference of the Italian Association for Artificial Intelligence (pp. 481-496). Springer. https://dx.doi.org/10.1007/978-3-031-47546-7_33
Walter, Y. (2024). Embracing the future of artificial intelligence in the classroom: The relevance of AI literacy, prompt engineering, and critical thinking in modern education. International Journal of Educational Technology in Higher Education, 21(1), 1–29. https://doi.org/10.1186/s41239-024-00448-3
Wang, M., Wang, M., Xu, X., Yang, L., Cai, D., & Yin, M. (2024). Unleashing ChatGPT’s power: A case study on optimizing information retrieval in flipped classrooms via prompt engineering. IEEE Transactions on Learning Technologies, 17, 629–641. https://doi.org/10.1109/TLT.2023.3324714
Wang, L., Chen, X., Wang, C., Xu, L., Shadiev, R., & Li, Y. (2024). ChatGPT’s capabilities in providing feedback on undergraduate students’ argumentation: Acase study. Thinking Skills and Creativity, 51, 1–14. https://doi.org/10.1016/j.tsc.2023.101440
White, J., Fu, Q., Hays, S., Sandborn, M., Olea, C., Gilbert, H., Elnashar, A., et al. (2023). Aprompt pattern catalogue to enhance prompt engineering with ChatGPT. ArXiv, abs/2302.11382. https://doi.org/10.48550/arXiv.2302.11382
Woo, D. J., Guo, K., & Susanto, H. (2023). Case of EFL secondary students’ prompt engineering pathways to complete a writing task with ChatGPT. ArXiv, abs/2307.05493. https://doi.org/10.48550/arXiv.2307.05493
Wu, T., Terry, M., & Cai, C. J. (2022). AI chains: Transparent and controllable human-AI interaction by chaining large language model prompts. In S. Barbosa, C. Lampe, C. Appert, D. A. Shamma, A. Drucker, J. Williamson & K. Yatani (Eds.), Proceedings of
the 2022 CHI Conference on Human Factors in Computing Systems (CHI ’22) (pp. 1-22). Association for Computing Machinery. https://doi.org/10.1145/3491102.3517582
Zamfirescu-Pereira, J. D., Wong, R. Y., Hartmann, B., & Yang, Q. (2023). Why Johnnycan’t
prompt: How non-AI experts try (and fail) to design LLM prompts. In A. Schmidt, K. Väänänen, T. Goyal, P. O. Kristensson, A. Peters, S. Mueller, J. R. Williamson&M. L. Wilson (Eds.), Proceedings of the 2023 CHI Conference on Human FactorsinComputing Systems (CHI ’23) (pp. 1-22). Association for Computing Machinery. https://doi.org/10.1145/3544548.3581388
Zhao, T. Z., Wallace, E., Feng, S., Klein, D., & Singh, S. (2021). Calibrate beforeuse: Improving few-shot performance of language models. In M. Meila &T. Zhang(Eds.), Proceedings of the 38th International Conference on Machine Learning (ICML‘21)(pp. 12697–12706). PMLR
