How word-image relationships in children's picture books adapt across levels: A multimodal analysis
Downloads
Children’s picture books apply a levelling system to adjust linguistic complexity according to the reader’s developmental stage. Despite having constitutive elements of text and images, their interdependent relationships within this system receive less attention. While the relationships of texts and images in picture books have been extensively studied, there is still a gap in understanding how these relationships adapt across different reading levels. Applying Halliday’s systematic functional linguistics (2004), Kress and Leeuwen’s multimodal approach (2021), and Nikolajeva and Scott’s approach to picture books (2006), this research qualitatively and quantitatively examined thirty-four text-image pairs from two children’s picture books at low and high reading levels. This study aimed to reveal whether such an adaptation exists and, if so, to what extent. The findings confirm that an increasing complexity in text-image relationships. The lower-level picture book exhibits three types of text-image relationships: enhancing, complementary, and counterpointing. The higher-level book, however, shows four types, incorporating a contradictory relationship alongside the other three. These findings suggest that the increasing complexity in children’s picture books occurs not only in linguistic aspects but also within multimodal relationships between texts and images, highlighting the need for picture book authors and educators to design or select picture books that progressively introduce multimodal complexity.
Al-Yaqout, G., & Nikolajeva, M. (2015). Re-conceptualising picture book theory in the digital age. Barnelitterí¦rt Forskningstidsskrift, 6(1). https://doi.org/10.3402/blft.v6.26971
Badan Standar, Kurikulum, dan Asesmen Pendidikan. (2022). Pedoman Perjenjangan Buku. Kementerian Pendidikan, Kebudayaan, Riset, dan Teknologi. https://buku.kemdikbud.go.id/
Bateman, John A. (2014). Text and Image: A critical introduction to the visual/verbal divide. Routledge.
Benjamin, R. G. (2011). Reconstructing readability: Recent developments and recommendations in the analysis of text difficulty. Educational Psychology Review, 24, 63–88. doi:10.1007/s10648-011-9181-8.
Callow, J. (2020). Visual and verbal intersections in picture books–multimodal assessment for middle years students. Language and Education, 34(2), 115–134. https://doi.org/10.1080/09500782.2019.1689996
Cresswell, John W., & Poth, Cheryl N. (2018). Qualitative inquiry research design: Choosing among five approaches (fourth edition). SAGE.
Damayanti, I. L., Moecharam, N. Y., & Asyifa, F. (2021). Unfolding layers of meanings: Visual-verbal relations in Just Ask"”a children's picture book. Indonesian Journal of Applied Linguistics, 11(2), 372-381. https://doi.org/10.17509/ijal.v11i2.39195
Edmonds, W. Alex., & Kennedy, Thomas D. (2017). An applied guide to research designs: quantitative, qualitative, and mixed methods. SAGE.
Elmiana, D. S., & Shen, S. (2024). Let pictures explain the world: text–image narration in picture books. Journal of Research in Childhood Education. https://doi.org/10.1080/02568543.2024.2367406
Green, C. (2025). Multi-word expressions occur infrequently in picturebooks: implications for early vocabulary instruction. Literacy Research and Instruction, 1–18. https://remote-lib.ui.ac.id:2075/10.1080/19388071.2025.2469068
Green, C., Keogh, K., Sun, H., & O'Brien, B. (2024). The children's picture books lexicon (CPB-LEX):A large-scale lexical database from children's picture books. Behavior Research Methods, 56(5),4504–4521. https://doi.org/10.3758/s13428-023-02198-y
Guijarro, Arsenio Jesus Moya, & Sanz, Maria Jesus Pinar. (2008). Compositional, interpersonal and representational meanings in a children's narrative: a multimodal discourse analysis. Journal of Pragmatics, 40(9), 1601–1619. https://doi.org/10.1016/j.pragma.2008.04.019
Halliday, M. A. K., Matthiessen, Christian M. I. M. (2014). Halliday's introduction to functional grammar: fourth edition. Routledge.
Haris, N. A., Febrianti, Y., & Yannuar, N. (2023). Exploring augmentation of meaning through intersemiotic complementarity in children comic book series. Indonesian Journal of Applied Linguistics, 12(3), 739-751. https://doi.org/10.17509/ijal.v12i3.39951
Hu, Y., & Qiu, Q. (2020). A study on verbal and image relations in multimodal texts from the perspective of intersemiotic complementarity. Canadian Social Science, 16(10), 50–56. http://doi.org/10.3968/11938
Jian, Y. C. (2022). Using an eye tracker to examine the effect of prior knowledge on reading processes while reading a printed scientific text with multiple representations. International Journal of Science Education, 44(8), 1209–1229. https://doi.org/10.1080/09500693.2022.2072013
Karatza, Styliani. (2020). Multimodal literacy and language testing: Visual and intersemiotic literacy indicators of reading comprehension texts, Journal of Visual Literacy, 39:3-4, 220-255, DOI: 10.1080/1051144X.2020.1826222
Kelly, Laura Beth, & Kachorsky, Dani. (2022). Text complexity and picturebooks: learning from multimodal analysis and children's discussion. Reading & Writing Quarterly, 38:1, 33-50, DOI: 10.1080/10573569.2021.1907636
Koutsikou, M., Christidou, V., Papadopoulou, M., & Bonoti, F. (2021). Interpersonal meaning: Verbal text–image relations in multimodal science texts for young children. Education Sciences. 11, 245. https://doi.org/10.3390/educsci11050245
Kress, Gunther, & Leeuwen, van Theo. (2021). Reading images: the grammar of visual design. Routledge.
Li, Linhui, Martin, Cortazzi, Liu, Bagoen, & Gao, Xiaomei. (2020). Moving eyes on pictures following visual grammar benefits meaning making: evidence from the independent reading of Chinese preschool children. Journal of Chinese Writing Systems, 4(1), 57–70. https://doi.org/10.1177/2513850219886109
Lewis, D. (2006) Reading contemporary picture books: Picturing text. Routledge.
Lepola, J., Kajamies, A., Laakkonen, E., & Niemi, P. (2020). Vocabulary, metacognitive knowledge and task orientation as predictors of narrative picture book comprehension: from preschool to grade 3. Reading and Writing. 33, 1351–1373, https://doi.org/10.1007/s11145-019-10010-7
Mikkelsen, Monica. (2016). Picture books as crossover literature: A study of how readers of different ages perceive iconotext and themes in picture books. [Unpublished Master thesis]. Western Norway University of Applied Sciences.
Morris, D., Trathen, W., Frye, E. M., Kucan, L., Ward, D., Schlagal, R., & Hendrix, M. (2013). The role of reading rate in the informal assessment of reading ability. Literacy Research and Instruction, 52, 52–64. doi: 10.1080/19388071.2012.702188.
Nikolajeva, Maria, & Scott, Carole. (2006). How picture books work. Routledge.
Pemerintah Indonesia. (2014). Undang-undang (UU) Nomor 35 Tahun 2014 tentang Perubahan atas Undang-Undang Nomor 23 Tahun 2002 Tentang Perlindungan Anak. https://peraturan.bpk.go.id/Details/38723/uu-no-35-tahun-2014
Pike, M. M., Barnes, M. A., & Barron, R. W. (2010). The role of illustrations in children's inferential comprehension. Journal of Experimental Child Psychology, 105(3), 243–255. https://doi.org/10.1016/j.jecp.2009.10.006
Ramos, Ana Margarida. (2020). Picture book format: Beyond the relationship between words and pictures an overview of the Portuguese editorial panorama. In Libri et Liberi (Vol. 9, Issue 1, pp. 61–74). Croatian Association of Researchers in Children's Literature, 9(1), 61-74. https://doi.org/10.21066/CARCL.LIBRI.2020.1.4
Serafini, F. (2024). The complex relationship of words and images in picture books. Journal of Visual Literacy, 43(3), 233–249. https://doi.org/10.1080/1051144X.2024.2394338
Serafini, F. (2023). How multimodality matters in children's literature scholarship. Australian Journal of Language and Literacy, 46(3), 245-256. https://doi.org/10.1007/s44020-023-00046-2
Serafini, F., & Reid, S. F. (2022). Analyzing picturebooks: semiotic, literary, and artistic frameworks. Visual Communication, 23(2), 330-350. https://doi.org/10.1177/14703572211069623
Shi, Dan. (2023). Intermodality and visual literacy: exploring visual-verbal instantiation in children's picture books on coronavirus, Journal of Visual Literacy, 42:3, 183-209, DOI: 10.1080/1051144X.2023.2258741
Shimek, C. (2021). Recursive readings and reckonings: kindergarteners' multimodal transactions with a nonfiction picturebook. English Teaching: Practice & Critique, Vol. 20 No. 2, pp. 149-162. https://doi.org/10.1108/ETPC-07-2020-0068
Yefymenko, Victoria. (2024). Multimodal text-image synergy in representing interpersonal relations in picture books. Cognition, Communication, Discourse, 28, 102–109. https://doi.org/10.26565/2218-2926-2024-28-07
Zhao, Jing, Zhu, Meifang, Ruiter, de Laura, & Chen, Si. (2022). Linguistic indicators for text complexity in picture books for young chinese children learning English as a foreign language. Frontiers in Education. DOI: 10.3389/feduc.2022.758736
Data Sources
Fauzia, Sarah, & Nufus, Alimah. (2022). Naik-naik ke puncak bukit. Pusat Perbukuan. Kementerian Pendidikan, Kebudayaan, Riset, dan Teknologi Republik Indonesia.
Widjati, Tyas, & Faza. (2022). Gambar lucu Mika. Pusat Perbukuan. Kementerian Pendidikan, Kebudayaan, Riset, dan Teknologi Republik Indonesia.
Copyright (c) 2025 Muhamad Fanni Al Farisi, Bernadette Kushartanti, Dwi Purwanto
This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.
The authors who publish this journal agree to the following requirements. The author retains the copyright regarding the work being simultaneously licensed below Creative Commons Attribution ShareAlike License.
Jurnal Diksi by Faculty of Languages, Arts, and Culture, Universitas Negeri Yogyakarta is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.
Based on a work at http://journal.uny.ac.id/index.php/diksi
2.png){kind=logo}
