From collection to connection: Teaching and learning science in an interactive multimodal learning environment

Dr Sally Stephens, Director of Science
Originally published in March, 2012

…the walls of the classroom are literally made irrelevant by the creation of communities of learners that span oceans, races, genders, and generations.
Richardson (2009, p.130)

Take a snapshot of how students conduct their out-of-school lives and you will see them using a kaleidoscope of digital technologies to communicate, collaborate and form social constructs with the world around them. They might tweet, text, chat or blog, buy and sell online, Skype or use Facetime. Or they might play online games. On their Facebook page they might update their status, write on their friend’s wall, upload and view still and video images, sort their ‘friends’ into social groupings, make their likes and dislikes known, and play games with other Facebook users. They might watch a video clip of One Direction singing ‘What makes you beautiful’ over and over (preferably wearing their headphones) or even, like thousands of other people, create a video of themselves dancing to ‘I’m sexy and I know it’ to broadcast on YouTube.

Are they learning while they are playing in their digital world? If so, what developmental dimensions are being enhanced? While research results are inconclusive, there is MRI evidence that excessive gaming (more than sixty hours per week) may cause parts of teenagers’ brains to atrophy, reducing their inhibitions and affecting their concentration, memory, and ability to make decisions and set goals (Yuan et al., 2011). On the other hand, American pop science writer, Steven Johnson, author of Everything Bad is Good for You, drawing from fields as diverse as neuroscience, economics and literary theory, argues that computer games require concentration, forward planning, lateral thinking and sustained problem solving and, as such, offer intellectual demands that can benefit overall cognitive development (2005). Similarly, Gee (2003) claims that there are better learning theories embedded in video games than many students experience in the classroom because online video games encourage them to be critical, reflective and strategic. In terms of social development, a study by Ellison, Steinfield and Lampe (2007) investigated the relationship between Facebook usage and the formation and maintenance of social capital and found the use of this social networking site to be of particular benefit for users experiencing low self-esteem and low satisfaction with life.

Now take a snapshot of these same students in the classroom. What you find is a hive of activity as students and teachers wrestle with the challenges of a variety of rich learning experiences. There will be episodes of whole-class teaching, small group work, one-on-one student-teacher interactions and quiet or individual study. Students could be working from the textbook or completing a worksheet. They might be engaged in an audiovisual activity or information and communication technologies (ICT) work. In most classrooms, the online delivery of activities will be blended with more traditional forms of classroom instruction via an array of effective instructional strategies. Learning objectives are being achieved, but it is clear that despite our best efforts, there remains a sharp disconnect between the learning strategies used inside classrooms and how students are learning in their out-of-school time. According to Hramiak (2012), ‘many pupils go home to better computing facilities than they have at school … and have much fewer restrictions in what they can and cannot access at home compared with school’. Communication is increasingly digital so digital technology is an intrinsic part of students’ lives. Young people are heavily invested in multimodal technology, yet classrooms remain primarily print-based. Some social researchers fear that students who live multimodal lives will soon become disenchanted with formal school practices that employ only traditional literacies (Green & Hannon, 2007).

Being literate is a transformational experience; it is our window to the world. For many years, being literate was defined as the ability to read and write (Richardson, 2009) but, according to Leu, Kinzer, Coiro and Cammack (2004), prominent researchers in the field of new literacies, it is clear that we are entering a new era of literacy in which all forms of communication are themselves undergoing a transformation. Leu et al. contend that ‘as the medium of the message changes, comprehension processes, decoding processes, and what “counts” as literacy activities must change to reflect readers’ and authors’ present-day strategies for comprehension and response’. Thus, if we are to maximize the potential of continuously emerging ICTs, we will require new literacies, in both our personal and professional lives.

Leu et al (2004) believe that traditional definitions of literacy are inadequate and any attempts to frame a new definition must consider the rapid changes we are experiencing today and will continue to encounter as new ICTs regularly emerge. The definition of literacy by Luke and Freebody (2000, p.9) is ‘the flexible and sustainable mastery of a repertoire of practices with the texts of traditional and new communications technologies via spoken, print and multimedia’, which identifies the characteristics of a literate person that should endure in the face of emerging technologies.

New literacies combine letters, symbols, colours, sounds and graphics to expand the ways by which we communicate. It is clear that expanding the boundaries of literacy has implications for literacy instruction and curriculum development as we seek to prepare students for the new multiple literacies that will characterise their future. There is a need for schools to assist learners to bridge the gap between their informal practices and the demands of academic study that incorporate ICTs (Hramiak, 2012). They can do this by adopting and integrating the types of resources made possible by new technologies, such as electronic textbooks and the instructional strategies that complement them. Although research into these types of texts is still in its infancy, Larson (2010) reports that their use is accompanied by the sorts of positive attitudes and behaviours that can promote literacy development. Other research has found that teachers and students who have adopted new literacies have replaced language-based pedagogies with those that enable them to take full advantage of multimodal learning styles (Bull & Anstey, 2010).

According to the New London Group (1996), multimodal texts communicate meaning through the juxtaposition of two or more of the five possible patterns of meaning: linguistic (oral and written language), visual (still and moving images), audio (music and sound effects), gestural (movement expression and body language), and spatial (organisation of objects in a setting). From this definition it is clear that multimodality in texts is not new. Any science text with illustrations would fit the bill. What is new, however, is the interactivity that accompanies electronic delivery. According to Moreno and Mayer (2007), an interactive multimodal text is not just an electronic version of the textbook, but a resource that contains design elements that allow active engagement with the text that is not possible in print-based documents. What happens in an interactive multimodal learning environment depends on the actions of the learner. In fact, the defining attribute of interactivity is responsiveness to the learner’s actions: the text transforms as the student interacts with it. On the other hand, a non-interactive communication might include text and pictures, but they are presented in a predetermined way, irrespective of the actions of the learner.

Science teachers have long used multiple representations of concepts to appeal to the variety of learning styles favoured by students, but they are still largely print-centric and tied to language-based pedagogy. That is not to say that Science teachers do not use interactive resources, we do. During lesson planning we constantly search for electronic resources, such as apps, games, applets, and simulations, which we use to help students make meaning of both fundamental and complex concepts. The very best get incorporated into the teacher’s individual ‘collection’. Effectively we are supplementing traditional teaching methods with multimedia instruction but clearly we need to do more than this if we wish to bridge the technological divide and ‘leverage the power of emerging technologies for instructional gain’ (Klopfer, Osterwell, Groff, & Haas, 2009, p.3).

So where are we headed with the use of digital technology in science education at Brisbane Girls Grammar School? We know that multimodal technologies are already permeating the workplace as productivity and development tools and that there is a reason that these technologies are so pervasive (Klopfer et al., 2009). Industries, businesses, medical training facilities, government departments and scientific research facilities are recognising the advantages of these tools and are exploiting them to enhance their work. Research in educational settings has found that multimedia education improves both motivation and comprehension, which translates into positive learning outcomes (Brady, 2004). So, just as non-educational institutions are able to see the benefits of these tools for their core business, we, as science educators, see enough evidence of improved learning outcomes to warrant the integration of emerging technology-based resources and pedagogies into our new curricula. There is a new wave sweeping across the educational landscape and we want to make sure that we are on the front end of it. We cannot let the emergence of new ICTs outpace pedagogy.

So that we are better able to adapt to the dynamics of our changing world, science teachers need to redefine ourselves in ways that will transform our repertoire of teaching strategies. Richardson (2009) argues that this will only be achieved if teachers commit themselves to the five different roles described here: 1) to be able to teach multiliteracies successfully to students, we must be able to use them competently ourselves, so we must learn to use new technologies effectively enough to become creators of content; 2) we must join with our students to become true collaborators in the learning process; 3) as students gain real-time access to trustworthy, authoritative information, we can no longer claim to be the most expert voice in the classroom, so we must learn to think of ourselves as coaches, those who train students to find their own information, and to critically evaluate and make sense of it; 4) we need to update our view of what it means to be a connector, the one who, in this new era of emerging multiliteracies, continues to use appropriate pedagogies to make the mandated curriculum accessible to all students; 5) finally, to transform the print-centric classroom to a multiliterate one, teachers have to become agents of change (Richardson, 2009). The time has come to move from ‘collections’ of resources to ‘connections’, to exploit the connectedness of interactive multimodal texts with all of the benefits that they afford our students.

Today’s students live in a highly interactive multimodal world where they appear, to many, to be skilled and confident consumers of emerging digital technologies. But are they as savvy as they seem? Our challenge as science curriculum developers is to channel the skills students are developing in their out-of-school lives into the academic arena so that they can receive guidance in the development of the desired ‘repertoire of capabilities’ they might not be able to acquire on their own. Interactive multimodal texts can promote new literacies practices and strengthen the connection between learners and text as engagement with, and manipulation of, text is enabled through the tools and features associated with electronic media. The ultimate goal is to future proof students’ multiliteracies.


Brady, J. (2004). More than just fun and games? Applied Clinical Trials. Retrieved April 29, 2012, from

Bull, G., & Anstey, M. (2010). Evolving Pedagogies; Reading and Writing in a Multimodal World. Retrieved April 3, 2012, from

Ellison, N. B., Steinfield, C., & Lampe, C. (2007). The benefits of Facebook “friends”: Social capital and college students’ use of online social network sites. Journal of Computer-Mediated Communication, 12(4). Retrieved April 3, 2012, from

Gee, J. P. (2003). What Video Games Have to Teach Us About Learning and Literacy? New York: Palgrave/Macmillan.

Green, H. & Hannon, C. (2007). Their Space: Education for a digital generation. Retrieved April 29, 2012, from

Hramiak, A. (2012). What does it mean to be literate in 2012? Retrieved April 3, 2012, from

Johnson, S. (2005). Everything Bad Is Good For You: How Today’s Popular Culture Is Actually Making Us Smarter. New York: Riverhead Books.

Klopfer, E., Osterwell, S., Groff, J., & Haas, J. (2009). The instructional power of digital games, social networking and simulations and how teachers can leverage them. Retrieved April 29, 2012, from

Larson, L. C. (2010). Digital readers: The next chapter in e-book reading and response. The Reading Teacher, 64(1), 15-22. Retrieved April 3, 2012, from

Leu, D. J., Kinzer, C. K., Coiro, J. L., & Cammack, D. W. (2004). Toward a Theory of New Literacies Emerging From the Internet and Other Information and Communication Technologies. Retrieved April 3, 2012, from

Luke, A., & Freebody, P. (1999). Further notes the four resources model. Retrieved April 3, 2012, from
Moreno, R., & Mayer, R. E. (2007). Interactive multimodal learning environments. Educational Psychology Review, 19, 309-326.

New London Group. (Spring, 1996). A pedagogy of multiliteracies: Designing social futures. Harvard Educational Review, 66 (1), 60-92. Retrieved April 3, 2012, from

Richardson, W. (2009). Blogs, Wikis, Podcasts, and Other Powerful Web Tools for Classrooms. 2nd ed. California: Corwin Press.

Read reflective commentary from Director of Science, Dr Sally Stephens, on her piece, ‘From collection to connection: Teaching and learning science in an interactive multimodal learning environment’.

I wrote this article in 2012 when the technology landscape of the Science Faculty looked very different from the way it does today. We had one trolley of 25 laptops on each of the three floors of the Science building, which allowed us to purchase digital probes that could measure scientific quantities such as time, temperature, pH, magnetic field strength, etc. with far more accuracy and precision than the instruments we were using. This was at a time when almost every household in Australia with children under 15 years of age had the technology to access the Internet at home, and we knew that personal computers’ capabilities were beginning to advance beyond those of computers provided at schools and universities.

The 2012 article was about connectedness, and in it I made three main points: first, during students’ formal education, we need to build on the multimodal approach to learning that they adopt in their out-of-school lives; second, we cannot let new technologies outpace our pedagogical practices; and third, we must future-proof students’ multiliteracies. I was clearly making an argument for increasing students’ access to laptop computers in the classroom because I felt they were necessary to take full advantage of multimodal learning.

Bring Your Own Device (BYOD) was introduced in 2014 and required a huge pedagogical shift. We searched for a digital platform to help us achieve the connectivity that we desired and found Cosmos Stile. Stile is a digital platform with an immense library of science units comprising lessons developed using ‘design elements that allow active engagement with the text that is not possible in print-based documents’. Its large collection of resources is curated into an easily navigable format; formative assessment is quick and easy to embed, with teachers able to add items during the lesson to gauge individual understanding, and its record keeping is excellent. Built-in analytics are exceptionally useful in tracking student progress. Stile’s ability to be customised by teachers empowers them to be proactive, responsive, flexible and adaptive in the learning environment.

Pedagogically, everything is going well but I cannot seem to shake this feeling of unease. The vast majority of our students have lived their entire lives in a digitised world. By the time our current Year 12 students were in Prep (in 2007), laptops and WiFi were commonplace and smartphones were entering the market. They are digital natives, and they are rarely in awe of technology because they expect it to evolve at a rapid pace. But, given the widespread use of digital devices in schools, how much do we know about the effect of digitisation of learning? I am concerned that we, as a society, have lost something as a result of the ubiquity of digital devices in our lives. Neuroscientists fear diminished capacity for analytical reasoning, memory, attention and creativity. I worry about the ability to imagine, ever important for science. Technology cannot prepare students for everything they will experience in life and it seems to have a dark side.

I am sure that naysayers have been voicing similar views since the invention of the printing press wrested hegemonic control of information away from the Church in 1451, and that resulted in many positive outcomes. The spread of the printing press ultimately led to the Renaissance, the Scientific Enlightenment and the Industrial Revolution. Still, I can relate to this research scientist and professor who anonymously responded to a survey about how changes in digital life will impact people’s overall wellbeing over the next decade:

The technologies that 50 years ago we could only dream of in science fiction novels, which we then actually created with so much faith and hope in their power to unite us and make us freer, have been co-opted into tools of surveillance, behavioural manipulation, radicalisation and addiction. (Anderson and Rainie, 2018)

In search of connectedness but with little understanding of the consequences, we have invited the digital world into the classroom. I am hopeful, however, that the positives of digital life will outweigh the negatives and I believe they will if we have the resolve to demand interventions to eradicate those aspects of technologies that are threatening our psychological, social, economic and political health.


Anderson, J. & Rainie, L. (April, 2018). The Future of Well-Being in a Tech-Saturated World. Retrieved from