Processing And Learning From Video
I recently read an online article suggesting that videos are best for teaching. The article said (lightly edited for anonymity, as I would rather not embarrass anyone): “We know from research that videos make processing and memory recall more efficient. They appeal to a wide audience and allow each user to process information in the way that’s best for them. No wonder video is best for teaching!”
I’ve seen similar—and usually far too simplistic—comments. Video is a good tool for some instructional purposes, and less so for others. And even when video is a good tool, it is far less so when it isn’t designed well. The edited quote implies that videos are good because they appeal to different learning styles, and although the learning styles myth has clearly been debunked (Pashler, 2008), this myth has achieved zombie status, and refuses to die. In Part 1 of my series on video and learning, I discussed choosing media and tools such as video for digital instruction. Some of the important points I made include:
- The purpose of this series of evidence-informed articles is to answer the question, “how should we create instructional videos to better foster learning?”
- Although some research often shows positive effects for video on learning, other research shows videos too often aren’t watched or are only partially watched.
- The best way to select digital tools and technologies is to match important teaching and learning activities to the tools that best support them. Video doesn’t support all important teaching and learning activities so many (most) times, we need more than one tool.
- Content and social interactions are often best supported by very different tools. Research points to the need for both content and social interactions, so again, we need more than one tool.
- While video has many strengths (such as the ability to stop, start, and review), it also has challenges (such as the ability for us to seemingly watch it without processing the content).
In Part 2 (this article), I’ll discuss how humans process video and the implications for designing it so we can learn from it. Although there is research showing video can help learning, poorly designed video can make learning harder, and we must avoid these poor designs! I’ll start by discussing how we process multimedia, especially video, and what can go wrong in poorly designed video instruction. Then I’ll discuss what research tells us about making video better for mental processing. Since better processing of video is critical to remembering, understanding, and use, knowing what we need to do is truly a big deal!
Processing Video For Learning
Multimedia, such as video, is defined as static or moving images plus auditory information presented simultaneously. Combining media types to create information or instruction—such as text, static images such as photographs and drawings, moving images such as video and animations, and audio—are common forms of multimedia. Instructional presentations showing static or moving images and auditory information simultaneously is referred to as multimedia learning (Mayer, 1997; 2001).
Mayer’s (2005) cognitive theory of multimedia learning (CTML) tells us how we process and learn from multimedia instruction. CTML is built on three assumptions: dual-channel, limited capacity, and active processing (Mayer, 2005).
- Dual-channel assumption
The dual-channel assumption tells us there are two separate channels—visual and auditory—that process information in working memory. The visual channel processes static and moving images as well as written words. The auditory channel processes narrations and other sounds. - Limited capacity assumption
The limited capacity assumption explains that each channel has a limited capacity to process information at any given moment. Learning is hindered or stopped when limits are exceeded. - Active processing assumption
The active processing assumption tells us that to learn, participants must actively process what they are watching and hearing (Mayer, 2005; 2009). Video may appear to be passive because we don’t see people mentally processing. But if they are actively mentally processing (making sense of the content, organizing it, integrating it with prior knowledge) it isn’t passive.
Working memory is where visual and auditory processing occurs, but it is very limited in amount and duration. Long-term memory is where information is stored for long-term retrieval and use, but the process of getting information into long-term storage is not simple. I discuss the process from seeing and hearing to processing to storing in my webinar, which you can watch for free.
Image 1: Patti Shank’s webinar recording
When participants aren’t processing the video or aren’t watching the video, they cannot learn from it. And unfortunately, research shows that this is commonplace. Guo and colleagues (2014) researched participant video viewing in four edX MOOCs, analyzing seven million video-watching sessions. When participants watched videos less than six minutes long, they watched the whole video. As videos lengthened, participants watched less and less. For example, the median watching time dropped to near 50% for 9–12-minute videos, and to about 20% for over 12-minute videos (Guo et al., 2014).
Video can enhance learning because processing with two channels can reduce cognitive load and help watchers integrate associated visual and auditory information. Those of us who build multimedia learning such as video can also help participants manage cognitive load, by editing the video to take out unnecessary sources of mental effort and add in effects that improve attention and understanding. For example, we can edit out unnecessary information and add annotations to cue participants to critical parts of the video. And participants themselves can manage cognitive load by stopping, starting, pausing, and reviewing the video. These user controls are extremely advantageous for learning from video (Tallent-Runnels, 2006).
Video not designed well may make it harder to adequately focus on or process video content (Costley and Lange, 2017; Mayer, 2014). We must therefore design video in alignment with how we process multimedia. Mayer (2014; 2020) introduced a series of multimedia learning principles to help us better design multimedia in three categories of learner processing: extraneous processing, essential processing, and generative processing. Extraneous processing damages learning and should be avoided. Essential processing is caused by the complexity of the materials and should be managed. And germane mental processing helps us more deeply understand and should be encouraged.
Multimedia Design Principles
The right kinds of mental processing are critical to learning. Mayer’s (2020) multimedia learning design principles are categorized into three types of mental processing: extraneous processing (harmful), essential processing (needed), and generative processing (very helpful). These principles are used to avoid extraneous processing, manage essential processing, and improve germane processing during instruction.
The information in the next three sections is adapted from Mayer’s third edition of Multimedia Learning (2020). In this book, he cites the wide variety of research on which his principles are based and the main conditions for these principles. If you want this additional information, I highly recommend getting the book.
1. Avoid Extraneous Processing
Extraneous mental processing wastes our limited processing ability and does not assist with meeting instructional goals. It leaves less mental effort available for essential and germane processing, which help learning. Here are five multimedia principles that help avoid extraneous processing.
- Coherence principle
The coherence principle says we learn best from multimedia when we include concise, critical material, and don’t include expanded, less critical material. This means we should generally avoid interesting but irrelevant details and materials that are distracting and not needed (such as background music). When unnecessary auditory information is added, for example, it competes with the narration for processing in the auditory channel. This means less capacity for paying attention to narration. - Signaling principle
The signaling principle says we learn best from multimedia when we add visual cues (e.g., an arrow) or auditory cues (e.g., a narrated sentence about what we are about to discuss) highlighting the organization of critical material. Signaling reduces the need to try to figure out what is critical and what is less so. - Redundancy principle
The redundancy principle says we learn best from multimedia when using images and narration rather than images, narration, and printed text, especially when the lesson is fast paced. When narration explains images, people do not have to use additional mental effort to go back and forth between the images and the printed text. - Spatial contiguity principle
The spatial contiguity principle says we learn best from multimedia when corresponding words and images are physically close together. When corresponding words and pictures are close, people do not have to use additional mental effort to search the page or screen to see what goes together. - Temporal contiguity principle
The temporal contiguity principle says we learn best from multimedia when narration and images play at the same time, rather than one after another. When corresponding narration and images are presented at the same time, people are better able to make mental connections between them.
An example of applying the spatial contiguity principle is shown below. The top illustration of clarinet parts requires you to match the numbers with the part names below, adding unnecessary mental effort. The spatial contiguity principle has been applied to the bottom illustration to make it easier to view the name of each piece without having to match them up.
Image 2: Wikimedia Commons (Gisbert König, CC0), adapted by Patti Shank
2. Manage Essential Processing
Essential processing is the mental processing needed to deal with the complexity of the materials while in the working memory. Essential processing is therefore needed to learn. Here are three multimedia principles that help manage essential processing.
- Segmenting principle
The segmenting principle says that we learn best from multimedia when it is presented in user-paced segments. This means that learning materials should be chunked into smaller, coherent sections, and participants should be able to control playback, especially when the material is complex and fast paced. This will help people process the material they are currently watching/hearing before moving on. - Pretraining principle
The pretraining principle says that we learn best from multimedia when we know the names (terms, definitions) and characteristics of important concepts first. Pretraining can help manage essential processing, by working through pretraining elements before main lessons. - Modality principle
The modality principle says that we learn best from multimedia when using images and spoken words rather than images and printed words. Images and spoken words use both channels (visual and auditory) instead of just one. In a narrated presentation, spoken words are processed by the auditory channel, allowing more effective processing of images in the visual channel.
3. Encourage Germane Processing
Germane processing is the mental processing needed to deeply understand and is related to the amount of effort the participant is willing to expend. These four multimedia principles are meant to encourage germane processing.
- Personalization principle
The personalization principle says we learn best from multimedia when we use a conversational style, rather than a more academic or formal style. When people feel that we are talking to them, they are more likely to put effort into making sense of what is being said. - Voice principle
The voice principle says that we learn best from multimedia when the narration is spoken in a pleasant, human voice, rather than in a machine voice. When people feel that we are talking to them, they are more likely to put effort into making sense of what is being said. - Embodiment principle
The embodiment principle says that we learn best from multimedia when an on-screen instructor uses social gestures, body movements, and facial/eye expressions while explaining. When people see these cues, they are more likely to put effort into making sense of them. - Generative activity principle
The generative activity principle says we learn best from multimedia when we are involved in learning activities such as summarizing, imagining, self-testing, self-explaining, teaching, or enacting while learning. These activities improve mental processing during learning and help people integrate new learning with relevant prior knowledge from long-term memory.
If the principles for germane processing look different than you remember, Mayer made some evidence-informed changes in his latest version of Multimedia Learning (2020).
Got It?
The information in this article is complex, so let’s use some multiple-choice questions to remember key points. Answers are at the end of the article.
Q1. What is multimedia learning?
a) The use of static or moving images, plus auditory information, that are used at the same time
b) Simultaneous static or moving images and auditory information used in instructional messages
c) Moving images (such as video or animation) used to deliver engaging information or instruction
Q2. What does the dual-channel assumption tell us?
a) We have two separate channels in sensory memory to process information: visual and auditory
b) We have two separate channels in long-term memory to process information: visual and auditory
c) We have two separate channels in working memory to process information: visual and auditory
Q3. Under which of these conditions are we using both channels at once?
a) Video with simultaneous narration
b) Animation with written text
c) A static image with written text
Q4. The limited capacity assumption tells us that:
a) We can process more when we use only one channel at a time
b) We are unable to process more than one visual object at a time
c) Each channel has a limited capacity to process information
Q5. The cognitive theory of multimedia learning (CTML) says we need to design multimedia such as video according to which assumptions about how our minds process multimedia?
a) Extraneous, essential, and germane processing
b) Dual-channel, limited capacity, and active processing
c) Sensory, working, and long-term processing
Q6. Learning requires which type of effort to store what we’ve learned for later use?
a) Physical effort
b) Mental effort
c) Both physical and mental effort
Q7. Where do we store what we have learned, so we can use it later?
a) Long-term memory
b) Working memory
c) Sensory memory
Q8. Do Mayer’s multimedia design principles tell us it is better to have concise and only relevant learning materials, or to also include added and expanded content?
a) Added and expanded
b) Concise and relevant
c) Depends on participant age
Q9. Do Mayer’s multimedia design principles tell us it is better to use narration to describe visuals or to use written text to describe visuals?
a) Narration should describe visuals
b) Written text should describe visuals
c) Narration or text can describe visuals
Q10. Designers and instructors should minimize which type of mental processing?
a) Essential
b) Germane
c) Extraneous
Next Time
This article discussed important insights about how we process multimedia, including video. Mayer’s principles for designing multimedia/video were presented so we know the best ways to process multimedia learning effectively. In the next article, we’ll specifically discuss application of design according to how people process video.
Correct Answers:
1(b), 2(c), 3(a), 4(c), 5(b), 6(b), 7(a), 8(b), 9(a), 10(c)
References:
- Brame, C. J. 2016. “Effective educational videos: Principles and guidelines for maximizing student learning from video content.” CBE Life Sciences Education 15 (4): 1–6.
- Çeken, B., and N. Taşkın. 2022. “Multimedia learning principles in different learning environments: A systematic review.” Smart Learning Environments 9 (19).
- Costley, J., and C. H. Lange. 2017. “Video lectures in e-learning: Effects of viewership and media diversity on learning, satisfaction, engagement, interest, and future behavioral intention.” Interactive Technology and Smart Education 14 (1): 14–30.
- Guo, P. J., J. Kim, and R. Robin. 2014. “How video production affects student engagement: An empirical study of MOOC videos.” [email protected] ’14 Proceedings of the First ACM Conference on Learning at Scale. New York: ACM, 41–50.
- Ibrahim, M. 2012. “Implications of designing instructional video using Cognitive Theory of Multimedia Learning.” Critical Questions in Education 3 (2): 83–104.
- Lange, C., and J. Costley. 2020. “Improving online video lectures: Learning challenges created by media.” International Journal of Educational Technology in Higher Education 17 (16).
- Mayer, R. E. 1997. “Multimedia learning: Are we asking the right questions?” Educational Psychologist 32 (1): 1–19.
- Mayer, R. E. 2001. Multimedia Learning. New York: Cambridge University Press.
- Mayer, R. E. 2005. “Cognitive Theory of Multimedia Learning.” In The Cambridge Handbook of Multimedia Learning, edited by R. E. Mayer. New York: Cambridge University Press, 31–48.
- Mayer, R. E. 2009. Multimedia Learning. 2nd Edition. New York: Cambridge University Press.
- Mayer, R. E. 2014. “Cognitive Theory of Multimedia Learning.” In The Cambridge Handbook of Multimedia Learning, edited by R. E. Mayer. New York: Cambridge University Press, 43–71.
- Mayer, R. E. 2020. Multimedia Learning. 3rd Edition. New York: Cambridge University Press.
- Mayer, R. E., and R. Moreno. 2000. “A Coherence Effect in Multimedia Learning: The Case for Minimizing Irrelevant Sounds in the Design of Multimedia Instructional Messages.” Journal of Educational Psychology 92 (1): 117–125.
- Mayer, R. E., and R. Moreno. 2003. “Nine ways to reduce cognitive load in multimedia learning.” Educational Psychologist 38 (1): 43–52.
- Paivio, A. 1971. Imagery and Verbal Processes. New York: Holt, Rinehart and Winston.
- Paivio, A. 1986. Mental Representations: A Dual Coding Approach. New York: Oxford University Press.
- Pashler, H., M. McDaniel, D. Rohrer, and R. Bjork. 2008. “Learning styles: Concepts and Evidence.” Psychological Science in the Public Interest 9 (3): 105–119.
- Risko, E. F., N. Anderson, A. Sarwal, M. Engelhardt, and A. Kingstone. 2012. “Everyday attention: variation in mind wandering and memory in a lecture.” Applied Cognitive Psychology 26: 234–242.
- Rudolph, M. 2017. “Cognitive theory of multimedia learning.” Journal of Online Higher Education 1 (2).
- Shank, P. 2018. Should We Use Background Music With Instruction? No.
- Shank, P. 2022. Creating Better Video For Learning, Part 1
- Sweller, J. 1989. “Cognitive technology: Some procedures for facilitating learning and problem-solving in mathematics and science.” British Journal of Educational Psychology 81 (4): 457–466.
- Sweller, J. 1994. “Cognitive load theory, learning difficulty, and instructional design.” Learning and Instruction 4 (4): 295–312.
- Sweller, J., J. J. G. van Merriënboer, and F. Paas. 1998. “Cognitive architecture and instructional design.” Educational Psychology Review 10: 251–296.
- Tallent-Runnels, M. K., J. A. Thomas, W. Y. Lan, S. Cooper, T. C. Ahern, S. M. Shaw, and X. Liu. 2006. “Teaching courses online: A review of the research.” Review of Educational Research 76 (1): 93–135.
