Visual Learning for Science and Engineering

[Timothy Gangwer]

A Visual Learning Campfire - Snowbird, Utah, June 1 – 4, 2002

Principal Authors: Ann Marie Barry, Drew Berry, Steve Cunningham, Julianne Newton, Marla Schweppe, Anne Spalter, Walter Whiteley, Rick Williams, contributions from the entire group

Edited by: Judith R. Brown

The computer is forcing a merging of disciplines

Introduction

Visual learning is especially important in the areas of engineering and science education, since students must, more than ever, learn to think visually and communicate their ideas visually to both peers and the general public. A Visual Learning Campfire (a highly interdisciplinary workshop-style event in the ACM SIGGRAPH and Eurographics co-sponsored Symposia Program) was held in Snowbird, Utah, June 1 – 4, 2002. This three-day workshop brought together researchers and practitioners from engineering, science, mathematics, psychology, computer science, art, design, photojournalism, and other areas where images play an important role in communicating ideas and enabling education. The twenty-six attendees represented eight countries.

Figure 1. A drawing of the view at Snowbird by Ken O'Connell (click for larger version)

A major goal of the campfire was to develop a white paper on the status of learning and the needs for the future in visual learning in science and engineering, including plans for future college curricula or program development. Since all participants submitted position papers, we started with four groupings, based on the position papers and interests of the attendees:

• story telling and visual images
• thinking and images (became visual cognition and neuroscience)
• design vocabulary (became design process in science and engineering)
• visual learning as a field (became technical visual thinking and communication)

 

Executive Summary – Visual Learning for Science and Engineering

Visual learning is the use of images and animations to enable and enhance learning at all levels.

Visual learning methods are: opening up new ways of problem solving providing new ways to think about science and engineering enhancing the education and practice of science and engineering

 

Visual learning is especially important in engineering and science because students, more than ever, must learn to think visually and communicate their ideas visually to both peers and the general public. This multi-disciplinary workshop brought together researchers and practitioners from engineering, science, mathematics, psychology, computer science, art, design, photojournalism, and other areas where images play an important role in communicating ideas and enabling education.

Our conclusions are that: Visual thinking, with particular connections to collaborative learning methodologies and distance learning or virtual learning environments, is crucial to the future of learning. We need to find ways to articulate and interpret the nuances of visual worlds. We need to collaborate across disciplines to articulate this methodology. We can blend old and new ways of knowing to build new "languages," or symbol systems, of visual communication. A visual language can communicate most effectively in some situations, just as verbal, mathematical, or musical "languages" communicate effectively in other situations. All science and engineering students should have some exposure to creating visualizations. Any science program should begin with a firm grounding in visual theory and practice from an intuitive perspective.

We strive to clarify science and engineering concepts, drawing on the successful approaches to problem solving, ways of thinking, and communication skills that have been developed in different disciplines. By integrating three-dimensional visual thinking into existing critical thinking studies across disciplines at all levels, we can create more effective learning environments and accentuate the visual expression and thinking of scientists. By providing course work on creating visualizations within existing science and engineering programs, students will be exposed to the basics of art and design and to a wide variety of visualizations.

Design can contribute to science and engineering education. In order to adapt good design principles and experience to science and engineering, students must learn skills outside their areas of specialty, thus gaining flexibility of perception and thinking and developing visual intuition. By gaining the ability to address problems through different thinking and problem solving skills, students can gain a new understanding of science concepts.

Also, advances in neuroscience within the last decade include better understanding of how the brain processes visual information and events. Knowledge in this area has grown exponentially due to the development of technologies that allow us to observe the brain in action, such as functional MRI. Cognitive neuroscience, particularly in those areas involving the intersection of emotion and higher mental processes, is therefore one of the most significant growth areas of knowledge in visual perception, thinking, and communication. Many assumptions are being shattered, and many surprising insights are being revealed.

One important curricular technique in the process of visual education is to permit students to engage in wide explorations. Stories illustrating core concepts provide a means for students to identify with the topic at hand, and interactive technologies, such as multimedia, virtual reality, and augmented reality, provide new ways of exploring.

Implications in terms of classroom pedagogy include the following:

• Lecture is the least effective way of reaching the whole learning being. To tap into the emotional learning systems, we must engage as many of the senses as possible.
• We must use visuals that suggest and reveal patterns, allowing students to explore, engage in, and complete the meaning of a scientific concept.
• For effective learning, the right brain, and preconscious visual systems, must be integrated with left brain attention through perceptual alerts such as change, color, aesthetics, and story.

 

Figure 2. Right brain/left brain learning model, courtesy of Walter Whiteley (click for larger version)

 

Overall Conclusion and Goal There is a field of computer graphics-based visual learning, and our specific goal is to set good guidelines for visual learning in science and engineering. Towards this end, we have looked at the components in a one or two semester course, as well as a studio design course for scientists and an interdisciplinary course that begins the task of developing both rational and intuitive intelligences. This field includes the following: visual cognition and neuroscience semiotic background, including applications of linguistic theory to the visual world social implications, as shown by the dominance of visuals in today's society visual communication, such as graphic design and fine arts, that provide underpinnings for visual education curriculum issues

Visual cognition and neuroscience

The future of education for visual learning depends on our awareness and utilization of the brain processing that underlies all visual functioning.

The model for brain processing is illustrated by the right brain/left brain learning model shown in the Executive Summary, provided by Walter Whiteley. This is primarily a visual metaphor for critical alternations in modes and levels of thinking which comes from his reflections on learning in geometry and builds on other hierarchical models. The selection of modes of work, visual vs. verbal, or integrated combinations of the two, is described as "meta cognition." This skill at picking your mode and knowing when to switch modes, is identified as a central skill of good problem solvers in mathematics.

We need to develop creators and users of the tools of visual thinking by teaching students to understand and effectively recognize, select, and use the appropriate cognitive modalities for the appropriate tasks. By using visual aspects of various intelligences and learning styles, we can develop exercises that promote the recognition, understanding, effective selection, retention, and use of appropriate cognitive modalities in one or more dimensions, in motion, and in interactive formats. Thus, we will support associative, imaginative, creative, and abstract thinking skills.

There are a number of basic misconceptions about visual perception and communication, as shown below. In contrast, we also provide basic principles that guide design, selection, and use of visuals from a neuroscientific viewpoint.

Basic misconceptions about visual perception and communication We see what is there. You see what I see. Mind and consciousness are the same. Reason is the primary cognitive experience. The tools for analyzing and understanding linguistic and logical processes are adequate for analyzing and understanding visual processes. If you are not conscious of using visual perception, you are not using it. I cannot be held responsible for unintended consequences of my use of visual media. Visual media has no effect on me. I can control the effect of visual media. Three-dimensional understanding is built from two-dimensional understanding. Visuals are trivial – you either get them or you don’t. Visuals are hard – if you don’t get them, there is nothing you can do.

 

Basic principles that guide design, selection, and use of visuals from a neuroscientific viewpoint We cannot design visual communication nor understand the meaning of visual communication fully unless we understand some of the basic neuroscientific underpinnings of visual cognition. Visual learning is facilitated by mechanisms that best fit innate brain mechanisms. Most visual learning occurs outside of our conscious awareness. Consciousness is informed after the fact. Visual processes form implicit (unconscious) memory that guides behavior. The emotional system makes no distinction between actual, mediated (generated through media such as print, radio, or TV), or imagined experience. The most important visuals in science are the images in our minds. We can change what we see through experience, reflection, imitation, and finding what works. Many visuals in science are layered on top of kinesthetic processes, and kinesthetic processes share many cognitive characteristics of visual processes.

 

Figure 3. The cognitive group at work, with the courses group in the background Cognitive Group (left to right) Walter Whiteley, Rick Williams, Ann Marie Barry, Anna Ursyn

 

Visual cognition and neurobiology

Within the last decade, the understanding of how the brain processes visual information and events has grown exponentially due to the development of technologies that allow us to observe the brain in action, such as functional MRI. Cognitive neuroscience, particularly in those areas involving the intersection of emotion and higher mental processes, is therefore one of the most significant growth areas of knowledge in visual perception, thinking, and communication. Many assumptions are being shattered and many surprising insights are being revealed, as shown in the sets of basic misconceptions and the guiding principles above.

The process of visual perception involves several basic parts, including the sensing of information, the use of past experience, and the processing of information along dual pathways. An understanding of this process is essential to realizing the power of visual images to move us emotionally and behaviorally, and to influence our conscious thought.

First, raw information is gathered from the external world in the form of light falling on and reflecting from surfaces. Both cognitive and emotional systems work in parallel, and both store separate memories of the experience. Next, templates of past experience are compared by executive functions in working memory to what is experienced now. These templates, built and stabilized by neural circuits firing again and again, act as maps for understanding new stimuli and events. In the process, emotional feelings, generated by different subsymbolic systems and involving many brain functions, provide the vast amount of material. Feelings, unconsciously developed and processed, prepare us to cognitively understand what we see.

The idea that visual communication operates cognitively, without the need for reason and before the rational mind is activated is now well documented in the literature of visual cognition, neurobiology, and communication theory. Cognitive neuroscientists and other researchers in many disciplines are providing quantitative data that supports the more intuitive sense about the complexity and significance of visual communication that has been expressed by many artists and scholars. For instance, in 1986, Joseph LeDoux explained the pre-rational nature of visual cognitive processes from a neurobiological perspective. He said “The sensory signals from the eye travel first to the thalamus and then, in a kind of short circuit, to the amygdala before a second signal is sent to the neocortex,” the structure of the brain where rational thought is processed. [LeDoux, 1986, pp. 237-248, 1996]. Antonio Damasio and others have furthered this concept with empirical evidence that suggests that the pre-frontal lobes of the brain are the repository of the unconscious memory that generates “…nonconscious biases (that) guide behavior before conscious knowledge does” and that rational behavior may be dependent upon access to unconscious biases [Bechara & Damasio, 1997, pp. 1293-1295, Damasio, 1999].

The integration of the work of these and other cognitive neuroscientists defines a relationship between the amygdala and pre-frontal lobes suggesting that they work together to mediate and guide pre-conscious behavioral motivations. The amygdala provides the more spontaneous, rudimentary response, and the pre-frontal lobes provide the more sophisticated, synthesistic response, both in pre-conscious formats before the rational, neocortex receives the impulse.

Simply put, visual cognition operates on pre-conscious levels to process visual information into knowledge that motivates behavior before the conscious processes of the neocortex receive the information.

This pre-rational, predominately pre-conscious cognitive ability is called “intuitive intelligence” because intuitive means to attain to knowledge without the use of reason. It is believed that visual intelligence is the primary intuitive intelligence. This does not suggest that visual intelligence is not learned or that visual information cannot be used rationally, but that the initial, primary visual cognitive response is pre-conscious. The subsequent integration of this intuitive, visual cognition with conscious cognitive processes generates expressions of whole-mind cognition that has the potential to foster greater creativity, more powerful cognitive and problem solving abilities, and balance between quality and quantity.

Any science program should begin with a firm grounding in visual theory and practice from an intuitive perspective. Traditional rational approaches to science must be infused with new work in intuitive intelligence and visual learning in order to fully open the window to creativity and visual intelligence. More details on the curriculum that supports this are given in the "curriculum" section below.

There are other, more conscious layers of ‘visual thinking’ as a mode of thought. Aspects include associational logic, the use of transformations to modify one image to another so we can ‘see' other associations, decomposition (visual analysis), and pulling pieces together into a new whole (synthesis). These visual skills can be learned and improved if we put in the time for conscious practice and guided experience.

We noted the principle that we can change what we see through experience. The book, "Drawing on the Right Side of the Brain," by Betty Edwards, makes the point that changing what we see is a central task of learning to draw. However, this possibility applies much more widely, and the ability to see in different ways is a major goal of ‘visual literacy’ and ‘visual education.’ Experts in science 'see' differently than novices. So, we need to create techniques to make these internal processes visible to the novice, and to find ways for students to practice these conscious and unconscious processes.

Story telling also addresses neuro-perceptual issues, contributions from art and design, and social concerns of diversity and communication.

Neuroscientist Gazzaniga (who refers to the location of the core story telling function of the brain as the "interpreter") and Ramachandran (who refers to this location as the "general") posit the existence of a module in the brain that is as old as humanity itself. The purpose of this module, built into the left hemisphere, is to give a unified sense of self to the being. It works by telling stories. The module weaves a story of the self, and it is the most basic function of the cortical part of the brain. It is the most basic way we communicate with one another.

Semiotic background

A basic introduction to semiotic theory and analysis would enable students to ask important questions about visual representations.

Semiotics is a philosophical/analytical theory of meaning and takes two forms: as a field of study in itself, and as a set of methodological approaches for analysis of signifying systems. It has been used within many disciplines. Fundamental to semiotics is the notion of the ’sign’ - being composed of the signifier (the representation of something) and the signified (the thing being represented), classically instantiated by language, where, for instance, the sign "red" is composed of the signifier -the word "red"- and the signified - the concept/percept "red." In story telling, we must consider cultural and cognitive differences in determining the signs and signifiers that encode the story.

One of the major criticisms leveled at semiotics is the primacy of written and spoken language as its principal model. Indeed, one of the principal challenges that confronts semiotics today is how to develop a convincing methodology and theoretical apparatus that can account for non-linguistic forms of meaning-making.

Like any form of knowledge, visual form becomes easier and more precise the more you do it: this is why it takes years for artists to develop drawing skills, for film-makers to develop unique filmic "languages," and for designers to refine designs. These principals of visual learning need to be applied in the area of scientific education. There is an assumption that pictures of things somehow give us "direct access" to them: that photographs are "realistic," that conventional narratives in films are like "real life," that certain types of diagrams and charts reflect more directly that which they propose to depict. It is important to understand the relative nature of representations and to understand that visual representations are artifacts embedded within a culture and within certain types of community within that culture.

In the development of a curriculum for teaching visual thinking to students of science and engineering, there could be no doubt as to the value to a basic introduction to semiotic theory and analysis. It would enable students to ask some important questions about visual representations, not only within the disciplinary area that they specialize in, but more broadly across disciplines and about the culture within which they live.

Part of the process of understanding is to find out how one knows things and how one can know things, and in pursuit of this, it is expeditious to be able to ask the right sorts of questions. In order to formulate these questions effectively, it is often useful to be able to step back from your object of enquiry and ask: "Exactly what am I looking at?" "How is it structured?" "How is it represented to me?" "How am I going to represent this?" and so on. Semiotic approaches can aid in this self-reflexivity in the process of interrogation, by causing us to ask questions of ourselves, not only as viewers, users, and consumers of images, but also as creators of them. In doing so, the very complex processes of visualization can be revealed to a viewer, and what so often seems simple can usefully be perceived as a complex structure, susceptible to analysis, and, crucially, susceptible to dialogue using mutually agreed-upon terminologies and methodological approaches. In this manner, semiotics can provide meaningful structural rigor to the understanding of images and their potentials of meaning.

Social and cultural implications

The ubiquity of visual materials may change the way we perceive and understand the world.

There are a number of questions we might ask about the social and cultural effects of the abundance of visuals in our lives. The ubiquity of visual materials may change the way we perceive and understand the world, including the fields of science and engineering.

As we learn to use visual learning effectively in science through all levels of education, we begin to include significant groups that were previously excluded from the practice of science. These visual learners have the characteristics (including characteristics associated with dyslexia and autism) of some of the most creative scientists of the past [West, Grandin]. The students who will engage in science and engineering through the big ideas and the larger context are also included through effective use of 'story telling' and visual forms in response to their questions [Tobias]. Taking advantage of the visual expression and thinking of scientists increases the development of science and engineering, as well as the effective communication within science. It also informs students about the character of practicing science, which is often highly visual.

Those who teach visual communication must go beyond the standard academic world. They must reach out to both working scientists and people in the schools. They can teach short courses on visual communication to working scientists and take feedback from those scientists to improve the teaching or the content of these courses; this feedback will inform the field and lead to better content and presentation of these ideas at all levels. At the same time, they must work with people in the schools to affect the curriculum at the grade school and high school levels, giving students an idea of the power of visual communication and visual problem solving for understanding science.

Techniques for better learning

Implications of neuroscientific finds in terms of classroom pedagogy include the following:

• Lecture is the least effective way of reaching the whole learning being. To tap into the emotional learning systems, we must engage as many of the senses as possible.
• We must use visuals that suggest and reveal patterns, allowing students to explore, engage in, and complete the meaning of a scientific concept.
• For effective learning, the right brain, and preconscious visual systems, must be integrated with left brain attention through perceptual alerts such as change, color, aesthetics, and story.

 

In accordance with the above implications, the following techniques should be used to enhance learning in science and engineering:

• engaging the attention through mild cognitive dissonance or ambiguity
• simplifying and abstracting to essential components
• eliminating distractions
• presenting materials in appropriate sequence beginning with the “big picture” context
• utilizing those aspects grasped immediately by the emotional system to further cognitive goals, such as color, shapes, angles, and 2-D or 3-D presentation
• remembering the perceptual tendency toward illusion, such as over-estimating height, assuming cause and effect, or assuming integral relationship
• approaching content experientially allowing for individual discovery and exploration
• using physical materials and 3-D objects for experience and experimentation
• taking the time to explore the multiple visual representations and interpretations that occur within a single classroom
• pausing to ‘unpack’ and repack compressed images that are not ‘obvious and transparent’, so students can learn the layering and conventions
• taking advantage of natural creativity in perceptual process by encouraging analogical thinking
• playing with symmetries and other transformations of an object
• beginning with the most basic and universal symbols and shapes and metaphors from direct experience and progressing through cultural to the idiosyncratic
• using illustrations that are abstracted representations of complex materials
• using examples from the history of science that are important sources of visualization in science
• encouraging illustrated note taking, as well as taking notes focusing on the gist, and then filling in the specifics

 

Visual communication

Story telling enhances science education by drawing on our most ancient means of communicating, and we can adapt lessons from design practice to create design vocabularies for science areas.

We looked at the need for better ways to teach science and engineering to increasingly diverse groups and the use of visual learning to enhance the teaching of science and engineering. Story telling can be an effective method, and we need to determine the new paradigms for story telling using images, as well as how story telling can improve visual learning in science and engineering.

Story telling enhances science education by drawing on our most ancient means of communicating, in terms of both human cognitive processing and narrative structures, and in finding pathways through known and unknown territories of knowledge. Telling stories with images creates highly effective paths for learning, and we are now developing new ways to tell stories interactively, via virtual reality techniques, creating new pathways that invite more diverse groups to participate. Thus, incorporating story telling imagery into curricula facilitates creative, visual, and holistic thinking among a great variety of learners

Figure 4. Storyboard that illustrates types of story telling, by Ken O'Connell (click for larger version)

 

Curricular innovations use the traditional guidelines for telling stories to develop new immersive story telling.

Story telling has been ubiquitous through time, and although the tradition of story comes from oral culture, new techniques involve more pictures than text. This in turn invites more diverse groups to tell the stories, making for better stories and stimulating creative thinking in all disciplines, including science and engineering. One important curricular technique in the process of visual education is to permit students to engage in wide explorations. As students explore via immersive story development, they extend their understanding of concepts. A story, where "story" is defined as a method for anything we want to communicate or teach, can be very powerful.

Power of the story engages the viewer facilitates common experiences focuses on the subject enables learner to identify with new ideas brings clarity broadens ways of knowing and creating encourages conscious awareness becomes a dynamic, organic "game" for learning can be both a means and an end provides a logical, organized structure enriches the learning environment by inviting diverse voices

 

Figure 5. Story telling group  Back row, left to right, Clare O'Leary, Julianne Newton, Berndt Lundz, Michael Hall, Ken O'Connell; front row, Clare Raby, Matylda Czarnecka

 

There are five key elements of a good story: audience, content, motivation, timing, and semiotics. The importance of the audience is that telling a story with no one around is not story telling. You have to be sensitive to the audience because the goal of a story is to give them new knowledge. We can use a story to help move people from one understanding to another understanding without awareness.

Content is crucial because stories must be about something. If it's a scientific concept, the scientist needs to learn how to build a context, how to segue into the topic, how to develop the best sequence to explain or explore the topic, and finally how to bring it together, make the concept exciting, and relate it to the larger context of scientific ideas.

Motivation is achieved when the story teller tells the story so that the audience feels "it's about us." Identifying with story components is the greatest motivation for attending to the story. The great advantage of interactive stories is that, as the actor, you become part of the story, and you help tell the story. Stories can engage many otherwise unmotivated learners.

It is important to have the right timing and tell the story with attention to progression (beginning, middle, and end). The right pacing holds the audience attention. People have different strengths in dealing with the three parts of the story: beginning, middle, and end. Some are very good with the beginning or setting the context or stage for the story. Others are very good at content development. Still others are very good at bringing together the summation or finish of the idea. Because it is rare that anyone is good at all three parts, small groups or teams working together strengthen a project.

A story teller must consider cultural and cognitive differences in determining the semiotics (signs and signifiers) that encode the story, no matter what the content. Immersive worlds have a unique opportunity to offer multi-vocalities, not only in the text of the story but in the symbols of class and culture – in the signifiers of the users. For example, the "women's gaze" is particularly pertinent if we are trying to attract girls and women into science and engineering. (A "women's gaze" refers to a woman's point of view, including the issues of power, subordination through visual media, objectification of the body, and other characteristics that are usually ascribed to the feminine, with a focus on relationship rather than domination.) In this case, it is important to identify ways to connect with young women's understanding of the context in their lives of learning science and engineering subjects.

Immersive story telling

Interactive technologies, including multimedia, virtual reality, and augmented reality, can provide new ways of story telling. The audience can not only consume the story, but can become an active part of the story. Immersive stories can be highly non-linear, which means that the story can evolve depending on the actions of the participants. For a teaching environment this has the additional value that:

• The story can change depending on the knowledge of the learner, focusing on new topics instead of already-known things.
• Learners have several ways to get to the same goal, giving learners with variable skills more possibilities for solving the task.
• Groups can be involved in the story in order to foster teambuilding

 

Design process for visual learning

Traditional design can contribute to science and engineering education. As part of the design process, students learn to express solutions to problems by creating models of the solution and developing communications that grow from the audience and the message being communicated. We can adapt lessons from design practice to create design vocabularies for science areas.

Successful design depends on a deep understanding of the message and the audience for the message, and the design decisions flow from these understandings. There are ways in which design for science and engineering will differ from some of the more traditional design areas, however, so there are challenges in understanding the needs of science and engineering and in adapting good design principles and experience to these fields.

The following two diagrams represent common approaches to design problems:

(click for larger version)

 

Both of these diagrams represent the interplay of different parts of the design process. In the left-hand diagram, a problem is approached by creating a model, developing an image, and evaluating the extent to which the image solves the problem. If it does not do so fully, then the evaluation attempts to find the point at which the solution breaks down, and we return to that point for further work. In the right-hand diagram, an idea, tools, and mental processes are brought to bear on a product, and they interact with each other to develop the product satisfactorily. Advancing the skill levels of all three vertex topics leads to mastery of the design process. In both cases, there is not one single path to a successful design. Rather, the design is a process that leads to a successful conclusion.

Another traditional concept from information design is that of encoding and decoding ideas, as illustrated in the diagram below. In this concept, a sender has an idea to communicate to a receiver. In order to communicate the idea, it is first encoded by the sender as language, image, sound, or another form, and it is then sent to the receiver via some form of media. The receiver must then decode the message to get the idea. A design is successful if the receiver’s decoding can successfully re-create the original idea.

(click for larger version)

 

In this model, effective communication involves more than one set of codes that are delivered simultaneously through different media. The media may be a mixture of formal codes, such as language, and informal codes, such as gesture. The multiple lines show that communication often takes place on several levels simultaneously. For example, verbal communication involves words, vocal inflections, body language, and gestures. In reading, the layout affects the reader's understanding, as well as the type size, line length, and font. In all media the code must be social; both the receiver and sender must be using the same system to encode and decode the message or the meaning will be lost.

Three Dimensions

In science and engineering, 3-D thinking predominates, and in the child’s visual development 3-D vision becomes primary around age two [Hoffman]. There is also increasing evidence that 2-D and 3-D thinking are different cognitive process, and the need to develop 3-D abilities is an important task in visual learning for science and engineering.

The question in science and engineering is not when we need to shift to 3-D, but when we would choose to move down to 2-D in the representation. Models of geometry learning, such as the van Hiele model, recognize the importance of passing through stages to learn to do proofs in plane geometry. This suggests that we need to pay attention to ensuring a solid 3-D base of experience for appropriate development of 3-D reasoning. This would include not just flat representations but work with physical models, and 3-D kinesthetic activities. If students arrive at a university classroom without that foundation, then it must be built up at that time, and not assumed.

Learning 2-D representations of 3-D objects requires extra layers of convention and coding, thus requiring extensive experience and learning. This also increases the chances of misunderstanding between teacher and learner, and makes some tasks of synthesis more difficult. There is a risk that experts, or learners, may confuse the representation with the actual structure. In the history of science, there is a 19th century example of expert chemists confusing 2-D representation of carbon with the physical structure of carbon – and resisting the switch to 3-D structure.

Also, in the history of science, and in anecdotal reports from classrooms, the most effective way of working with dimensions involves a high capacity to shift dimensions and reason among dimensions. [See Gooding’s writings on Faraday.] This powerful learned ability should be one goal of visual work within dimensions. Our base experience of visuals in 3-D is wrapped up in our larger experiences with objects, motion, and planning. We can learn significant design principles for visualization in 3-space from the design principles for 3-D objects, such as the work of [Norman].

The effect of stress

The most effective visual learning occurs in an experiential context, utilizes the workings of the emotional system to prepare for proper conscious cognition, and allows for play as the most effective preparation for and method of learning. Mild stress enhances conscious learning, but too much stress, especially for too long a time, prevents it. Stress speaks primarily to the emotional learning system, and there it works primarily in a negative way. Extreme stress, caused by too much different information, unrelated information, or information too rapidly introduced or presented within too short a space of time, adds to a negative emotional reaction and clicks in a fear response. This memory is engraved below the level of awareness and becomes conscious as an attitude toward or feeling about the situation or topic. This emotional memory is engraved by stress and is responsible for aversion attitudes toward the subject matter and, in the extreme, phobias.

On the other hand, having time to reflect, to integrate, and to sleep on new information stabilizes material in memory. Also, testing in the same mode and in the same learning circumstances aids recall through cueing.

Visual vocabularies

The first step in creating effective design in any field is to understand the audience and the message for the design. We believe there are indeed significant challenges in creating effective visual communication design in the sciences and engineering. However, we believe that there are instances of either general design principles or design experience in other fields that can be adapted to science and engineering. In doing this adaptation, there will be many alternatives that can be considered, so we would expect that there could be many alternate ways to create any particular design or set of designs. We must help students understand the number of options available in design and gain skill in the evaluation of different approaches, as we discussed above.

An example of this may be found in thinking of the meaning of color for the sciences. There are many ways to use color effectively: as an identifier (the red line and green line can be seen as different so they may be compared), as an indicator of relative value (synthetic color from a color ramp can be applied to elements such as surfaces to indicate the relative values of a parameter), as an indicator of exact value (again, colors from a color ramp can be used, although the user can usually only get an approximate value), or as a part of a synthetic scene where color is intended to show the actual color in the real analog of the scene. There are issues in having color as an identifier because there are (possibly unintended) secondary meanings of individual colors or of color combinations. There are also issues in having color as an indicator of relative or exact value because of differences in the ways color ramps are perceived by the audience. For example, color ramps that go from red to blue are often perceived as having meanings of heat, but may simply be used because of contrast, and rainbow color ramps can cause confusions in meaning. There are also different cultural meanings of color. So there are important considerations in how we use color for effective communication and in how color is associated in different ways with different vocabularies in various disciplines. Color is also used to attract or shift attention, a critical visual thinking skill. Programs like dynamic geometry problems use color and changes in color to guide students into more expert patterns of attention.

Design is important in the sciences and engineering because it provides the viewer with guidance in selecting information from the often complex set of materials in a problem. Good design helps the viewer find the clues that show specific things or things with a particular relationship in that complex environment.

Two design areas are less well covered by traditional design education: interface design and interaction design. Interface design is being rapidly developed by game designers. There is very good reason to look at these design decisions for games because game developers must create self-teaching interfaces that enable users to learn how to control game components quickly and fluently. Some of this new work is becoming available to the rest of us as the games community shares its work through traditional presentation and publication channels. Interaction design is probably best done by treating the interaction as if it were with a physical object and modeling the interactions we would have with that object. Design disciplines such as industrial and consumer product design have many things to teach us in this area.

Curriculum

Science and engineering students must be exposed to visual concepts of art and design.

Each of the groups produced some recommendations for visual learning curricula in science and engineering. One group defined a studio design course for science and engineering, a second group created a template for content within existing science and engineering programs, and a third group disussed a visual literacy course. The story telling group proposed a book on story telling for computer graphics that is detailed in Appendix B.

Studio design course

One approach to the issue of visual learning in the sciences and engineering is to create a course based on the methods of a studio design course in art and design, but focused on teaching visual communication and problem solving in the sciences. The studio design course is chosen as a model because students need to encounter visual communication and design concepts and examples whenever they meet communication issues. They need to see examples of both successful and unsuccessful design, along with a comparative analysis that says why one works and the other does not, and for the viable design(s) they need to understand why they work and what underlying principles make them succeed.

This studio model for teaching the design concepts underlying visual communication and visual thinking requires a great deal of collaboration between students. Its studio processes, such as team projects and critiques, help build both teamwork and the students’ abilities to evaluate design work. It should include a multifaceted approach to design, demand a variety of skills, and engage varied learning styles. It should be highly multidisciplinary so the students can get the variety of viewpoints needed to understand the design principles. It is important for students to understand the tools for visual design, to develop the skills necessary to effectively use the tools, and to make informed choices about the media and the tools appropriate to any visual communication task. For students in science and engineering, the course should focus on giving a holistic experience of nonlinear thinking rather than on a linear design process, and it should be accompanied by a survey of related principles from the literature.

Figure 6. Drew Berry presents work from the studio design group

A key concept in this education is that students should learn outside their areas of specialty to expand both their skills and their learning styles. For example, some football players learn ballet so they can expand their range of physical skills. Students in all areas, but especially in the sciences and engineering, should expand their visual vocabulary by experiencing different kinds of design and communication, and particularly by experiencing alternative visualization skills. This will lead to flexibility of perception and thinking and to visual intuition development. Revealing the hidden visual traditions of science, as well as thinking outside traditional science patterns, can have another benefit. These may give students the ability to address problems where the main issue is the understanding of a science concept.

As an example of the value of learning to think outside normal channels, in an experiment one group of Harvard medical students studied art history in an intensive way that included a great deal of visual analysis and learning visual cues, while a control group had no such course. Members of the experimental group were much better at listening and observing patient symptoms from visual and verbal clues. There are a number of teaching tools that are, or can be, developed to assist in teaching design issues, such as interactive textbooks, Web-based courseware, and computer-based laboratories and exercises.

Members of the studio design group were Drew Berry, Steve Cunningham, Sybille Hambach, Frank Hanisch, Mary Stieglitz and Steve Wroble.

Curriculum within a science and engineering program

Another group discussed visual learning in science and engineering from three angles:

• A program in an art and design school, similar to medical illustration but in support of science and engineering instead of medicine, where students learn to produce visualizations for science and engineering.
• A program in a computer science or information technology department where students learn to work in developing visualization materials for science and engineering.
• A program within science and engineering departments where science and engineering students learn about visualization and learn some of the basic skills for producing visualizations.

 

The first two cases were discussed only briefly. However, it was felt that this type of student would need expertise in the production techniques as well as experience in the areas of science or engineering at a rudimentary level.

In the art and design master's program, students would have a background in one of the content areas or would have a background in visual arts. Depending on the particular background, some courses in a degree program or bridge courses would be required to fill in the areas where the student did not have strong exposure. These would be either art and design courses or courses in science and engineering. Additional curriculum would include techniques used in creating visualizations in science and engineering, study of information design, techniques for working with a content provider, methods for testing the effectiveness of a visualization, and some courses in programming.

A master's degree in computer science or information technology would be similar, but would focus on providing programming support to visualization efforts. It's possible that a single degree would be more interesting, a degree that would be interdisciplinary and include courses in the art/design and programming areas. However, it's likely that students will be stronger in one area or the other.

Figure 7. The courses group at work

Courses group members, clockwise from left foreground: Peter Morse, Pavel Slavik, José Teixeira, John McDonald, Andy van Dam, Marla Schweppe. Not shown, Anne Spalter.

The curriculum within science and engineering programs is of particular interest. This option is to provide course work within existing science and engineering programs. This group felt that all science and engineering students should have some exposure to creating visualization and assumed that they will have some experience with visualizations in their other courses, as those visualizations would be used in presenting other content.

While, with this limited exposure, their skill level in this area might not be strong enough for them to excel at creating visualizations, they would at least have the appropriate vocabulary and some ability to converse effectively with support staff (educated in the other two programs perhaps) who would be creating the visualizations. Or they might decide to pursue further study in a program similar to the two described above.

In discussing what the content of a course of this type might be, it was felt that science and engineering students should be exposed to the basics of art and design including:

• elements
• principles
• basic drawing techniques
• hands-on exercises
• the critique process

 

See Appendix A for a full description of the context and suggested content of a course in technical visual thinking and communication.

Students should also be exposed to a wide variety of visualizations drawn from science and engineering. The course is designed as modules, rather than specifically as a quarter or semester long course, so that it can be adapted to various situations. (See Appendix A) Certain threads go through all the modules and represent ideas that will develop over duration of the course or courses. These threads are:

1. Establish basis of traditional 2-D, 3-D, and motion-based art (including story telling) and design concepts and processes.
2. Provide examples and motivation from science and technology from a wide diversity of applications.
3. Stress the power of images and visual thinking, as well as when not to use them.
4. Bring in relevant materials from perception and cognition literature.
5. Bring in relevant materials from semiotics.
6. Emphasize the role of planning and due diligence-type research.
7. Conduct critiques.
8. Establish metrics of success.

 

A visual literacy course

Asignificant challenge that education and culture face today is the challenge to redesign curriculum so that we develop individuals who understand and value the integrated use all of our cognitive abilities toward greater creativity in problem solving and in the development of culture. Toward this end, Rick Williams from the cognitive group has developed and taught a course that he believes begins the task of developing both rational and intuitive intelligences as equal and complementary components of whole mind cognition. Because all cognitive modalities have significant visual components, visual communication provides an ideal medium through which to teach theories of cognitive balance and is central to both the theory and creative exercises in this course design.

This interdisciplinary course integrates theory in art, neurobiology, psychology, education, visual communication, media and society and other disciplines with creative exercises that are designed to apply the theories in an Omniphasic model of balanced cognition. He has taught this course to more than 5,000 students in two major universities over seven years. The results, as indicated by student’s work and self evaluations suggests that learning to overcome our culture’s rational bias in favor of whole mind cognitive function can increase creativity and problem solving abilities in a wide range of activities and disciplines including art, communication, science and technology. Because all facets of education and culture benefit from enhanced creativity and problem solving abilities, this model should be equally applicable to curriculum in multiple disciplines and cultural and work environments.