Module 1: What is multimedia?

What is (or are) the media

Before launching into a study of multimedia, it seems reasonable to first understand what we mean by "media".

Definitions of media (and its singular form, medium) are as diverse as the disciplines they are used in. Consider the definitions below of medium and media provided by different sources.

(Note: Although "media" is strictly speaking the plural form of medium, media can be used as a singular noun.)

There is plenty of variety among these definitions; however, there is generally agreement that a medium carries, represents, or transmits something, but is not that thing itself. And what is that thing? In biology, it would be microorganisms or organic compounds, as indicated in Definition 3 of Table 1. Because you are a student of multimedia, the thing that is transmitted is information.

Because people mean different things when they talk about media, let's create some classifications that can help us share a common language when talking about the various phenomena that are part of media.

• Media modality: Text, audio, video, graphics, animation

• Media encoding type: Digital or analog

• Media storage material: Optical (CDs, DVDs), magnetic (typical hard drives), other (flash drives)

• Media transmission strategy: Electromagnetic (radio waves), optical, physical

• Mass media classifications: TV, radio, print, Internet, telephone/mobile

So now we have a more expanded vocabulary that we can use. Instead of saying, "The media should be held accountable for the accuracy of the news that they report," we can say, "The mass media should be accountable for the accuracy of the news that they report." Instead of observing, "Webpages often present information using different media," we can more accurately say, "Webpages often present information using different media modalities."

Using this idea of multimedia as simply meaning "different, unrelated media", consider the case of a student studying the subject of Newtonian physics using information as presented through different media modalities.

• Video: Watch a recording of professor Walter Lewin at the Massachusetts Institute of Technology give a lecture in front of a class of undergraduate students.

• Text: Reading about Newton's laws in a traditional college physics textbook.

• Animation: Watching a video that shows simulated interactions between physical objects

• Audio: Listening to a physics expert discuss Newton's Laws on an audio file recording, a podcast, or on a CD-ROM.

Multimedia, new media, rich media, digital media

When the term "multimedia" first started gaining currency, it was during a point when it was inconceivable to transmit and receive the quantities and types of data over the Internet that we now do today. Barfield (2004) recounts the history of the term, and how the Internet changed how we looked at information was delivered:

'Multimedia' used to mean the design of systems authored with tools such as Macromedia Director and distributed using cd-roms as a carrier medium. In the mid-1990s the developments surrounding the [I]nternet and the [W]eb... meant that the focus of multimedia development shifted from the static physical carriers like the CD-ROM to dynamic and updatable delivery methods on the web. Even now this shift is getting more and more pronounced, pushed along by the developments on the web, the increase in bandwidth available and the explosion in access by the public. Distribution based on CD-ROMS will always have a niche in the market, but the main focus of multimedia will be online content on the web... Many classical multimedia courses are introducing their students to the Web and including Web design and construction as part of the curriculum. This trend will gather pace as courses restructure to follow developments on the Internet. [emphasis added]

Multimedia from the point of view of one discipline

Let us look at the topics currently studied by people who identify themselves as multimedia experts. For instance, let's look at multimedia from the point of view of one discipline--computing science. In computing science, multimedia is "a research field that includes computing methods in which different modalities are integrated and combined, with the aim to take advantage from each data source." (Institute for Systems and Technologies of Information, Control and Communication, 2019) You can see this definition in action by reviewing the article topics accepted by the peer-reviewed journal, the Journal of Multimedia:

This extensive list helps us understand the broad range of academic and industry interests in multimedia, and we can see that disciplines such as engineering, computing science, psychology, law, and education all have a stake in multimedia. What that list allows us to do is look at definitions proposed by those who identify themselves as authorities in multimedia. Consider the following passage from the book Multimedia and Communications Technology (Heath, 1999):

If there is a term or phrase that has appeared in more diverse publications than any other over the last few years, it must be multimedia. The number of definitions for it are as numerous as the number of companies working on it. If this is the case, what is multimedia?

In essence, it is the use or presentation of data in two or more forms. The combination of audio and video in the humble television was probably the first multimedia application, but it has been the advent of the PC, with its ability to manipulate data from different sources and offer this directly to the consumer or subscriber, that has sparked the current interest.

As a result, multimedia for many people conjures up the image of a PC with a SoundBlaster card playing interactive games or searching through an encyclopedia with all the information supplied on a CD-ROM. While this is undoubtedly a valid multimedia application, it is only part of the story. This definition is pragmatic and based on reality; other prophecies for the industry have ranged from the televisions becoming PCs, and vice versa, and that one day, everyone will have a combined television-PC-phone-fax, capable of doing everything you could possibly want and more. Oh and yes, you will never buy software anymore, as you will be able to access it from your televisual-PC-phone-fax form the network and only pay for what you need!

The reality is somewhere between the extremes. Undoubtedly, with the ever-improvising ability of the PC to provide TV quality audio and video, the television and PC are becoming very close. Add the ability to provide graphical overlays and the difference is very small indeed. With cable TV companies providing telephone connections and the increasing combination of PC with a modem to access the Internet and thus provide an intelligent telephone, the forecasts for the universal widget are a logical progression.

Compare that passage with the following excerpt from Ralf Steinmetz & Clara Nahrstedt’s book, Multimedia Systems (2004):

Multimedia is probably one of the most overused terms of the 90s... The field is at the crossroads of several major industries: computing, telecommunications, publishing, consumer audio-video electronics, and television/movie/broadcasting. Multimedia not only brings new industrial players to the game, but adds a new a dimension to the potential market… Similarly, not only the segment of professional audio-video is concerned, but also the consumer audio-video market, and the associated TV, movie,and broadcasting sectors.

As a result, it is no surprise when discussing and establishing multimedia as a discipline to find difficulties in avoiding fuzziness in scope, multiplicity of definitions, and non-stabilized terminology. When most people refer to multimedia, they generally mean the combination of two or more continuous media, that is, media that have to be played during some well-defined time interval, usually with some user interaction. In practice, the two media are normally audio and video, that is, sound plus moving pictures. (Steinmetz & Nahrstedt, 2004)

Finally, consider this passage from Ze-Nian Li & Mark Drew’s book, Fundamentals of Multimedia (2005):

People who use the term "multimedia" often seem to have quite different, even opposing, viewpoints. A PC vendor would like us to think of multimedia as a PC hat has sound capability, a DVD-ROM drive, and perhaps the superiority of multimedia-enabled microprocessors that understand additional multimedia instructions. A consumer entertainment vendor may think of multimedia as interactive cable TV with hundreds of digital channels, or a cable-TV-like service delivered over a high-speed internet connection.

A computer science student [...] likely has a more application-oriented view of what multimedia consists of: applications that use multiple modalities to their advantage, including text, images, drawing , drawings (graphics), animation, video, sound (including speech), and, most likely, interactivity of some kind. The popular notion of "convergence" is one that inhabits the college campus as well as the culture at large. (Li & Drew, 2005)

Multimodality and interactivity

There are at least two points to note from the preceding excerpts. First, all three sets of authors suggest that “multimedia” was mobilized as a buzzword in the 1990s by technology and entertainment industries to push their respective (and largely profit-driven) agendas. The authors seem to suggest that there was no clear agreement of what properly lies under the umbrella of “multimedia”, since the stakeholders in the term differed in their approaches to integrating and innovating on mass media types, media transmission strategies, and media storage technologies.

However, we also do see a common thread among all three excerpts that is particularly useful for our attempt to understand multimedia from an instructional design perspective. Consider some selected excerpts from these passages, presented in the table below.

• Heath, 1999: the use or presentation of data in two or more forms.

• Steinmetz & Nahrstedt, 2004: the combination of two or more continuous media, that is, media that have to be played during some well-defined time interval, usually with some user interaction. In practice, the two media are normally audio and video, that is, sound plus moving pictures.

• Li & Drew, 2005: a more application-oriented view of what multimedia consists of: applications that use multiple modalities to their advantage, including text, images, drawing , drawings (graphics), animation, video, sound (including speech), and, most likely, interactivity of some kind.

The previous tabledraws out what these authors consider as multimedia when they consider media modalities. From this admittedly small selection of expert opinions, we can see that multimedia can be characterized by at least two general features:

1. Simultaneity of media modalities: Multimedia presents data simultaneously in two or more modalities (such as audio+video, text+images, animation+sound). The simultaneous use of two or more media modalities to present information is known as multimodality.

2. Interactivity: Users can affect the state of a multimedia product through their actions, which are mediated by some sort of interface.

(Note that Steinmetz and Nahrstedt's definition requires "continuous media" that have to be "played during some well-defined time interval", suggesting that multimedia also has to be time-based. For this course, we don't need to be bound by that definition.)

Analog and discrete information

Throughout the course of your studies, you will be learning concepts, approaches, and techniques from computing science and information systems that are relevant to the understanding, creation, and manipulation of digital media. We now examine a fundamental property of digital media--that it is encoded using discrete units of information.

Immersion, interactivity, narrativity, hypertextuality

A broader take on multimedia was proposed by Randall Packer in an article from 1999, and in the following activity you will read up on and restate his views, and then integrate them with your own prior understanding of multimedia.

After reading Packer's article, you will notice that in addition to interactivity and the simultaneous presentation of media modalities, Packer proposes that multimedia is characterized by immersion, an integration of disciplines, narrativity, and hypertextuality/hyperlinkedness. Immersion and disciplinary integration work together to envelop a user in the multi-sensory experience of a multimedia environment. Hyperlinkedness allows users (at least in theory) to find their way through the environment using a navigational logic that suits them.

What multimedia is–an individual view

We've gone through a lot of material, and you're probably still unsure about what multimedia is. Well, as we've seen, most people aren't sure either! Here's a set of pointers that I propose you use to understand multimedia.

• Multimedia is a term that was created in 1990s. Why is this important? Newer terms are also being used to refer to multimedia as it has been affected by changes in the landscape of computer, communication, and media over the past 20 years, and many recent developments--such as augmented and virtual reality--build on the creation of what is properly known as multimedia technologies.

• Multimedia doesn't have to digital... but it often is. For this course (and indeed, for the entire BAMS program) we are interested in digital multimedia, i.e., multimedia that was produced using digital tools.

• Digital multimedia involves the simultaneous presentation of information using one or more different digital media modalities. A book might present information using both analog text and analog images, whereas a website will present information using digital text and digital images.

• Multimedia is interactive. A user should be able to navigate through the information in the way they want to. The user's choices are involved in a multimedia experience. This is one reason why TV, which is a one-way affair, can't be considered multimedia.

• Multimedia is best thought of as a description, not a thing. Multimedia is most useful as an adjective, not a noun. Things can possess a quality of being multimedia. Gadgets and devices are not "examples of multimedia". It doesn't seem right to say, "A cellphone is a kind of multimedia." But rather, you can say, "A cellphone can present multimedia content," or "This video game console runs multimedia applications."

• Just because something involves digital technology doesn't make it multimedia. Think about a microwave ovens, which often contain simple embedded software in it to make it run. But a microwave oven is not multimedia. Why? Microwaves don't present information; they cook food!

I hope that these pointers will be relevant not just for the rest of this course but for your other MMS courses. Keep in mind, however, that other people will have different definitions for multimedia. For some, any digital media modality (such as digital text, digital audio, digital images, or digital video) is multimedia.

Many other terms such as digital media, rich media, interactive media, social media, and new media have arisen since the 'multimedia' was first coined, and these refer to many of the same aspects of multimedia that you will be studying in BAMS For instance, new media is commonly applied to media delivered through the Internet, while rich media refers specifically to bandwidth-heavy content such as audio and video. However, these distinctions are not set in stone, and you can expect to see them used interchangeably.

Summary

In this module, we looked at various experts' attempts to define multimedia, a term that is still very contested. First we looked at we meant by media. Then we looked at how multimedia is not simply the same as "multiple media". Throughout the discussion, I hope you got a glimpse of how personal, professional, academic, and corporate interests shape the field of study of multimedia. We then highlighted media modalities as salient to our study of educational multimedia design and evaluation, and spent some time looking at Randall Packer's ideas around multimedia, which I claim are useful for our study of educational multimedia design and evaluation. We looked at a particular example of a slippage in definitions: multimedia can be both a digital product and a tool that is used to create a digital product. Finally, we looked at some common properties of all digital media modalities.

Throughout this course, you will find that I will be using the terms "multimedia", "digital tools", "technology", and "software" more or less interchangeably. If I do, I mostly do it for reasons of style. I do not want to leave you with the impression that they these terms identical. Of these terms, "technology" is the broadest. Technology can be seen as a way of doing something coupled with the tools needed to actually do it. "Digital tools" can refer to both hardware (such as desktop computers, laptops, mobile devices, and other tangible electronic objects) and software (which are made out of code and run on hardware) that rely on data to be encoded in binary form.