Horizns – an augmented reality, collaborative storytelling game

Horizns logo 1_croppedHorizns is my thesis project for the M.A. Program in Digital Media Design for Learning at NYU, created (in 2015) using the ARIS (Augmented Reality Interactive Storytelling) platform.

Here’s the back-of-the-box description:

Playable by anyone aged 12 and up, Horizns is a narrative-based, augmented reality (AR) game ultimately designed for collaborative storytelling in grades 7-12 ELA and/or Social Studies classrooms.

Players begin by participating in the (fictional) “Horizns Rewards Program,” an AR tour of the history of Times Square, NYC. The plot takes a dark turn, however, as players must “dystopify” the world around them; and everyone’s best chance at escaping a dire future means interacting with the dystopian visions of others.

If you’re interested in my (six-minute) talk introducing Horizns to attendees of the ECT-DMDL Design Expo (5/15/15), you can find it here. [Warning: Contains spoilers! ;-)]

My general aim with this project was to make something that was a) genuinely constructivist and constructionist; and b) a genuinely engaging gaming experience. More specifically, as far as learning theory goes, the  game’s design is ultimately driven by the notion of “Social Imagination,” which Maxine Greene defines as learners’ “capacity to invent visions of what should be and what might be in our deficient society, on the streets where we live, and in our schools” (Releasing the Imagination, 2005, p. 5). And for a bit more on the theory behind Horizns‘ design, please feel free to check out my Design Expo poster (pdf).

“Through My Eyes” – Oculus Rift + interactive fiction (+ Harry Potter)

Created for:

Project length:

  • Medium-long (approx. last quarter of the term, final group project design document).

Team:

  • Diondra Arroyo
  • Enid Brown
  • Kelsey Buttendorf
  • Jullie Harten
  • Matt McGowan
    • my principle contributions: “Persona,” “Context Scenario” and “Narrative Flowchart”

Challenge:

  • As a group, produce a full, thorough design document on a project of mutual interest.

For our final project, my group came up with an Oculus Rift-driven, interactive fiction learning game experience titled “Through My Eyes.” Here’s the concept brief:

Through My Eyes - concept brief
(Layout by Jullie Harten)

The full design document is here:

“Through My Eyes” – full design document

and these are my principle contributions:

Persona:

TME - persona
(Layout by Jullie Harten)

Context scenario:

TME – context scenario (pdf)

Narrative flowchart:

(Page layout by Jullie Harten)
(Page layout by Jullie Harten)

Teaching design at the AMNH… with POV Mad Libs!

During a Game/Mobile Design internship at the American Museum of Natural History (Fall 2014) I worked on a program called “The Neanderthal Next Door,” which was

a 27-session youth program for 21 12th-graders that’s designed to develop and implement a digitally augmented (augmented reality-enhanced) print activity guide that explores the topic of human evolution through the frame of Neanderthals.

The piece of the program I worked on the most was developing a design-thinking approach that would guide the 12th-graders in their work. Given the time we had with the students, I thought an approach that used a selection of Stanford d.school’s Bootcamp Bootleg cards would work best. Below is a post about a few of the sessions that the program’s director, Barry Joseph, asked me to write for his blog, mooshme.org:

“‘PEOPLE NEED A CHANGE IN LIGHTING BECAUSE THEY WALK TO THE RIGHT’ – USING DESIGN-BASED LEARNING WITH MUSEUM TEENS”

Instructor facilitating POV Madlibs for high school students
The POV Mad Libs discussion – lead by the program’s Science Educator, Marissa Gamliel.

One of the many shuffles I did of d.school's Bootcamp Bootleg deck.
One of the many shuffles I did of d.school’s Bootcamp Bootleg deck.

And here’s an earlier part of the program curriculum focused on design (using the Bootcamp Bootleg deck):

d-school cards for sessions 9-16 (pdf)

“The Mural” – a point-and-click adventure game about restoration

Institute exterior with protagonist and Foundation head

Created for:

Project length:

  • Medium-to-long (approx. the last third of the term; first/only group project in the course).

Team:

  • Matt McGowan–game concept, narrative design;
  • Maria Jose Saint Martin–project management, UI;
  • Jeremy White–art; and
  • Allen Yu–lead programming.

Some of the proudest and most educational–and most stressful–experiences I had in pursuit of the DMDL degree came from the NYU Game Center‘s Narrative Game Studio course. As one of only two non-MFAs in a class of nearly 20 students, I often felt like an outsider or interloper, but was beyond psyched and honored that the idea I pitched for the final project was selected (by fellow students) as one of the four to be worked on for the course’s only group assignment. This (in somewhat more polished prose than I ended up speaking) is what I pitched:

In this point-and-click adventure game, the player character (PC) is a painting conservator who has been called in by a somewhat secretive, private foundation to clean a mural recently discovered in a long-unused wing of the foundation’s headquarters (HQ). The request is strange, as this foundation is not known for its art collection, but rather for its involvement in developing tokamak nuclear fusion reactors.

Upon arrival at the headquarters, the PC is introduced to two non-player characters (NPCs): the current foundation president and the caretaker of this part of the HQ (a historical structure that the foundation has owned for over a hundred years, around which is built numerous, state-of-the-art facilities). The PC learns from these two NPCs that the mural was created by the foundation’s former chief scientist, who suffered some kind of mental-emotional breakdown shortly before he died (over twenty years ago). Care for this painting (cleaning the various layers/areas of dirt, mold, etc. which have accumulated on the work) has to be conducted in a very specific way, however—with custom-made solutions designed by the painter himself. The formulae for these solutions must be gleaned from texts found in the chief scientist’s former library/office (in an a room adjacent to the room containing the mural). The mural itself is a utopian cityscape, which shows a future in which the foundation has solved the world’s energy crisis.

The game’s first principle mechanic, then, is deciphering the code for the solutions’ formulae, which must be created with precise measurements and timing (and with certain penalties for mistakes). As the PC cleans the mural, section by section, the top layer of paint is stripped away to reveal another mural underneath the utopian scene—a dystopian view of the same scene, with the foundation at the center of the world’s decline. Clues to the story behind this dystopian vision are contained in the marginalia of the books in the library. These marginalia also reveal that, in the course of his experiments, the scientist experienced a series glimpses into the future. His first vision was of the utopia—which then turned into a dystopia. Despite what he believed was the ultimate truth, however, the scientist painted the former vision over the latter one after he, under scrutiny by the foundation, came to fear for his life.

The second principle mechanic of the game is deciphering the differences between the two murals, i.e. determining the course of events, the narrative, of how this vision of utopia turned into a dystopia. Once the PC has determined all the pieces of this narrative (or as many as they can, if they’ve damaged the mural at all with any erroneously created solutions) they must enter the sequence of events into an old (1980’s-era) computer in the library.

The ending of the game is determined by the PC’s choice of whom they discuss their findings with—the president or the caretaker. An incorrect choice will result in PC death (a shot to the back of the head, represented by a sudden blood splatter on the computer screen once they input the correct narrative), with an epilogue of the murderer (the “bad” NPC) declaring that the truth must be hidden. A correct choice causes the “bad” NPC to flee and the good PC to reveal a hidden room filled with hundreds of paintings of the dystopian vision, which, they state, must be deciphered and revealed to the world.

So, with the advice “Kill your darlings!” firmly planted in the back of my head, perpetual concerns about scale chirping on my shoulder, and determined that every team member felt thoroughly invested in the project (i.e. it was no longer just “my” idea), I gulped down my apprehensions, gratefully accepted my team members’ generous goodwill, and we went for it.

We didn’t end up with everything I first pitched, of course, but we did get a lot of it, relatively speaking. Having one (or 25%) of our team members effectively drop out during the final stage of production (due to unforeseen, unfortunate personal circumstances) was a pretty intense challenge; but, fortunately, adjusting to this became largely a matter of further scaling back cuts we had already made. Did we end up with the game we wanted? Not quite. Did we learn a ton and explore a bunch of new stuff and ideas and have more than a little fun? Absolutely.

In brief, the main/major changes from the original pitch were the following:

  1. in the interest of realism/scale–we cut the (second) mechanic of deciphering and inputting the differences between the two murals, in favor of creating a more complex puzzle around the discovery and creation of the (singular) paint-stripping solution;
  2. in the interest of ease of game play, given our platform (AGS)–we simplified the chemical solution puzzle a bit by removing the timing aspect and reducing the stakes for failure; and
  3. in the interest of a more interesting/compelling/involving story–we changed the protagonist’s gender to female and making her the–long-estranged–niece of the late chief scientist.

And here’s some screenshots of the results:


As far as my writing/narrative design duties went, here are a couple of screens of the–non-linear/branching–dialogue (created in Dialog Designer, which integrates fairly well with AGS):

Intro dialogue between Miranda Butler, the conservator, and Dr. Margaret White, the head of the Ceir Institute. (Note: “MrCurator” is the name Miranda’s character got coded with early on, and it stuck.)
Screen Shot 2015-06-23 at 1.05.53 PM
Dialogue between Miranda and Frederik Jessup, the Caretaker (originally “Librarian”) at the Institute’s HQ.

Choose Your Own (Grammatical) Adventure – Ambiguous Pronoun References audio narrative

Created for:

Project length:

  • Short (one week).

Team:

  • Matt McGowan (solo project/assignment).

Challenge:

  • “Create an audio piece that uses narrative to help teach a topic you know well, but that is typically narrative-free.”

For my “typically narrative-free” topic I chose ambiguous pronoun references, an error I saw time and again when I taught undergraduate composition and literature courses. Here’s the audio narrative (with me reading the script):

And here’s a brief sketch of the target audience, learning goals, and applicable learning theories:

Target audience

Middle school-age, language arts class.

Learning goals

By being immersed in a narrative environment, students will listen for and learn about the lack of clarity that can result from ambiguous pronoun references. As the narrative is audio-only, in addition to learning the grammatical issue addressed, students may also improve their listening (and therefore audio-processing) abilities.

The attached example merely shows one player-student choice for one example of an ambiguous pronoun reference. Further examples/encounters will allow for a certain amount of branching of the storyline. Player-student choice is not about making the right decision, but rather exploring different options for editing and/or rewriting so as to avoid this grammatical problem.

Learning theories

Affective Learning: The relative immersiveness of the (focused, audio-only) narrative aspect of this lesson is designed to engage and motivate students on a certain emotional level.

Gamification: The mild (chose-your-own-adventure-style) gamification of the lesson is designed to increase student motivation and encourage review via replayability.

Information Processing Theory: The focused, deep listening that students are required to do for this lesson should aid in the development of their listening skills–that is, their ability to process, store and eventually draw from information and stimuli that the audio narrative provides.

Self-determination Theory: The choose-your-own-adventure-style format enables students to have a certain amount of autonomy in the way they progress through the lesson.


Audio credits:

“StoryPix” – a digital storytelling card game

StoryPix app - Interpret path - order images

Created for:

  • Digital Media Design for Learning (DMDL) degree; Cognitive Science and Educational Technology II course (Fall 2013).

Project length:

  • Medium-long (approx. last quarter of the term, final group project design document).

Team:

  • Robyn Berland
  • Heather Kim
  • Matt McGowan
    • my principle contributions: “Summary of Project,” “Points of Viewing Theory,” and “Wireframes.”
  • Cooper Wright

Challenge:

  • As a group, produce a full, thorough design document on a project of mutual interest.

The full design document for this project can be found here:

“StoryPix” design document (pdf)

But here’s an excerpt from the “Summary of Project”:

StoryPix is a platform-agnostic, digital card game designed for middle-school-age children and older. It is an image-based storytelling environment that both visualizes and verbalizes the multitude of ways that people interpret the world. Like other social media platforms, StoryPix users will share images with the intention of telling their “story.” Unlike other social media platforms, StoryPix users will be encouraged to present their images in a more deliberately sequential manner, to present them more like a story, and to create an interpretation of the visual stories of others.

The learning objective of the game is to encourage players to think about their own and others’ thinking. Players will create a story constructed from a series of “cards” or “frames,” static images being either self-created or found (photos or drawings). These images are then sequenced to tell a story (as do, for example, comic strips and photo essays). Players then add a title (of no more than three words) and a descriptive, narrative text (of no more than 200 characters) to accompany their picture sequence. Once a story is completed, the player “publishes” the story and shares it with another player. The recipient of the story, however, only sees the story’s title (not the original narrative text) and a “stack” of images/cards/frames that have been “shuffled” and placed in a random order. The recipient player must then guess the originating player’s story in the original order of the cards and send their guess back to the originating player, along with their own (under 200-character) narrative text. Recipient players are allowed three tries to “get it right.” Players also have the option of sharing one story with multiple recipients, allowing the creator of the visual story to view the multiple ways in which their story is being interpreted. Finally, the player has the option to publish their story, giving other friends or the community a chance to interpret the story as well.

and a few of of the wireframes we included:

“Mama Hen Is Sick!” – an interactive fiction game

Created for:

Project length:

  • Medium (approx. the middle third of the term).

Team:

  • Matt McGowan (solo project/assignment).

Challenge:

Create a game using Inform where the player must solve a mystery based on the traces left in the environment. As part of the concept of the world, establish events that happened in the mystery and that the player will have to reconstruct.

Constraints:

Your game should at least have one NPC [non-player character] to talk to, and there should be a set of mechanics that allows the player to input the solution of the case.

In the coding, the length of the strings is 140 characters. You can generate text dynamically that is longer than that on the screen, though.

The final game should be a short scene that players can solve in 7-10 minutes.


Here’s part of the description of “Mama Hen Is Sick” as originally conceived (with inspiration swiped from The Space Merchants and–the Sonmi~451 story in–Cloud Atlas):

Setting

The not-too-distant future, in a factory that grows and processes vat-grown meat, the mass of which is named “Mama Hen.” The processing is attended by three dozen human clones (all male), supremely loyal and compliant laborers who live in the factory where they work. Unbeknownst to the clones, however, the factory is about to get bought out, meaning the fate of the entire operation is up in the air. All senior management staff are off at a “retreat” (a Machiavellian buyout negotiation preparation summit), leaving the janitor (a non-clone human who also lives in the factory) and the factory AI in charge.

The story takes place at night, when all the other clones are asleep. The central mystery revolves around finding a saboteur who has infected Mama Hen with some kind of flesh-eating disease that’s causing her to rapidly (and disgustingly) deteriorate. She could be dead by morning. The player character (PC) is awakened from his sleep cycle by the factory janitor, who says he needs help cleaning up the unholy mess that Mama Hen’s deterioration is creating. The janitor also wants to see if the PC knows anything about what’s happening to the meat.

The principle/iterative difference between the finished product and the outline for the game is that I ended up getting rid of the janitor NPC, as the AI NPC served all the story purposes I had in mind just fine (and I only had 7-10 minutes of play time to tell that story).

“Mama Hen Is Sick!” is entirely a text-based game, but this is the “map” I had in mind when making it:

outline draft 3.001

And here’s a game play screen from the opening scene:

Mama Hen Is Sick!


This interactive fiction assignment was my first-ever intensive coding experience. I’ve since learned that Inform 7 is a…let’s say “weird”…place to start such a journey. It felt it. This fact, combined with the assignment’s constraints–the 140-character string limit, in particular–, made for an extremely challenging design scenario. It also made for one of the most educational experience I had in the DMDL program.

If you’d like to play “Mama Hen Is Sick!”, the zipped Inform file is here on Dropbox. (And if you don’t have an “interpreter” to play interactive fiction on, here’s a list of a bunch.) I plan on going back to work on–expanding, tightening, polishing–the game in the not-too-distant future; but, until then, if you do play it, feedback (good and bad) is most welcome!

Narrative and wearables – the Digital Creator Jumpsuit

Created for:

Project length:

  • Short (one week, design document).

Team:

  • Heather Kim
  • Matt McGowan
    • my principle contributions: “Research prototype description,” “Illustrative example” and “Storyboards”
  • Cooper Wright

Challenge:

  • The class was asked: “How can you enable kids to experience an artistic or scientific concept on a visceral or kinesthetic level?”

Our answer to this question was the Digital Creator Jumpsuit. Below are a couple of storyboard sketches and an illustrative example I wrote for the

Digital Creator Jumpsuit group design document (pdf):

Illustrative example

Anna, a 4th-grader, has decided to create, along with two of her friends, a short–digital–movie about a mouse who learns how to dance by watching the human family whose house he lives in. Anna is responsible both for the film’s set and for the background sound and music to be played during several of the short’s scenes.

Working on the sets first, Anna orients her jumpsuit to “painter mode” and calls up–on the sleeve of her non-dominant arm (in Anna’s case, her right)–the palate of colors that she and her friends agreed upon for the mouse’s hole-in-the-wall home. Anna decides she wants to use her left arm to paint with blue, so she taps that color on her palate and swipes up her arm, watching the color flow up her right arm, across her chest and down her left arm–“filling up” the entire left sleeve of the jumpsuit with that color. For her right arm, Anna needs yellow, so she taps on that color and swipes down, filling up her right sleeve. Done with color selection for the time being, Anna triple-taps the palate to close it. Now she’s ready to paint!

Suddenly remembering one of her art classes from last week, in which she and her classmates learned about Vincent van Gogh’s The Starry Night, Anna decides to go with a “giant swirly” pattern for the mouse’s living room wall. Facing the wallscreen and touching her hands to her shoulders (which signals to the screen that she’s about to paint with her arms), Anna walks from the left of the screen to the right in long, loping strides–swirling, twisting and waving her arms as she does, seeing her multi-colored movements registered on the giant digital canvas in front of her.

Once she reaches the right side of the screen, Anna surveys her work and thinks that the pattern she’s made looks like “some crazy waves.” “So maybe,” she muses, “the house the mouse lives in is by the ocean, and he can hear it from his home in the wall!”

Wanting to play around with the idea before she forgets about it, Anna puts up the hood of her jumpsuit, which is her setting to automatically switch to the suit’s “sound and music” mode. Next she triple-taps her right arm to call up her music and sound effects board to search first for “waves on the beach” and then for “pebbles on the beach”–two of her favorite ocean sounds. Anna then tasks (by selecting and swiping different sound/music icons) “waves on the beach” to both of her arms and “pebbles on the beach” (which she thinks will go well with the star-dots she’s planning on painting in and around the swirly pattern she’s already created) to both of her legs/feet, as she wants to be able to mix the two sounds in real time. She’ll make the wave sounds rise and fall with the movement of her arms and make the pebble sounds by running in place with “quick little baby steps,” listening to the mix through the surround-sound array embedded in her jumpsuit’s hood.

As she’s planning all this out in her head, Anna realizes that she’s probably going to look pretty ridiculous doing it; but she doesn’t really care–that’s why the museum gave each of the students in this program their own “studio.” And, anyway, she knows her friends wouldn’t laugh. Actually, she can’t wait to finish this sound piece (and maybe the star-dots, too) so she can call up a chat window and have her two co-creators come down the hall to see what she’s got so far. Their movie is going to be so awesome–she just knows it!

Making art history come to life with “Frieze Tag”

Frieze Tag wireframe - Figures assembled

Created for:

Project length:

  • Short-medium (final two weeks of class, design document).

Team:

  • Matt McGowan (solo project/assignment).

Challenge:

  • Design a learning game; produce a design document that contains a detailed software analysis (or landscape audit), related classroom activities, and a mockup or prototype.

My response to this assignment was a design for a multimedia, augmented-reality learning game/app called “Frieze Tag!”

The full design document can be found here:

“Frieze Tag” design document (pdf)

These are a few screens from the mockup (or rigged-path prototype) I created for the prototype:

And here are some images from the “Klepsydra” segment of the 2004 Summer Olympics Opening Ceremony in Athens: