Category: GDAP

Overview  

Problem Statement & Our Goal

The city of Philadelphia is rich with history and culture. Unfortunately, many of the ways to experience the city are not designed for children.

With our interactive exhibit, Next Stop, Philadelphia, we gave children a tour of Philadelphia that was designed just for them! At our exhibit, kids engaged with educational games, crafts, and activities that let them experience the culture and history of the city in an age-appropriate, exciting way.

Roles and Responsibilities

• Becky Liu, Project Manager
• Mikayla Lisiy, UI Lead & Animation
• Leanne Chin, Story Lead & Product Design
• John Greaves, CS Lead
• Ziad Karameth, Art Lead
• Tara Boonngamanong, XR Lead
• Aman Kartha, Programmer
• Victoria Stauffer, Product Design Lead & UX Lead

Timeline

Scope and constraints

There were no scopes or constraints to the project, as long as we are developing anything that relates to the realm of Digital Media. However, because the project consists of User Experience & Interaction Design (UXID), Game Design & Production (GMAP), and Computer Science (CS) students, the project must include aspects of game design and development. The UXID students are focusing on the Storyline, User Interface of the digital components, UX Research, and Physical Production of the exhibition space. The GMAP students will be working on the digital game designs and building the assets needed for each game that is developed. The CS students will be working on testing different sensors, and programs, and developing our ideas into physical games that can be played throughout the exhibit.

We started the project in August 2021 and it was completed by June 2022.  

Research

User Survey

The first step of our research process was to send out a user survey. We gathered anecdotal data and recruited highly qualified teachers for our user interviews. Here are some insightful quotes from the survey:

Tell us about a memorable field trip. What makes it stick out to you?

“Annapolis… The students had a guide to take them through an experience, which kept them engaged in the lesson”

“The Glencairn Cathedral and Castle… They used different medieval characters who told their stories and hosted activities for the students in different rooms of the castle. It really brought the history to life and kept students engaged.”

What books do your students engage with?

“As teachers, we try to read books from a variety of genres and levels”

What resources can you recommend to us as we create an exhibit and story for children to interact with?

“Know the elements of plot. The importance of the relationship between an inciting incident, conflict and climax. Look at Freytag’s Pyramid for narrative text structure.”

“Kindergarten kids often enjoy books with colorful illustrations, repetitive text, and humor.”

User Interviews

Our next step was to conduct User Interviews. We met with teachers who had over 20 years of experience teaching grades K-8, and a tour guide who had 3 years of experience at the Constitutional Walking Tour of Philadelphia. Below are some of the questions we asked:

• How do you manage classrooms & keep students engaged?
• What advice do you have for writing stories and creating characters for different age groups?
• Can you tell us about a game you play in the classroom? How does this enhance learning & engagement?

Special thanks to Anna Leong, who assisted with the User Interviews.

Our Findings

After conducting our interviews, we spent time sorting through our transcripts. We looked for patterns in what we were told and took note of valuable quotes. Here are some of our key findings:

Learning Objectives

First, we learned that establishing learning objectives for our stations would be foundational to our process. We had multiple interviewees speak to this point. Start with the goal of the station, and reverse engineer the activity from there.

“It’s that reverse engineering. What is your goal… what do you want (kids) to get, and then you walk it back.”

Fostering Engagement

In our interviews, we learned that there’s a lot of variety in the needs of elementary-aged children. At our exhibit, we needed to accommodate a broad spectrum of ages, learning levels, and personalities. We also learned that teachers had an equal preference for digital and hands-on activities. With this, we decided that for each of our 4 stations, we would design a digital game and a hands-on activity, or craft. Giving kids choice helped to maintain their engagement throughout the whole experience.

“I try to use as many modalities as we can because we think that different things will reach different kids and that way you get more of a chance in reaching all of them.”

Story & Conflict

A key idea that developed from our User Interviews was to write a story for our exhibit. A story helped make our stations more cohesive, and motivated visitors to move through the whole exhibit.

In our research, we learned the best approach when writing for our target audience. We learned about Freytag’s Pyramid, which maps out a basic story structure, highlighting the importance of using conflict to engage the reader. We were also told about Lexile Scores. To accommodate kids ages 5-10, we should “Shoot for the middle,” and write for a 3rd grade Lexile level. Finally, we got advice for writing the instructions in our exhibit. We were told to create engaging, simple, and clear instructions. The faster kids were able to pick things up and start engaging, the less likely we were to lose them.

“What’s going to engage the kids is the conflict, the problem.”

“I think if you are going to balance between kindergarten and fifth grade you’d shoot for the middle. And we would shoot for third grade.”

Story Research

The first step when starting out this project was finding research on how to develop an exhibit specifically for children. The first thing we did was go to the best source possible on how Philadelphia is being presented to children: The Library. We found as many books as possible that were about Philadelphia that were specifically geared towards the ages of 6 to 11.

By researching children’s books, we were able to see how to tailor our language to our audience and learn what landmarks in Philadelphia are most prominently taught in books for younger ages and also see what landmarks are not being as well-represented as they should be.

We also researched what type of characters are generally in children’s shows. “Are these characters humans, people, objects, etc.?”  We learned about anthropomorphism where objects or animals are given human traits to provide some emotional disconnect to the audiences, while also being able to connect enough to provide them with lessons.

After many different iterations of animals, we designed animals that were native to Philadelphia. Those animals were then drawn using Procreate to create sketch-like illustrations of our characters: an opossum, squirrel, pigeon, and rabbit.

Story Outcome

The story had gone under several different revisions ever since the first concept. Initially, the exhibition was to have different landmarks of Philadelphia where children engaged with games associated with each station. Through many discussions, we began to develop a more cohesive storyline that was able to better guide children through the exhibit using a narrative and incentives to retrieve each station.

Our final story allows visitors to help our 4 characters find their missing souvenirs in Philadelphia, guiding them through our physical exhibit by meeting each of the corresponding characters at specific stations.

Event Coordination

Event Planning/Organizing

When planning and organizing our event, we found our exhibit location, made reservations and managed our project budget. We coordinated with each other and the Digital Media Department to identify the most ideal event date. Then, we worked with Drexel Event Services Coordinator, Nate Grossi, to ensure that all event requirements, like children’s clearances and COVID restrictions, were met, and secured a reservation at the Drexel Bossone Lobby.

Project Budgeting

For this project, our team had a budget of $3,500. We used a GoFundMe Campaign to raise $2600, and were awarded $1,000 from the Drexel Swift Fund.

We used a spreadsheet to work out a budget for our project. We used the information from Drexel Event Services to estimate the cost of our exhibit space rental, estimated the cost of our supplies list, and kept track of every order, receipt, and expense that occurred in the months leading up to our exhibit.

Take a closer look at how we organized and managed our Project Budget.

Exhibit Floor Plan

After sketching some ideas, we developed 3D model scenes of each of our stations. We used Maya to construct the 3D models and rendered the scene out to communicate our vision for each station. This helped us understand the scaling of each element that went into the experiences and see what we needed to be prepared for.

We also created a 3D map of the final exhibition layout using the floor plan that we have received from the event service. We used this map to give the visitor an overview of the space as well as for the team to plan the position and rotation of staff on exhibit day.

Product Design

Our product design team brought our exhibit to life! We combined our physical and digital design skills to create a space that was 100% original. We built installations, painted murals, and designed every piece of our exhibit with our branding in mind. Later, we’ll explain how we designed each station to be unique, but for now, let’s see how we worked to design a cohesive experience.

The Mural

For our exhibit exit, we painted a mural of the Philadelphia City Skyline. This mural served as a photo-op station in our exhibit. Visitors could pose in front of it and take polaroids and personal photos.

This 6’x4′ mural was painted with acrylic paint on canvas. The Next Stop, Philadelphia banner was created separately – a canvas that was cut, gessoed, painted, and then hand-stitched onto the mural. The canvas was originally 9′ tall, so we hemmed it to be about 6′ and gave it a curtain rod pocket.

Reading Terminal Market Activity

The Reading Terminal Market activity was inspired by the spinning tic tac toe boards at playgrounds.

At Reading Terminal Market, we wanted visitors to discover the many cultures of Philadelphia through their food. This playground-style game was a matching game that challenged visitors to match the foods that were in the game with the correct cultural categories.

This installation was constructed with wood, PVC pipe, Styrofoam, and paper. First, we built the wooden base frame. Then, we strung the blocks onto the PVC pipe and inserted the three rows of blocks into the frame. Next, we built and attached the outer frame and doors on hinges. The last steps consisted of painting the doors, attaching the handles, and adding the finishing touches, like pictures and category names.

Reading Terminal Market Stand

The controller for the Terminal Reading Market station was the simplest out of all the three. For this station, we decided to use arcade buttons as the main controllers for the station. With these buttons, the player would be able to select which store they would like to visit.  

We used Arduino Leonard Board as our hardware for the store switcher controller. Three buttons were used, one being assigned to each store in the game. These buttons were then placed on the surface of a food stall that we constructed in the wood workshop.  

The biggest problem that we faced for this station was building the food stall. With limited time and resources, we were not able to build it at the scale that we were hoping for and had to settle for a simpler version. Originally, all we had for the stall was just a table. We then constructed a cardboard structure around it and we then used felt fabric to cover and decorate the surface. We also built a wooden box for where the buttons would go on.  

Uniting Designs with our Branding

In order to maintain consistency throughout the exhibit, we developed a consistent design amongst the exhibit whether they were showcasing instructions for crafts or signs that directed people where to stand for a game. We used the same font and colors throughout the exhibit for any products that were to be printed onto physical media.

Digital Design

User Interface Design

We designed the overall UI of our exhibit to stay consistent throughout our station games as well as our physical assets and activities. Our goal was targeted towards designing a fun and educational environment that made children feel welcomed when they entered our exhibit. Because our exhibit includes diverse elements from different cultures that are covered in our station learning objectives, we made sure our overall design would be inclusive of everything that contributes to the educational importance of Philadelphia. Our overall event design includes a very vivid and bright color palette as well as painted brushstroke visuals. We felt the overall aesthetic would connect with our younger audience and engage them to learn more about Philadelphia.  

Animations

Our animations were designed to guide our users throughout the exhibit and let them know when they were reaching their learning objectives while interacting with our games. Our 30^th street station animation was targeted toward getting children to understand how to navigate the exhibit and teaching them the importance of going to each station to complete the map. Our animation introduces a few characters (Rell, Po, Pidge, & Nia), who have lost their souvenirs while exploring Philadelphia. They ask the user to help them find the lost souvenirs which are scattered throughout all 4 stations on the map. The characters direct the user to go to each station in order to collect all souvenirs and complete the map. Our 30^th street station storyline engages our users to interact with our exhibit by giving them a sense of heroism for helping our characters on their journey throughout Philadelphia while fulfilling their learning objectives.  

Next Stop, Philadelphia! Intro Video 

Our game tutorials and station animations were designed with the same purpose to lead our users to complete each game while following the storyline. The game tutorials were designed with the intent to get the user to understand how to play the games. Our station introduction animations use elements from the map that make the player aware of the station they are currently interacting with when referring to the map. Depending on the user’s progress after playing the game, if they fulfill their learning objectives, the user will be presented with a win screen that displays the lost souvenir they have collected for that station. If the user does not complete the game, the user will instead be presented with a screen that shows they did not collect the lost souvenir, and that they must try again in order to continue.  

3D Modeling

Reading Terminal Market Station

For the digital game at the Reading Terminal Market station, we wanted to design a game that used leap motion technology.  

It took us a while to decide how we wanted our game to be designed. We were going for a painterly and kid-friendly art style, for both the textures and the 3D models. When researching similar games, we looked into shopping games and games that had a painterly art style. One of the most inspiring games was Alba — the art style of this game is exactly what we were trying to go for, so we used screenshots of the game as references in our Art Design Document.

As for the 3D art, we started designing and making our models first. We picked a variety of foods found at the Reading Terminal Market and started 3D modeling:

Then we started adding the textures based on our overall art style:

During this process, we presented our station concepts to our class and advisors. Feedback made us realize that a Juice Stand does not represent the Reading Terminal Market very well, especially when considering the learning objective of this station: to help the kids learn about different cultures through their food.

Therefore, we decided to scratch the idea and changed it to a Dutch Kitchen stand. After researching and looking at the menus of the food courts at Reading Terminal Market, we picked the four most representative foods of this stand and developed 3D models:

Then we added the new textures:

We also redesigned the stand to look more like the ones at the Reading Terminal Market:

We added more objects to the background so all the stands look more realistic:

Based on the feedback we got from the presentation, we noticed that a lot of the players didn’t recognize the names of the dishes while playing. To solve this issue, we made a pop-up UI at the top right which showed up when the players hovered their hands over the objects:

Liberty Bell Station 

For this station, we wanted to show the process of how the Liberty Bell was recast. To show this process clearly, we broke the game into two mini games: stacking the mold and melting the metal. The art team came up with the full asset list, and did research to get a sense of how to design the setting of the game. We found images of 18th Century factories for reference.  

With our reference images, we started to design and texture our 3D models for the environment:

The art team also created major models, for example, the bell and the furnace, as well as the particle effects for both mini-games.

Philadelphia Museum of Art

Learning Objective

At the Philadelphia Museum of Art (PMA) station, visitors learned about the principles of art: shapes, colors, and patterns.

UI

The puzzle pieces to each painting in the Philadelphia Museum of Art game are separated by abstract shapes and scattered throughout the middle of the canvas space using Adobe Photoshop. The idea of having a canvas-styled space was designed to mimic the experience of viewing artwork in a museum since this is a museum-based game.

As the players work to match each painting, they are presented with a clock that visually shows the players the limited amount of time they are given to match each painting. We decided since this game focuses on visuals, we did not want to take attention away from the game by displaying a numerical timer. We instead implemented the clock to show the player visually how much time they have left by having the clock disappear clockwise. When the player completes the game, they are displayed with a gallery of paintings that the player can touch and learn more about each artwork. This gives the player some historical context to each painting and allows them to learn more about colors and mediums.  

​​For our Philadelphia Museum of Art Station, we also created 2D assets for the game. Our UI and assets are painterly to match the aesthetic of our 3D games. We made some hand-written assets for the game along with some made in Photoshop (background, frames, etc.):

We decided that the PMA sign was too serious and looked weird with the shadows. We also changed the colors of the Start sign to make it more legible.

We thought the hand-written font we used was a bit difficult to read, especially for children, thus we opted for a computer-generated font.

Programming

Github link: https://github.com/jsg339/NextStopPhilly/tree/New 

The digital game for the Philadelphia Museum of Art is a puzzle game where a set of paintings are separated by shape and spread throughout the playing field. Each of these paintings was edited using Adobe Photoshop so that they would be easy to separate and put back together. These pieces are randomized so that every time the player loads the game, a different set of paintings will appear.

The goal of the game is to move the art pieces via touch, to the correct canvas on the edge of the screen. When the piece hits the canvas, it will appear as the real-life version of the painting. Finally, once all of the pieces are on the canvas, a digital gallery will appear so the player can view all of the paintings in the game and read information about each painting. This game is played on the ViewSonic touch screen TV, which allows for multiple players to play at once which really enhances the fun of the game.  

In order to make this game, we used Unity Engine. Unity helped build the overall foundation of the game since it has a very intuitive framework as well as many helpful tools for use. We used C# in order to code the various gameplay elements we have in the game, such as the physics of the pieces, the game object interactions, and the scene transitions.

Finally, we used TouchScript, which is a free open source extension that can be installed into unity in order to handle both touch input and multi-touch input.

There were minimal problems we ran into for this game. The main issue was working with the view sonic tv since we had never worked with this device previously. The main issue was using the different inputs that the game would need to handle, which ended up being a bigger problem than we anticipated.

We went through many different solutions, first, we tried to build an android application directly to the TV, however, that proved unaffected since we weren’t able to build the android OS onto the TV. Then we tried to create a webpage that would host the game build since the TV can hook up to the computer running the game and play the game via that webpage. Finally, we settled on using the TouchScript extension which would handle all of the multi-touch input as long as we create the correct gesture. This ended up working great and was the main savior of the product as a whole.  

The digital game started as quite a simple game, as seen in this demo. We initially had a tech demo where we laid out all of the art pieces in a unity scene and moved the pieces to the square that it corresponds to. We also started the gallery out by simply laying out all of the pieces in the game along with the information simply displayed on the screen.  

Next, we focused on making the game more visually interesting and actually interacting with the UI. We added a timer which would help add tension and a lose state to the game. If the player is unable to rebuild all of the art pieces in time then they would get sent to the lose screen and need to restart the game. We also changed it so that there are only four canvases on the screen at a time since. Not only is it easier to understand (one painting in each corner), but we also wanted to add one more painting since it would correspond to the story of the station.

At this station, the characters lost their postcard and the player will need to rebuild it along with the rest of the paintings in the game. This was also when we wanted to test the web build version of the game since we needed to experiment with the multi-touch features of the view sonic tv.  

View our second iteration of the PMA game demo here!

Now that the flow and all of the main gameplay elements have been defined moved towards the development of the UI and refining the look of the game. We also changed the gallery so instead of swiping between each of the paintings, we would simply click on the painting in order to learn more about them. Finally, we had a UI overall on everything, from the background to the main menu screen you see at the beginning, to even the end and lose screen. This was also when we were testing an android build for the game since we wanted to try building the game to the TV’s os

The new iteration can all be seen in this demo here on the android build!

Finally, we had many rounds of user testing which affected the final product in a great way. We changed the timer since we found that the streak of paint was very confusing for players. Also, we changed the physics of the game so that it was more consistent and easy to understand since a lot of players got confused when playing the game. We wanted to standardize all of the UI assets for our games since we wanted it all to feel like it belongs together and comes from the same developer. This helps bring our stations together and make them feel like they belong together.

All of these assets are videos that are played until the user presses the corresponding button or when the video is finished playing since these are all animations exported to an mp4. Finally, we added music to the game, which was made by our music producer.

Look here and watch the final Game Demo for the PMA!

Hands-On Activity

At the PMA station, we designed and hand-screen printed postcards with questions such as “What did you learn today” and “What did you learn at the PMA station” to help reinforce our learning objectives for the PMA and other stations at the exhibit.

During the exhibit, visitors were able to answer the question and decorate their postcards with a variety of different mediums such as crayons, watercolor paints, markers, and acrylic paints.

Liberty Bell

Learning Objective

At the Liberty Bell stations, visitors learned about how the historical Liberty Bell was broken and rebuilt.

UI

Our Liberty Bell game is an interactive experience that takes players through the process of making the Liberty Bell. In the first half of the game, the player must mold the bell by matching the moving parts of the bell within the mold. If the player successfully fits the bell part within the mold, green confetti will surround the screen letting the player know they succeeded. The hp bar in the top left corner is also an indicator of the player’s progress throughout the molding process. If the player does not align the bell part within the boundaries of the mold, red confetti will surround the screen letting the player know they did not match the bell parts correctly within the mold. This will also be reflected in the player’s hp bar, as they will lose a portion of their hp bar. The bell icon on the hp bar is also a visual representation of their progress. If the player starts to lose hp points, the bell icon will start to visually crack. The color of the hp bar will also start to change based on the number of mistakes the player makes. The blue color lets the player know they are making perfect progress; the green color lets the player know they are still making good progress but have made a few mistakes; the red color lets the player know that their progress is critical and that they are close to losing the game. The combination of both the bell icon and colors in the hp bar gives the player an accurate representation of the progress they are making and visually shows the repercussions of making a mistake, which motivates the player to make better progress.  

The second part of the Liberty Bell game takes the player through the melting and cooling process while forming the Liberty Bell. The player is presented with both a thermostat and meter on the right side of the screen. The player must control the temperature within the range of the arrow. The arrow is an indicator where the player must control the temperature on the thermostat. There are three different thermostat states. The frozen icicles on the thermostat show the player that the temperature is cool, the regular thermostat shows that the temperature is moderate, and the thermostat which projects flames shows that the temperature is hot. The meter to the right of the thermostat shows the amount of progress the player is making. Once the meter reaches the top, the player will have successfully melted the metal used to create the Liberty Bell. This accurately measures the player’s progress as they learn to control the temperature within the limits of the arrow.  

Programming

Github Link: https://github.com/Aman-Kartha/BellGame 

This game is the first part of the Liberty Bell digital game, it is a stacking game in which the player would need to create the mold of the bell bit by bit. Players will need to accurately stack five pieces of the bell together so that they all fit in the correct shape. When the player stacks incorrectly, the bell will lose health and “crack”. Once the player loses all of their health, then the game would send them to the lose screen and they would need to try again.

Take a look at the stacking game here!

This game was created via the Unity Engine in order to build the scene and handle all of the models in the game. We used Unity’s built-in C# in order to create the physics and interactions of all the models and parts in this game. Finally, we used Arduinos as the controllers for controlling when to stop each piece.  

Most of the problems we faced were figuring out how we initially wanted to play this game. Initially, we were thinking about creating a game where there was no actual lose state. The players would need to stack the pieces of the bell, if they missed or misplaced a piece of the bell it would fall off and stay where they were. This would cause the bells to vary in shape and size but led to many technical questions about how to move forward with the game. We ultimately decided it was best to have the pieces be sent to their correct position and give missing pieces a crack since this was on theme with the Liberty Bell.

We started with a scene that was representative of an old Philadelphia workshop. This would allow the players to feel like they are stepping into Old City and become immersed in the station. We then separated a bell model into five separate entities and moved these entities across the screen. If the player hits the button, then the piece currently moving will stop in its place. If the piece is in the incorrect spot, the player would lose some life depending on how wrong the placement was. This can all be seen in this demo!  

After we got this piece of the game working, it was time to make the game more visually interesting. We added textures to all of the models and gave the UI some life and theming. We implemented the system where if the player was wrong in the position of the piece then that piece would crack similar to the liberty bell. Finally, we added dynamic colors to the health bar, where it starts as a soft blue, but moves to an alerting red in order to show that the player has only a few more mistakes left before starting over again. Check out the new iteration here!

Finally, we added a particle system which would be another indication of the player’s successful and unsuccessful placements. If the player succeeds in placing the bell piece in the correct spot, then the particles will be green. However, if the player is unsuccessful, then the particles will be red and the screen will shake. This provides an indication that the player was not supposed to do that and they need to try to place the piece in the correct position. Later when we combined all of the scenes of the liberty bell digital game together, we added various menus and screens to the game as well as music.  

View the stacking game here!!

Github Link: https://github.com/Aman-Kartha/BellGame 

We wanted to have the Bell game split into two mini-games, with the second game being a balancing game.  

The initial demo of the second game was a simple balancing game where the player clicked on these four buttons to increase or decrease the temperature to keep it at a certain level.

This evolved into a 3D game where a pool of molten metal adapts and changes depending on the temperature. The gauges on the right were inspired by the fishing mini-game from “Stardew Valley” through game research. The thermometer showed the current temperature, while the green square showed the goal. The player pressed a button to increase the temperature and would slowly cool down overtime when the player stopped pressing the button. A progress bar fills up while the thermometer is on top of the green square.  

Next, we experimented and learned how to use shaders in Unity. The molten lava in the center was linked to the numerical values of the current temperature which would change the speed and color of the shader.

The final round of changes was to animate the 3D objects to show how the two mini-games relate. Firstly was to animate the bellow to move similarly to the custom controller we made.  

Then we created a pouring animation using scripts that would trigger certain parts: first to move the furnace and bellow, then to tilt the bucket and pour the metal into a mold. To create the pouring animation, a 3D model of the pour was created and made invisible, only to appear when the bucket is tilted. Then the shader was shifted so that the y-axis would move instead of the X which makes it look like a liquid is moving from the top to the bottom.

Liberty Bell Controllers

The controllers that we have created for the Liberty bell station were inspired by the main process of how bells were constructed – creating mold and melting the metal.

Because these two processes in real life can be very risky and unsafe for kids, we wanted to create a kid-friendly version of the experience. We decided to build two controllers for each step. The first is the molding process, here we created a hammer controller where the player can control the position of the molding piece. For the melting of the metal process, we created a bellow shape controller which the player can use to control the temperature of the melting pot.

We used the Arduino Leonard Board as our main hardware for the two controllers. Both controllers are connected to one board. We used a Piezo vibration sensor for the hammer and conductive tape to detect contact inside the bellow. The hammer sensor was connected to an analog pin while the bellow sensor was connected to a regular pin. We used light wood to create the handle of the hammer and felt fabric wrapped around a cylinder of hard paper to form the head. The board on which the hammer sat was constructed using laser-cut cardboard that we have designed specifically for this experience. Lastly for the bellow, we used pine wood as the main material. After drafting the shapes and sizes, we cut them out using a bandsaw. We then installed wire and conductive tape to the wood, gluing the felt to two handles as a final touch. The wires that we used for both controllers were recycled from an old electronic cable. With its strong outer rubber tube, it was perfect for our controllers since we wanted to make sure that it would not get disconnected when young kids were interacting with them.

Since the Piezo sensor was cheap, we had to replace it a couple of times. Another thing that we noticed was that when the sensor was being placed on flat surfaces, the sensor would not detect the hit properly. With this, we created a curved platform for the sensor to sit on so that no matter how hard the player was hitting, it would detect vibrations.

Originally all parts were constructed using real wood. But after multiple rounds of play testing and feedback about the realness of our controllers, we have decided to use different materials to make it more kid-friendly. Since some of our target users may not be in full control of their mobility and ability to use force, we wanted to eliminate any chances of them getting hurt or hurting anyone by accident.

Hands-On Activity

The Liberty Bell’s activity was inspired by a craft that we found when researching in-person events in Philadelphia. The initial activity that we based the craft on was a moving horse using just printouts, paper fasteners, and string. We used these similar mechanics and developed a craft that showed the bell breaking.

The product design team printed and cut out the paper pieces, and assembled the string and popsicle sticks so that day, visitors only had to assemble 2 pieces together and focus more on the decorating portion.

Reading Terminal Market

Learning Objective

At Reading Terminal Market, we wanted visitors to explore the many cultures in Philadelphia through their food.

UI

Our Reading Terminal game is an interactive experience that takes players through the Reading Terminal Market to look for food items. The player is given a randomized list that details what food items the player must collect in order to complete the game. The idea of having a shopping list gives the player the incentive that they are shopping for these food items. As the player navigates Reading Terminal Market with their list, they must make sure they are obtaining the right items, or else they will lose one of the three coins displayed in the top left corner. These coins represent the number of tries the player has throughout the game. Since the game takes the player through a shopping experience, it felt accurate to have currency representing the number of tries the player has instead of implementing hearts or other visual elements that represent a player’s lifeline.  

Programming

Github link: https://github.com/Aman-Kartha/TerminalGame

The first step for making the Reading Terminal Market game was to learn how to use the Unity modules that LeapMotion provides us. We had to play around with all the assets provided to get a good idea of how to use it. Detecting what the hands were doing was a matter of proximity, direction and which fingers were extended. For example, if you wanted to check if the user’s hands were clapping, you would first need to check that all fingers are extended on both hands, then check the proximity of each hand to each other, and finally check and see if the two palms are facing each other. A simple Boolean can add them all up. There were a few hitches here and there mainly to do with tweaking sensitivity and then trying to find a sweet spot to set the camera in. A big one was that sometimes the camera wouldn’t be able to pick up certain fingers since they were blocked (The index mainly).  Interaction was a little more complicated, like trying to juggle.

At the start, we had the idea of incorporating cooking into the Reading Terminal Market game. This concept was scrapped after realizing that since Reading Terminal Market itself didn’t have any cooking, the idea would stray away from the concept of our project, and the station’s learning objective.

For the first game demo we developed, we focused on camera movement controlled by the leap motion user. The original idea was to make the camera move or pan towards the next stall when the user pressed a button (using their hands) but we couldn’t get it smooth enough, so we opted to use a technique where we placed two sensors at the opposite ends of the camera that would constantly scan its proximity for the hands and then turn when activated. We had to center the player and place the stalls all around, and limit turns to be 90 degrees at a time.  

Next we made the food items look more appealing. For this we were inspired by the art style of Cooking Mama. We made the objects float and rotate on their own axis, and made them slightly shine and expand. This was quite easy but took a little time to get the fine-tuning right so that there were no clipping issues. We also programmed physics so that when a hand got near the food, they would either spin faster or get knocked away. This was challenging because we couldn’t find a way to control the amount of force the hands put on the food items. Our solution was to limit movement on the axis, and only allow food to rotate.  

Above is the first iteration of what one stall would look like. This shows the problem with attaching camera movement to the model hands. This iteration also revealed that using interaction between the hands to place the items into a basket would cause too many issues in future development.

Below is the updated version of the stands that minimized the issues of our first iteration.  

We moved on to adding UI and art elements, testing to see which approach was best. The hearts were replaced with the cash icons, the game now showed a start screen in the beginning and the list has the notepad as its background. The font was changed as well. We started adjusting the background of the scene as well to make it feel fuller. We experimented with new walls, more stalls and adding humanoid figures to it.

We noticed during our tests that the difficulty of grabbing items varied. This was due to how the collision boxes rotated with the object. Our solution was to change all the collision boxes so that they had a uniform size. To do this we moved around all the positions of the food items so that they were evenly placed and adjusted model sizes to be smaller than the actual collision boxes. This created a trick where we could place the Leap Motion projected hands much further back than usual and not have to worry about the z-axis of the camera, whose range was quite limited.

Next was to clean up and polish our design.

Multiple objects were added to the scenes to make them more complete. Each stall had a design theme to indicate which culture they represented. All food items hovered over plates to make sure the user knew which objects were interactive. The notebook UI had been updated to scratch out the items picked up. We swapped to Text Mesh Pro so that the text would automatically adjust its size depending on the length of the text.

Luckily Leap Motion provided methods to change the look of the projected hands. We opted for the “ghost hands” as they were transparent allowing the player to see what they are grabbing. In order to handle transitioning between the stalls, we used custom physical buttons similar to those found in arcades that were mapped to specific keyboard keys such as “w” and “q”.  

At this point we were ready to playtest this station. User feedback showed issues with transitioning where the game would rubber band while trying to move between stalls. This was fixed by disabling the ability to take inputs while the camera was moving to a new stall. There were also issues with grabbing the items. We found that at certain angles the Leap Motion controller would not be able to “see” the index figure. The fix was to disable the need for the index finger to be extended. We programmed a grab to happen when the middle, ring and pinky finger were folded.  

Hands-On Activity

The Reading Terminal Market (RTM) activity was inspired by the spinning tic tac toe boards at playgrounds. At RTM, we wanted visitors to discover the many cultures of Philadelphia through their food. This playground-style game was a matching game that challenged visitors to match the foods that were in the game with the correct cultural categories.

This installation was constructed with wood, PVC pipe, Styrofoam, and paper. First, the wooden base frame was built. Then, the blocks were strung onto the PVC pipe and inserted the three rows of blocks into the frame. Next,  the outer frame was built and attached to the doors on the hinges. The last steps consisted of painting the doors, attaching the handles, and adding the finishing touches, like pictures and category names.

Chinatown

Learning Objective

At the Chinatown Station, we wanted children to learn about the story behind the ancient Chinese Lantern Festival that is celebrated annually in our city of Philadelphia.

UI

Our Chinatown game is an interactive game that takes players through the Chinese lantern festival. This game works similar to old iPhone game, Flappy Bird, except two players will control the head and tail of the dragon. Players must work together to guide the dragon throughout Chinatown without hitting the lanterns and buildings. The game UI is designed to replicate the culture and style of Chinatown by using both traditional and modern Chinese buildings and overall art style. The progress bar at the top shows the player a live view of where they are at in the game and how much they have left to navigate before collecting their souvenir. The Chinatown archway icon shows the starting point of the game since that is where the game starts, the dragon head symbolizes the player, and the dragon coin symbolizes their token of victory as they reach the end. As the player progresses through the game, the dragon head will move along the progress, giving the player a visual representation of how much they have left to complete. The dragon coin is also shown in the top left-hand corner. The three dragon coins represent how many tries the player has throughout the game. Hitting an obstacle will result in the player losing one of these dragon coins. Since the dragon coin is the player’s symbol of victory, having the dragon coins shown as the number of tries the player has emphasizes the importance of the player holding onto their dragon coins.  

Programming

Github link: https://github.com/jsg339/NextStopPhilly/tree/New 

The game for the Chinatown Station is a two-player game where each player controls a different part of a dragon. One player controls the head, while the other player controls the tail. The goal of the game is to survive an obstacle course that resembles Chinatown and retrieve the lantern at the end of the game. The physics in this game was inspired by “Flappy Bird” in which the players are constantly battling gravity in order to move throughout the screen space and avoid the obstacles. If the player gets hit three times by a thing on the field, then both players will need to restart the game.

As for the development of this game, we used Unity as the engine in order to develop the game. We also used Unity’s C# in order to program the game and control all of the physics and interactions. Other than that, we used three controllers that were created via Arduino technology to control the head and the tail of the dragon.

The biggest issue that we noticed was that it was difficult to play. We had to have a multitude of user testing to get to its final state since many players were not able to complete the game. We had to adjust both the physics of the game, as well as the number of lives the player got since the game ended up being that difficult. Thankfully, we ended up in a pretty great state on the day of the exhibit since it was pretty challenging but fair.  

The game started off quite simple. The first thing that we worked on was creating a foundation for the two-player flappy bird aspect, as well as the physics we would use for the game. We created a landscape that would coastally change and differ in size, that way we can test the physics of the game as well as the logistics of playing with another person. It ended up being pretty fun, and we really liked the two-player aspect of the game, it added a lot to the experience.  

Check out the first demo here!

Next, we wanted to make the player feel more like a “dragon”, so the two players would need to seem connected. The main problem is that the player is constantly moving, so whatever the body would be would need to be dynamic and stretch around the screen. Luckily we ended up figuring out a very cool algorithm that would be constantly updating the scale and rotation of the body based on the midpoint and distance of the two players. This can be demonstrated in the following demo here!  

Now that we have our foundation, it was time to actually make the game itself. We replaced all of the empty boxes that are in the game with actual sprites for the final game. This would include the various buildings, the dragon, and the sky.  We used lanterns as the upper obstacles and the buildings as the lower obstacles, we then spread them out throughout the screen with increasing difficulty in order to fit the challenge of the game. We also decided to add a curve to the body of the dragon, since it would seem more natural and “dragon-like”. This was a little harder than expected, but with Unity’s “Spline” objects we were able to make a spine for the two players that would move dynamically to where the positions of the players were. This gave it a more polished and realistic feel to the game which you can see here!

After this update, we had a very long playtesting session where many people played the game. We really noticed the difficulty in the game since none of the play testers were able to finish the game in a reasonable amount of time. We ended up creating many quality-of-life changes to the game. We added a life system where the player would be able to take three hits from the obstacles before having to restart the game. We also added a tracking system into the game so the players would be able to know where they are on the map and how much longer they need to play in order to beat the game. Finally, we changed the order of the obstacles so that it was more forgiving and easier for the players to get through everything.

View our next iteration of the game here!

This ended up having great responses and the game was in a very good place. So the final updates that we made to the game were fixing all of the UI elements as well as adding a proper title, lose, and win screen into the game. These screens were similar to all of our screens for the final versions of our games since we wanted everything to feel like they belonged together. We also added music by our music producer, which added much atmosphere to the game. Overall, we are very proud of this game and it had a ton of great reception from the visitors at the exhibit.

View our Final Chinatown game here!

Chinatown Controllers

The controllers that we have created for Chinatown’s station were designed to give the players a glimpse of what it is like to be a part of the Chinese Lantern Festival. Since the main character of the game is the Chinese dragon, we have looked into other elements that come with the experience. Chinese dragon performances were traditionally accompanied by traditional Chinese musical instruments, the main being drums, and cymbals, incorporating the two instruments as our main controllers.  

Both controllers’ main component was the Arduino Feather Board ESP32, which allowed them to be connected to the computer wirelessly via Bluetooth. The Feather Board ESP32 also came with Capacitive Touch Sensor which allowed the boards to recognize the user’s skin contact as input. We decided to use capacitive touch as our main user input because it provided the player with a more hands-on experience with the controllers compared to vibration sensors. We wanted to challenge ourselves and experiment with its uniqueness due to the capacitive touch’s newness in the industry. Additionally, with the vibration sensors, anything could activate that input but with capacitive touch, only skin contact would send the signal.

We used conductive tape and tread to make the vision of creating a hand-on and textile experience possible. Conductive thread was sewed into the felt fabric onto the area where the player would place one hand on the controller. The thread was then connected to the ground pin on the Feather board while a signal from another hand would be connected to the Touch pin. The moment when both hands are in contact with the ground pin and the touch pin wiring, the board would then send a signal to the computer that the user was giving an input. The board will then send an output to the computer as “key pressed down” where in this case we were sending the key S and W for each controller.  

The body of the controllers we constructed with plastic plates and containers. We used these materials because of their affordable cost and easy access to replacement parts. Due to the materials being lightweight, they were perfect to make kids-friendly controllers. With the main body originally being plastic plates and containers, we sanded down the outside of the object and sprayed black matte to give it a clean look. In addition to forming the controllers, we also used felt fabric and nylon straps to build a secure holder area for the controllers.  

There were three main problems that came with this unique set of controllers. The first thing was that the lithium batteries that we ordered were manufactured in a way that the polarities were wired reversely. This caused the board to be non-functional and parts to short circuit. We spent a lot of time trying to understand what we did wrong, diagnosing our codes and our setups, but we completely ignored the possibility that it was the manufacturer’s fault.  It took us about 3 weeks of debugging to realize that it was the battery wiring, but once we confirmed the source of the issue, we cut the end wires of the batteries and swapped the ground with the pin.  With capacitive touch still being pretty new, there was a lot of fine-tuning of the sensitivity to get the controllers to send out signals properly when different users were using it. We noticed that age and gender would affect how much signal the board was detecting. For example, the controller was more sensitive to some male playtesters than female playtesters. Without their hand making direct contact with the controllers, the capacitive touch would detect the electrons. And in some cases, the controllers would not detect touch at all which we hypothesized that it could be caused by the dryness/thickness of the player’s skin. Lastly, with the nature of the dynamic of the game and the physical impact that our controllers were getting, hitting the drum and the cymbals, parts of the controllers would get loose and wires would get disconnected. This required us to reconnect parts and troubleshoot disconnections regularly.  

Over time our controllers evolved. In the beginning, the prototype of the controllers was tested using an Arduino Leonard board and foil paper to test out the connection and dynamic. After we were confident that the dynamics of the controllers fit with this station, we then constructed it and tested the materials – plastic plates and containers. At this stage, the controllers were still wired directly to the computer and it was clear that the controllers were household objects. After multiple rounds of playtesting, we then moved on to construct the final version of the prototype. We replaced all the materials with newer ones, sanded and repainted the body, as well as built support for the hardware. Lastly, we replaced some of the wires with thicker ones so that they could withhold more force. Because we were aware of the unexpectedness of our controllers, we constructed a simple backup controller in case the original controllers malfunctioned. We used simple arcade buttons along with Arduino Leonard.

We were very happy to hear that the playtesters were very excited by the uniqueness of our controllers. Many of them loved the musical element that the controllers were providing and the textile feeling that the material had. The positive feedback pushed us to continue to develop the controllers and fine-tune them so that they would remain to be the main part of the station’s experience.  

Hands-On Activity

The craft for the Chinatown Station was to have visitors create their very own Chinese lanterns. This is a craft that many elementary school children learn to do in school when learning about Chinese culture, making it a familiar craft that is both fun and easy to assemble, while also teaching about the culture.

Event Day!

Event Outcomes

In the end, on May 7th, 2022, the exhibit was quite successful!

We had 60 visitors, with 98% working through the entire exhibit and completing their map. Our Learning Objectives were met with above-average scores, ranging from 72% to 87% and we had a visitor satisfaction rating of 4.8/5.

We are pleased to say that we achieved our goal of immersing our visitors in the history and culture of our city at this fun-filled exhibit, Next Stop, Philadelphia!