CCC DMC199 Research

Download

(DMC199) Hololens Development Course

Exploring the Integration of HoloLens Technology within Academia for Trade-Based Education

About the Course & Grant

Partners: Microsoft, Intel, Oregon Storyboard & Clackamas Community College

Grant Funds: $100,000

Matched Funds: $100,000 (Intel), $15,000 (CCC), $100,000 (Volunteer Pro Development Hours)

Grant Team: Shelley Midthun, Andy Mingo, J Bills, Nikki Dunsire, Tawny Schlieski

Course Instructors: J bills, Nikki Dunsire, Thomas Wester

Core Student Builder: Philip Modin

Research & White Paper prepared by: Anthony Pires, Shelley Midthun

In October 2015, Clackamas Community College and four other universities (Carnegie Mellon, Dartmouth, Virginia Tech, and UC Berkeley) were chosen as recipients of the Microsoft’s HoloLens Academic Research Grant. I was a part of this team. Microsoft established this grant to better understand the role and possible applications for holographic computing, which would revolutionize the computing experience. Additional goals were to stimulate and advance academic research in mixed reality and encourage applications of holograms for new and unique purposes. This opportunity allowed for early access to an emerging technology and early exploration of its potential within a learning environment.

In the spring of 2016, prior to this initial public release, Microsoft made HoloLens Developer Kits available to early adopters and industry pioneers. Oregon Story Board (OSB), Clackamas Community College (CCC), and Intel partnered to create an educational opportunity for developers and storytellers for the HoloLens and the broader Augmented, Mixed and Virtual Reality fields, within that early adoption space.

To better understand the technology that we would be interfacing with, Andy Mingo, Shelley Midthun, J Bills and I were provided an opportunity to attend the Microsoft Hololens Academy on the Microsoft campus.

This experience was unlike any other, providing in-depth training on the headset’s nuances, baseline coding requirements, and the hand gestures recognizable by the head-mounted display’s camera. During the training, we identified a few potential challenges:

While impressive, the field of view was extremely limited.
Building and loading content required patience and expertise in C#, creating a skills barrier.
The head-mounted display (HMD) was uncomfortable.

As part of our proposal for augmented trade-based education, OSB welcomed students and professionals eager to work with this emerging technology. The initial CCC/OSB grant application proposed the development of "project-centered, studio classes" that would bring together students from various Portland-area schools to work in cross-disciplinary teams with industry professionals. Using software such as Unity3D, Maya, and Microsoft’s HoloLens, students created an interactive holographic model of an automotive transmission. This final project was designed as a supplemental teaching tool for the Automotive Services Training program at CCC. I was one of three instructors leading this course, alongside John Michael Bills (co-instructor) and Thomas Wester, who took over my position when I returned to full-time responsibilities at The Art Institute of Portland.

The DMC199 class represented a controlled venture into uncharted territory, lookig to leverage the collective talent of the HoloLens grant team and some of Portland’s brightest minds to develop content for this new mixed reality medium. The team included students and professionals with a range of skills, prioritizing Unity and C# expertise, as well as digital design skills such as modeling and texturing.

Assessing Immersive Build Curriculum

The primary objective of this grant was to integrate Microsoft HoloLens technology into a competency-based, academically structured, exploratory environment. While many academic institutions show enthusiasm for adopting emerging technologies, early champions often face obstacles such as limited support, technical challenges, unclear student learning outcomes, and difficulties demonstrating a clear return on initial investments.

In their white paper, The Digital Promise: Transforming Learning with Innovative Uses of Technology, Dr. Michael Levine, founding director of the Joan Ganz Cooney Center, and Jeanne Wellings (2009) emphasize the importance of preparing students with advanced technology skills. They note, “Schools need to function in a 21st-century work environment where employees are expected to collaborate on projects, incorporate feedback from a work group and a supervisor, and make connections between new and existing knowledge.” The collaborative, exploratory approach of CCC’s DMC199 classes reflected these principles. The classes were offered through Clackamas Community College, though they took place at the Oregon Story Board facilities in downtown Portland.

The DMC199 series engaged students and participants in a nine-month journey to understand the HoloLens, develop content for it, and create holographic applications for trade-based education. Based on student testimonials and qualitative interviews, the following areas emerged as strengths of the program:

Professional and Industry Partnerships: The DMC199 series brought together participants from diverse professional backgrounds, including guest speakers specializing in Virtual Reality (VR) and Augmented or Mixed Reality (AR/MR) business models, instructors with VR/AR/MR content creation skills, technology enthusiasts, early adopters, and visual media students. This diversity of perspectives enriched the classroom environment, promoting valuable problem-solving approaches and essential skills for content creation.

Learn-by-Doing with HoloLens Training: Participants reported high satisfaction with hands-on HoloLens training. Through direct engagement with the technology and troubleshooting development challenges, students gained competencies that were highly valued throughout the DMC199 courses.

Exploratory Classroom Environment: The DMC199 series coincided with the pre-release phase of HoloLens, meaning that the hardware, software, and development processes were still in an exploratory stage. Consequently, the students’ learning experience was also exploratory. They collaborated to solve problems, applied their strengths to the project, and built upon prior knowledge.

Levine and Wellings (2009) highlight that leaders across academia, industry, and philanthropy agree that “…the key to engaging the 21st-century digital generation involves harnessing its passion for media and technology and incorporating it into rigorous, more participatory learning experiences.”

The Evolution of Exploratory Pathways

The DMC199 classes were laid out to achieve the following student learning outcomes:

During the assessment of student learning outcomes for DMC199, the curriculum specialists discovered that the development process for creating HoloLens content includes specific learning pathways. These pathways evolved when students utilized their initial skillsets to build upon the project-‐based outcomes. These exploratory pathways were categorized into three main groups: Developers, 3D Modelers, and Experience Designers (Instructional and User Experience).

While the participants gravitated to their respective development roles holistically, participant feedback indicated that they would have derived greater benefit from a more concentrated track in which they could expand upon their existing knowledge of programming, 3D modeling, or user experience design.

“I would suggest a more prescriptive approach in methodology. If instructors knew multiple skill sets were coming in, there should’ve been thought behind how the class could benefit from each development skill.“

-Student Testimonial of DMC199 series

Integration of emerging technologies faced hurdles in the classroom, primarily with achieving the originally desired student learning outcomes. Multiple skillsets were required for development with the Microsoft HoloLens, so students would benefit from training within key areas of the development process prior to product development as a team. With student learning outcomes focused in areas of user-‐experience design, 3D development, and programming for HoloLens development, competencies should align to enhance team collaboration and generate a stronger team project outcome.

Challenges with Emerging Technology in the Classroom

Many institutions view their technology department as a “support” resource for their campus. Dr. Willard R. Daggett, author of Preparing Students for Their Technological Future, stated that as educators, “…we are unwittingly failing to prepare our students for a technology-‐driven world that is nothing like the place that many of us graduated into.” To positively impact DMC199 classroom instruction, OSB and CCC had to embrace and embed new technology resources. During the initial phase of setup for OSB’s grant collaboration with Microsoft and CCC, OSB’s instructors and technology support staff had to build a fully equipped computer lab and IT infrastructure from the ground up. Emerging technology requires adaptation in an ever changing environment.

The integration of emerging technologies embedded within classroom instruction is much more complex than delivering information about an already established topic. Some of the main challenges that this grant project encountered resonate with the complexities of teaching new technology and their processes. The following were the top challenges gathered from participant interviews:

Lacking Defined Assessment Methods: The DMC199 series was built as an exploratory class with less focus on grading and more on experimentation with HoloLens technology. While exploration with emerging technology has benefits, it left some participants with an unclear assessment of how they were progressing in the class.
Software/Hardware Challenges: One of the biggest challenges was the variety of collective software used in the pipeline of HoloLens production. HoloLens development required different types of software that needed to be integrated to work as a whole system. Some of the software suites were Unity, Visual Studio, and Windows 10. The software often updated at randomized intervals, which impacted the production pipeline workflow and halted project productivity if updates were not addressed consistently. These pauses in productivity during instruction meant making unplanned modifications to original lesson plans, which affected classroom workflow and frustrated instructors and students.
Oversaturated Communication Methods: The DMC199 classes intended to integrate real-‐world production processes within their structure, using project management and communication methods to interact and to organize processes with participants. The software used to communicate with the participants included Basecamp, Slack, Google Docs, GitHub, personal and CCC email accounts. The variety of communication methods confused some participants at times. This confusion led to project development setbacks and feelings of frustration.
Class Structure Inconsistencies: During the nine (9) month period of the DMC199 series, new instructors were assigned every three months to teach competencies for specific areas of HoloLens development. Each instructor had different teaching methodologies and classroom management approaches. These transitions created an additional learning curve for participants. Another inconsistent element was the hand off of the holographic transmission progress. Each instructor had a different approach to creating transmission content and translating instructional design elements and process.

These challenges can be addressed when setting up a learning environment and prepping classroom instructors and technical support. The challenges above reflected a negative impact in student satisfaction but, if taken into consideration during curriculum development and classroom planning, they can be improved upon in future iterations.

What is the Educational Value of HoloLens Content in the Classroom?

Early adopters of the Microsoft HoloLens will most likely convey that first-‐time users need to experience the device to understand its full potential. This perspective can be attributed to both the development of holographic content, as well as use during experiential learning in the classroom. In his research study on experiential learning, John Richardson noted that our educational program delivery systems should include delivery methods that provide opportunities for clientele to gain sensory, exploratory experience with the information being presented (Richardson, 1994).

The holographic transmission was deployed using three (3) primary methods:

Faculty lecture using the HoloLens
Individual review during lab with HoloLens
Faculty led, group review using the HoloLens

Our goals were to build an experience that:

Helps students better understand and imagine the inner workings of a car
Helps instructors explain the front-‐wheel transmission without creating new curriculum
Increase comprehension without the need for more instruction hours

Trade-‐Based Classroom Study

Instructors from the DMC199 series and an automotive instructor from CCC collaborated with a curriculum specialist to develop and integrate the holographic transmission into the appropriate automotive lesson plan. The students in the CCC automotive class were learning about the front wheel transmission assembly, which was comprised of an eight (8) gear system. The holographic transmission built for HoloLens contained the first gear components, animated the assembly process, and included an “x-‐ray” command function to view the internal workings of the transmission assembly parts.

The holographic transmission was labeled and color-‐coded to match the automotive manual that faculty and students used in the classroom. A Likert-‐Scale questionnaire was then used to evaluate the experience of students in the CCC automotive classroom while they used the holographic transmission produced within the DMC199 class as a learning tool. The sample size of automotive classroom participants was nine (9) in total.

Findings

The demographic analysis indicated that most respondents were adults from 19 to 23 years old. All of the participants were enrolled at Clackamas Community College. About two-‐thirds were full-‐time students, 13% were part-‐time auto repair technicians, and the remainder indicated their primary occupation was outside of the automotive field. Overall, 75.7% of participants strongly agreed that the HoloLens and the Holographic Transmission, had a positive impact on learning about trade-‐based automotive content.

Student Learning Perspective

To determine the effectiveness of the holographic transmission, students were given a survey to evaluate their time spent with the HoloLens and their understanding of content associated with the holographic transmission. The Likert-‐Scale ranged from one (1) through seven (7); with one (1) being “Strongly Disagree” and seven (7) being “Strongly Agree.” Data indicated an overwhelming degree of positive satisfaction from participants as shown in the Tables below.

Participants were asked if the features within the holographic transmission, particularly the “x-ray” mode, increased comprehension and understanding of the content. In their responses, an average of 75.7% of participants agreed that the holographic transmission increased knowledge and understanding of the content. These findings, shown in Table 2, also indicated that an average of 90% of participants thought that the x-‐ray mode enhanced their ability to comprehend moving transmission components and their interactions with each other.

With respect to hardware accessibility and gainful employment, Table 3 indicated that an average of 80.0% of participants strongly agreed that access to the HoloLens in the classroom improved their technical awareness and potential job opportunities. Table 4 showcased that an average of 87.1% of participants would like to see HoloLens technology used more frequently in their classes. This data provided insight into the students’ demand for emerging technologies, particularly with interests toward potential job prospects.

Findings indicated that students who used the HoloLens content gained a stronger understanding of the concept than learning through the traditional training medium of text and lecture. An average of 94.3% of participants indicated that they strongly agreed that the HoloLens transmission matched their existing instructional manual. The instructor also emphasized the correlation between the manual and holographic transmission during his HoloLens-‐assisted lecture. The instructor was involved in the DMC199 holographic transmission development process and thus was familiar with how to use the hardware and associated content. Based on participant response, an average of 87.1% of students strongly agreed that the instructor used the HoloLens effectively for in-‐class lecture.

Summary

HoloLens Curriculum (DMC199)

The results of participant interviews indicated primarily positive views towards the DMC199 series. Participant feedback indicated high satisfaction with learning how to develop content for the HoloLens. Through hands on engagement with the technology, as well as trouble-‐shooting through augmented reality development challenges, students valued competencies learned throughout the DMC199 courses.

Participant feedback expressed desire to expand upon student learning outcomes in areas of user-‐experience design, 3D asset development, and programming for HoloLens applications. Expanding upon these student learning outcomes during curriculum design will enhance team collaboration, as well as generate stronger project-‐based outcomes.

HoloLens Holographic Transmission Learning Content

Findings demonstrated a high degree of acceptance and engagement with HoloLens technology in trade-based education. The enhanced learning experience provided opportunities for students to observe, interact, and discuss content within a traditional classroom setting. Developing a program that incorporates these interactive components will benefit both instructors and students.

Evidence from the pilot project indicated that HoloLens technology had a positive impact on the student learning experience in the automotive class at Clackamas Community College. After the class, instructor Rick Lockwood shared his insights on how the HoloLens improved student comprehension and engagement:

“Traditionally, in my class, [auto transmission] powerflow was taught through power point presentation and hands on with actual transmission components. Powerflow has always been a huge obstacle for students to overcome. The average student would spend a minimum of three weeks mastering this concept and some as much as five weeks. I often found myself demonstrating powerflow to the students more times then I care to count. After the introduction of HoloLens, most all of my students were able to demonstrate first gear powerflow in the first week. In my opinion, HoloLens made a significant difference in the student’s ability to learn powerflow.”

-Rick Lockwood

Clackamas Community College Automotive Service Technology

DMC199: Hololens Academic Research Grant