CCC DMC199 Research

Download

(DMC199) Hololens Development Course

Exploring the Integration of HoloLens Technology within Academia for Trade-Based Education

About the Course & Grant

Partners: Microsoft, Intel, Oregon Storyboard & Clackamas Community College

Grant Funds: $100,000

Matched Funds: $100,000 (Intel), $15,000 (CCC), $100,000 (Volunteer Pro Development Hours)

Grant Team: Shelley Midthun, Andy Mingo, J Bills, Nikki Dunsire, Tawny Schlieski

Course Instructors: J bills, Nikki Dunsire, Thomas Wester

Core Student Builder: Philip Modin

Research & White Paper prepared by: Anthony Pires, Shelley Midthun

In the spring of 2016, Microsoft released the HoloLens, a mixed-‐reality headset that revolutionized the computing experience. Prior to this initial public release, Microsoft made HoloLens Developer Kits available to early adopters and industry pioneers. Oregon Story Board (OSB), Clackamas Community College (CCC), and Intel partnered to create an educational opportunity for developers and storytellers for the HoloLens and the broader Augmented, Mixed and Virtual Reality fields, within that early adoption space.

In October 2015, Clackamas Community College and four other universities (Carnegie Mellon, Dartmouth, Virginia Tech, and UC Berkeley) were chosen as recipients of Microsoft’s HoloLens Academic Research Grant. I was a part of this team. Microsoft established this grant to better understand the role and possible applications for holographic computing. Additional goals were to stimulate and advance academic research in mixed reality and encourage applications of holograms for new and unique purposes. This opportunity allowed for early access to an emerging technology and early exploration of its potential within a learning environment.

To better understand the technology that we would be interfacing with, Andy Mingo, Shelley Midthun, J Bills and I were provided an opportunity to attend the Microsoft Hololens Academy on the Microsoft campus.

This experience was unlike any other. We were introduce to the nuances of the headset, code needed to implement baseline functions and visuals, and training on the hand gestures recognizable to the camera on the head-mounted display. Right away we recognized some potential challenges:

Though still impressive, the field of view was incredibly small
Building and loading content would require patience and skill, particularly in coding (C#)
The HMD was not comfortable

As part of the proposal for augmented trade-‐based education, Oregon Story Board opened its doors to interested students and working professionals excited to take part in this emerging technology. Part one of the CCC/OSB grant application proposed that we create "project-centered, studio classes,” which would allow students from various Portland-area schools to work with industry professionals in cross-disciplinary teams to develop content for the HoloLens. Using software such as Unity3D and Maya and Microsoft’s HoloLens, students created a holographic interactive auto transmission. The final HoloLens project would then be used as a supplemental teaching tool within the Automotive Services Training program at CCC. I was one of the three instructors who taught the course, along side John Michael Bills (co-instructor) and Thomas Wester who stepped into my position when I was pulled back onto endeavors at The Art Institute of Portland in a full-time capacity.

The DMC199 class was a controlled adventure into the unknown, with a goal to apply the combined talent of the Hololens grant team with some of the best minds in Portland, towards a new visual communication medium utilizing this new mixed reality device. The grant team engaged students and professionals with a full range of skills; Unity and C# were a priority, as were digital design skills such as modeling and texturing.

Assessing Curriculum for Emerging Technologies

A main objective of this grant was to embed the Microsoft HoloLens technology within an academically structured, competency-‐based, exploratory environment. Many academic institutions express enthusiastic interest in bringing emerging technologies to their institutions, however an early champion’s initial intent is often derailed due to lack of support, technical understanding, obtaining clear student learning outcomes, and/or the ability to demonstrate a clear value in the initial investment.

In their whitepaper, “The Digital Promise: Transforming Learning with Innovative Uses of Technology,” Dr. Michael Levine, founding director of the Joan Ganz Center, and colleague, Jeanne Wellings (2009) say that students need preparation through advanced technologies: “Schools need to function in a 21st century work environment where employees are expected to collaborate on projects, incorporate feedback from a work group and a supervisor, and make connections between new and existing knowledge.” The collaborative and exploratory classroom environment for the CCC DMC199 classes touched on exactly these principles. The DMC199 classes were run through Clackamas Community College, but held at the Oregon Story Board facilities in downtown Portland.

The DMC199 series engaged participants and students in a nine-‐month journey to understand the HoloLens, develop content for it, and apply those development practices towards created holographic content for trade-‐based education. Centered on student testimonials and qualitative interviews, the following categories were areas with positive student feedback:

Professional/Industry Partnerships: The DMC199 series included various participants with a range of professional backgrounds: guest speakers with a focus in Virtual Reality (VR) and Augmented or Mixed Reality (AR/MR) business models, instructors with skillsets that aligned with VR/AR/MR content creation, enthusiasts and early adopters of the technology, and college students enrolled in visual media programs. This diversity in perspective added value throughout the classroom environment and positively influenced problem-‐solving and necessary skillsets required for content creation.
Learn by Doing w/ HoloLens Training: Participant feedback indicated high satisfaction with learning how to develop content for the HoloLens. Through hands on engagement with the technology, as well as trouble-‐shooting through development challenges, students valued the competencies gained throughout the DMC199 courses.
Exploratory Classroom Environment: The DMC199 series ran in conjunction with the pre-‐release of the HoloLens technology. The hardware, software, and development processes were in a stage of exploration. In turn, the students’ educational experience was also partially exploratory in nature. Students had to work together to problem solve, apply personal strengths to the project, and build upon preexisting knowledge.

Wellings and Levine (2009) noted that researchers across many prestigious universities, industry leaders, and philanthropies, have concluded that “…the key to engage the 21st century digital generation involves harnessing its passion for media and technology and incorporating it into rigorous, more participatory learning experiences.”

The Evolution of Exploratory Pathways

The DMC199 classes were laid out to achieve the following student learning outcomes:

During the assessment of student learning outcomes for DMC199, the curriculum specialists discovered that the development process for creating HoloLens content includes specific learning pathways. These pathways evolved when students utilized their initial skillsets to build upon the project-‐based outcomes. These exploratory pathways were categorized into three main groups: Developers, 3D Modelers, and Experience Designers (Instructional and User Experience).

While the participants gravitated to their respective development roles holistically, participant feedback indicated that they would have derived greater benefit from a more concentrated track in which they could expand upon their existing knowledge of programming, 3D modeling, or user experience design.

“I would suggest a more prescriptive approach in methodology. If instructors knew multiple skill sets were coming in, there should’ve been thought behind how the class could benefit from each development skill.“

-Student Testimonial of DMC199 series

Integration of emerging technologies faced hurdles in the classroom, primarily with achieving the originally desired student learning outcomes. Multiple skillsets were required for development with the Microsoft HoloLens, so students would benefit from training within key areas of the development process prior to product development as a team. With student learning outcomes focused in areas of user-‐experience design, 3D development, and programming for HoloLens development, competencies should align to enhance team collaboration and generate a stronger team project outcome.

Challenges with Emerging Technology in the Classroom

Many institutions view their technology department as a “support” resource for their campus. Dr. Willard R. Daggett, author of Preparing Students for Their Technological Future, stated that as educators, “…we are unwittingly failing to prepare our students for a technology-‐driven world that is nothing like the place that many of us graduated into.” To positively impact DMC199 classroom instruction, OSB and CCC had to embrace and embed new technology resources. During the initial phase of setup for OSB’s grant collaboration with Microsoft and CCC, OSB’s instructors and technology support staff had to build a fully equipped computer lab and IT infrastructure from the ground up. Emerging technology requires adaptation in an ever changing environment.

The integration of emerging technologies embedded within classroom instruction is much more complex than delivering information about an already established topic. Some of the main challenges that this grant project encountered resonate with the complexities of teaching new technology and their processes. The following were the top challenges gathered from participant interviews:

Lacking Defined Assessment Methods: The DMC199 series was built as an exploratory class with less focus on grading and more on experimentation with HoloLens technology. While exploration with emerging technology has benefits, it left some participants with an unclear assessment of how they were progressing in the class.
Software/Hardware Challenges: One of the biggest challenges was the variety of collective software used in the pipeline of HoloLens production. HoloLens development required different types of software that needed to be integrated to work as a whole system. Some of the software suites were Unity, Visual Studio, and Windows 10. The software often updated at randomized intervals, which impacted the production pipeline workflow and halted project productivity if updates were not addressed consistently. These pauses in productivity during instruction meant making unplanned modifications to original lesson plans, which affected classroom workflow and frustrated instructors and students.
Oversaturated Communication Methods: The DMC199 classes intended to integrate real-‐world production processes within their structure, using project management and communication methods to interact and to organize processes with participants. The software used to communicate with the participants included Basecamp, Slack, Google Docs, GitHub, personal and CCC email accounts. The variety of communication methods confused some participants at times. This confusion led to project development setbacks and feelings of frustration.
Class Structure Inconsistencies: During the nine (9) month period of the DMC199 series, new instructors were assigned every three months to teach competencies for specific areas of HoloLens development. Each instructor had different teaching methodologies and classroom management approaches. These transitions created an additional learning curve for participants. Another inconsistent element was the hand off of the holographic transmission progress. Each instructor had a different approach to creating transmission content and translating instructional design elements and process.

These challenges can be addressed when setting up a learning environment and prepping classroom instructors and technical support. The challenges above reflected a negative impact in student satisfaction but, if taken into consideration during curriculum development and classroom planning, they can be improved upon in future iterations.

What is the Educational Value of HoloLens Content in the Classroom?

Early adopters of the Microsoft HoloLens will most likely convey that first-‐time users need to experience the device to understand its full potential. This perspective can be attributed to both the development of holographic content, as well as use during experiential learning in the classroom. In his research study on experiential learning, John Richardson noted that our educational program delivery systems should include delivery methods that provide opportunities for clientele to gain sensory, exploratory experience with the information being presented (Richardson, 1994).

The holographic transmission was deployed using three (3) primary methods:

Faculty lecture using the HoloLens
Individual review during lab with HoloLens
Faculty led, group review using the HoloLens

Our goals were to build an experience that:

Helps students better understand and imagine the inner workings of a car
Helps instructors explain the front-‐wheel transmission without creating new curriculum
Increase comprehension without the need for more instruction hours

Trade-‐Based Classroom Study

Instructors from the DMC199 series and an automotive instructor from CCC collaborated with a curriculum specialist to develop and integrate the holographic transmission into the appropriate automotive lesson plan. The students in the CCC automotive class were learning about the front wheel transmission assembly, which was comprised of an eight (8) gear system. The holographic transmission built for HoloLens contained the first gear components, animated the assembly process, and included an “x-‐ray” command function to view the internal workings of the transmission assembly parts.

The holographic transmission was labeled and color-‐coded to match the automotive manual that faculty and students used in the classroom. A Likert-‐Scale questionnaire was then used to evaluate the experience of students in the CCC automotive classroom while they used the holographic transmission produced within the DMC199 class as a learning tool. The sample size of automotive classroom participants was nine (9) in total.

Findings

The demographic analysis indicated that most respondents were adults from 19 to 23 years old. All of the participants were enrolled at Clackamas Community College. About two-‐thirds were full-‐time students, 13% were part-‐time auto repair technicians, and the remainder indicated their primary occupation was outside of the automotive field. Overall, 75.7% of participants strongly agreed that the HoloLens and the Holographic Transmission, had a positive impact on learning about trade-‐based automotive content.

Student Learning Perspective

To determine the effectiveness of the holographic transmission, students were given a survey to evaluate their time spent with the HoloLens and their understanding of content associated with the holographic transmission. The Likert-‐Scale ranged from one (1) through seven (7); with one (1) being “Strongly Disagree” and seven (7) being “Strongly Agree.” Data indicated an overwhelming degree of positive satisfaction from participants as shown in the Tables below.

Participants were asked if the features within the holographic transmission, particularly the “x-ray” mode, increased comprehension and understanding of the content. In their responses, an average of 75.7% of participants agreed that the holographic transmission increased knowledge and understanding of the content. These findings, shown in Table 2, also indicated that an average of 90% of participants thought that the x-‐ray mode enhanced their ability to comprehend moving transmission components and their interactions with each other.

With respect to hardware accessibility and gainful employment, Table 3 indicated that an average of 80.0% of participants strongly agreed that access to the HoloLens in the classroom improved their technical awareness and potential job opportunities. Table 4 showcased that an average of 87.1% of participants would like to see HoloLens technology used more frequently in their classes. This data provided insight into the students’ demand for emerging technologies, particularly with interests toward potential job prospects.

Findings indicated that students who used the HoloLens content gained a stronger understanding of the concept than learning through the traditional training medium of text and lecture. An average of 94.3% of participants indicated that they strongly agreed that the HoloLens transmission matched their existing instructional manual. The instructor also emphasized the correlation between the manual and holographic transmission during his HoloLens-‐assisted lecture. The instructor was involved in the DMC199 holographic transmission development process and thus was familiar with how to use the hardware and associated content. Based on participant response, an average of 87.1% of students strongly agreed that the instructor used the HoloLens effectively for in-‐class lecture.

Summary

HoloLens Curriculum (DMC199)

The results of participant interviews indicated primarily positive views towards the DMC199 series. Participant feedback indicated high satisfaction with learning how to develop content for the HoloLens. Through hands on engagement with the technology, as well as trouble-‐shooting through augmented reality development challenges, students valued competencies learned throughout the DMC199 courses.

Participant feedback expressed desire to expand upon student learning outcomes in areas of user-‐experience design, 3D asset development, and programming for HoloLens applications. Expanding upon these student learning outcomes during curriculum design will enhance team collaboration, as well as generate stronger project-‐based outcomes.

HoloLens Holographic Transmission Learning Content

The results of these studies indicated that integrating HoloLens technology within academic, trade-‐based education has a strong degree of acceptance from its participants. The learning process was enhanced by providing opportunities for students to see, interact, and discuss the information while in their traditional classroom setting. Thus, development and implementation of a comprehensive program that included these components will be most effective for all participants involved in both delivering and receiving the HoloLens curriculum and content.

Evidence indicated that students had an enhanced learning experience with the HoloLens technology during the holographic transmission pilot project. Upon completion of the automotive class at Clackamas Community College, instructor Rick Lockwood gave this statement about his experience with the HoloLens and its effect on student learning:

“Traditionally, in my class, [auto transmission] powerflow was taught through power point presentation and hands on with actual transmission components. Powerflow has always been a huge obstacle for students to overcome. The average student would spend a minimum of three weeks mastering this concept and some as much as five weeks. I often found myself demonstrating powerflow to the students more times then I care to count. After the introduction of HoloLens, most all of my students were able to demonstrate first gear powerflow in the first week. In my opinion, HoloLens made a significant difference in the student’s ability to learn powerflow.”

-Rick Lockwood

Clackamas Community College Automotive Service Technology

DMC199: Hololens Academic Research Grant