CMU METALS Capstone Project
Client: Virginia Tech
Duration: 8 months
Team: Yali Chen, Junyu Huang, Nicholas Lewis, Yilin Ying
My Role: Team Lead
Domain: Education, Technology
A collaboration between
Every year, over 1,600 undergrad students at Virginia Tech are taking Calculus as one of the required courses. Virginia Tech tries to facilitate higher rates of access and student success through tutoring service and office hours. The challenge is to create a learning resource to facilitate more effective tutoring in individual and group sessions, specifically in foundational calculus courses.
Our Solution: Create a better peer-learning community
Through user-centered research methods, an iterative design process, and prototype development, we have created the mobile app Together that enhances peer-learning and group tutoring experiences in calculus. This solution will help build a sustainable learning community for students in Math 1225.
A mobile app that will help Virginia Tech improve peer learning and tutoring experiences, facilitate human connections, and build a learning community. Together connects students with peers and provides valuable information to tutors, instructors, and administrators.
The process outlined below provided the means to systematically evaluate the experiences of both students and tutors at Virginia Tech in order to determine opportunities for improvement. It is a systematic and iterative approach that combines user needs with academic research and creative solutions, focused on details without losing sight of the bigger picture.
1. Literature Review
Before conducting field research at Virginia Tech, we performed a literature review to explore as much as we could about calculus for first-year university students. Additional competitive analyses were conducted on tutoring programs and services in other universities. These contexts (outside of Virginia Tech) provide a look into similarities and differences across other universities and products.
2. User Research
With the help of Virginia Tech, we identified our target audience of students who had received tutoring for Math 1225, the tutors themselves, and coordinators for both tutoring services and calculus. These observations were critical for discovering the student and tutor experience. Pairs of team members, one acting as the note taker and the other as the interviewer, spoke with 25 participants over a period of three days. Our structured interviews covered general topics which led to deeper discussions focusing on details that were particular to the participant being interviewed.
While our interviews provided a strong foundation, we discovered we lacked the broad perspective of typical students that take Math 1225. We had talked with a handful of students who had received tutoring, but we were also interested in those who did not seek help or found help from other resources. A survey was developed to gather opinions about tutoring services, the classroom environment, attitudes about resources, and help-seeking behavior from a sample of all students who took Math 1225 in the previous year.
3. User Testing
A user testing session is dependent on creating good tasks for users through our user stories we developed. All tasks were described as specific end goals signifying something that the participant is trying to accomplish. We tried to phrase tasks in familiar ways that were related to the users lives. We started this process with simple mockups, began testing with simple prototypes and continued iterations while watching the design become more effective with each cycle. The following highlights the refinement of the prototype over time.
Our goal for this part is to answer the following three questions:
What do students need to be successful in Calculus?
What do tutors need to better support students’ success in Calculus?
What does the system need to support students’ success in Calculus?
These needs are directly synthesized from the insights that we found from user research and which were validated by literature reviews.
We used the activity affinity diagramming as a bottom-up method to explore and understand the natural relationships among the information collected in the user research. Raw interview data were turned into transcripts and we began sorting and gathering relevant quotes into a format we could easily examine.
The flow model helped us visualize different categories of findings from user research data and to identify where the system could get improved.
Current user's challenges and pain points
Students need help that is convenient and on their own schedule. Students want more tutor availability and accessible resources. But currently, there is limited time, space, and resources.
Students need more training on self-regulated problem-solving. Students note that they want advice that is more practical rather than purely conceptual, which includes strategies and skills for studying.
Students need more practice materials on the required procedural skills and concepts. Students often find the materials too difficult and taught at too fast of a pace resulting in inadequate preparation for assignments and tests.
Students need additional outside resources and help that targets material relevant to their courses. Peers and online resources are a major source of help outside of class and tutoring. They can access these resources easily and work at their own pace.
Students need personalized help, guidance, and pacing to support their learning. Students want help that targets their individual needs and problems. But currently, standardized classroom teaching cannot meet students’ variety of needs.
Students need comfortable environments and emotional support. Students prefer a supporting environment where tutors are encouraging and warm so they feel comfortable with asking questions and making mistakes.
Based on our user research, we categorized 3 types of students that are in need of tutoring services or other types of personalized help according to their prior knowledge and skills, motivations, help seeking behaviors, learning experiences and struggles. They are:
(1) "Help Seeker" who highly relies on outside help;
(2) "Resource Gatherer" who highly relies on outside resources;
(3) "Help Avoider“ who has low reliance on outside help or resources.
Our approach to ideation can be conceptualized as a form of brainstorming. Initially, ideation is an entirely generative process where in key themes and concepts from our research take shape into solutions. In later stages, we work to develop and communicate promising solution concepts that will best meet the needs of our client and target users.
In each idea, we targeted different needs from users and generated ideas based on the needs. We also tied our research findings to the process of ideation to make sure the ideas are addressing the real problems and aiming at users’ needs.
Co-design Session with Virginia Tech Administrators
At the end of Spring semester, we presented the research and key findings that inspired our ideation to Virginia Tech administrators. Our client provided insightful feedback and comments about each idea that was pitched. We also held a co-design session with them so that we have a clearer idea about what directions we would take during the summer.
Feature 1: Events
All events, which include peer-to-peer sessions, group tutoring sessions, and office hours, are presented in one place. This creates a convenient feature for students and helps them find events that fit their needs.
Feature 2: Study friends
After taking a matching preference quiz, students can search study friends and get recommendations based on shared learning preferences. These searches can be filtered by major, class section, or residence hall.
Feature 3: Comments
Students can leave comments and respond to other’s comments before tutoring events. This feature also benefits tutors by allowing them to understand students’ questions and problems ahead of time.
Feature 4: Help queue
The help queue in Together facilitates offline interactions in tutor-led group study sessions. When students join a session they can add themselves to a queue based on their topic of interest.
Feature 5: Data Report
A separate report for administrators and instructors to understand the peak hours and peak time for tutoring sessions, office hours or peer-to-peer events; learn about frequently asked topics; view comments from students and tutors.