Research Products

We plan to make available the research products and resources that enable us to advance our understanding of collaborative interactions among student learners. This includes Annotation Schemes and Code. For more information, please contact our team at Info.AI-Institute@Colorado.edu.

Data Annotation Frameworks & Schemes

*Signifies a new annotation framework developed in-house.

Name

Description

*Nonverbal Interactions in Collaborative Learning Environments (NICE)

The NICE framework is used to analyze nonverbal behaviors during collaborative learning. The coding scheme is split between nonverbal actions (gaze, gesture, posture) and nonverbal emotions (positive, negative, bored, confused/concentrating).

Generalized Collaborative Problem-Solving Model (GCM)

Collaborative Problem-Solving addresses both the cognitive abilities required to problem-solve as a team and the social dynamics between group members as they coordinate their knowledge and actions. The GCM framework identifies three core facets of CPS measurable in verbal discourse: constructing shared knowledge, negotiation/coordination, and maintaining team function. Each facet is constructed of sub-facets, which are in turn defined by indicators. Transcripts of collaborations are annotated at the indicator level alongside video. A separate coding scheme is used for nonverbal indicators of CPS which is being combined with NICE (see below).

Abstract Meaning Representation (AMR)

AMR represents the logical meanings of sentences, abstracting away from their syntax. Advantages of AMR include its relative simplicity, ease of annotation, and available corpora.

*Gesture Abstract Meaning Representation (GAMR)

Gesture AMR (GAMR) is an extension to AMR that captures the meaning of gesture as used in multimodal communicative interactions. GAMR focuses on how gesture form and meaning relate, how gesture packages meaning both independently and in interaction with speech, and how the meaning of gesture is temporally and contextually determined.

*On-Topic/Task

The On-Topic/Task annotation scheme identifies whether communication during collaborations is relevant to the specific problem-solving situation confronting the group in that moment and to the context of the curriculum they are working through.

*Dependency Dialog Acts (DDA)

DDA represents the functional, discourse, and response structure in multi-party multi-threaded conversations. It prioritizes the relational structure of the dialogue units and the dialog context, annotating both dialog acts and rhetorical relations as response relations to particular utterances. It also embraces overloading in dialogues, encouraging annotators to specify multiple response relations and dialog acts for each dialogue unit.

*Moments of Support Analysis in Collaboration (MOSAIC)

MOSAIC is used to analyze moments of support in collaborative classrooms. A moment of support occurs when an external person, typically a teacher, offers guidance to students in a small group that is engaged in a curricular task. The aim of analyzing these moments using the MOSAIC framework is to identify the most common forms of support provided and to explore whether these are associated with particular kinds of tasks. These analyses inform the design of AI partners, curriculum routines, and curricular materials.

*Community Agreements (CA)

In iSAT’s classroom curricula, students discuss how they want to behave and collaborate in their classroom. These discussions are guided to coalesce around specific Community Agreements: Respectful, Equitable, Committed to Community, and Moving Our Thinking Forward. The CA annotation scheme identifies behaviors observable in small group interactions that indicate adherence to each of these Community Agreements.

Team Communication Style (TC)

The TC annotation scheme includes indicators of motivational, informative, timely, and collective speech. This coding scheme was developed to measure communication quality and style with a domain-agnostic method.