The behavioural data analysis from Adaptive Learning Platforms (ALPs) is used to create Learning Analytical Dashboards (LADs), which can provide teachers with an overview and recommend pedagogical actions as feedback on students’ behaviour. Existing research focuses on demonstrating ALP value through retaining student engagement and prediction of performance, designing multi-modal data, applying algorithms, and improving software and learning systems. Still, it lacks methods for evaluating ALP’s LADs as they are applied with a pedagogical aim. Our study addresses the empirical research gap of pedagogically grounded ALP LAD prototypes. The pro-totypes analyse student behaviour, aiming at enhancing ALP functionality and assisting teachers in pedagogical reflection. We analysed activity logs from 397 nursing students of Denmark’s University College Absalon from Fall 2020 to Fall 2023 using R. Our study explains students’ behaviour, content difficulty, meta-cognition, and performance through two LAD prototypes and aligns insights with learning theories. Results show that while ‘Activity Labyrinths’ occurrences are infrequent, our first LAD prototype enables teacher awareness of autonomy-related motivational issues. In the second LAD prototype, teachers find the LAD difficult to comprehend. Additionally, the study addresses the black box issue which hinders the design of ALP LADs by utilising behavioural data along with qualitative data in prototyping pedagogically founded LAD prototypes.

Se artiklen her

This article presents findings from a research project that aimed to replace physical books with an adaptive learning platform, Area9 Rhapsode™, for nursing students’ class preparation. As the project expanded, it became evident that students employed diverse preparation practices, prompting an investigation into their preparation habits. This is done through the following research questions: (1) how do students feel about class preparation? (2) what learning resources do students choose? (3) when and where do they prepare for class? This is conducted through a mixed methods design that draws on empirical data from interviews, mobile ethnography, surveys, and log data. The results highlight the wide range of resources students utilise, including digital and analogue tools, the various temporal patterns of preparation, and students’ spectrum of feelings in their preparation. The findings suggest the importance of understanding individual students’ preparation habits to facilitate targeted teaching approaches that cater to diverse learning needs. The article contributes to the existing literature on study habits and strategies, emphasising the need for educators to recognise and adapt to students’ unique preparation behaviours in order to enhance their academic success.

Se artiklen her

Existing methods for software requirements elicitation, five-point Likert scales and voting methods for requirements prioritization, and usability and  user experience evaluation methods do not enable prioritizing the learning analytics dashboard requirements. Inspired by management and product design field, this research applies Kano’s two-factor theory to prioritize the features of learning analytics dashboards (LADs) of adaptive learning platform (ALP) called Rhapsode^TM learner, based on students’ perceived usefulness to support designers’ decision-making. Comparing usability and user experience methods for evaluating LAD features, this paper contributes with the protocol and a case applying Kano method for evaluating the perceived importance of the dashboards in ALP. The paper applies Kano’s two-factor questionnaire on the 13 LADs features of Rhapsode^TM learner. Responses from 17 students are collected using a questionnaire, which is used to showcase the strength of the two-factor theory through five tabular and graphical techniques. Through these five tabular and graphical techniques, we demonstrate the application and usefulness of the method as designers and management are often carried away by the possibilities of insights instead of actual usefulness. The results revealed a variation in the categorization of LADs depending on the technique employed. As the complexity of the techniques increases, additional factors that indicate data uncertainty are gradually incorporated, clearly highlighting the growing requirement for data. In the case of RhapsodeTM learner platform, results based on the students responses show that 11 of 13 LADs being excluded due to low significance level in categorization (technique 1) and low response rate.

Se artiklen her

This research paper highlights and addresses the lack of a systematic review of the methods used to evaluate Learning Analytics (LA) and Learning Analytics Dashboards (LAD) of Adaptive Learning Platforms (ALPs) in the current literature. Addressing this gap, the authors built upon the work of Tretow-Fish and Khalid (2022) and analyzed 32 papers, which were grouped into six categories (C1-6) based on their themes. The categories include C1) the evaluation of LA and LAD design and framework, C2) the evaluation of user performance with LA and LAD, C3) the evaluation of adaptivity, C4) the evaluation of ALPs through perceived value, C5) the evaluation of Multimodal methods, and C6) the evaluation of the pedagogical implementation of ALP’s LA and LAD. The results include a tabular summary of the papers including the categories, evaluation unit(s), methods, variables and purpose. While there are numerous studies in categories C1-4 that focus on the design, development, and impact assessment of ALP’s LA and LAD, there are only a few studies in categories C5 and C6. For the category of C5), very few studies applied any evaluation methods assessing the multimodal features of LA and LADs on ALPs. Especially for C6), evaluating the pedagogical implementation of ALP’s LA and LAD, the three dimensions of signature pedagogy are used to assess the level of pedagogy evaluation. Findings showed that no studies focus on evaluating the deep or implicit structure of ALP’s LA. All studies examine the structural surface dimension of learning activities and interactions between students, teachers, and ALP’s LA and LAD, as examined in categories C2-C5. No studies were exclusively categorized as a C6 category, indicating that all studies evaluate ALP’s LA and LAD on the surface structure dimension of signature pedagogy. This review highlights the lack of pedagogical methodology and theory in ALP’s LA and LAD, which are recommended to be emphasized in future research and ALP development and implementation.

Se artiklen her

 

In the present study, we followed a group of students over the course of a three-semester project aimed at increasing their abilities to express their competence development. Extensive data (video, transcripts, documentation of artifacts and communications, and field notes) was analyzed in a framework analysis based on cultural–historical activity theory (CHAT). We understood contradictions, either within or between students’ activity systems, as drivers for expansive learning. Based on the in-depth analysis of three exemplary students, this study adds to the understanding of competence development in higher education by pointing out its individuality and situatedness, as well as the social dimension of expansive learning. Implications call for researchers and educators not to take competence development for granted when designing learning opportunities, but deliberately to create spaces in which competence expression can be facilitated.

 

Se artiklen her

The restrictions during COVID-19 pandemic resulted in gaining more experience on video conferencing systems (VCS) and continued adoption during post-pandemic teaching scenarios. Designing and installing video conferencing systems in various classroom scenarios are expensive, increase complexity, and reduce interaction opportunities unless the designs for learning activities are well-defined and well-executed. For improving the quality of contact hours, the EdViCon Erasmus + project’s aim is to explore the existing diversity of scenarios of video conferencing systems’ use, various methods, and didactics applied for engaging students using various software and VCS features and define and develop two portable VCS toolkits and training. This paper reports the empirical case of Technical University of Denmark by applying phonomyography [1] as the methodology for exploring the VCS use scenarios, methods, and didactics for exploring problems and good practices. Despite increased adoption and number of papers on VCS, the insights on the diversity of the scenarios of use during contact hours has not been sufficiently reported from the viewpoint of HCI researchers. This study applies Shuman’s [2] concept of signature pedagogies as the theoretical viewpoint for defining scenarios of the use of VCS in teaching and David Benyon’s [3] PACT (people, activities, context, technologies) framework for analysing functionalities, methods, and didactics. The causes and effects of problems with VCS-mediated contact-hour activities were grouped into seven and 12 categories respectively. Seven types of teaching environments, three types of video-conferencing systems (fixed, on-wheels, and mobile), and the various tools used for engaging students were identified.

Du kan finde paperet her

This study contributes with a case study on redesigning three Learning Analytics Dashboards (LADs) of the adaptive learning platform Rhapsode™ with instructions for pedagogical actions. Applying self determination theory’s elements of competence and relatedness and mental models in a design thinking process, the differences among the teachers perceptions and the designers intentions are highlighted through several methods to answer the questions of: How might we improve the learning analytics dashboards by prioritizing course instructors’ perceived competence and relatedness? and How might we redesign learning analytics dashboards by including course instructors’ purpose, insights, and recommending actions?

These questions are answered first by developing three Role-based Personas of Alina Action, Niels Novice, and Paul Privacy along with scenarios and user stories. Second, prototypes of interfaces are designed and tested in three iterations showing insights, recommended actions, and explanation of mechanics. Feedback from the tests on the prototypes receives positive feedback from all teacher personas. The teacher persona of Niels Novice also supplies a criticism of the insights and recommended actions on the basis of creating undesired interpretation, potential bias, taking away freedom of interpretation, and authoritative system that “instructs/orders” action. Additionally, the scope of the study cannot meet the persona of Paul Privacy’s reservations on students’ possible experience of surveillance.

Du kan finde paperet her

Abstract:

From a usability perspective, designing and evaluating an adaptive learning system involve complexities associated with adaptivity and the diverse requirements from content designers, educators, and students. Moreover, students and educators are increasing getting subjected to educational quality studies including the evaluation of digital learning technologies, which the desired participants typically do not perceive as valuable. Furthermore, very few case studies report the results of the usability evaluation of adaptive learning systems. Thus, this study addresses the research questions: (1) Considering low participation and time constraint, which established usability evaluation methods might be applied for mixed-methods study of adaptive learning systems? (2) What is the students’ satisfaction score of Rhapsode^TM Learner and implications for redesign? The result is the selection and application of system usability scale (SUS) and co-Discovery with concurrent interviews for the evaluation of Rhapsode^TM Learner. The average score of 53.8 (n = 15) shows low marginal acceptability with good internal consistency (10 items, Cronbach’s α = 0.863). Analyses of the co-discoveries using 12 interaction design principles enable identifying the scopes for redesigning technological factors. For the evaluation of pedagogy and content aspects, further instruments should be developed.

Du kan finde paperet her

Abstract

There is currently no systematic overview of with what purpose Learning Analytics (LA) and Learning Analytics Dashboards (LAD) are evaluated on  Adaptive Learning Platforms. This work in progress systematic review provides the preliminary results of this endeavor. The paper establishes an overview of the current research field from two reviews. From this foundation we provide an analysis of seven papers. The preliminary results show four different purposes for evaluating LA and LAD on Adaptive Learning Platforms. These are: 1) Evaluation of LA and LAD design and framework, 2) Evaluation of LA and LAD performance, 3) Evaluation of perceived value, and 4) Evaluation of adaptivity. Through examining these papers, we see that when LA and LAD are evaluated on Adaptive Learning Platforms there are both single and multiple purpose of applying an evaluation method. These categories might change as the work in progress develops and more papers gets added in the synthesis.

Du kan finde paperet her