Methods for Evaluating Learning Analytics and Learning Analytics Dashboards in Adaptive Learning Platforms: A Systematic Review

This research paper highlights and addresses the lack of a systematic review of the methods used to evaluate Learning Analytics (LA) and Learning Analytics Dashboards (LAD) of Adaptive Learning Platforms (ALPs) in the current literature. Addressing this gap, the authors built upon the work of Tretow-Fish and Khalid (2022) and analyzed 32 papers, which were grouped into six categories (C1-6) based on their themes. The categories include C1) the evaluation of LA and LAD design and framework, C2) the evaluation of user performance with LA and LAD, C3) the evaluation of adaptivity, C4) the evaluation of ALPs through perceived value, C5) the evaluation of Multimodal methods, and C6) the evaluation of the pedagogical implementation of ALP’s LA and LAD. The results include a tabular summary of the papers including the categories, evaluation unit(s), methods, variables and purpose. While there are numerous studies in categories C1-4 that focus on the design, development, and impact assessment of ALP’s LA and LAD, there are only a few studies in categories C5 and C6. For the category of C5), very few studies applied any evaluation methods assessing the multimodal features of LA and LADs on ALPs. Especially for C6), evaluating the pedagogical implementation of ALP’s LA and LAD, the three dimensions of signature pedagogy are used to assess the level of pedagogy evaluation. Findings showed that no studies focus on evaluating the deep or implicit structure of ALP’s LA. All studies examine the structural surface dimension of learning activities and interactions between students, teachers, and ALP’s LA and LAD, as examined in categories C2-C5. No studies were exclusively categorized as a C6 category, indicating that all studies evaluate ALP’s LA and LAD on the surface structure dimension of signature pedagogy. This review highlights the lack of pedagogical methodology and theory in ALP’s LA and LAD, which are recommended to be emphasized in future research and ALP development and implementation.

Se artiklen her

 

Researching student competence development through the lens of cultural historical activity theory

In the present study, we followed a group of students over the course of a three-semester project aimed at increasing their abilities to express their competence development. Extensive data (video, transcripts, documentation of artifacts and communications, and field notes) was analyzed in a framework analysis based on cultural–historical activity theory (CHAT). We understood contradictions, either within or between students’ activity systems, as drivers for expansive learning. Based on the in-depth analysis of three exemplary students, this study adds to the understanding of competence development in higher education by pointing out its individuality and situatedness, as well as the social dimension of expansive learning. Implications call for researchers and educators not to take competence development for granted when designing learning opportunities, but deliberately to create spaces in which competence expression can be facilitated.

 

Se artiklen her

Scenarios, Methods, and Didactics in Teaching Using Video-Conferencing Systems and Interactive Tools: Empirical Investigation on Problems and Good Practices

The restrictions during COVID-19 pandemic resulted in gaining more experience on video conferencing systems (VCS) and continued adoption during post-pandemic teaching scenarios. Designing and installing video conferencing systems in various classroom scenarios are expensive, increase complexity, and reduce interaction opportunities unless the designs for learning activities are well-defined and well-executed. For improving the quality of contact hours, the EdViCon Erasmus + project’s aim is to explore the existing diversity of scenarios of video conferencing systems’ use, various methods, and didactics applied for engaging students using various software and VCS features and define and develop two portable VCS toolkits and training. This paper reports the empirical case of Technical University of Denmark by applying phonomyography [1] as the methodology for exploring the VCS use scenarios, methods, and didactics for exploring problems and good practices. Despite increased adoption and number of papers on VCS, the insights on the diversity of the scenarios of use during contact hours has not been sufficiently reported from the viewpoint of HCI researchers. This study applies Shuman’s [2] concept of signature pedagogies as the theoretical viewpoint for defining scenarios of the use of VCS in teaching and David Benyon’s [3] PACT (people, activities, context, technologies) framework for analysing functionalities, methods, and didactics. The causes and effects of problems with VCS-mediated contact-hour activities were grouped into seven and 12 categories respectively. Seven types of teaching environments, three types of video-conferencing systems (fixed, on-wheels, and mobile), and the various tools used for engaging students were identified.

Du kan finde paperet her

Prototyping the Learning Analytics Dashboards of an Adaptive Learning Platform: Faculty Perceptions Versus Designers’ Intentions

This study contributes with a case study on redesigning three Learning Analytics Dashboards (LADs) of the adaptive learning platform Rhapsode™ with instructions for pedagogical actions. Applying self determination theory’s elements of competence and relatedness and mental models in a design thinking process, the differences among the teachers perceptions and the designers intentions are highlighted through several methods to answer the questions of: How might we improve the learning analytics dashboards by prioritizing course instructors’ perceived competence and relatedness? and How might we redesign learning analytics dashboards by including course instructors’ purpose, insights, and recommending actions?

These questions are answered first by developing three Role-based Personas of Alina Action, Niels Novice, and Paul Privacy along with scenarios and user stories. Second, prototypes of interfaces are designed and tested in three iterations showing insights, recommended actions, and explanation of mechanics. Feedback from the tests on the prototypes receives positive feedback from all teacher personas. The teacher persona of Niels Novice also supplies a criticism of the insights and recommended actions on the basis of creating undesired interpretation, potential bias, taking away freedom of interpretation, and authoritative system that “instructs/orders” action. Additionally, the scope of the study cannot meet the persona of Paul Privacy’s reservations on students’ possible experience of surveillance.

Du kan finde paperet her

Usability Evaluation of Adaptive Learning System Rhapsode Learner

Abstract:

From a usability perspective, designing and evaluating an adaptive learning system involve complexities associated with adaptivity and the diverse requirements from content designers, educators, and students. Moreover, students and educators are increasing getting subjected to educational quality studies including the evaluation of digital learning technologies, which the desired participants typically do not perceive as valuable. Furthermore, very few case studies report the results of the usability evaluation of adaptive learning systems. Thus, this study addresses the research questions: (1) Considering low participation and time constraint, which established usability evaluation methods might be applied for mixed-methods study of adaptive learning systems? (2) What is the students’ satisfaction score of RhapsodeTM Learner and implications for redesign? The result is the selection and application of system usability scale (SUS) and co-Discovery with concurrent interviews for the evaluation of RhapsodeTM Learner. The average score of 53.8 (n = 15) shows low marginal acceptability with good internal consistency (10 items, Cronbach’s α = 0.863). Analyses of the co-discoveries using 12 interaction design principles enable identifying the scopes for redesigning technological factors. For the evaluation of pedagogy and content aspects, further instruments should be developed.

Du kan finde paperet her

Evaluation of Learning Analytics on Adaptive Learning Systems: A work in progress Systematic Review

Abstract

There is currently no systematic overview of with what purpose Learning Analytics (LA) and Learning Analytics Dashboards (LAD) are evaluated on  Adaptive Learning Platforms. This work in progress systematic review provides the preliminary results of this endeavor. The paper establishes an overview of the current research field from two reviews. From this foundation we provide an analysis of seven papers. The preliminary results show four different purposes for evaluating LA and LAD on Adaptive Learning Platforms. These are: 1) Evaluation of LA and LAD design and framework, 2) Evaluation of LA and LAD performance, 3) Evaluation of perceived value, and 4) Evaluation of adaptivity. Through examining these papers, we see that when LA and LAD are evaluated on Adaptive Learning Platforms there are both single and multiple purpose of applying an evaluation method. These categories might change as the work in progress develops and more papers gets added in the synthesis.

Du kan finde paperet her

 

Må digte være sjove?

https://www.podbean.com/media/share/pb-sdnxc-10d8f57

I dette afsnit undersøger vi, om man må være sjov i sine digte. Er man useriøse og indholdsløs, hvis man bruger humor i sine digte? Er der fortilfælde af digtere der har brugt humor? Lyt med og find ud af, hvordan du måske kan bruge humor i dine egne skriverier. Følg med i opdateringer på Facebook, Instagram og vores hjemmeside https://32w.dk/Poesi101. Vil du have dine egne digte med i udsendelserne, så send dem til poesi101@32w.dk.

Kan man skrive sammen med andre?

https://www.podbean.com/media/share/pb-g5pdh-10d8ec0

I dette afsnit undersøger vi, hvordan man kan skrive sammen med andre. Hvad kan man opnå ved at sætte sig sammen med en anden og skrive? Er det lutter lagkage eller kan det være svært? Hvordan griber man det an? Følg med i opdateringer på Facebook, Instagram og vores hjemmeside https://32w.dk/Poesi101. Vil du have dine egne digte med i udsendelserne, så send dem til poesi101@32w.dk.

7. Intermezzo – Hvad er hverdagspoesi?

https://www.podbean.com/media/share/pb-7kj5c-10d8ebe

I 7. Intermezzo undersøger Tobias og Claus begrebet hverdagspoesi. Begrebet kommer i mange afskygninger alt fra Gruk til graffiti. Nogle gange kommer det fra hjertet, nogle gange fra hovedet. Men hvad gør sig gældende for hverdagspoesi? Og hvordan kan man genkende den? Følg med i opdateringer på Facebook, Instagram og vores hjemmeside https://32w.dk/Poesi101, hvor du også kan deltage i vores skriveudfordringer. Vil du have dine egne digte med i udsendelserne, så send dem til poesi101@32w.dk.

Kladde af en skærmtrold

Som en skærmtrold kan virke ovenpå og i kontrol

Som han vrider under sit joystick-vold

Som han vrider og han griber og skriger i iver

Smelter mit ansigt ud i rammen og glider

Ned i en smeltedigel, hvor det smelter sammen

Med en superbeige baggrundsmentalitet

Teknicolor dråber

Af mit øjes blå

Bløder på min trøje og erobrer mit orange

Åh!  Se det flyder nu fra brystet langs mit ærme

Og i et frame

Lander det i støvlen.