- Title
- A toolkit for successful workplace learning analytics at software vendors
- Creator
- Whale, Alyssa Morgan
- Subject
- Computer-assisted instruction
- Subject
- Intelligent tutoring systems
- Subject
- Information visualisation
- Date Issued
- 2024-04
- Date
- 2024-04
- Type
- Doctoral theses
- Type
- text
- Identifier
- http://hdl.handle.net/10948/64448
- Identifier
- vital:73713
- Description
- Software vendors commonly provide digital software training to their stakeholders and therefore are faced with the problem of an influx of data collected from these training/learning initiatives. Every second of every day, data is being collected based on online learning activities and learner behaviour. Thus, online platforms are struggling to cope with the volumes of data that are collected, and companies are finding it difficult to analyse and manage this data in a way that can be beneficial to all stakeholders. The majority of studies investigating learning analytics have been conducted in educational settings. This research aimed to develop and evaluate a toolkit that can be used for successful Workplace Learning Analytics (WLA) at software vendors. The study followed the Design Science Research (DSR) methodology, which was applied in iterative cycles where various components of the toolkit were designed, developed, and evaluated by participants. The real-world-context was a software vendor, ERPCo, which has been struggling to implement WLA successfully with their current Learning Experience Platform (LXP), as well as with their previous platform. Qualitative data was collected using document analysis of key company documents and Focus Group Discussions (FGDs) with employees from ERPCo to explore and confirm different topics and themes. These methods were used to iteratively analyse the As-Is and To-Be situations at ERPCo and to develop and evaluate the proposed WLA Toolkit. The method used to analyse the collected data from the FGDs was the Qualitative Content Analysis (QCA) method. To develop the first component of the toolkit, the Organisation component, the organisational success factors that influence the success of WLA were identified using a Systematic Literature Review (SLR). These factors were discussed and validated in two exploratory FGDs held with employees from ERPCo, one with operational stakeholders and the other with strategic decision makers. The DeLone and McLean Information Systems (D&M IS) Success Model was used to undergird the research as a theory to guide the understanding of the factors influencing the success of WLA. Many of the factors identified in theory were found to be prevalent in the real-world-context, with some additional ones being identified in the FGDs. The most frequent challenges highlighted by participants were related to visibility; readily available high-quality data; flexibility of reporting; complexity of reporting; and effective decision making and insights obtained. Many of these related to the concept of usability issues for both the system and the information, which is specifically related to System Quality or Information Quality from the D&M IS Success Model. The second and third components of the toolkit are the Technology and Applications; and Information components respectively. Therefore, architecture and data management challenges and requirements for these components were analysed. An appropriate WLA architecture was selected and then further customised for use at ERPCo. A third FGD was conducted with employees who had more technical roles in ERPCo. The purpose of this FGD was to provide input on the architecture, technologies and data management challenges and requirements. In the Technology and Applications component of the WLA Toolkit, factors influencing WLA success related to applications and visualisations were considered. An instantiation of this component was demonstrated in the fourth FGD, where learning data from the LXP at ERPCo was collected and a dashboard incorporating recommended visualisation techniques was developed as a proof of concept. In this FGD participants gave feedback on both the dashboard and the toolkit. The artefact of this research is the WLA Toolkit that can be used by practitioners to guide the planning and implementation of WLA in large organisations that use LXP and WLA platforms. Researchers can use the WLA Toolkit to gain a deeper understanding of the required components and factors for successful WLA in software vendors. The research also contributes to the D&M IS Success Model theory in the information economy. In support of this PhD dissertation, the following paper has been published: Whale, A. & Scholtz, B. 2022. A Theoretical Classification of Organizational Success Factors for Workplace Learning Analytics. NEXTCOMP 2022. Mauritius. A draft manuscript for a journal paper was in progress at the time of submitting this thesis.
- Description
- Thesis (PhD) -- Faculty of Science, School of Computer Science, Mathematics, Physics and Statistics , 2024
- Format
- computer
- Format
- online resource
- Format
- application/pdf
- Format
- 1 online resource (xxii, 248 pages)
- Format
- Publisher
- Nelson Mandela University
- Publisher
- Faculty of Science
- Language
- English
- Rights
- Nelson Mandela University
- Rights
- All Rights Reserved
- Rights
- Open Access
- Hits: 1
- Visitors: 2
- Downloads: 0
Collections
NMU School of Computer Science, Mathematics, Physics and Statistics
| Thumbnail | File | Description | Size | Format | |||
|---|---|---|---|---|---|---|---|
| View Details Download | SOURCE1 | Whale, AM.pdf | 11 MB | Adobe Acrobat PDF | View Details Download |