photo by Lee Bennett cc licence on Flickr
There are a number of challenges we are now facing with learner analytics. In our 2016 LTIF project we have a number of courses providing meaningful feedback to students, however, we are not able to at present provide meaningful comparisons of activity between these courses. The great advantage of the x-Api standard is that it can provide a consistent language and standard for comparing online behaviours across different platforms, through the consistency of data created by Learning Record Stores (LRS). As well as helping us provide a consistent experience for students it can help contexts where multiple platforms or possibly two or more LMSs are being used at the same time.
Recently we approached IT Services about the potential for x-Api to be introduced as a standard for using with Learner Analytics. Technical Architect Dhugal Fletcher kindly undertook to examine x-Api and how it might be applicable. Below is his summary. Any comments or questions to his response would be most appreciated.
A Learning Management System contains multiple functional components that work together to provide the overall learning environment. Once of these components controls how learning content is distributed to students and tracks the results of their learning experiences working through multiple content packages in a sequence. The Sharable Content Object Reference Model (SCORM) provides a common definition on how an LMS performs this role and is in very common use globally within multiple LMS technologies.
A Learning Record Store (LRS) represents the evolution of the SCORM model. It extends and expands the standard to also cater for integration between multiple disparate systems in a form that is independent of the particular LMS technology in place. This means student activity within many systems that contribute to their learning can be centrally recorded for feedback and analysis.
The Tin Can API or Experience API (xAPI) is an implementation of an LRS system with broad acceptance at universities globally. The current review of the overall Learning Management System (LMS) solution and approach needs to incorporate more capabilities afforded by evolving and new technologies.
LRS Functionality Overview
Layered view of Functionality:
- Layer 1: Modernised SCORM
The LRS delivers a modernized version of SCORM that frees us from the obsolete constructs of old to provide a foundation for educational analysis in the age of big data.
- Layer 2: Recording Experiences
The LRS allows us to record any learning experience, including informal learning, giving us a much richer picture of an individual’s learning path. Recording many diverse experiences allows a view of student progress that has never been available before.
- Sets triggers on student activity on multiple platforms, including social media and any 3rd party sites that allow customization and API integration.
- Layer 3: Freed Data
The LRS frees your data from the confines of your siloed LMS. It is designed from the ground up to be accessible to, and integrable with, multiple systems to enable the free flow of data.
- Provides responses to students such as ‘badges’ or other learning milestones using native integration with the Open Badges standard.
- Layer 4: Data Correlation and Analysis
The LRS is capable of achieving the correlation of job performance data with training data to assess training effectiveness and much more. This is the foundation of Learning Analytics, and covers descriptive, predictive and prescriptive analysis. A quick definition of these types includes:
Descriptive Analytics, which use data aggregation and data mining techniques to provide insight into the past and answer: “What has happened?”
Predictive Analytics, which use statistical models and forecasts techniques to understand the future and answer: “What could happen?”
Prescriptive Analytics, which use optimization and simulation algorithms to advice on possible outcomes and answer: “What should we do?”
Learning Analytics can then combine previous student experience with current progress to give each individual real data on how they are tracking.
- For example, track how many hours a student has been using a particular application in a Lab to provide them a view of progress.
- Use data gathered over years to provide indicators to students, eg “Students who spend over 200 hours using this Application in the Lab during the semester have a 95% chance of passing.”
The Tin Can API or Experience API (xAPI) is a widely used implementation of the LRS capability. The xAPI is a set of RESTful web services. The services carry a JSON payload that allows a ‘Learning Activity Provider’ to make a series of ‘Statements’ to a ‘Learning Record Store (LRS)‘. Each ‘Statement’ describes a learning experience and consists of three parts, an ‘Actor,’ a ‘Verb’ and an ‘Object’. A ‘Statement’ conveys something like ‘Mike passed Introduction to REST’
The buildup of these statements for all students creates the raw data set for feedback reporting and analysis.
A detailed capability table is provided in Appendix A that compares SCORM with xAPI.
‘Fitbit for Education’
Learning Record Store (LRS) Capability Overview:
The Fitbit wristwatch device tracks activity for its wearer without requiring any special interaction or even awareness of the user. The resulting data is then presented to the wearer primarily via smartphone interface that shows raw data, graphs and charts designed to provide a view of progress towards fitness goals.
An LRS provides this same capability for the education industry, with added benefits of group data analysis over time allowing staff to understand and react to actual, measured learning outcomes.
An LRS provides a complex set of capabilities that are best explored and explained through describing the use cases already envisaged to improve learning and teaching experiences.
Use case: Student Activity Tracking
A student interacts with teachers and students in multiple digital environments. This might include Facebook, Twitter, Learning Management Systems, Email, instant messaging and specialist software installed on Lab machines. Defining the activity in environments that are relevant to learning activity creates many entries in the LRS.
“Student viewed task in LMS”
“Student logged into Lab PC”
“Student used Application for 50 minutes”
“Student put comment on course Facebook page”
“Student watched video, stopped at 40% of total”
This could also receive data from mobile devices about position and movement if required.
The collection of these defined activities builds into a profile of information about their activity that can then be used to provide instant feedback and reminders via various communication media. A student would have access to a dashboard interface to view reports on their activity. This could be made into a competitive system to provide comparisons between current and past individual students as required.
“You need to complete 200 hours working on this application, you have completed 12% of the task”
“You interacted with other students on the course site 50 times, you’ve earned the communicator badge”
As students complete courses at RMIT, the LRS data could also be used to automatically populate a dynamic ePortfolio.
Use case: Teaching Staff Reporting
Once data has been collected by the LRS, reporting becomes available to teaching staff.
“These students are under the required interaction level and may need reminders”
“This student as already completed the tasks assigned and is now helping others”
“This class is performing 6% above the average class level at this stage”
Use case: Teaching Staff Analysis
As more data is collected by the LRS about multiple courses over multiple years, more detailed analysis is possible to uncover insights.
“Based on their activity in the first month of this course, this student will fail, assistance is needed”
“Based on the amount of time spent in Labs and interacting on the course site, this student is a top performer who could achieve more”
“Based on comments in social media, these students regard the teacher very highly”
“Based on comments in social media, there is a problem with the application installed on computers in this Lab”
Adoption of an LRS to improve the learning and teaching experience requires multiple factors for successful deployment at RMIT.
- A common standard for data from various technology implementations is needed
- A standard technology implementation providing LRS capabilities is needed across RMIT globally. This must be agreed on early and enforced across all Colleges and business groups.
- The system will require high availability and disaster recovery as it must be available to receive the learning record updates from many diverse systems concurrently.
- The system must integrate natively with all Learning Management Systems.
- The Business Intelligence reporting platform should be used as the primary tool for reporting and analysis of the raw data.