Just as you need to get up early to climb Mt Fuji, if you want to start an LTIF project in semester one, it helps to get the ethics application early. Cathy Leahy and Peter Saunders have been working hard with Blackboard data to extract hopefully meaningful feedback to be given to students about their performance. This data is from Dr Kat Daley’s Research Strategies for Social Science class. The data details basic information about students’ activity and presence in the course which is being made available to them. This includes information such as hits per day and the hours per day spread across days of the week.
There have been many challenges extracting the data. The Blackboard data does not present in consistent ways, requiring it get “massaged” a great deal before it can be used. Furthermore gathering the hours per day for each student has to be done individually for each student. With 180 students this has been extremely time consuming. It will be interesting to get student feedback on how useful this data is for their self efficacy.
Another issue is that report formats are not compatible with each other, requiring that staff data and students no longer active, be extracted before useful comparisons can be made. As well report names do not indicate what data will be extractable. They are not at all clear or intuitive or indicative of use. To quote our data analyst, Cathy Leahy, “I couldn’t believe how messy Blackboard data comes out. There’s no consistency of files or formats.”
Further processing of the data is required to make it compatible for R Studio which creates visualisations for the students.
At the moment the student data is being emailed in pdf format due to the short time frames of starting early in the year. This is not at all ideal for getting students attention. It is hoped that in the future this might become a dashboard. Alternately these analytics could be made available to students using Google add-ons or scripts, however taking their data into personal gmail accounts, which is what this work-around would currently require, is ethically tricky.
While we have shown that collating analytics out of the current instance of the LMS is doable, it is certainly not sustainable or scaleable. As well we are only capable of showing students what they are doing, rather than howthey are doing. We are still working more with statistics than with learning analytics. We will report students’ feedback to this first phase soon, as well as detail the steps taken, should anyone want to follow the arduous route we have taken. As the Japanese say “Anybody would be a fool not to climb Mount Fuji once – but a fool to do so twice”.
This doesn’t mean that we give up mountain climbing. The project committee will review the value of this approach for Kat’s class in second semester, based on student feedback and the time it takes for an individual to do all the processing. There are particular challenges that Kat’s style of class delivery presents. Her students are not on Blackboard much, as a lot of the learning and teaching occurs in Google Communities. There are differences in experience with online and offline delivery and her students are also using the TTM tool to evaluate their learning as they progress. This is a good example of where using the xApi standard for describing behaviour across different platforms could be implemented.
We are hoping the next phase of the project may prove more positive. Dr Ehsan Gharaie has been collating data from Blackboard for his Project Planning and Management classes for several years. He does weekly assessment as well as collecting in-class feedback weekly. Students are using Blackboard most of the time and there will be additional data from Turnitin we can analyse. Interestingly, Ehsan uses the previous year’s class data as a motivating factor for current students to improve (almost compete with) previous years’ performance.