What the research says – LKL Big Data and Learning Analytics session #wtrs8

This Thursday March 21st, I attended the eighth “What The Research Says” session at the London Knowledge Lab on Big data and Learning Analytics.

The first presentation was by Jenzabar, a service provider from Boston, USA about predictive modelling of student performance. Main objectives of the project involves academic achievement and at-risk-students. The speaker talked about some developments through the years on predictive modelling, touching upon point-in-time assessments/surveys, lagging indicators and early warnings based on observations. The second presentation was about Arbor, an adaptive system. Apparently, they sport the first NoSQL database system, using tags to capture data. Other systems can connect to their API. The third presentation by Alexandra Poulovassilis was about several tracking tools, most concerned with the Migen project. She described the evolution of teacher tools for student progress within the system. She showed an, in my opinion, very interesting visualization of student progress, essentially a logfile. I recognized a lot of the difficulties with analyzing logfiles of student progress in my PhD. Good to see they’re working on a web-based version as well. The fourth presentation showed Maths-Whizz work. I was impressed by their dashboard and visualization. Less impressed by the actual maths content I saw in the sample. For example, why do I get 0 points if my third step in the equation in the figure?

After this I attended a more detailed session by Jenzabar. Very interesting to hear more about the Learning Analytics (or is it data mining? ;-)) process. Familiar terms like logistic regression were touched upon. It resembles some of the work I’m doing now, looking at models and seeing what recall, precision etc. are. As a system, Jenzabar looks great. Visiting their website I can read Jenzabar describes themselves as “Software, strategies, and services empowering higher ed institutions to meet administrative and academic needs.”. That explains a lot; as an educator I’m more interested in what actually happens in classrooms, rather than the admin surrounding them. Algorithms behind the system range from uni-variate models to  multivariate, naive-Bayes and regression. They aim for at least 85-95% correct predictions.

I did not have much time to look at many other systems. The discussion at the end of the session primarily touched upon privacy and ethical aspects of big data. Like other topics there seems to be quite some polarization in this discussion. on the one hand you have the people (the USA seems to be pretty easy with data) who don’t seem to see anything non-ethical about collecting data from students. The other extreme is that you would have to ask permission about anything. I don’t recall that teachers who conduct pen-and-paper tests or check homework had to ask students whether they could make a judgment based on the data collected. I think the answer (again) is in the middle: we can and should use student data (I prefer the more qualitative data) but must use it sensibly. The day finished with someone suggesting we should look into ‘teaching analytics’. I agree, that’s why we’ve put this in a European bid.