Big Data and Education

By Michael Covert

bigdataandeducationEducation today is changing very rapidly. The new paradigm for many schools is being formed by the introduction of computer technology into the classroom. Many refer to this as the data-driven classroom. In the past, many curriculums were based on the premise that students all started at a relatively common level, and that most progressed at an even, steadily paced rate. These concepts are being challenged and the trend today is towards more individualized curriculums. Students that are found to be deficient in certain areas of understanding can be diverted into specialized curriculums, given remedial or corrective training, and then reestablished within the mainline curriculum.

Additionally, another trend is towards a more continuous evaluation process. In the past, the typical life cycle was lecture, read, repeat…, which then culminating in a series of periodic tests (of course with copious homework and quizzes thrown in!) Today, since much work is computer based and it is captured electronically, there exists a capability to assess progress continuously, and not just through episodic testing. These two trends go together quite nicely. The learning process becomes continuous, customized, and dynamic, and the theory is that the student will be enabled to achieve a better final result and learning experience. At each step, sometimes daily, a student can be assessed for progress and understanding.

While these trends are being implemented, data for student, curriculum, class, teacher, and school is being amassed and analyzed. It is individualized data at the student level. A student’s electronic profile will contain a rich and enormous amount of data ranging from personal information (age, gender, race, ethnicity, etc.), curriculum, teachers, family/school mobility, testing scores and assessments, medical (including behavioral) information, achievement and awards, attendance, and even disciplinary data. In a typical US state with a million or more students, the amount of data can be quite large. A set of technologies generally referred to as Big Data is increasingly being used to process this data to aid teachers, administrators, parents, and students to make better educational decisions and to improve outcomes. Big Data allows a tremendous amount of information to be analyzed across multiple dimensions (such as student age, region, school type, etc.) and can produce a holistic view of many areas of interest.

Big Data is not completely new to education. It has been used for a while now in various ways. Early usages were to mine the Web and online resources looking for evidence of plagiarism – the copy-and-paste syndrome. Additionally, much regional and national data has been assembled and ranking reports, student segments, and other statistical data has been assembled at a previously unprecedented level of accuracy, detail, and speed of delivery.

But this technology now goes so far beyond previous capabilities by allowing sophisticated data mining (discovering things we didn’t know before) and predictive analytics (based on what we know, what is likely to occur next) to be used. It can now help determine at a student level the most appropriate curriculum for the student. It can now quickly determine the areas of deficiencies in student understanding by correlating multiple interactions and responses, thereby allowing dynamic intervention to drill the student using lessons that augment these deficiencies. And even more so, using technologies called “machine learning” (using computer programs to recognize patterns in data that allow accurate prediction of future outcomes), the data can be used to predict varied future outcomes even before testing makes them apparent. Longitudinal studies (studying the same subjects over long periods of time) can also be conducted that can reveal those educational practices that most closely correlate with success both academically, and also from a career perspective.

There is a pure social side as well that uses Big Data to match students with potential entitlements (lunch programs, uniforms, customized tutoring, book lending programs, etc.) that uses the power of Big Data to match students up to these programs that would otherwise go undetected or undiscovered. In some cases, even the local school district can benefit by reaching certain enrollment thresholds, and also by tapping into financial resources that can help with their very funding.

Broadly speaking, Big Data is addressing three primary goals for education:

  • Produce meaningful insight for teachers, administrators, students, and parents
  • Make these insights “real-time”
  • Use these insights to transform education to produce “personalized learning”

It has been said that Big Data therefore gives a teacher a “small view of each student.” As we move into an ever more electronically defined society, as we become more connected through our mobile devices, our wearable biometric devices, and through more and more connected and integrated data, the possibilities are immense. While we must always be cognizant of privacy concerns, it is clear that Big Data will have a major impact on education and learning and will quite possibly be the enabling technology that will allow these goals to be achieved.


Sorry, comments are closed for this post.