News

There is an instant appeal to the notion of using technology to improve academic outcomes in our schools, particularly in key gateway subjects like mathematics. The pronouncements of new tech-based initiatives often generate a significant amount of hype and excitement which is also often accompanied by bold proclamations of what technology is able to achieve. For the past 4 years, we’ve been using tech-based solutions to improve senior phase mathematics at OLICO Youth in Diepsloot, north of Johannesburg. While we’re seeing strong improvements in academic performance, it is clear to us that technology is no silver bullet. In fact, in our experience, there are a number of important tensions we wrestle with on an almost daily basis. Some of these tensions are discussed below and shared in the hope that these experiences might be helpful to others too. We’d love to hear your experiences.

The Advantages of Technology

There are, without doubt, many positive aspects of technology in education and many reasons to be excited about what technology can help achieve. Perhaps most notably, and obviously, learners LOVE using tech-based solutions and adapt very quickly to whatever is put in front of them. Using technology, 45 Grade 7 learners from Diepsloot have tackled a combined total of over 100,000 maths questions this year alone (Feb-Aug 2015). It certainly would have been a lot harder to achieve anything of a similar magnitude using paper-based alternatives. So from the perspective of a significant improvements in “time-on-task”, technology offers exciting opportunities.

In addition, the interactivity of technology enables learners to receive immediate and personalised feedback to questions they’re attempting. The ability to watch video explanations and pause and repeat as necessary creates a safe self-paced learning environment. It is also clear that the wealth of analytical data available to a skilled teacher/tutor can have extremely positive implications for future lesson plans and targeted interventions.

Tensions we experience and things we’re beginning to learn.

The way in which we work at OLICO Youth has involved a cycle of planning, implementation, reflection, revision and improvement. Here we document some of the tensions we have had to face in the process and some of the lessons we’ve extracted in dealing with these tensions.

Initially, we relied heavily on Khan Academy content (www.khanacademy.org) and learners were guided in terms of which Khan Academy videos to watch and then practiced using the online exercises that followed. However, this quickly led to our first major tension.

Tension 1: Using already existing materials vs. developing our own

Khan Academy (henceforth: KA) is an impressive resource, but increasingly we found that we wanted videos and questions more suited to the South African context and curriculum. We also found it increasingly difficult to keep track of learners and prevent them from engaging on activities we didn’t want them to. For example, learners would often follow the automated prompts from KA which is aligned to the US common core curriculum and not always helpful to our context. For learners new to computers and the internet, sites like KA can also be quite overwhelming and difficult to self-direct. This is further complicated by the different uses of maths terminology which generally created as much confusion in our learners as we were trying to solve. Our broad conclusion is that KA is a decent resource for learners with well-developed meta-cognition skills. The over-riding priority for our learners however, is to first bridge the gaps in foundational content knowledge before learners can effectively make use of sites like KA and other existing online resources.[1]

Our solution then has been to use the open-source learning management system, Moodle, and build our own much more restricted course content and structure. All the content we’ve subsequently developed is published under a creative commons license and free to share. The core of the online programme is still videos and questions banks (although these are now custom-built for our learners), but the use of the Moodle has also enabled us to include online games, where appropriate, and responsive web-based tools for visual representation of topics like number lines and fractions. Our Moodle is hosted online at http://learn.olico.org[2] where interested parties can self-register on many of the courses. Enrollment keys are available from learn@olico.org.

Tension 2: Working on computers vs. working with pen and paper

Early on, we discovered that it is possible for a learner to achieve a high level of mastery on the computer in a particular topic, but then struggle with the same topic on a paper based test. There are a couple of possible reasons for this: Firstly it seems that it is possible to get into a rhythm on the computer that, if not checked, can create an illusion of conceptual understanding. Secondly, short answer or multiple choice responses tend to predominate in computer-based environments. This means the kind of responses that require a clear exposition of the steps or a logical argument are backgrounded. However, the “working out” is a very important part of what learners will be assessed on in mathematics, and the ability to reflect on it is crucial to the learning process.[3]

To counteract this, the importance of learners completing homework in their OLICO booklets and regular sets of mixed written exercises, termed ‘5-a-day’, is emphasised. In addition, the “pause and review” section of the lesson plans involves a paper-based checkpoint for learners to complete offline and only later feed into the computer for feedback.

The dual nature of the checkpoints (i.e. written with pen and paper offline, but then fed into the computer for immediate feedback) provides very useful information for the facilitator. Moodle is able to provide immediate statistics on class performance – including a breakdown by question. The facilitator is able to easily identify problem areas that need to be worked on with the class as a whole. In addition, the individual written work allows the facilitator to scrunitise in detail the work of any learners who are having difficulty and to use the working provided there to identify individual misconceptions or problems. Checkpoints are accorded a high value because a virtual badge is awarded to learners passing the checkpoint quiz.

Tension 3 – Learner mastery vs. working to a set timetable

In an ideal world, learners would be afforded an opportunity to take as long as they need to gain full mastery of a topic before proceeding to the next section. However, given that most learners (in our experience) are already 2 to 4 grades behind where they should be, time is a highly valuable commodity and in limited supply. This is especially true for learners who are far behind. The nature of mathematics dictates that they need to master the content in order to build knowledge, but they also need to keep moving – and without this impetus, many learners have a tendency to just drift.

In 2014, we allowed learners to wholly dictate the pace, but this created a wide and disparate spread across topic areas and made it very difficult for facilitators to track, monitor and effectively intervene. In 2015, we’re experimenting with a more hybrid approach that still allows learners to work largely at their own pace, but requires learners to cover a set number of lessons before a scheduled checkpoint quiz. Those learners who are struggling and need extra time attend additional sessions over and above their minimum of 2 sessions a week. It is still early, but the initial sense is that facilitators have a much better idea of how the class is progressing with this approach.

An ongoing challenge, particularly in relation to the Senior Phase (grades 7 – 9) intervention, is the identification of the core content and skills to focus on. This is complicated by the large backlogs learners arrive with and the limited time available in an afterschool environment. Identifying precisely which foundational elements to emphasise is not simple, and is one of the key questions we continue to work on in the development and improvement of the programme.

Tension 4: The need for remediation vs. current curriculum support

With a large number of learners initially ill-equipped to engage with grade appropriate curriculum content, there is a real need for remedial assistance. Many of the learners arrive with poor number sense, lack of experience in dealing with shapes and an orientation to mathematics as a set of arbitrary rules. There is thus an urgent need to provide learners with an experience of some of the fundamental basics of mathematics in a way that is connected and has meaning. Yet at same time, there is pressure from learners and parents to ‘cover’ content currently being addressed at school – a quandary we haven’t yet adequately resolved. While we have pushed the need to get the foundations in place and focused much of the computer-based work on developing a solid conceptual understanding, we have – by necessity – reverted to exam-specific drilling at points when learners have upcoming tests or exams. This kind of drilling has the advantage of offering some immediate rewards (normally a small uptick in the results for the test), and this improves confidence and buy-in to the programme where the really hard work of getting a solid grasp of the mathematics is done.

Tension 5: Individualised learning vs. creating communities of learning

One of the inherent advantages of a targeted use of technology is that the learner engages in a highly individualised learning experience. The computer will provide feedback based exclusively on what the individual learner inputs and the learner can respond appropriately in each case. However, there are also strong advantages to creating opportunities for extensive learner interactions. The value of peer-to-peer collaborative support is often vastly underestimated.

To this extent, we now look to start each class with a 5-minute game or challenge for learners to collaborate on. In addition to the homework learners receive based on where they are in the Moodle, we also assign the ‘5-a-day’ set of mathematics questions for learners to complete at home. Learners then share their experiences of the ‘5-a-day’ questions before they begin their computer-based sessions. Using feedback on learners’ performances from the Moodle, we are also able to identify groups of learners who need help with a particular area of content. Targeted small group sessions can then be structured for these learners where they can work with their peers and a tutor.

Tension 6: Knowledge retention vs. gaming the system

Knowledge retention is a complex issue and hot topic in all forms of learning, since retention most likely only comes with a clear conceptual understanding of a topic. It is possible to master a single skill (e.g. adding fractions) in isolation without understanding the process at all. The gamification of our computer-based mathematics system has the potential to create a perverse incentive to simply get through the lessons by any means necessary, e.g. by figuring out the ‘trick’ to the right answer (even if the learner doesn’t understand why), or worse, simply copying from a fellow learner.

To prevent this and assess for learner retention, we have included regular mixed exercise sets to circle back on content previously covered and ensure that learners regularly have to deal with mathematics in situations where it isn’t immediately obvious which steps need to be applied. The checkpoints are also crucial as they are strictly supervised to prevent cheating and cover a range of skills. They thus allow us to pick up where learners are “gaming” their way through the online exercises.

Tension 7: Personal agency vs assumed control

The biggest learning and in some sense a bit of a contradiction is that learners only really start to fly once they take responsibility for their learning and see a relationship between personal effort and outcome. It is vitally important that learners want to understand the mathematics, take responsibility for asking when they don’t understand, and persevere with a difficult concept until it makes sense.

However, at the same time, we discovered in our initial explorations with computer-based learning that we need to play a very strong guiding (controlling) role in the initial stages. When learners had options to choose where to go on Khan Academy, some learners would spend most of their time working on easy exercises where they could have the gratification of getting everything right, but learning very little. Alternatively, learners would bounce off into topics completely disconnected from where they need to be. It thus requires a fine balancing act between temporarily taking more control (and needing to in order to get learners to find productive ways of working) and letting go enough that they can start to develop their personal agency and take responsibility for their own learning. Ultimately this is the clearest indicator of whether a learner is going to overcome their circumstances and succeed at mathematics.

Background information

OLICO Youth caters to 95 learners in grades 7-11 who attend computer-based mathematics classes for an hour after school, twice a week. Learners also attend life skills, study skills and literacy programmes on Saturday mornings. Participation in the programme is voluntary and learners are accepted on a ‘first-come first-served’ basis. However, strict attendance requirements are adhered to and learners who do not attend regularly are asked to leave the programme. OLICO only accepts a new intake of learners at the beginning of Grade 7 each year. Results of initial diagnostic tests each year suggest that the majority of learners entering the programme are between 2 and 4 years below grade level in mathematics.

The design of the programme is straightforward. Learners arrive at the centre after school; show completed homework to gain entry to the computer lab; follow the lesson-prompts on the computer; and collect new homework at the end of the session. The actual computer-based lessons are divided into 5 Ps.

1. Pre-Quiz
2. Presentation (video)
3. Post-Quiz
4. Practice mixed set
5. Pause and review

 

[1] On a related point, we have been unable to replicate the example Salmon Khan uses in his TED talk of a struggling learner making dramatic gains to catch up with the rest of the class. In our experiences to date, the progression is much more gradual and requires sustained intervention since the skills that are lacking are rooted in core number proficiency gaps.

[2] There is a far greater variety of pre-existing options available to FET Phase learners already suited to the South African context. We have enrolled our Grade 10 and Grade 11 learners on Siyavula’s Intelligent Practice system (www.everythingmaths.co.za) and supplemented this with our own videos, lesson plans and checkpoint assessments.

[3] In this our experience echoes that of a New Zealand-based online programme, mathsbuddy.co.nz, and we have incorporated some of their suggested strategies into our work to deal with this.