Reports

OLICO’s Diepsloot Matrics beat the odds with an 82% Pure Maths Pass-rate

January 5, 2018 By Andrew

It is with great pride that OLICO announces our Diepsloot matrics achieved an 82% pass-rate for pure mathematics with two-thirds scoring in the Top 20% of maths performers nationally. Overall, OLICO learners achieved 55% Bachelor Level passes and 27% Diploma Level passes.

The Diepsloot matric class of 2017 first joined OLICO in 2013 as Grade 8 learners seeking extra maths support. On first enrolment, these learners were approximately 3-4 years below their expected maths grade level (which coheres with Kotze and Spaull, 2015). In Grade 9, this group scored an average of just 24% on the 2014 Annual National Assessments (ANAs) for maths.

OLICO’s multi-year approach is to build core foundational skills and gradually deepen conceptual understanding over an extended period of time. OLICO’s matric class of 2017 dedicated significant proportions of their after-school lives, over many years, to improving their maths competencies.

OLICO’s Programme Coordinator, Andrew Barrett adds, “Given the realities of the schooling maths situation in South Africa, the time and support available during school hours is simply not sufficient to cover all that needs to be covered so OLICO’s after-school support and blended use of technology is one way in which we can supplement the work of the teacher.”

Ultimately, 82% of OLICO’s Diepsloot matric learners passed pure mathematics compared to only 19% of their Diepsloot peers.

Further to this, two-thirds of OLICO’s learners scored above 50% in pure mathematics which puts them into the top 20% of maths achievers across South Africa.

This 2017 matric class in Diepsloot is part of a larger group of 155 learners between Grades 7-12 who receive extra maths support after-school 2-3 times a week. The improvements are the result of intensive work on number skills and supplementary curriculum support with OLICO’s freely available online maths tool and tailored tutoring support.

OLICO learner, Boledi Mampa believes OLICO made a huge difference to her maths results, “I joined OLICO 5 years ago and today I got my Bachelor Level matric certificate. It’s been great being part of OLICO as it’s helped me pursue my dream which is to study Environmental Science at the University of Johannesburg this year.”

Likewise, her classmates Lebogang Mafa and Mokgadi Morwasehla believe OLICO has helped them unlock their potential and achieve her goals. Says Lebogang,

My maths was poor when I first joined but by practicing with OLICO’s support, my results have really improved. I want to start a new life at university where I’m looking to pursue a BSC degree.

Intervening Early.

There is no quick-fix or simple solution to improving South African maths results but there are clear indications that intensive multi-year support in an after-school setting can achieve substantial improvements. At OLICO, we use a custom-built online maths programme (http://learn.olico.org) with our Grade 7-9 learners alongside skilled maths tutors to supplement learning. This online maths tool is published under a creative commons licence and contains over 22,000+ unique interactive maths questions and 330 tutorial videos specifically tailored for South African senior phase maths. The belief is that establishing firm foundations and good number sense is essential for future maths success.

The South African Education challenge

Some excellent articles on the maths situation in South Africa:

Nicky Roberts: https://www.dailymaverick.co.za/article/2017-01-08-school-maths-what-is-our-story/#.WHSakxt96Uk

Paul Berkowitz: http://mg.co.za/article/2017-01-06-the-difference-between-achievement-and-excellence-mathematics-results-for-the-nsc

Nic Spaull: https://nicspaull.com/2018/01/05/explaining-the-matric-results-in-7-gifs/

Beyond Diepsloot

In addition to the 155 Grade 7-12 learners in Diepsloot, OLICO is also working in 5 schools in the Western Cape on the Year Beyond programme and has over 5000 learners enrolled on the FREE online maths program for Grades 7-9 at http://learn.olico.org. OLICO is also working on a younger intervention through Maths Clubs in collaboration with the SA numeracy Chair Project at Rhodes University and the Maths Clubs Collective (www.mathsclubs.co.za).

With grateful thank to our supporters:

OLICO is deeply grateful to our donor partners who have helped make this happen. including: The Datatec Education and Technology Trust, The Learning Trust, Hatch, The Oppenheimer Memorial Trust, The Claude Leon Foundation and the Nedbank Private Wealth Educational Foundation.

100% Matric Pass-rate. Top 20% Maths Nationally.

January 10, 2017 By Andrew

It is with great pride that OLICO announces a 100% matric pass-rate (83% Bachelor Level passes) with our learners achieving in the top 20% maths results nationally.

In 2012, OLICO began working with a group of Grade 8 learners in Diepsloot who have now graduated as members of our first-ever matric cohort of 2016. The matric class in Diepsloot is part of a larger group of 145 learners between Grades 7-12 who receive extra maths and literacy support twice a week. On first enrolment, our 2016 matric class had a mathematics average of just 18%. The latest matric results show that OLICO learners have dramatically increased their maths marks with 83% achieving marks of above 50% in pure mathematics which puts them into the top 20% maths achievers across South Africa. These improvements are the result of intensive work on foundational number skills and supplementary curriculum support with OLICO’s freely available online maths tool and tailored tutoring support.

OLICO learner, Patience Moyo believes OLICO made a huge difference to her maths results, “When I first joined OLICO, my maths was poor but by practicing with OLICO’s support, my results have really improved. I want to start a new life at university and graduate as a psychologist to help people and be an example of what success is to my community.”

OLICO’s Maths Coordinator, Dr Lynn Bowie adds, “Over the past 5 years, these learners have worked tirelessly to improve their maths skills and deserve all the credit they’re now receiving. In short, their commitment and dedication has been outstanding and they are reaping the rewards. For many learners, the time and support available in school hours is simply not sufficient to cover all that needs to be covered so OLICO’s after-school support and blended use of technology is one way in which we can supplement the work of the teacher.”

Likewise, her classmates Roda Chitegha and Christine Ngwenya believe OLICO has helped them unlock their potential and achieve her goals. Says Roda, “Coming to OLICO in the afternoons since Grade 8 has helped to keep myself away from trouble and temptation that faces the youth of Diepsloot. OLICO has helped ensure my results at school are always good and I can go to university. I have registered for a BSC but this is not my only dream, I also want to see myself as a role model to others that matric success in maths is indeed possible.”

image1-lr — OLICO’s Thandiswa Nxelle (middle) with matric learners Christine Ngwenya and Roda Chitegha

No Short-cuts to improving Maths Results.

There is no short-cut, quick-fix or silver bullet to improving South African maths results but there are clear indications that intensive multi-year support in an after-school setting can achieve substantial improvements. At OLICO, we use a custom-built online maths programme (http://learn.olico.org) with our Grade 7-9 learners alongside skilled maths tutors to supplement learning. This online maths tool is published under a creative commons licence and contains over 20,000+ unique interactive maths questions and 330 tutorial videos specifically tailored for South African senior phase maths. The belief is that establishing firm foundations and good number sense is essential for future maths success.

The South African Maths challenge

Some excellent articles on the maths situation in South Africa:

Nicky Roberts: https://www.dailymaverick.co.za/article/2017-01-08-school-maths-what-is-our-story/#.WHSakxt96Uk

Paul Berkowitz: http://mg.co.za/article/2017-01-06-the-difference-between-achievement-and-excellence-mathematics-results-for-the-nsc

Nic Spaull: https://nicspaull.com/2017/01/06/my-take-on-matric-2016/

Beyond Diepsloot

In addition to the 145 Grade 7-12 learners in Diepsloot, OLICO is also working in 8 schools in the Western Cape on the Year Beyond programme and has over 2000 learners enrolled on the FREE online maths program for Grades 7-9 at http://learn.olico.org.

With grateful thank to our supporters:

OLICO is deeply grateful to our donor partners who have helped make this happen. including: The Datatec Education and Technology Trust, The Learning Trust, Hatch, The Oppenheimer Memorial Trust, The Claude Leon Foundation and, most recently, the Nedbank Private Wealth Educational Foundation.

Seven Tensions we’re experiencing using technology

October 11, 2015 By Andrew

There is an instant appeal to the notion of using technology to improve academic outcomes in our schools, particularly in key gateway subjects like mathematics. The pronouncements of new tech-based initiatives often generate a significant amount of hype and excitement which is also often accompanied by bold proclamations of what technology is able to achieve. For the past 4 years, we’ve been using tech-based solutions to improve senior phase mathematics at OLICO Youth in Diepsloot, north of Johannesburg. While we’re seeing strong improvements in academic performance, it is clear to us that technology is no silver bullet. In fact, in our experience, there are a number of important tensions we wrestle with on an almost daily basis. Some of these tensions are discussed below and shared in the hope that these experiences might be helpful to others too. We’d love to hear your experiences.

The Advantages of Technology

There are, without doubt, many positive aspects of technology in education and many reasons to be excited about what technology can help achieve. Perhaps most notably, and obviously, learners LOVE using tech-based solutions and adapt very quickly to whatever is put in front of them. Using technology, 45 Grade 7 learners from Diepsloot have tackled a combined total of over 100,000 maths questions this year alone (Feb-Aug 2015). It certainly would have been a lot harder to achieve anything of a similar magnitude using paper-based alternatives. So from the perspective of a significant improvements in “time-on-task”, technology offers exciting opportunities.

In addition, the interactivity of technology enables learners to receive immediate and personalised feedback to questions they’re attempting. The ability to watch video explanations and pause and repeat as necessary creates a safe self-paced learning environment. It is also clear that the wealth of analytical data available to a skilled teacher/tutor can have extremely positive implications for future lesson plans and targeted interventions.

Tensions we experience and things we’re beginning to learn.

The way in which we work at OLICO Youth has involved a cycle of planning, implementation, reflection, revision and improvement. Here we document some of the tensions we have had to face in the process and some of the lessons we’ve extracted in dealing with these tensions.

Initially, we relied heavily on Khan Academy content (www.khanacademy.org) and learners were guided in terms of which Khan Academy videos to watch and then practiced using the online exercises that followed. However, this quickly led to our first major tension.

Tension 1: Using already existing materials vs. developing our own

Khan Academy (henceforth: KA) is an impressive resource, but increasingly we found that we wanted videos and questions more suited to the South African context and curriculum. We also found it increasingly difficult to keep track of learners and prevent them from engaging on activities we didn’t want them to. For example, learners would often follow the automated prompts from KA which is aligned to the US common core curriculum and not always helpful to our context. For learners new to computers and the internet, sites like KA can also be quite overwhelming and difficult to self-direct. This is further complicated by the different uses of maths terminology which generally created as much confusion in our learners as we were trying to solve. Our broad conclusion is that KA is a decent resource for learners with well-developed meta-cognition skills. The over-riding priority for our learners however, is to first bridge the gaps in foundational content knowledge before learners can effectively make use of sites like KA and other existing online resources.[1]

Our solution then has been to use the open-source learning management system, Moodle, and build our own much more restricted course content and structure. All the content we’ve subsequently developed is published under a creative commons license and free to share. The core of the online programme is still videos and questions banks (although these are now custom-built for our learners), but the use of the Moodle has also enabled us to include online games, where appropriate, and responsive web-based tools for visual representation of topics like number lines and fractions. Our Moodle is hosted online at http://learn.olico.org [2] where interested parties can self-register on many of the courses. Enrollment keys are available from learn@olico.org.

Tension 2: Working on computers vs. working with pen and paper

Early on, we discovered that it is possible for a learner to achieve a high level of mastery on the computer in a particular topic, but then struggle with the same topic on a paper based test. There are a couple of possible reasons for this: Firstly it seems that it is possible to get into a rhythm on the computer that, if not checked, can create an illusion of conceptual understanding. Secondly, short answer or multiple choice responses tend to predominate in computer-based environments. This means the kind of responses that require a clear exposition of the steps or a logical argument are backgrounded. However, the “working out” is a very important part of what learners will be assessed on in mathematics, and the ability to reflect on it is crucial to the learning process.[3]

To counteract this, the importance of learners completing homework in their OLICO booklets and regular sets of mixed written exercises, termed ‘5-a-day’, is emphasised. In addition, the “pause and review” section of the lesson plans involves a paper-based checkpoint for learners to complete offline and only later feed into the computer for feedback.

The dual nature of the checkpoints (i.e. written with pen and paper offline, but then fed into the computer for immediate feedback) provides very useful information for the facilitator. Moodle is able to provide immediate statistics on class performance – including a breakdown by question. The facilitator is able to easily identify problem areas that need to be worked on with the class as a whole. In addition, the individual written work allows the facilitator to scrunitise in detail the work of any learners who are having difficulty and to use the working provided there to identify individual misconceptions or problems. Checkpoints are accorded a high value because a virtual badge is awarded to learners passing the checkpoint quiz.

Tension 3 – Learner mastery vs. working to a set timetable

In an ideal world, learners would be afforded an opportunity to take as long as they need to gain full mastery of a topic before proceeding to the next section. However, given that most learners (in our experience) are already 2 to 4 grades behind where they should be, time is a highly valuable commodity and in limited supply. This is especially true for learners who are far behind. The nature of mathematics dictates that they need to master the content in order to build knowledge, but they also need to keep moving – and without this impetus, many learners have a tendency to just drift.

In 2014, we allowed learners to wholly dictate the pace, but this created a wide and disparate spread across topic areas and made it very difficult for facilitators to track, monitor and effectively intervene. In 2015, we’re experimenting with a more hybrid approach that still allows learners to work largely at their own pace, but requires learners to cover a set number of lessons before a scheduled checkpoint quiz. Those learners who are struggling and need extra time attend additional sessions over and above their minimum of 2 sessions a week. It is still early, but the initial sense is that facilitators have a much better idea of how the class is progressing with this approach.

An ongoing challenge, particularly in relation to the Senior Phase (grades 7 – 9) intervention, is the identification of the core content and skills to focus on. This is complicated by the large backlogs learners arrive with and the limited time available in an afterschool environment. Identifying precisely which foundational elements to emphasise is not simple, and is one of the key questions we continue to work on in the development and improvement of the programme.

Tension 4: The need for remediation vs. current curriculum support

With a large number of learners initially ill-equipped to engage with grade appropriate curriculum content, there is a real need for remedial assistance. Many of the learners arrive with poor number sense, lack of experience in dealing with shapes and an orientation to mathematics as a set of arbitrary rules. There is thus an urgent need to provide learners with an experience of some of the fundamental basics of mathematics in a way that is connected and has meaning. Yet at same time, there is pressure from learners and parents to ‘cover’ content currently being addressed at school – a quandary we haven’t yet adequately resolved. While we have pushed the need to get the foundations in place and focused much of the computer-based work on developing a solid conceptual understanding, we have – by necessity – reverted to exam-specific drilling at points when learners have upcoming tests or exams. This kind of drilling has the advantage of offering some immediate rewards (normally a small uptick in the results for the test), and this improves confidence and buy-in to the programme where the really hard work of getting a solid grasp of the mathematics is done.

Tension 5: Individualised learning vs. creating communities of learning

One of the inherent advantages of a targeted use of technology is that the learner engages in a highly individualised learning experience. The computer will provide feedback based exclusively on what the individual learner inputs and the learner can respond appropriately in each case. However, there are also strong advantages to creating opportunities for extensive learner interactions. The value of peer-to-peer collaborative support is often vastly underestimated.

To this extent, we now look to start each class with a 5-minute game or challenge for learners to collaborate on. In addition to the homework learners receive based on where they are in the Moodle, we also assign the ‘5-a-day’ set of mathematics questions for learners to complete at home. Learners then share their experiences of the ‘5-a-day’ questions before they begin their computer-based sessions. Using feedback on learners’ performances from the Moodle, we are also able to identify groups of learners who need help with a particular area of content. Targeted small group sessions can then be structured for these learners where they can work with their peers and a tutor.

Tension 6: Knowledge retention vs. gaming the system

Knowledge retention is a complex issue and hot topic in all forms of learning, since retention most likely only comes with a clear conceptual understanding of a topic. It is possible to master a single skill (e.g. adding fractions) in isolation without understanding the process at all. The gamification of our computer-based mathematics system has the potential to create a perverse incentive to simply get through the lessons by any means necessary, e.g. by figuring out the ‘trick’ to the right answer (even if the learner doesn’t understand why), or worse, simply copying from a fellow learner.

To prevent this and assess for learner retention, we have included regular mixed exercise sets to circle back on content previously covered and ensure that learners regularly have to deal with mathematics in situations where it isn’t immediately obvious which steps need to be applied. The checkpoints are also crucial as they are strictly supervised to prevent cheating and cover a range of skills. They thus allow us to pick up where learners are “gaming” their way through the online exercises.

Tension 7: Personal agency vs assumed control

The biggest learning and in some sense a bit of a contradiction is that learners only really start to fly once they take responsibility for their learning and see a relationship between personal effort and outcome. It is vitally important that learners want to understand the mathematics, take responsibility for asking when they don’t understand, and persevere with a difficult concept until it makes sense.

However, at the same time, we discovered in our initial explorations with computer-based learning that we need to play a very strong guiding (controlling) role in the initial stages. When learners had options to choose where to go on Khan Academy, some learners would spend most of their time working on easy exercises where they could have the gratification of getting everything right, but learning very little. Alternatively, learners would bounce off into topics completely disconnected from where they need to be. It thus requires a fine balancing act between temporarily taking more control (and needing to in order to get learners to find productive ways of working) and letting go enough that they can start to develop their personal agency and take responsibility for their own learning. Ultimately this is the clearest indicator of whether a learner is going to overcome their circumstances and succeed at mathematics.

Background information

OLICO Youth caters to 95 learners in grades 7-11 who attend computer-based mathematics classes for an hour after school, twice a week. Learners also attend life skills, study skills and literacy programmes on Saturday mornings. Participation in the programme is voluntary and learners are accepted on a ‘first-come first-served’ basis. However, strict attendance requirements are adhered to and learners who do not attend regularly are asked to leave the programme. OLICO only accepts a new intake of learners at the beginning of Grade 7 each year. Results of initial diagnostic tests each year suggest that the majority of learners entering the programme are between 2 and 4 years below grade level in mathematics.

The design of the programme is straightforward. Learners arrive at the centre after school; show completed homework to gain entry to the computer lab; follow the lesson-prompts on the computer; and collect new homework at the end of the session. The actual computer-based lessons are divided into 5 Ps.

Pre-Quiz

Presentation (video)

Post-Quiz

Practice mixed set

Pause and review

[1] On a related point, we have been unable to replicate the example Salmon Khan uses in his TED talk of a struggling learner making dramatic gains to catch up with the rest of the class. In our experiences to date, the progression is much more gradual and requires sustained intervention since the skills that are lacking are rooted in core number proficiency gaps.

[2] There is a far greater variety of pre-existing options available to FET Phase learners already suited to the South African context. We have enrolled our Grade 10 and Grade 11 learners on Siyavula’s Intelligent Practice system (www.everythingmaths.co.za) and supplemented this with our own videos, lesson plans and checkpoint assessments.

[3] In this our experience echoes that of a New Zealand-based online programme, mathsbuddy.co.nz, and we have incorporated some of their suggested strategies into our work to deal with this.

OLICO Youth Academic Report (to Oct 2014)

January 1, 2015 By Andrew

OLICO Youth continues to explore the effectiveness of computer-based mathematics solutions with a group of South African township-school youth. This year, sixty-five learners from Diepsloot between grades 7-10 are participating on the programme. Between Feb and Sept 2014, OLICO learners attempted over 76,609 maths questions and achieved an average 35% increase in basic number sense as well as an average 47% increase on foundational curriculum content. The focus now shifts to improving the speed at which learners progress through these lessons and improving the retention of newly-acquired knowledge. OLICO is committed to finding a genuine solution that is effective, sustainable and replicable.

Introduction OLICO Youth is a community-based academic support programme for township school youth with a focus on senior phase mathematics. The aim of OLICO Youth is to provide learners with a pathway into high school mathematics by bridging the gaps in foundational understanding, thereby serving as a conduit into algebra and geometry. There are currently sixty-five Diepsloot learners between grades 7-10 enrolled on the OLICO Youth programme. Learners attend 2 computer-based after-school sessions twice a week as they progress through the OLICO curriculum at their own pace. To remain on the programme, learners must attend at least 75% of their sessions and complete their homework assignments. Learners also attend on Saturday mornings for non-academic input.

At each mid-week session, learners follow OLICO’s 5-Step Process:

Arrive on time for scheduled session

Check in and show completed homework

Log onto computer with unique username

Follow OLICO’s 5 P’s

Check out and receive new homework

Learners work at their own pace once they log onto the OLICO learning management system. The personalised learning environment allows learners who score 80% or more in the short pre-lesson quizzes to skip these sections. Learners who achieve less than 80% however follow OLICO’s 5 P’s which is made up of the following:

Pre-Quiz

Presentation (video)

Practice exercises

Post-Quiz

Pause and review

The most pressing need for the majority of learners joining OLICO Youth is to substantially improve their general number sense skills and mathematical fluency. There are major challenges across South Africa in the shift from foundation phase mathematics (grades 1-3) to intermediate phase mathematics (grades 4-6) with most learners enrolling at OLICO Youth needing to catch up 2-4 years worth of content. Given this context, OLICO focuses on building foundational skills as a gateway to deeper curriculum level understanding. Key lessons to date:

Learners are highly enthusiastic about technology. There is little doubt that the computer-based approach leads to an increase in practice attempts, time-on-task, positive attitude towards maths and commitment to progressing through the lessons. The sheer volume of questions attempted by OLICO learners this year is impressive. In total, over 76,609 maths questions have been attempted this year alone… and counting.

OLICO learners achieved substantial improvements in their basic number sense abilities. Since February this year, 83% of the learners completed the number sense curriculum with learners improving by an average of 35%. The number sense curriculum includes timed exercises focusing on improving mental arithmetic with questions on: single/double-digit addition/subtraction; number bonds; finding differences; friendly numbers; multiplication and division tables.

OLICO learners also demonstrated substantial improvements on the foundational curriculum. Mastery for foundation-level topics is set at 80% since the lessons are fundamental requirements for future mathematics understanding. Learners are permitted to progress to subsequent lessons only once they have achieved this level of mastery. As a result, the speed at which learners progress through these lessons is an important additional indicator of effective learning. Given the amount of content learners need to cover, there is a need to increase the speed at which learners are progressing but, this concern aside, the results are impressive:

On average, learners improved on their pre-assessment results by a whopping 47%:

Equally pleasing is the progress of the OLICO learners who were part of the original pilot phase in 2012. These learners joined while in Grade 8. They are now in Grade 10 and have consistently improved their curriculum-level results. The grade 10 learners have displayed impressive levels of commitment, dedication and a positive attitude towards mathematics which is beginning to show in their results over time:

In addition to the clear benefits of an increase in the amount of question-attempts and time-on-task, technology also offers instant feedback to the learner which is particularly important for effective learning. Learners can thereby shape their personal learning experiences by responding appropriately to the cues provided by the computer.

Finally, the numerous advantages of technology do not replace the need for a skilled facilitator alongside a context-appropriate programme. At the very least, the facilitator must have a well-developed number sense in order to intervene when learners get stuck. Technology enables a good facilitator to interact and assess learner needs (with real-time feedback) on a highly individualised and targeted basis.

5 core focus areas going forward:

Over the forthcoming 6-9 months, the OLICO team is focused on finding ways to increase the speed at which learners progress through the OLICO lessons and thereby increase curriculum coverage.

The OLICO team is also placing a greater emphasis on regular checkpoint interventions to ensure and better assess knowledge retention.

Since English literacy remains a barrier for many of the OLICO learners, a literacy strategy is to be incorporated as core to the programme from 2015.

A new custom-designed database is being built to simplify the OLICO processes and reduce administrative complexity.

OLICO will continue to develop and publish open-source mathematics materials on our resources page and build a programme around Siyavula’s Everything Maths for our Grade 10 and 11 learners in 2015.

Conclusion: A significant amount of progress has been achieved over the past year, both in terms of learner progression through the OLICO maths curriculum and in the development of context-appropriate open-source content. There are strong positive indications that OLICO Youth is addressing a number of the deep-rooted foundational challenges facing SA’s learners. The key remaining challenge over the next 6-9 months is for OLICO to achieve progress on the “5 core focus areas” (as identified above). Success in this regard will determine whether the OLICO solution is indeed ready and effective enough to begin replicating in new environments.

Some programme background and history: Initially, the OLICO Youth project began by experimenting with freely-available online mathematics videos and exercises from Khan Academy. However, there are a number of challenges to using the Khan Academy platform for OLICO’s specific purposes and a decision was made to build OLICO’s own customised learning management system (LMS). The OLICO LMS thus pools together a variety of relevant open-source materials and the OLICO team has been developing additional content when context-appropriate resources do not already exist (published as creative commons). For the older grades, it also serves as a gateway into the Siyavula’s Everything Maths. The broader mathematics context: Although it is a very small sample, the profiles of the OLICO Youth learners reflect many of the broader South African mathematics challenges:

The nation-wide results of the Annual National Assessments (ANAs) provide clear indications that the majority of learners require significant remedial and foundational support in mathematics. The complexities of curriculum-level content increases greatly because learners are often still grappling with basic maths fundamentals. See also: SAHRC and UNICEF (2014) Poverty traps and social exclusion among children in South Africa. Pretoria: SAHRC.

The shift from foundation phase mathematics (grades 1-3) to intermediate phase (grades 4-6) appears to be particularly problematic. The transition has a deep impact on mathematics understanding as it involves a shift from additive reasoning to proportional reasoning; from concrete representations of numbers and concepts to more abstract representations; and from mother tongue instruction to English. For more on the difficulties in the shift to proportional/multiplicative reasoning, see: Hiebert, J. And Behr, M. (eds) (1988) Number Concepts and Operations in the Middle Grades, Vol 2. Research Agenda for Mathematics Education. NCTM; and Harel, G. And Confrey, J. (1994) The development of multiplicative reasoning in the learning of mathematics, SUNY press, Albany.

Experimenting with Khan Academy in Diepsloot (Report)

April 23, 2013 By Andrew

In mid-April 2012, OLICO embarked on a 6-month experiment to gauge the effectiveness of freely-available on-line learning solutions with a group of South African township-school youth. Forty-eight Grade 8 learners from Diepsloot were enrolled on the program. Twenty-nine learners completed the six months with an average increase of 22% on their curriculum content compared to their initial pre-project assessments.

To download the PDF version of this report, click here.

Introduction:

In mid-April 2012 OLICO set out to determine whether there is evidence to suggest that computer-based content such as the freely-available Khan Academy could successfully supplement the mathematics education of South Africa’s township-school youth. Khan Academy is a free online repository of educational videos and exercises across a wide range of disciplines with a particularly extensive collection of mathematics content.

The OLICO project enrolled a group of 48 Grade 8 learners from surrounding Diepsloot high schools. The learners were provided with lesson notes (designed by educators from the mathematics department of St Davids Marist Inanda) that incorporated a selection of Khan Academy videos and exercises covering a range of topics on the Grade 8 curriculum. In total, the lessons covered approximately 15% of the national curriculum. In addition, learners were assigned paper-based homework sheets for each lesson and were required to complete tutorials and an assessment at the end of each module (comprising of up to 5 lessons).

For the first month of the project, learners received computer literacy training which included directions on basic email and internet usage and assistance with registering as a Khan Academy student. In addition, each learner wrote a baseline assessment designed by the teachers from St Davids. The teachers described the assessment as one upon which they would be very concerned if any of their own grade 8 learners scored below 40%. None of the Diepsloot learners managed to pass. The group average was 18% and the highest mark was only 32%.[1]

It was soon apparent that remedial work was required before the learners could continue with curriculum-specific assignments. In general, learners struggled with basic tasks such as “addition with carrying” or “subtraction-with-borrowing” and when asked to complete elementary calculations like multiplying 6 by 7, learners would draw 6 rows of 7 “sticks” and then count them. That such basic concepts related to times-tables knowledge and ‘mental maths activities’ were not previously instilled to these learners despite 8 years of schooling is highly troubling, but is precisely the problem that this experiment is attempting to address.

In formulating our approach we implemented a 10-Step-Process for learners to follow (the details of which can be found below). Local facilitators from Diepsloot were employed to ensure that learners were following this process. In the event that learners needed assistance, the facilitators were given strict instructions to refer learners back to the “Khan Academy” videos or exercise hints, as the facilitators themselves had limited mathematics skills to avoid contaminating the experiment.

It is encouraging that by the end of the 6-month experiment, learners achieved an average increase of 22% on the curriculum content compared to their initial baseline results. While this improvement only lifted the group average to 40% it still reflects significant progress and suggests that computer-based learning solutions like Khan Academy can indeed be effective in supplementing mathematics education.

Of particular interest is that every learner in the group showed a positive improvement on completing the assigned curriculum. No learner achieved less than a 10% increase with the movement of the group as a whole strikingly illustrated in the chart below.

As encouraging as these improvements are, it is unlikely that technology on its own is capable of delivering a solution to SA’s educational challenges, although it seems entirely possible that the proper application of certain kinds of technology has the potential to make a marked positive contribution. At the very least, the growing prevalence of technology in educational contexts across the spectrum offers increasing opportunities for collaboration and shared learning. It is in this spirit that we offer further details of the project below:

Infrastructure Involved

1 x computer laboratory with 20-networked computers with individual headsets.

We used NComputing L300-series terminals with a mid-level i7 desktop computer running Windows 7 Enterprise as a server.

Each learner works on their own computer. On the current model, 1 computer can serve 5 learners attending after school twice a week for 1 hour long sessions in the afternoons. With 20 workstations our current capacity is 100 learners.

Internet connection is required for Khan Academy exercises and learner progress-tracking.

We used a standard 3G connection since we had good network coverage. It was expensive although Cell C now have a 100GIG prepaid option for R2500 which we are using in 2013. There is also Khan Academy Lite which is designed to work offline as a self-contained solution to low-bandwidth problems.

We used approximately 12 GIGs of bandwidth per month.

The Khan Academy videos were pre-downloaded onto the server to preserve bandwidth usage.

Learner Selection Process:

48 learners from Diepsloot secondary schools enrolled in response to a once-off distribution of pamphlets to surrounding secondary schools.

The invitation was exclusively for Grade 8 learners. Interested learners were encouraged to self-select onto the programme. We discouraged schools and teachers from pre-selecting learners to avoid learners attending against their will.

Learners were informed they would be accepted on a first-come, first-served basis although this wasn’t implemented as we accepted all 50 applications we received (2 learners applied and were accepted but did not begin the program).

There was no prior academic merit requirement and no English comprehension assessment.

Commitment Required of Learners:

Learners were expected to attend at least two sessions per week of at least 1 hour per session.

The centre was open in the afternoons after school between 2:30pm and 4:45pm.

Learners booked their places in advance to ensure availability of computers.

We originally intended to run the pilot for three months and assess the situation thereafter, but extended this by a further three months as a result of the greater than anticipated need for remedial work.

Learners were required to attend computer literacy training with an emphasis on internet, email, browsing and the skills required to effectively navigate through the Khan Academy website.

At the end of the experiment, 60% of the learners completed all the modules assigned. The remaining 40% either dropped out or were excluded due to inadequate commitment levels. There was no learner retention strategy applied and those who did not meet the minimum requirements were simply excluded.

Pilot Facilitators:

The pilot facilitators were relatively low-skilled Diepsloot residents with instructions to follow the 10-Step-Process and direct learners back to the Khan Academy videos or exercise hints in the event that they needed help.

10-Step-Process

1. Learner arrives for pre-booked slot

2. Learner Checks-in with facilitator:

Marks the Attendance Register

Hands in Homework

3. Learner logs onto computer with unique username

4. Completes times-tables exercise (5mins)

5. Opens lesson on desktop and proceeds with video/Khan Academy exercises

6. Learner aims to correctly answer approximately 8/10 khan academy exercises in-a-row

7. Facilitator monitors progress and advises next step based on homework submitted & Khan Academy progress

Learner either revises lesson homework or begins new lesson

8. Time up – Learner ends session & logs out

9. Learner Checks Out

Collects marked homework sheet

Collects new homework sheet

Learner books slot for next session and collects sweet

10. Everyone travels home safely

Key Lessons to date:

Very little extrinsic motivation was offered to keep learners motivated and attending. None the less, 60% of the learners who enrolled on the project met the minimum attendance requirements with no retention strategy applied.

Perhaps the most important practical lesson we learnt is that learners need constant supervision to ensure they are on the correct exercise and watching the correct video. Khan Academy is a very open system, which makes it very confusing for our learners to navigate. Left to their own devices, with only instructions about which videos to watch and which exercises to complete, the learners were soon lost within the Khan system and attempting problems that were well beyond their capabilities. They did not have the prerequisite skills to understand where they were within the Khan system itself and how this related to what they needed to learn.

Learners also required considerable prompting to move on once they understood a concept. Learners had a tendency to remain on topics (or revert back to topics) on which they had achieved success – often completing far more exercises than necessary (sometimes hundreds of exercises and on occasion into the thousands). This may be partly explained by the built-in affirmation the Khan Academy site provides and the concomitant boost to a learner’s self-esteem when repeatedly getting certain concepts right. Their dedication to the task was admirable, but not correctly channeled when considering the amount of ground that has to be covered.

Learners often don’t have the basic background knowledge to understand the concept that they are currently studying. Here we need to look at adequate testing to identify prerequisite knowledge before a particular new concept is attempted.

Since learners are only able to attend two sessions a week at the centre itself, the development of homework material to be used between the computer-based sessions is vital to the success of the program.

To improve the impact of the Khan Academy videos, learners benefitted from pre-prepared worksheets that corresponded directly to the examples in the Khan videos they were watching. Learners were asked to attempt the exercises prior to watching how they were completed in the video.

The Khan Academy videos are generally useful, although there are a number of limitations, most notably the following:

Some videos confused South African learners with references to specifically American themes like dollars, quarters and dimes.

Certain mathematical mechanisms are different too, for example Khan Academy uses the mnemonic, ‘PEMDAS’ whereas most South African schools use ‘BODMAS’.

Some of the terminology in the videos is confusing for non-first-language English speakers, e.g. the use of word “parentheses” instead of “brackets”.

Some of the videos are focused on procedural skills and not on conceptual knowledge. This appears to be a common critique of the Khan Academy videos although possibly overstated.

A number of the videos are out of sync with the corresponding exercises. For example, certain videos explain a difficult mathematics skill, but the linked exercise is very simple, and vice versa.

Provision will need to be made for sections of the South African curriculum that are not featured on the Khan Academy platform.

By registering as a coach on the Khan Academy website, the facilitators are able to monitor the progress of learners in real-time and examine progress on a sum-by-sum basis. The coaching dashboard is very useful but not particularly user-friendly when trying to monitor a number of learners simultaneously. For example, there is no way to filter learners according to those currently logged-on. We’ve since discovered Always Prepped offers a more intuitive layout for higher-level monitoring of individual learner progress.

For future development, a better pre-assessment is necessary to determine a learner’s prior mathematical ability and to establish an appropriate entry point to the Khan Academy. As stated previously, learners often needed to return to very basic concepts to establish a good grounding.

Translations / Other examples of Khan Academy Projects in South Africa:

The University of the Cape Town, Click Maths and Numeric have translated a number of videos into isiXhosa. Find the Xhosa content here. We used only the English videos in our experiment but it will be interesting to see what impact this has in the future particularly with rural learners.

Numeric is currently using the Khan Academy with a variety of groups and schools in the Western and Eastern Cape. The Numeric website is full of useful tools and tips for anyone wishing to experiment in their own South African context.

The process involved in the OLICO experiment above is slightly different to that of the Numeric approach in that Numeric focuses primarily on Khan Academy as a group activity. This has the potential advantage of fusing a strong social bond amongst learners and is also likely to further promote peer-to-peer learning, a much under-rated educational tool as we discovered during our time with IkamvaYouth. The approach described in the OLICO experiment however is designed specifically for learners to proceed at their own pace. Correspondingly, this has the potential advantage of ensuring that quicker learners are not held back by struggling learners or those who are struggling need not be left behind when a group moves to the next topic.

There is no doubt in our mind that there is considerable merit to both approaches.

Next Phase Questions:

During the next stages of the project there are a number of questions still to be resolved including, but not limited to the following:

It would be interesting to compare the improvements displayed by the group to the progress (or lack thereof) of a control group.
Of particular interest is how we design a pathway into algebra that ultimately equips a learner to take Pure Mathematics (as opposed to Maths Literacy) when selecting subjects at the end of Grade 9?
It remains an open question as to whether there is a minimal level of prerequisite skill required for Khan Academy to be effective; how we measure this; and how we bridge this gap?
A key element is therefore how we most effectively test a learner’s prerequisite knowledge before proceeding with a particular Khan exercise?
There are issues pertaining to how best to contextualize the Khan Academy aspects that are not suitable to the South African context?
And finally, given that we have had learners beginning at extremely low levels of knowledge, can computer-based solutions like Khan Academy also move learners up from 40% to 60% or higher?

Sustainability Factors:

Currently the funding for the project comes from private donations and corporate sponsorship. We have begun experimenting with additional income-generating opportunities, including a low-cost learner fee similar to how we operate our adult education programs. It’s important to note that this fee is not for access to the Khan Academy (which is free), but to cover the operational costs of the local facilitator, lesson notes and the centre. There will of course be a number of learners who cannot afford even a low-cost option and we invite learner sponsorships to either supplement or fully cover the costs.

In addition, the operating costs of the centre are supplemented by using the computers in the mornings for additional income generating activities. These include: adult education, post-matric bridging year programmes and IT/internet services for community members.

Commitment to Creative Commons and ‘Open Source’ Content:

Any lessons learnt, notes developed, processes implemented or software built will be published under the creative commons licence and we hope to learn as much from others as from our own experiments. The invitation is thus for everyone to participate, either in an implementation in a local environment or through further development of the programme content itself. The long term vision is to contribute to a network of individuals and groups around the country working together to keep the content current and adapted to the local situation.

Any queries or comments can be directed to Andrew Barrett: andrew@wearegrowing.org.

[1] This is indicative of the wider deficiencies within the South African education system which ranks 139th out of 144 countries in the World Economic Forum’s 2013 Global IT Report. South Africa also ranks second last in maths and science education. See: http://www.weforum.org/issues/global-information-technology.

For information on Diepsloot, we highly recommend Anton Harber’s book Diepsloot.