JA Africa prides itself as an organization with integrity and transparency. We have global and regional key performance indicators by which we measure the success and impact of our programs. These include student enrollments, volunteer engagement and program completion, to name a few.
JA programs are generally delivered within a school year and students surveys are compared before and after the program. Student surveys can show that students moved toward a favorable attitude on dimensions such as 1) the importance of gaining skills for their future careers now or 2) The importance of being attentive to the industry they aspire to work in as examples of practical impact. Attention is given to avoid an inflated number of schools or students reached (i.e. double counting schools and universities). Data is also captured and disaggregated by gender and age in order to be able to produce useful analytics.
The delivery of in-school programs takes place inside of the traditional school day (i.e. is carried out in schools) and is provided directly for the students. Care is taken to ensure that this data is backed up with paper-based survey forms and audited for accuracy, consistency and quality. Multiple points of reference are used to ensure accurate reporting, such as the number of students per class.
JA’s approach for evaluating its programs is consistent with established best practices within the evaluation industry/community. During the pilot or initial release of a new program, we conduct a formative assessment (or baseline survey) which is either paper-based or technology-based depending on the target group). The formative assessment explores the general nature of the program, establishes baselines for future evaluation, and provides an important feedback mechanism for our curriculum developers to suggest improvements to the program. At the end of the programs, JA conducts summative evaluations to evaluate program impact. These studies explore and assess students’ knowledge gain, skills acquisition, attitude change, and indication to adopt or engage in new behaviors (“behavioral intentionality”) relative to the program’s content. The methodology employed for the summative evaluation is rigorous, and allows JA to calculate meaningful measurements that are statistically sound and can be compared to other student development programs.
Examples of questions we seek to answer through our evaluations are:
1) What are the impacts of the programs on participating students?
2) What is the impact of participation in program on students’ knowledge of financial literacy and other knowledge components embedded in the curriculum?
3) What is the impact of participation in a program on students’ acquisition of skills related to the curriculum?
Beyond the students who will be directly reached through this program JA posits that diffusion happens in its programs as research indicates that young people are heavily influenced by their peers. When they engage in activities in which they find value, their behaviors and practices have a diffusive effect that can be farther-reaching, in terms of volume, quality and impact. JA believes therefore that the spillover effect of this program can be extensive.
Our experiences over 100 years as a global organization show the following:
1. Project training attracts more students than just those targeted, often word of mouth gets the attention of different age cohorts and peer groups.
2. Sharing learning and knowledge takes place in higher proportions than direct formal transfer or knowledge, learning by doing, practice of peers are great sources of learning. This happens more than JA is able to document.
From the beginning of a project, strong collaboration will be achieved with all levels and departments of the government. Local government agencies and district development committee are involved ensuring their commitment to assisting the families involved. Stakeholders (such as Ministries of education and of youth) are accessed to obtain support. This also strengthens the capacity of the schools and educators.
JA programs aim to see marginalized students increase their access to and utilization of new skills to enhance their future career prospects.
3. Projects can attract more resources to a community/school as there is greater attention, organization, interest and commitment. Some of these resources are other interested stakeholders such as financial institutions, local government services, etc. In this way, the initiative garners a critical mass of interest that propels it to be self-replicating.