SHAIGLE

Simulations for Equity in Education


Tutorial

 

Figure 1. Screen-shots of the user interface of the SEE model – main menu (dark blue cells); scenario setting matrix (light blue section); viewing results (green cells); entering data (orange cells).

 

Setting scenarios

A scenario is a hypothetical set of future interventions and the outcomes of those interventions.  The general parameters for a scenario are ideally set at a stakeholder meeting, with the participants providing ideas about possible future interventions.  The advantages of using a model to compute the outcomes of scenarios are that it can incorporate many pieces of the whole situation in its calculations and can provide a consistent framework for alternative scenarios.

Setting a scenario. In SEE, a scenario consists of the selection of one or more interventions, setting the scale of each of the intervention(s), the time-frame for its or their implantation, the distribution over the different risk-groups and targeting particular barriers within the risk groups (more on this later).  All the scenario setting is done in one table, shown in the top panel of Figure.

 

Viewing results.  The results are shown in graphs below the scenario-setting table (see top panel Figure).  The graphs are flexible, and the user can decide which graphs to make visible, and can adjust which results to display, which year to display and whether to see results by risk group or national averages.

 

Entering data.  Data is entered on one page, in a series of small tables, a sample of which are shown in the lower panel of Figure .  Data entry is done in the ‘Data and Calculations’ page.

 

 

 

Formal description of the SEE model

 

Getting started 

 

 

Before you start making scenarios by adding interventions, there are a few help full options for setting up your work. These basic options are available on the menu bar at the left when you open SEE, as shown and explained here:

 

 


Select language

 

You can select the language you want to work with – English, French or Portuguese– with the ‘Select language’ drop-down menu.

 

 

‘Choose simple SEE’ or ‘Choose full SEE’

Simple SEE is a smaller version of the tool for use in districts or in very small countries. The simpler interface that includes only one risk group (but still allows you to define barriers within that group) and five interventions. Data can be entered on the ‘Interface’ page. Full SEE is intended for a more detailed analysis of equity. It includes four risk groups and up to ten interventions.  It also includes the MoRES and sensitivity analysis modules. Data must be entered on the ‘Data and calculationspage.

 

About SEE’

Provides basic information about the developers of SEE.

 

‘Data summary’

Clicking on this button opens columns with basic information about the country or region – number of pupils, teachers, classrooms and books and the prevalence of different marginalization factors within the risk groups, and about the effectiveness of interventions at reducing four major gaps – non-entry, repetition, drop out, and failure to learn.

 

‘Show/Hide MoRES view’ and ‘Show/Hide sensitivity analysis view’

These are two additional modules for special analysis that can be shown or hidden with the buttons shown at left.

 

 



Basically, the scenario setting happens in the table in the blue section, which looks like the diagram below. You can add up to 10 different interventions in this table. The list of interventions comes from the data page. In this diagram, one intervention has been added, Mother-tongue interventions (Balochi), in the second row.  Balochi is a mother tongue of Balochs.

 

Now let us add a second intervention here.

 

 

When you click on an empty cell in the ‘Interventionscolumn of the scenario table (the second cell for us in this example) a pop-up menu with all of the interventions appears. (On a PC, the box is smaller than the one shown here, which is from a Mac.) Click on one of the interventions. This automatically adds it to the scenario table. We are going to select the intervention ‘Mother-tongue instruction’.

 


Once you have selected an intervention, you can find out more about it–in particular, the units used for the intervention and how many children are affected by each unit–by clicking on the ‘infobutton to the left of the intervention name. The ‘infobutton for ‘Mother-tongue instruction’ shows that the unit of this intervention is the teacher, and that for each teacher trained, 40 children benefit (per year).

 

 

 

Next, se the parameters of the intervention.

 

The parameters of the interventions tell the model about the timing of the programme, the scale and the targeting towards risk groups and barriers.

 

Timing.  To the right of the intervention name are three cells where users can input when the programme will start; how many years the roll-out period lasts, and (if any) what the last year of the programme is. In our example(shown below in the next figure),the first mother-tongue schools will start in 2015 and the roll out period is 8 years, and the program will continue to run at least for the projection period of 10 years.

 

Distribution over risk-groups.  Next, you can input how many units will be provided for each of the risk-groups.   How many units will go to girls in the northern regions? To boys in the northern regions? The rest of Balochistan? In our example, 14,000 children in the north, evenly distributed between boys and girls, will be reached by the programme.


 

 

Targeting barriers within risk groups.   Within risk groups, how well is the intervention targeted to reduce particular barriers faced by vulnerable children? In our example, we assume that 90% of the children receiving mother-tongue instruction will be children who benefit from it, i.e., who speak the language of mother-tongue instruction at home.

 

The diagram below shows you where to add all the settings.

Text Box: 1. Add interventionsText Box: 2. Add timing – year when intervention starts, duration of roll-out, last year of programme.

 


1  

 



 

 

 

 

 

 

 

 

 



Text Box: Provide the number of units (e.g., schools, meals, books, teachers) to go to each risk group. Hint: If the risk groups include boys/girls, the intervention will often be split 50/50.

Text Box: Add how well the intervention will be targeted to reduce specific barriers.

 

 

 

 

 

 

 

 

To see what your scenario looks like – how your intervention is implemented over time, according to your settings–click ‘Info’ button again. The graph below shows the scenario for ‘Mother-tongue instruction’ and that by 2015, 100% of children in the North (boys and girls) are reached by it (effective coverage).  Once the effective coverage is 100% further increases to scale will not bring any more benefits. 

 

To hide the graph, click on the ‘ Info’ button again.

 

 


Keep adding interventions until you are satisfied with this scenario.


 

Saving scenarios

 

 

 

 

Once you are satisfied with your scenario, which can consist of one or multiple interventions, you can save it.   To the right of the scenario setting table, you’ll find a box with a ‘Save scenario’ button next to it.  To save a scenario, give it a new name in this box, and click the button.  You scenario will automatically be saved.  

 

 

 

Retrieving saved scenarios

 


If, later, you would like to present your scenario, or work with it again, you can retrieve it.   Below the “Save scenarios” button, you will see another box with a button ‘Get scenario’ next to it.  To retrieve your scenario, click on the box, to open a drop-down box with all the saved scenarios. Select your scenario, and click on ‘Get scenario’. All your scenario settings should be back in the table!

 

 

 

Deleting scenarios

 

If you save many scenarios, you may get to the point in which the list of scenarios is too long and cumbersome, or you may want to get rid of scenarios for another reason. To remove saved scenarios from SEE, go to the ‘Saved Scenarios’ worksheet. The scenarios are saved in column format with the name of the scenario at the top of the column in row 1. Simply delete the columns with the scenarios that you no longer need. The list of scenarios will be automatically updated.

 

Pre-entered, standard scenarios

 

There are four standard scenarios, which you cannot delete: ‘Blank’, ‘Pilots 1’, ‘Pilots 2’, and ‘Pilots 3’. 

·         ‘Blank’ = this scenario returns a blank intervention table and is useful for starting new work.  It is like erasing the white-board.

·         ‘Pilots’ = these scenarios are useful to provide a starting point for cost-effectiveness and impact.  The assumption in these scenario is that one unit of each intervention is provided to each risk-group (it is like running many small pilots, hence the name).  The scale of the interventions is small so that there are negligible interaction effects.  There are three ‘Pilot’ scenarios to cover the entire maximum list of 30 interventions (each ‘Pilot’ scenario covers 10 interventions).


 

Customizing the results interface (green section)

 

 

One of the features of SEE is the visual link between setting the scenarios and seeing the results:  whenever you make a change to a scenario in the intervention table, this is immediately reflected in changes to the results charts.

 

You can customize and select which results to see, and can easily switch between different views.  This section discusses the different options.

 

There are seven different graphs. They are:

 

 

1.       Bottleneck chart–a high-level summary chart of selected key outcome indicators showing the initial values and the values in a target year for the different risk groups or for all groups together.

2.       Cost chart–shows the costs of the interventions (by intervention) over time for the risk groups or all groups together; can include initial costs at start-up or only the added (marginal) costs of the interventions.

3.       Compare scenarios chart – this chart is useful if you have made multiple scenarios and want to compare the outcomes.

4.       Scenario cost effectiveness chart–shows how many children reach three education benchmarks – school entry, completion, and learning – per $1000 spent on each intervention in the scenario. 

5.       Impact chart -- shows the additional children who will enter school, complete a level, or reach a benchmark learning level as a result of the interventions in the scenario.

6.       Equity chart–compares progress over time of selected scenario outcomes for all risk groups and the average, together on the same chart.

7.       Out-of-school children chart–shows the distribution of out-of-school children by four categories, children who are out of school because: they will never enter; they will enter late; they dropped out; or they are in preschool.


 

Setting up the bottlenecks chart

 

 

The bottlenecks chart is a graph that should be set up to track the most important outcomes in your policy dialogue– whether the discussion applies to an effort to reduce bottlenecks overall, a specific initiative to improve learning outcomes, a UNICEF country office MoRES effort or an initiative to reduce the number of out-of-school children.

 

To see the bottlenecks chart, click on the ‘View Bottleneck Results’ button.

 

When you click on this button, you will see the arrangement shown below.  In this chart, you can:

1.       Select which risk group to show results for, by clicking on the ‘select risk group’ box and selecting from the drop down list.

2.       Values for which target year to show (the start year is always shown), by clicking on the ‘select target year’ box and selecting a year from the list.

3.       Which (up to six) bottlenecks or high-level indicators to show.    The indicators are selected by clicking left of the chart and selecting from the drop down box.  The list of indicators is provided in Table 7.

 

 


 

 

 

 

 

 

 


Table  List of the outcome indicators in SEE (any of these can be selected for the bottleneck chart)

 

Additional entrants due to interventions

% children entering overage (only with OOS)

Number of pupils

Net enrolment rate (only with OOS)

Additional completers due to interventions

Gross enrolment rate

Additional children NEA Math pass

Survival to end of primary

Additional children NEA English pass

Primary completion rate

Number of teachers

Expected primary completion (% entry x survival rate)

Number of trained teachers

% Actual NEA Math pass

Additional trained teachers due to interventions

% Actual NEA English pass

Number of untrained teachers

% expected NEA Math pass

Number of classrooms

% expected NEA English pass

Additional books due to interventions

% Expected children with NEA Math pass (% completers x pass rate)

Books per pupil

% Expected children with NEA English pass (% completers x pass rate)

Pupil teacher ratio

% children in school (incl. CBE)

Pupil trained teacher ratio

% OOS children who will not enter school

Percent trained teachers

% OOS children who will enter late

Pupil classroom ratio

% OOS children who dropped out

Enabling social norms

% OOS children in preschool

Enabling legislation and policy

Survival to end of primary

% schools receiving capitation grant on time (budget)

Primary completion rate

% schools with SMC meeting >1/term (governance)

Expected primary completion (% entry x survival rate)

Student attendance rate

% Actual NEA Math pass

Teacher attendance rate

% Actual NEA English pass

% desired books (compared to national target

% expected NEA Math pass

% desired teachers

% expected NEA English pass

% desired qualified teachers

% Expected children with NEA Math pass (% completers x pass rate)

% desired classrooms (compared to national norm

% Expected children with NEA English pass (% completers x pass rate)

% children with school nearby

% children in school (incl. CBE)

School utilization rate (if school is available)

% OOS children who will not enter school

% children who will enter school

% OOS children who will enter late

% children age 12 who ever-attended school

% OOS children who dropped out

Gross intake rate

% OOS children in preschool


 

Setting up the cost chart

 

 

This chart shows the costs of the scenario.  Along the top of the chart, you see the total costs of all the interventions over the entire projection period of 10 years.  The chart itself shows the costs per year, subdivided by intervention.  In the chart below, we see that the total costs of the 730 formal Class rooms, 2003 School report cards, 8330 SMC,s and 13,000 Mother-tongue Instruction classrooms is $1,199 million over the course of the ten year projection.  The chart shows how these costs vary by year.  Mother-tongue instruction (shown in the red portion of the bars) has costs mainly for the 2 initial years (for training) and then again 5 years later (for re-training).  The formal Class rooms have costs every year – initially for the building of the schools; then to pay the salaries of the teachers and materials.

 

1.       Select for which risk group to show costs by clicking on the drop down menu for risk groups.

 

2.       The chart as shown below does not include the initial costs for education for existing schools, teachers and programs.  Sometimes, it is useful to see costs for existing programs plus the costs of the interventions.  In this case, you can select ‘Yes’ from the ‘Show start year costs’ selection box.

 


 

 


Setting up the compare scenarios chart

 

Often, when you have made different scenarios, it is useful to compare the results.  This can be done easily with the ‘Compare scenarios chart’.  Click on the button to show this chart.

 

To set this chart up, first select the scenarios you want to compare, by clicking on the drop down menus in the boxes next to the y-axis of the chart.   You can do this for up to six scenarios. In the chart below, we compare two scenarios, ‘Wing and mother T’ and ‘Schools only’.  We have chosen to look at ‘Additional completers due to interventions’.  With the first scenario, 362,000 additional children are predicted to complete primary school due to the interventions; in the second, 164,000 additional children complete primary school.

 

Next, select the result you want to compare. 

 

 

 

 

 

 


 

 

 

 

 


Setting up the cost-effectiveness chart

 

Cost-effectiveness is an important criterion in selecting interventions.  This chart shows how many children reach six education benchmarks – children with sufficient books, teachers, and schools as well as children who enter school, complete and learn – per $1000 spent on each intervention set in the scenario. 

 

The cost effectiveness chart has three customizable options:

 


1.       Set up which for which benchmark to see the cost effectiveness, by clicking on the drop down box for ‘Select result’.

2.       Set up whether to see average cost effectiveness for all risk-groups together (‘National’) or separately for each risk group (‘Equity’)


 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 


3.        Set up whether to see only direct impacts of the intervention on the selected result or also indirect impacts.  For example, building schools contributes directly to more children entering school.  But, because more entrants also (usually) translate into more completers, an indirect impact of building schools is more completers.  If we have selected the result ‘Additional children to complete primary per $1000’, and we select ‘Yes’ to including indirect impacts, the chart will show the additional children who complete as a result of higher entry.  If we select ‘No’ then the chart only shows additional children to complete as a result of higher retention rates.  It is recommended to usually keep the setting on ‘Yes’ as this captures the full impacts of interventions.

 

 

 




Setting up the impact chart

 

Impact is as important as cost-effectiveness.  The impact chart shows the absolute number of additional children who will reach six education benchmarks -- children with sufficient books, teachers, and schools as well as children who enter school, complete and learn – because of the interventions in the scenario setting. 

 

This chart has three of the same customizable options as the cost-effectiveness chart:

1.       Select which benchmark to show results for.

2.       Select whether to show outputs for all risk groups together or for the risk groups separately.

3.       Select whether to show indirect effects (recommended: ‘Yes’).

 

4.       In addition, you can select which year to show the results for.  For example, you can select how many additional children will complete primary school in 2015.  It is also possible to select ‘All years’ for a sum over all the years of the projection.

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 


Setting up the equity progress chart

 

 

This chart shows comparative progress over time for all four risk-groups and the overall national value.Youcanusethischarttocomparehowinequitygapsarebeingreduced.

 

You can choose which indicator will be in the chart out of the list of all of SEEs indicators (See Table 9).

 

The chart below shows the pupil-to-classroom ratio (PCR) over time in the four risk groups in Ghana, with our wing school and mother-tongue instruction scenario.Theadditionof200wing schools in the north and 1,000 in the rest of Ghana is not sufficient to reduce the pupil-classroom ratios in the country as a whole. In fact, rather than improve access in a disadvantaged area, in this scenario the north would see arise in PCR. This is because the Wing schools only provide additional classrooms up to grade 3; the existing schools would have to take up the growing influx of pupils in grades 4-6, leading to an increase in PCR inequality overall. This result may prompttheusertoconsiderascenariowithmoreschoolstargetedtothenorthernregionsinorder to equalize the pupil-classroom ratio across the country.

 


 

Setting up the chart for children out of school

 

 

 

 

The seventh chart shown below has the percentage of primary-school-aged children who are out-of-school (OOS) by risk group and by reason for being out of school: never entry, late entry and dropout. It includes a special category of ‘OOSchildren, namely those who are of primary- school age but still in kindergarten or preschool. Children OOS are computed from age-grade specific projections. This chart can only be used if you have entered age-specific school attendance data.

 

This chart has three customizable elements:

1.       Select whether to see the development of out-of-school children over the time of the projection (10 years) or, for a selected year only, but by age-group.  Both views can be valuable, but they answer different questions.

2.       Select the risk group to show.

3.       If you have selected to show OOSC by age-group, you need to select for which year.

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 


 


The chart above shows ‘OOS by age in selected year’ for our Wing schools and Mother-tongue scenario. You can also select which region to show (‘Boys north’ selected), and the year for which the result is shown (‘2020’ selected). The red bars show that a little more than 20% of children will never enter school (in this scenario). Children gradually enter school between the ages of 6 and 9 (dark blue bars of ‘children in school’ are progressively higher from left to right), and at the younger ages, some are in kindergarten (aquamarine portion of the bars). This significant group of school- age kindergartners reflects a general trend in Ghana where about 10% of school-aged children are in kindergarten.

 


Ref: Simulations for Equity in Education (SEE): Model description and user’s guide