Kenya - Hunger Safety Net Programme Impact Evaluation 2010-2011, First Follow-up Round

The first follow-up round covered 4,637 households. The sample size at the first follow-up was smaller than at baseline due to sample attrition.

The evaluation sub-locations were selected from a sample frame of all secure sub-locations in each district. In each district 12 sub-locations were selected with PPS (Probability Proportional to Size) with implicit stratification by population density such that there is an even number of selected sub-locations per new district.

The evaluation sub-locations were sorted within districts by population density and paired up, with one of the pair being control and one being treatment.
The sampling strategy for the quantitative survey was designed in order to enable a comparison of the relative targeting performance of three different targeting mechanisms. These are:

- Community-based targeting (CBT): The community collectively selected households they consider most in need of transfers, up to a quota of 50% of all households in the community;
- Dependency ratio targeting (DR): Households were selected if individuals under 18 years old, over 55 years old, disabled or chronically ill made up more than a specified proportion of all household members;
- Social pension (SP): All individuals aged 55 or older were selected.

For both the treatment and control sub-locations there are an equal number of CBT, SP and DR sub-locations. Assignment of targeting mechanisms to sub-locations was done randomly across the same pairs that were defined to assign treatment and control status.
In all the evaluation sub-locations, the HSNP Admin component implemented the targeting process. In half the sub-locations the selected recipients started receiving the transfer as soon as they were enrolled on the programme - these are referred to as the treatment sub-locations. In the other half of the evaluation sub-locations, the selected recipients were not to receive the transfer for the first two years after enrolment - these are referred to as the control sub-locations.

The households in the treatment sub-locations that are selected for the programme are referred to as the treatment group. These households are beneficiaries of the programme. In control sub-locations the households that are selected for the programme are referred to as the control group. These households are also beneficiaries of the programme but only begin to receive payments two years after registration. The targeting process was identical in the treatment and control sub-locations.
The following population groups can thus be identified and sampled:
- Group A: Households in the treatment sub-locations selected for inclusion in the programme;
- Group B: Households in control sub-locations selected for inclusion in the programme but with delayed payments;
- Group C: Households in treatment sub-locations that were not selected for inclusion in the programme;
- Group D: Households in control sub-locations that were not selected for inclusion in the programme.

Because targeting was conducted in both treatment and control areas, households were sampled in the same way across treatment and control areas. Selected households (groups A and B) were sampled from HSNP administrative records. Sixty six beneficiary households were sampled using simple random sampling (SRS) in each sub-location (in two of sub-locations this was not possible due to insufficient numbers of beneficiaries in the programme records). In cases of household non-response replacements were randomly drawn from the remaining list of non-sampled households. This process was strictly controlled by the District Team Leaders.

Non-selected households (groups C and D) were sampled from household listings undertaken in a sample of three settlements within each sub-location. These settlements were randomly sampled. The settlement sample was stratified by settlement type, with one settlement of each type being sampled. Settlements were stratified into three different types:
1. Main settlement (the main settlement was defined as the main permanent settlement in the sub-location, often known as the sub-location centre and usually where the sub-location chief was based. As there was always one main settlement by definition, the main settlement was thereby always selected with certainty).
2. Permanent settlements (permanent settlement is defined as a collection of dwellings where at least some households are always resident, and/or there is at least one permanent structure).
3. Non-permanent settlements.

As concern community level data, community questionnaires were conducted in every community for which at least one household interview was attached. A community was defined as a settlement or a sub-section of a settlement if that settlement had been segmented due to its size. Due to missing data, a small proportion of households are not linked to any community data.

The above explanation is taken from "Kenya HSNP Monitoring and Evaluation Component: Impact Evaluation Final Report 2009 to 2012". For more details please refer to this report in Related Materials section.


Two versions of the sampling weight are provided:

1) hh_wt sampling weights produce estimates for all households living in sub-locations covered by the evaluation (i.e. the study population). They do not provide estimates for any larger population.
Weights are given by the inverse probability of being selected by strata. For selected households, the weights are given by: wi = Ni /ni, where:
- ni is the number of beneficiary households interviewed in the ith sub-location;
- Ni is the number of beneficiaries listed in the HSNP administrative data for that sub-location.

For non-selected households, the weights are given by:wijk = 1 / [ (aijk/Aijk) *(1/bij)*(1/cij) ], where where:
- Aijk is the total number of non-beneficiary households of residency status k in the selected segment of the selected type j settlement in sub-location i;
- aijk is the number of households of residency status k in the selected segment of the selected type j settlement in sub-location i that were interviewed;
- bij is the total number of segments in the selected type j settlement in sub-location i (often bij=1);
- cij is the total number of settlements of type j in sub-location i.

2) hh_wt_original sampling weights produce representative statistics for the entire population of secure sub-locations within each district.

Community-level variables can be weighted using community weights (cmq_wt), which equal the sum of household weights across the households lnked to that community.

The above explanation is taken from "Kenya HSNP Monitoring and Evaluation Component: Impact Evaluation Final Report 2009 to 2012". For more details please refer to this report in Related Materials section.