To design the project's monitoring and evaluation system (M&E system) you use the second and third columns of the logical framework, with the Indicators and Verification Sources respectively.
Once you've listed the indicators that you will need to follow-up each goal, purpose or output (and activity if you want to), you can use the Details pane to choose the type of indicator and other options such as the value unit, the value range, etc. You can also register the baseline and set targets.
With all this information you can create the Monitoring tool in MS Excel. This tool allows you to follow-up the progress of your project over time:
More often than not, organisations use value-based indicators to monitor the progress of their project and assess its effects and broader impact. This is certainly a valid way to work, but that doesn't mean that ‘indicator’ must be synonymous with ‘values’. Logframer offers you many different types of indicators:
This means that Logframer gives you the flexibility to do ‘strict’ logical framework methods with ‘hard’ values. But you can also use different approaches such as Outcome mapping, using open ended questions for your proxy indicators.
Logframer offers you many options to specify units, value ranges, scores, etc., but that doesn't mean you are obliged to use all these options. Select the tools that best fit your way of working – don't let the tools prescribe how you should do things.
To add indicators to a goal, purpose, output or activity, you just start typing in the empty cell to its right in the Indicators column.
Indicators can have sub-indicators. This enables you to split more complex indicators into their basic components. You can add as many levels of sub-indicators as you want.
Adding sub-indicators and parent indicators:
The total value or total score of the indicator is calculated automatically based on the values of its sub-indicators:
For each type of indicator you can register a baseline value. The baseline is the measurement you make before your project (or at least any of the project's activities) starts.
Some types of indicators use targets, while others have none. If you use targets, you can specify different targets according to the calendar of the Monitoring Deadlines.
Some information can be found or registered centrally, for instance impact indicators that take information from government statistics. These can be entered by someone at your office.
In other cases you may have different teams in the field that register information about their beneficiaries or intervention zone. To get the total results at project level you may have to make a total of the numbers of the various teams.
It's also possible that you need to register information for every beneficiary (or a sample of the group of beneficiaries). The total number/score for that indicator at project level is then calculated as the total sum of the beneficiaries.
In the Indicator Details pane, select the Registration tab. Use the Registration drop-down box to select one of these three options:
Select the target group for which the indicator will be used. You can also use an indicator for all target groups
Use the Aggregate drop-down list to choose how you want to aggregate the information from the different teams or beneficiaries:
For value-based indicators, you can choose how you will define your targets:
In the Indicator Details pane, select the Scoring tab and go to the Targeting box (on the right). Select one of the three options.
To set the actual targets, go to the Targets tab.
If you use a simple target value, you can enter it directly in the cells
If you use a value range, a pop-up window will appear (double-click on the cell if it doesn't) where you can set a minimum value and/or a maximum value. To do so, select the sign (<; >; <=; >=) and enter a value. If you don't select a sign, the min/max value won't be used:
For questions with targets, you can choose which (total) value the indicator will produce:
To set the actual scores:
Value-based indicators are probably the most popular type of indicators used for monitoring projects. However, they may not always be the best choice. Logframer offers you no less than 17 different types of indicators to make sure you have the right indicator for the right occasion. They are grouped in five main groups:
Before you start detailing your indicators, the first choice you have to make is on the right type of question to use. On the Scoring tab, select one of the options from the Question type drop-down list:
Indicators without targets |
Ask a question and specify how much space the respondent (or interviewer) gets to answer |
|
|
The respondent gets a number of different options and can indicate which is the best and the worst option |
|
|
Include a picture of the situation before the project. For each report the respondent has to include a follow-up picture |
|
Value-based indicators |
Specify baseline and target values; choose the unit (items, kg, liters…) and specify a value range if necessary (min. and maximum values) |
|
|
Same as the above, but expressed as a percentage |
|
|
The respondent has to provide two numbers and then the ratio is calculated (for instance ratio of girls to boys in schools) |
|
|
Allows the respondent to give a series of numbers. You can specify a formula to give you the total number you want. Useful for sums of a series of items, complex ratios etc. |
|
Multiple options |
Respondent/interviewer can tick the box next to yes or no |
|
|
Identify different options so the respondent/interviewer can tick one or more of the options |
|
|
Identify different options so the respondent/interviewer can tick (only) one of the options |
|
Ordinal questions |
Identify different options, which the respondent has to rank according to his/her preference |
|
|
A typical Likert scale has five options (but can also be less or more), for example: strongly disagree; agree; neutral; disagree; strongly disagree. This indicator has only one statement to score |
|
|
Similar to a Likert type scale, but with a series of pairs: interesting – not interesting; practical – not practical; smurf – not smurf |
|
Expressing opinion |
Uses a series of statements on which the respondent can indicate if he/she agrees or disagrees with the statement. Each statement expresses a more or less positive attitude towards the concept that’s being analysed. These statements have been issued a score before based on a preliminar survey. This score is then used to measure the attitude of the respondent. |
|
|
Similar to Thurstone scale in design, but instead of a choice between agree/disagree, the respondent can give a more nuanced answer by indicating how strongly he/she agrees or disagrees. |
|
|
Similar to Thurstone scale, but the statements are ranked, meaning that if you agree with the fourth statement, it is understood that automatically you agree with the three that came before. |
|
|
A likert scale that is useful for working with groups, organisations, etc. because instead of indicating what the answer is of one person, you can register how many people have chosen that particular option (frequency). |
|
Other types |
Not only can you show an image of the baseline situation, but you can also include images of how the situation will look like from one moment to the other. |
|
|
Mixed sub-indicators |
If you add sub-indicators of different question types to a main indicator, the question type of the latter will be set to ‘Mixed sub-indicators’. The total score of this main indicator is always a score (never a value or percentage). |
Set the amount of white space that you want to give to the respondent/interviewer on the questionnaire.
On the Scoring tab:
On the Baseline tab:
On the Scoring tab:
On the Targets tab:
With scoring:
Beneficiary level monitoring:
Beneficiary level monitoring with scoring:
On the Scoring tab:
On the Targets tab:
Beneficiary level monitoring:
To calculate a ratio, the respondent has to answer to two questions. For each question, you can specify the number of decimals, unit and value range.
On the Scoring tab:
On the Targets tab:
With scoring:
Beneficiary level monitoring:
Beneficiary level monitoring with scoring:
On the Scoring tab:
On the Targets tab:
With scoring:
Beneficiary level monitoring:
On the Scoring tab:
On the Targets tab:
Beneficiary level monitoring:
With the Multiple Options question type, the respondent (or interviewer) can select one or more options from a list. If you want to ensure that he/she can only select a single option, use the Multiple Choice question type.
On the Scoring tab:
On the Targets tab:
Beneficiary level monitoring:
With the Multiple Choice question type, the respondent (or interviewer) can select only one option from a list. If you want to ensure that he/she can only select a one or more options, use the Multiple Option question type.
On the Scoring tab:
On the Targets tab:
Beneficiary level monitoring:
On the Scoring tab:
On the Targets tab:
The difference between the Likert-type scale and a full-blown Likert scale is that the Likert-type scale only uses a five-point (or seven-point, or whatever you prefer) to answer on a single question. A full-blown Likert scale on the other hand uses a series of statements that explore different dimensions of a subject.
On the Scoring tab:
On the Targets tab:
Beneficiary level monitoring:
A semantic uses pairs of adjectives to measure a person's attitude towards something. For instance: ‘The workshops on sustainable organic fertilizers were:’
To answer, the respondent can use a series of options that allow him/her to express different degrees of agreement with one of the two extremes. If you use an uneven number of options you can use a ‘Neutral’ option in the middle.
On the Scoring tab:
On the Targets tab:
Beneficiary level monitoring:
The method of equal-appearing intervals or the Thurstone scale is used to measure attitudes of people. Logframer allows you to use a Thurstone scale to measure an indicator, but developing such a scale is not so simple.
This kind of scale is used to measure people's attitude towards a fairly clear and unidimensional concept, using a number of statements that vary in how they express a positive or negative opinion about the main concept. We'll briefly explain the steps of developing a Thurstone scale:
You can now use the scale to measure attitudes 'in the field': enter the list of statements and use them to interview people or present them in a document. For each item, they should express whether they agree or disagree. The total score of a person is calculated by making the sum of the values of all the statements they agreed with, divided by the number of items he agreed on (in other words, the average of the statements they agreed with).
On the Scoring tab:
On the Targets tab:
The difference between a full-blown Likert scale and a Likert-type scale is that the Likert-type scale only uses a five-point (or seven-point, or whatever you prefer) to answer on a single question. A full-blown Likert scale on the other hand uses a series of statements that explore different dimensions of a subject.
The Likert scale can also be used to measure attitudes of people. When responding to a Likert questionnaire item, respondents specify their level of agreement or disagreement on a symmetric agree-disagree scale for a series of statements. Thus, the range captures the intensity of their feelings for a given item. As with the Thurstone scale, the development of a Likert scale takes some effort.
A Likert scale uses a number of Likert items, which are statements to which a respondent expresses his agreement or disagreement. Generally, a five-point or seven-point scale is used, for instance:
As you can see, each item is bipolar: it measures either someone's agreement or disagreement to the statement and allows you to give some measure of how much you agree or disagree. There is a variety of possible response scales (5-point; 7-point; 4-point…). All of these odd-numbered scales have a middle value is often labelled ‘Neutral’ or ‘Undecided’. It is also possible to use a forced-choice response scale with an even number of responses and no central neutral choice. In this situation, the respondent is forced to decide whether they lean more towards the agree or disagree end of the scale for each item.
The basic steps of developing a Likert scale are:
You can now use the scale to measure attitudes in the field: enter the list of statements and use them to interview people or present them in a document. For each item, they should express to what degree they agree or disagree.
The final score for the respondent on the scale is the sum of their ratings for all of the items (this is why this is sometimes called a summative scale). On some scales, you will have items that are reversed in meaning from the overall direction of the scale. These are called reversal items. You will need to reverse the response value for each of these items before summing for the total. That is, if the respondent gave a 1, you make it a 5; if they gave a 2 you make it a 4; 3 = 3; 4 = 2; and, 5 = 1.
On the Scoring tab:
On the Targets tab:
The cumulative scale or Guttman scale measures to what degree a person has a positive or negative attitude to something. It makes use of a series of statements that are growing or descending in how positive or negative a person is towards the subject. If for instance on a scale with seven statements the respondent agrees with the fifth statement, it implies that he or she also agrees with the first four statements, but not with statement number six and seven.
To create a Guttman scale, you need to:
You can now use the scale to measure attitudes 'in the field': enter the list of statements and use them to interview people or present them in a document. For each item, they should express whether they agree or disagree. Each scale item has a scale value associated with it (obtained from the scalogram analysis).
To compute a respondent's scale score we simply sum the scale values of every item they agree with.
On the Scoring tab:
On the Targets tab:
If you interview a whole group of people at a time or you work with total numbers of different groups (classes) of something, you may want to work with this version of the Likert scale. It works exactly as the Likert scale, but you enter numbers instead of checking options.
On the Scoring tab:
On the Targets tab:
On the Targets tab:
Logframer offers the following tools to design and manage your monitoring system:
The Monitoring tool is a very powerful and interactive tool. It allows you to monitor your project for its entire duration, using the different types of indicators you've selected. Basically, you'll get a complete monitoring and evaluation system at one click of a button.
The Monitoring tool is a report that you can export to MS Excel. In the Import/export section of the File toolbar, click on the Create Monitoring Tool button.
You can select if you want to view all indicators or the indicators of a specific level (goals; purposes; outputs; activities). You can also select a specific target group, or the indicators for all target groups.
Note that it may take some time to export this report, especially if you have many indicators. When Logframer opens Excel, don't do anything in Excel (clicking in a cell etc.) because this will terminate the exporting process.
One final remark is that the different scales (Thurstone, Likert, Guttman) are only partially included, in the sense that you can only register the total scores. The actual tables with the statements and response scales are not included. The idea here is that you print the list of indicators, fill the forms out in the field and then you enter the results in this table.
Underneath the usual settings of the Print Settings bar, you can set different options to select what information you want to include in the questionnaire.
You can select:
The Performance Measurement Framework is tool used for Results Based Management. It is a table that presents an overview about:
The Performance Measurement Framework is a report that you can either print or export to MS Word or MS Excel.
You can select if you want to view all indicators or the indicators of a specific level (goals; purposes; outputs; activities).
The Verification Sources column is the third column of the logical framework. The verification sources describe where you (or an external evaluator) can find the information necessary to verify your indicators. For each indicator you can specify one or more verification sources. With multiple verification sources you can triangulate data: verify if the data is accurate by comparing different sources.
Click on a verification source in the logframe to get the following Verification Source Detail panel:
Here you can indicate how the data will be collected, and who will be responsible for the collection of this information.