The advantage of an open-ended question is that it allows people to express themselves freely, thus providing you with information that you might not have found otherwise. Open-ended questions are also interesting to get information about how a person feels about something. Often, the answers you get are more objective than direct or closed-ended questions.

Typically, open-ended questions start with 'Why...', 'How...', 'How do you feel about...' and so on.

The disadvantage of open-ended questions is that by definition, the replies are not standardised. This means that collecting, treating and analysing the information can be more complex, especially when you interview a lot of people.

In terms of the logical framework, open-ended questions are interesting to measure outcomes (expected or unexpected effects; wanted or unwanted effects) and impact (expected or unexpected; wanted or unwanted).

You can enter one or more questions in the 'Label' column of the table. In the 'White space' column you can set how much space the respondent or the interviewer has to note down the answer. This way you have some measure of control to the length of the answer. The amount of white space can vary from 1 to 7 lines, or from a quarter page to a full page.

In many cases, you will want to register hard data in the form of absolute values. Often, Objectively Verifiable Indicators are seen as synonymous to numbers, however this needn't be the case. Also take into account that numeric indicators may look good on paper, but may be very difficult to measure in practice. In many countries, statistics are notoriously hard to get by, and people are far less 'number-oriented' than in highly developed societies, where everything is measured and everything has got a price.

• In the 'Label' column, you can specify your question(s).
• The 'Decimals' column allows you to specify to what degree of exactitude the value has to be measured, i.e. the number of decimals behind the decimal point/comma.
• You can specify the unit of your values using the drop-down list of the Unit column. You can choose from a wide range of metric units, US customary units, time units and others. If you can't find the unit you need, simply type it into the field and it will be added to the list as a user-defined unit.

Multiple choice and multiple options questions allow you to identify a series of standard responses from which your respondent or interviewer/observer can choose from. With multiple choice, you can only select only one of the options. With multiple options, you can select more than one option at a time.

Some questions can be answered by a simple yes or no. This type of question is easy to use, and a series of yes/no questions can be used to measure more complicated indicators. For each question, you can indicate the correct answer, and you can also set a target in terms of what percentage of the population should get it right.

Add questions and indicate the preferential answer

Go to the first row and enter the first question in the 'Label' column. Then you can indicate what would be the correct or preferred answer from your respondents, by clicking in that column. A check mark will appear.

Set the target

You can set the target in terms of how many beneficiaries out of the total group (population) will achieve this target. Click on the  button, or double-click on the target cell in the table to open the following dialog window:

With ranking questions, you present the respondent with a number of options, and ask him or her to indicate which one is the most important, second most important, third most important (and so on).

When you use best/worst scaling or MaxDiff scaling (Maximum differential), you present the respondent with a number of options. Then you ask him or her to indicate which one in his/her opinion is the most important (or the best), and which on is the least important (or the worst).

Compared to other rating scales, best/worst scaling pushes people to make a clear-cut choice between a number of options. Instead of asking people to choose the best out of one pair and then move on to the next pair and the next until you've got a definite choice, this kind of scaling allows the respondents to choose directly from a whole set, which makes it easier to use and to take a decision. Another benefit from MaxDiff scaling is that there are no numbers involved, people just have to indicate the best and the worst choice in their opinions.

The method of equal-appearing intervals or the Thurstone scale is used to measure attitudes of people. Logframer allows you to use a Thurstone scale to measure an indicator, but developing such a scale is not so simple.

This kind of scale is used to measure people's attitude towards a fairly clear and unidimensional concept, using a number of statements that vary in how they express a positive or negative opinion about the main concept. We'll briefly explain the steps of developing a Thurstone scale:

1. Determine the focus: what concept are you going to measure (see what people's attitudes are toward it)?
2. Ask a group of people (or a person) to write down different statements about this concept, reflecting different opinions or attitudes about the subject. Make sure you have a large number of statements, making sure that people can either degree or disagree with them (no - open - questions for instance).
3. Rating the scale items: the next step is to have your group rate each statement on a 1-to-11 scale in terms of how much each statement indicates a favourable attitude towards the concept. The members of the group must not express their own opinion, they must only indicate how favourable they feel each statement is. You can use a scale with 1 = extremely favourable attitude towards the subject (focus) and 11 = extremely unfavourable attitude towards the subject.
4. Compute the median and interquartile range for each statement. Create a table with these values and sort by the median.
5. Select the items for the actual scale: you should select statements that are at equal intervals across the range of medians. Within each value, you should try to select the statement that has the smallest Interquartile Range. This is the statement with the least amount of variability across judges. You don't want the statistical analysis to be the only deciding factor here. Look over the candidate statements at each level and select the statement that makes the most sense. If you find that the best statistical choice is a confusing statement, select the next best choice.

For a detailed example, see http://www.socialresearchmethods.net/kb/scalthur.php

You can now use the scale to measure attitudes 'in the field': enter the list of statements and use them to interview people or present them in a document. For each item, they should express whether they agree or disagree. The total score of a person is calculated by making the sum of the values of all the statements they agreed with, divided by the number of items he agreed on (in other words, the average of the statements they agreed with).

The Likert scale can also be used to measure attitudes of people. When responding to a Likert questionnaire item, respondents specify their level of agreement or disagreement on a symmetric agree-disagree scale for a series of statements. Thus, the range captures the intensity of their feelings for a given item. As with the Thurstone scale, the development of a Likert scale takes some effort.

A Likert scale uses a number of Likert items, which are statements to which a respondent expresses his agreement or disagreement. Generally, a five-point or seven-point scale is used, for instance:

As you can see, each item is bipolar: it measures either someone's agreement or disagreement to the statement and allows you to give some measure of how much you agree or disagree. There are a variety possible response scales (1-to-7, 1-to-9, 0-to-4). All of these odd-numbered scales have a middle value is often labelled Neutral or Undecided. It is also possible to use a forced-choice response scale with an even number of responses and no middle neutral or undecided choice. In this situation, the respondent is forced to decide whether they lean more towards the agree or disagree end of the scale for each item.

The basic steps of developing a Likert scale are:

1. Determine the focus: what concept are you going to measure (see what people's attitudes are toward it)?
2. Ask a group of people (or a person) to write down different statements about this concept, reflecting different opinions or attitudes about the subject. Make sure you have a large number of statements, making sure that people can express their degree of agreement or disagreement on a five or seven-point scale.
3. Rating the scale items: the next step is to have your group rate each statement on your five-point (or seven-point, or 10-point) scale in terms of how much each statement indicates a favourable or unfavourable attitude towards the concept. The members of the group must not express their own opinion, they must only indicate how favourable or unfavourable they feel each statement is. All statements (Likert items) in the Likert scale must use the same number of points on the scale (so either a 5-point scale, or a 7-point scale, but not a mix of the two).
4. Select the items for the actual scale: The next step is to compute the intercorrelations between all pairs of items, based on the ratings of the judges. In making judgements about which items to retain for the final scale there are several analyses you can:
   
   1. Throw out any items that have a low correlation with the total (summed) score across all items
   2. For each item, get the average rating for the top quarter of judges and the bottom quarter. Then, do a t-test of the differences between the mean value for the item for the top and bottom quarter judges.

Correlation between the items and the total score: In most statistics packages it is relatively easy to compute this type of Item-Total correlation. First, you create a new variable which is the sum of all of the individual items for each respondent. Then, you include this variable in the correlation matrix computation (if you include it as the last variable in the list, the resulting Item-Total correlations will all be the last line of the correlation matrix and will be easy to spot). How low should the correlation be for you to throw out the item? There is no fixed rule here -- you might eliminate all items with a correlation with the total score less that .6, for example.

T-test: Higher t-values mean that there is a greater difference between the highest and lowest judges. In more practical terms, items with higher t-values are better discriminators, so you want to keep these items. In the end, you will have to use your judgement about which items are most sensibly retained. You want a relatively small number of items on your final scale (e.g., 10-15) and you want them to have high Item-Total correlations and high discrimination (e.g., high t-values).

For a detailed example, see http://www.socialresearchmethods.net/kb/scallik.php    

You can now use the scale to measure attitudes 'in the field': enter the list of statements and use them to interview people or present them in a document. For each item, they should express to what degree they agree or disagree. The final score for the respondent on the scale is the sum of their ratings for all of the items (this is why this is sometimes called a summative scale). On some scales, you will have items that are reversed in meaning from the overall direction of the scale. These are called reversal items. You will need to reverse the response value for each of these items before summing for the total. That is, if the respondent gave a 1, you make it a 5; if they gave a 2 you make it a 4; 3 = 3; 4 = 2; and, 5 = 1.

The cumulative scale or Guttman scale measures to what degree a person has a positive or negative attitude to something. It makes use of a series of statements that are growing or descending in how positive or negative a person is towards the subject. If for instance on a scale with seven statements the respondent agrees with the fifth statement, it implies that he or she also agrees with the first four statements, but not with statement number six and seven.

To create a Guttman scale, you need to:

1. Determine the focus: what concept are you going to measure (see what people's attitudes are toward it)?
2. Ask a group of people (or a person) to write down different statements about this concept, reflecting different opinions or attitudes about the subject. Make sure you have a large number of statements, making sure that people can either degree or disagree with them (no - open - questions for instance).
3. Rating the scale items: the next step is to have your judges rate each statement, indicating whether the statement expresses a positive (favourable) or negative (unfavourable) attitude towards the concept. The members of the group must not express their own opinion about the concept, they must only indicate whether the statement is favourable or unfavourable.
4. Developing the scale: construct a matrix or table that shows the responses of all the respondents on all of the items. Then sort this matrix so that respondents who agree with more statements are listed at the top and those agreeing with fewer are at the bottom. For respondents with the same number of agreements, sort the statements from left to right from those that most agreed to those that fewest agreed to.
5. If there are lots of items, you need to use a data analysis called scalogram analysis to determine the subsets of items from our pool that best approximate the cumulative property. Then review these items and select the final scale elements. There are several statistical techniques for examining the table to find a cumulative scale.
6. Because there is seldom a perfectly cumulative scale you usually have to test how good it is. These statistics also estimate a scale score value for each item. This scale score is used in the final calculation of a respondent's score.

For a detailed example, see http://www.socialresearchmethods.net/kb/scalgutt.php

You can now use the scale to measure attitudes 'in the field': enter the list of statements and use them to interview people or present them in a document. For each item, they should express whether they agree or disagree. Each scale item has a scale value associated with it (obtained from the scalogram analysis). To compute a respondent's scale score we simply sum the scale values of every item they agree with.

A semantic differential is useful to measure what the attitude is of the respondent towards something. In a semantic differential, you have two opposing characteristics  at either end of a scale (also called a bipolar pair). This scale in the middle typically has five or seven items. When the number of items on the scale is uneven, the middle one expresses a neutral point between the two extremes. But this is not necessarily so, you can also have a scale with an even number of items - sometimes up to ten items are used in the scale.