A Logical Framework for Monitoring and Evaluation : A Pragmatic Approach to M & E Professor

The following article discusses theoretical approaches to Monitoring and Evaluation that over time come to be streamlined and modified for implementation in the real world. Logical Frameworks (logframes) and pragmatic approaches to Monitoring and Evaluation (M&E) frequently facilitate the need to modify processes that can be used in instances where there are limited resources, limited financial capital and limited human capital to determine whether programmes and projects have had an impact. The practice of M&E often moves from detailed log frames being used for measuring impact, to the implementation of orderly pragmatic approaches and finally resorting to utilizing simple fundamental principles of M&E that entail identifying measureable objectives, performance measurement indicators, targets and periodic reporting. The United Nations Environment Programme (UNEP), for example, offers a detailed framework for insuring that intervention logic is measured in terms of verifiable indicators. The framework, however, requires the establishment of a means of verification to determine whether activities are being accomplished and outputs are being achieved. An alternative to such a detailed approach is to assess the risk of projects and programmes, and then report on the likelihood or unlikelihood of objectives being achieved. In the case of limited resources, the organization should at the least report on measureable objectives in terms of performance indicators. This paper concludes by emphasizing that no matter what approach is used in monitoring and evaluation, establishing targets is necessary in order to conduct meaningful Monitoring and Evaluation for impact.


Introduction
There are two approaches to conducting daily operations in the real world of management, monitoring and evaluation.There is the logical approach where frameworks are used for formal systematic analysis (Norad, 1999).Then there is the pragmatic approach (JSB, 2007) that while being inherently orderly and structured, allows for greater flexibility and immediate reactions to achieve outputs.Having those characteristics, action [oriented] research lends itself to pragmatism, in contrast to the application of a logical framework where clearly defined indicators and outputs contribute to a rigid time consuming structured logframe. 1As it relates to implementation, practitioners can be researchers whose portfolio includes responsibility for achieving outputs.In other words, a portion of their responsibility is to evaluate, critique and make recommendations designed to improve operational efficiency.However, due to time constraints they will be inclined to take a pragmatic approach based on action research conducted.Alternatively, a responsible manager, also a practitioner, will be concerned with long-term outcomes that supersede short-term [immediate] outputs.The use of a LogFrame and a logical framework approach suits the manager who, by the nature of their job description, is allowed the luxury of time to achieve outcomes.In the realm of education, for example, a researcher would be concerned with how to increase the number of successful graduates -i.e., maximizing throughput.In contrast, the department manager would be concerned with post university careers of students, such that they are gainfully employed and absorbed into the job market to become contributors to the tax base, thus being less dependent on the social services network -i.e., long term outcomes to be realized many years after graduation.Monitoring and Evaluation (M&E), therefore, can be accomplished with a view towards and emphasis on the short-term or the long-term -concern for immediate outputs or concern for future outcomes.Moreover, the practitioner, researcher or responsible manager in the real world is faced with immediate performance goals and may not have time to implement a prescribed LogFrame.They may then opt for a pragmatic approach to monitoring and evaluation.
This article discusses the merits of two approaches to monitoring and evaluation.In principle, managers are compelled to develop frameworks [logframes] that should be used throughout the organization to evaluate projects and programmes to determine to what extent goals, objectives, outputs and outcomes are being achieved -this being a logical approach to M&E.In practice, managers opt and resort to shortened forms of an imposed logframe, finding it necessary to take shortcuts and a more pragmatic approach to M&E.The article further discusses the implementation of a principled framework, followed by a discussion on what is actually put into practice in the real world.Resorting to pragmatism, therefore, is a coping mechanism that is used when time and resources do not allow for the implementation of detailed logframes.What then follows is: 1) using a logframe for planning, management and M&E; 2) using a LogFrame under duress or by design; and 3) moving from the logical, to the pragmatic and eventually to what is practiced in the real world.The conclusion will be drawn that no matter what approach is used, the identification of a target to be achieved is crucial to implementing any system of monitoring and evaluation.

A Logframe for Planning and M&E
As with any framework, a logical framework approach is a tool for designing, monitoring and evaluating projects.It, essentially, entails goal and objective oriented planning for project implementation.The written planning or internal policy document is essentially the logframe.The two terms (logframe and logical framework approach) are at times used interchangeably in referring to the underlying processes for M&E.Nevertheless, the logical framework approach (LFA) itself includes the M&E processes and again the logframe is the actual policy document.The discussion here is of those processes included and discussed in a planning document.
One thing to keep in mind about an LFA is that it can vary in look and structure, depending on the implementing organisation.For example, an LFA for United Nations projects will differ from an LFA for, say, Nordic Development Projects (Norad, 1999).While the example here is that of the United Nations Environment Program (UNEP), all LFAs will have common components, one way or another, consisting of objectives that can be measured, indicators and targets subject to periodic reporting requirements.In fact, a pragmatic approach is sublimated by those basic components due to time constraints and limited resources.This remains to be discussed.In the mean time, the UNEP (2005:92) LFA is a matrix that has three (3) main features: firstly, it describes how a programme or project is meant to work on a practical basis; secondly, the LFA matrix identifies the components required to assure that the stated results and objectives are achieved; and thirdly, the matrix lists specific indicators to be used to measure actual performance.The following table exemplifies the elements of the UNEP LFA matrix: Note above the elements of a Logical Framework.Those elements include: 1) the intervention logic that, as an objective, remains to be achieved, verified and measured; 2) results that are comprised of the intervention logic, performance indicators that serve to objectively verify that the objective has been achieved, by way of a specific means of verification that addresses the need for a Framework for Managing Programme Performance Information (National Treasury,  2007:5); and a statement of assumptions that qualify the achievement of the intervention logic; 3) outputs; and 4) activities.Notably, the notion of "logic" in a logical framework and the logframe for that matter implies that there should be a sequential and an understandable flow from results through to the relative subcomponents, leading to desired outputs that can only be achieved by performing activities that have some relationship or simply something to do with the desired results and output.In other words, there must be a relationship and alignment amongst and between the various elements indicated above.For example, national safety and security strategy policies (SSSPs) will emanate from national ministries and those ministries will be the environment within which the surveys will be conducted.A misalignment would occur if the means of verification (conducted surveys) for integration and coherence were to be administered in local communities or industrial sectors, considering that the environment under review includes national departments and the policies therein.The survey would be optimized by being administered in the departments themselves because the objective (intervention logic) is that of improved inter-departmental cooperation.Logic reflects order and relevance as there is movement from the objective to the indicators; activities must have "something to do with" the stated objectives as well.Alignment is an important integral requirement in a logical framework approach for Monitoring and Evaluation.This, therefore, requires planning and forethought with the aim of optimising project and programme implementation.

A Pragmatic Approach and Planning for M&E
The basis of this approach to monitoring and evaluation is of course the pragmatic philosophical school of thought that professes linking practice and theory.Indeed, in this instance theory arises out of practice or what should be done in the real world.This arises from the notion of intelligent practice as theory can be clearly traced back to daily practice. 3Pierce (1898) referred to "normative recommendations," implying what should be done to achieve clarity and comprehension.Applying a normative approach to M&E, the Judicial Studies Board's (JSB) approach is to collect information and use data collected to measure effectiveness and efficiency.Circumventing the need to formally evaluate each and every JSB objective, a pragmatic approach to M&E would entail measuring risk by considering two factors: • Factor 1: The impact of the measureable objective under consideration.
• Factor 2: The likelihood that objective will have an impact.Note that impact and likelihood would be determined by an analysis of risk and then indicating the results of an analysis on a grid marking or indicating in one of nine quadrants to reflect low, medium or high risk.Simply, an analysis of risk would be done and the results indicated on the following grid in terms of two parameters -impact and likelihood: 3. Of course the danger here is that the practice itself maybe flawed.The subsequent theory can therefore be expected to be flawed and not sound.An assumption then is that the theorist will only proceed to formulate a theory based on a tried and true practice proven to have some merit.

426
The grid aims to readily illustrate and reflect the outcome of the risk analysis. 4An approach to risk analysis that might be used is a decision tree that includes decision points, alternatives, chance points, states of nature and payoffs.The results of the analysis would then be used to reflect the likelihood and impact of the payoff.After evaluating, summarizing and illustrating risk in this manner, the next step would be to determine the resources required to facilitate the achievement of the objective.
Essentially, a pragmatic approach to M&E could first involve evaluating risk, say using some decision making framework and then modeling the results of the analysis by way of a two parameter grid indicating low, medium or high outcomes on an X and Y axis.While practitioners will find it beneficial to measure risk as part of the M&E process, limited time and resources may cause them to further simplify the monitoring process.Practitioners will then be forced to consider the implementation of M&E programmes and processes either by design or by duress.

Implementation by Design or by Durress
Gasper (2000) commented that while the uses of logical frameworks have spread enormously, the approach itself has had very little systematic evaluation.It, therefore, is being suggested that the efficacy of organized, pragmatic and logical approaches to monitoring, evaluating and reporting is questionable.Indeed, much effort is put into writing internal M&E frameworks and implementation plans only to find the problems and challenges still remain -even after extensive monitoring and reporting takes place.It is not unusual for the documents mentioned (frameworks and implementation plans) being written are not fully implemented or accepted for use by directorates and departments.In other words, they are planning documents written without buy-in or commitment to meet the objectives of monitoring and evaluation -to monitor projects and programmes and evaluate predetermined measureable objectives and targets.Monitoring and evaluation under these conditions will then be implemented by duress.In isolation, an M&E specialist draws up the plans, disseminates information to departmental managers and provides training on principles of M&E.There is an expectation that a decentralised approach will insure that M&E is conducted throughout the organization.In reality, M&E takes a back seat to other departmental activities.Report writing on projects (progress reports) is construed to be monitoring and evaluation.However, narratives on programmes and projects without evaluation in terms of measurable objectives and targets should not be considered to be monitoring and evaluation.Monitoring and evaluation becomes so pragmatic, so bureaucratic that senior managers do all they can to avoid not conducting M&E in a systematic manner.Moreover, logframes where risk analysis was first conducted and where an analytical tool was used to monitor performance are so intricate and involved that, again, departmental managers do all they can to avoid complying with any requirements to engage in structured systematic monitoring and evaluation.And why should departmental managers comply when, as noted above: The approach itself has had little systematic evaluation.
Arguably, the most important component of any approach to M&E is the involvement and support shown by the Chief Executive Officer (CEO).Importantly, an internal M&E programme must be driven from the top, with delegations from the CEO.The CEO must be seen to be part and parcel of any internal logical framework decided upon.It [M&E]  would even be most effective by way of a separate directorate or department unto itself, reporting to the CEO and empowered as an operational department.This will be a requirement for any logical framework, pragmatic or otherwise in order to be implemented by design.Unfortunately, M&E starts out to be implemented by a logframe designed in isolation, only to move towards a less pragmatic real world approach.

Logframe to Pragmatism in the Real World
Surprisingly, a less pragmatic approach includes consideration of assumptions for the subsequent implementation of interventions for successful achievement of measureable objectives.The process is relatively organized and structured, giving the appearance of being pragmatic.A pragmatic approach to M&E will involve risk analysis where the findings and outcomes are illustrated by way of a grid or matrix to summarise low to high potential for achieving objectives.At best, these approaches can be time consuming, requiring capital and human resources for programme implementation.This entails M&E under the most ideal scenario, with support from the CEO and the executive management team.Monitoring and evaluation will then be raised to the highest level of importance.
Monitoring and evaluation can, however, take a back seat to seemingly more important organizationsal functions.

MCSER Publishing, Rome-Italy
Vol 4 No 14 November 2013 427 Total responsibility and operationalisation usually rests with one mid-level manager -an M&E Specialist -who must rely on a decentralized approach, with others being tasked to implement an organizational [policy] M&E framework written in isolation.Under such conditions, processes might be necessarily streamlined, with M&E being implemented by way of the most basic components -namely, recognition of measureable objectives, performance indicators, stated targets and a formal system for reporting.Note that the common component or factor that exists whether there is a pragmatic approach bounded by a logframe -that is, the notion of a target.Indeed, it is that thing to be achieved and reported on in a responsible manner.Thus no entity can be said to be engaging in M&E in the absence of a stated target for an individual programme or project objective.Table 3. illustrates a basic straight forward reporting tool that when wholeheartedly utilized meets the essential requirement for monitoring and evaluation, and subsequent reporting.The simplicity of monitoring and evaluation to evaluate achievement often escapes organizational managers.Indeed, this is M&E as it should be in the real world.The question raised is whether pragmatic approaches and the obsession with logical frameworks dissuade the use and implementation of the most basic components of M&E.This may be the case, as formal policy documents and frameworks are drawn up, only to be shelved to accumulate dust.In the absence of CEO commitment, organizational buy-in and adherence to targets, monitoring will remain a fleeting thought in organizations seeking to achieve organizational efficiency and maximize service delivery.5

Conclusion
This paper has discussed how the application and process of monitoring and evaluation progresses from a logical [framework] approach, to being altered to a more flexible, orderly pragmatic process and finally to a point where in the real world it becomes necessary to streamline processes due to a limitation of resources.Evaluations and assessments move from being rigid through to becoming flexible enough to enable managers to monitor, evaluate and report.While there are indeed varying logical frameworks (logframes) that may be referred to, one such as the (UNEP, 2005) illustrates the complexity and challenges in implementation.Time, human resources and management buy-in are required to optimally determine if project and programme objectives are being met.Indeed, this is M&E being implemented in the most perfect circumstance -ideal circumstances that often do not represent what occurs in the real world.With wholehearted intentions of implementing rigid practices for M&E, it may become necessary to make modifications by focusing on risk assessment and evaluation as an alternative to a rigid logframe.The Judiciary Services Board's approach is to assess the risk associated with programme objectives and then quantify and map the risk on a grid indicating likelihood on an X axis and impact on a Y axis.This is nothing less than a pragmatic approach to M&E.In the real world, however, practitioners will be limited by constraints that will preclude their continued used of either a logframe or some overly pragmatic approach to M&E.At the least the basic principles for M&E (measureable objective, performance indicator, target and periodic reporting) can be used in a reporting tool.The advantage here is simplicity and efficiency in data collection, recording and reporting.Implementation, as in all cases, is subject to buy-in and support from the top down.Optimal monitoring and evaluation will be assured when driven either as an operational department or out of the office of the chief executive.

Table 1 .
UNEP Logical Framework Example. 2ject Description: Capacity building to strengthen policy for safety and security.Strategic Goal: Improved policy coherence in the context of national development through Safety and Security Strategy Policies (SSSPs) and implementation of multilateral environmental agreements (MEAs).

Table 2 .
M&E Grid to Summarize Risk Analysis