Knowledge Center Catalog

Local cover image
Local cover image

Can impact analysis be used for research evaluation?

By: Contributor(s): Material type: TextTextPublication details: Mexico, DF (Mexico) CIMMYT : 2003Description: p. 90ISBN:
  • 970-648-076-5
Subject(s): Summary: Many impact studies tend to relate changes in a particular impact indicator (e.g., output) with a measure of research investment. This requires the implicit assumption that the link between research outputs and impact indicators dominates all other relationships. However, this is true only for minor improvements along stable technological paradigms (like successive replacement of modern varieties). For most technologies, such as new crops or crop management, other factors, such as policies and markets, influence adoption and, consequently, impact. Allocation of impact ( or lack of) to research assumes that all other factors were unimportant in the generation and adoption processes.||Since many factors influence adoption, research impacts must be analyzed within a broader framework. Such a framework is provided by complexity theory, which posits that impacts depend on the technology's own evolution, on external forces (e.g., markets and regulations), and the direct and indirect interactions of networks of agents (e.g., researchers, input suppliers, policy makers and financial institutions). Some of these interactions are formal and some informal; some are planned while others are-random. These interactions also depend on the maturity and nature of the technology. Newer and/ or more complex technologies are more uncertain and, thus, require greater collaboration. In the development of simpler or better- known technologies, each agent is more aware of the role of other agents and of technology standards; thus, direct relationships are less important than market interactions. The problem is occasionally compounded because the impacts appear after many years and often cannot be measured.||The complexity framework has broad consequences for agricultural and research policies. Impacts result from the technology's own evolution, from changes in the networks that generate it and from the actions of competing and complementary networks. Since impacts ensue from the ways the whole network reacts to internal and external signals, they cannot be allocated to individual agents. In evaluating networks, the relevant parameters to study are the mechanisms that govern their reaction to new technological and market environments. These include rules for generating, collecting and sharing information, financing procedures, intellectual property right regulations and availability of human and financial resources. What can be evaluated for individual agents are their patterns of participation in particular networks and the factors that determine those patterns. These factors are (1) benefits and costs of participation, (2) criteria for promotions, (3) evaluation criteria, (4) financial arrangements, and (5) institutional cultures.||Since technology generation and adoption are random processes, any impact is, in part, the result of chance and could not have been be fully foreseen by any agent. Because of this feature, impact analysis should concentrate more on the analysis of development and adoption process and less on sophisticated rules to allocate the result to particular agents. Additionally, since research impacts cannot be predicted, an ex-ante estimation of impact is a poor instrument to allocate resources. Again, the emphasis should be put on the particular mechanisms that enable networks to be efficient: collective learning routines and the inter- and intra-institutional incentives to participate in innovation networks.
Tags from this library: No tags from this library for this title. Log in to add tags.
Star ratings
    Average rating: 0.0 (0 votes)
Holdings
Item type Current library Collection Call number Copy number Status Date due Barcode Item holds
Conference proceedings CIMMYT Knowledge Center: John Woolston Library CIMMYT Staff Publications Collection CIS-3806 (Browse shelf(Opens below)) 1 Available 632445
Total holds: 0

Abstract only

Many impact studies tend to relate changes in a particular impact indicator (e.g., output) with a measure of research investment. This requires the implicit assumption that the link between research outputs and impact indicators dominates all other relationships. However, this is true only for minor improvements along stable technological paradigms (like successive replacement of modern varieties). For most technologies, such as new crops or crop management, other factors, such as policies and markets, influence adoption and, consequently, impact. Allocation of impact ( or lack of) to research assumes that all other factors were unimportant in the generation and adoption processes.||Since many factors influence adoption, research impacts must be analyzed within a broader framework. Such a framework is provided by complexity theory, which posits that impacts depend on the technology's own evolution, on external forces (e.g., markets and regulations), and the direct and indirect interactions of networks of agents (e.g., researchers, input suppliers, policy makers and financial institutions). Some of these interactions are formal and some informal; some are planned while others are-random. These interactions also depend on the maturity and nature of the technology. Newer and/ or more complex technologies are more uncertain and, thus, require greater collaboration. In the development of simpler or better- known technologies, each agent is more aware of the role of other agents and of technology standards; thus, direct relationships are less important than market interactions. The problem is occasionally compounded because the impacts appear after many years and often cannot be measured.||The complexity framework has broad consequences for agricultural and research policies. Impacts result from the technology's own evolution, from changes in the networks that generate it and from the actions of competing and complementary networks. Since impacts ensue from the ways the whole network reacts to internal and external signals, they cannot be allocated to individual agents. In evaluating networks, the relevant parameters to study are the mechanisms that govern their reaction to new technological and market environments. These include rules for generating, collecting and sharing information, financing procedures, intellectual property right regulations and availability of human and financial resources. What can be evaluated for individual agents are their patterns of participation in particular networks and the factors that determine those patterns. These factors are (1) benefits and costs of participation, (2) criteria for promotions, (3) evaluation criteria, (4) financial arrangements, and (5) institutional cultures.||Since technology generation and adoption are random processes, any impact is, in part, the result of chance and could not have been be fully foreseen by any agent. Because of this feature, impact analysis should concentrate more on the analysis of development and adoption process and less on sophisticated rules to allocate the result to particular agents. Additionally, since research impacts cannot be predicted, an ex-ante estimation of impact is a poor instrument to allocate resources. Again, the emphasis should be put on the particular mechanisms that enable networks to be efficient: collective learning routines and the inter- and intra-institutional incentives to participate in innovation networks.

Socioeconomics Program

English

0309|R01CIMPU|AGRIS 0301|AL-Economics Program

Juan Carlos Mendieta

CIMMYT Staff Publications Collection

Click on an image to view it in the image viewer

Local cover image

International Maize and Wheat Improvement Center (CIMMYT) © Copyright 2021.
Carretera México-Veracruz. Km. 45, El Batán, Texcoco, México, C.P. 56237.
If you have any question, please contact us at
CIMMYT-Knowledge-Center@cgiar.org