Abstract
As the impact of strategic decision-making at the corporate, sectoral and national levels increase, there are growing demands for high quality and solid Foresight outputs. In this regard, a timely detection and elimination of problems in Foresight projects is of great importance. A thorough evaluation of criteria and methods used in Foresight analysis would permit the improved effectiveness of Foresight activities. The results could be set against the aims to decide on the feasibility of projects and identify ways to improve them. Despite great interest in Foresight evaluation demonstrated by stakeholders at various levels, the general principles for conducting it have not yet been formulated, which hinders its development and the diffusion of successful expertise.
The purpose of this paper is to identify the basic elements-criteria and methods—used in evaluating national Foresight. It presents the most interesting results for specific tasks, the themes examined, the methods and the findings, in general. In addition, the study allowed some inference about theory and practice of project management. The study reveals that a key motivation for evaluation of Foresight projects is to provide feedback to national Foresight organizations and identify areas for further development. Evaluation is used to guide Foresight (in varying degrees) and provide lessons learned for future projects (for example, simplifying implementation, involving business and social organizations, recognizing the need for better compliance with the methodology and objectives of the study participants).
The paper serves to develop a general methodology for assessing national Foresight programmes. A further agenda is to develop a comprehensive analysis of Foresight, ensure comparability of results of Foresight evaluation initiatives in different countries, promote standardization of evaluation procedures.
References
Aichholzer G. (2001) Delphi Austria: An Example of Tailoring Foresight to the Needs of a Small Country. Vienna: ITA.
Alsan A., Öner M.A. (2004) Comparison of National Foresight Studies by Integrated Foresight Management Model//Futures. Vol. 36. № 8. P. 889-902.
Amanatidou E., Guy K. (2008) Interpreting Foresight Process Impacts: Steps Towards the Development of a Framework Conceptualising the Dynamics of 'Foresight Systems'//Technological Forecasting and Social Change. Vol. 75. № 4. P. 539-557.
BMBF (2002) Futur Lead Visions Complete Document. Berlin: German Ministry for Education and Research.
Bennett C. (1975) Up the Hierarchy//Journal of Extension. Vol. 13. № 2. P. 7-12.
Bezold C. (2010) Lessons from Using Scenarios for Strategic Foresight//Technological Forecasting and Social Change. Vol. 77. № 9. P. 1513-1518.
Björn L., Lübeck L. (2003) Swedish Technology Foresight -A Successful Project With Many Lessons Learned. Paper presented at the Second International Conference on Technology Foresight, 27-28 February, Tokyo.
Bohanec M., Rajkovič V., Semolič B., Pogačnik A. (1995) Knowledge-Based Portfolio Analysis for Project Evaluation//Information & Management. Vol. 28. № 5. P. 293-302.
CROWN (2009) Multi-Criteria Analysis: A Manual. London: Department of Communities and Local Government. http://www.communities.gov.uk/documents/corporate/pdf/1132618.pdf (accessed 29 August 2012).
Calof J., Smith J.E. (2008) Critical Success Factors for Government Led Foresight. Paper presented at the Third International Seville Seminar on Future Oriented Technology Analysis: Impacts and Implications for Policy and Decision-Making, Seville, October 16-17.
Cuhls K. (2003) From Forecasting to Foresight Processes -New Participative Foresight Activities in Germany//Journal of Forecasting. Vol. 22. P. 93-111.
Cuhls K., Georghiou L. (2004) Evaluating a Participative Foresight Process: Futur -The German Research Dialogue//Research Evaluation. Vol. 13. № 3. P. 143-153.
Daim T., Basoglu N., Dursun O., Saritas O., Pisek G. (2009) A Comprehensive Review of Turkish Technology Foresight Project//Foresight. Vol. 11. № 1. P. 21-42.
Destatte P. (2007) Evaluation of Foresight: How to Take Long-Term Impacts into Consideration? Paper presented at FOR-LEARN Mutual Learning Workshop-Evaluation of Foresight, 19 September, Brussels.
Dursun O., Türe T.E., Daim T.U. (2011) Post-Evaluation of Foresight Studies: Turkish Case//International Journal of Foresight and Innovation Policy. Vol. 7. № 4. P. 311-337.
Eilat H., Golany B., Shtub A. (2008) R&D Project Evaluation: An Integrated DEA and Balanced Scorecard Approach//Omega International Journal of Management Science. Vol. 36. № 5. P. 895-912.
European Commission (1999) Evaluation of Socio-Economic Programmes. Glossary of 300 Concepts and Technical Terms. Luxembourg: Means Collection.
FOR-LEARN -On Line Foresight Guide. http://forlearn.jrc.ec.europa.eu/guide/7_cases/futur.htm (accessed 27 July 2012).
Georghiou L. (2003) Evaluating Foresight and Lessons for Its Future Impact. Paper presented at The Second International Conference on Technology Foresight, 27-28 February, Tokyo.
Georghiou L., Acheson H., Cassingena Harper J., Clar G., Klusacek K. (2004a) Evaluation of the Hungarian Technology Foresight Programme (TEP). Report of an International Panel. Manchester: PREST, University of Manchester.
Georghiou L., Cameron H., Keenan M., Miles I., Saritas O. (2006) An Evaluation of the UK Foresight Programme: Final Report. Manchester: PREST, University of Manchester.
Georghiou L., Clarysse B., Steurs G., Bilsen V., Larosse J. (2004b) Making the Difference -The Evaluation of Behavioural Additionality of R&D Subsidies. Brussels: IWT Vlaanderen, IWT Studies 48.
Georghiou L., Keenan M. (2006) Evaluation of National Foresight Activities: Assessing Rationale, Process and Impact//Technology Forecasting and Social Change. Vol. 73. № 7. P. 761-777.
Giesecke S. (2008) Futur -The German Research Dialogue//Giesecke S., Crehan P., Elkins S. (eds.) The European Foresight Monitoring Network. Collection of EFMN Briefs. Part 1. Luxembourg: Office for Official Publications of the European Communities.
Godinho P.C., Afonso A.R., Costa J.P. (2004) On the Use of Multiple Financial Methods in the Evaluation and Selection of Investment Projects//Investigação Operacional. Vol. 24. № 1. P. 1-20.
Guba E., Lincoln Y. (1989) Fourth Generation Evaluation. London: Sage Publications.
HM Treasury (2003) The Green Book: Appraisal and Evaluation in Central Government. London: TSO.
Habegger B. (2010) Strategic Foresight in Public Policy: Reviewing the Experiences of the UK, Singapore, and the Netherlands//Futures. Vol. 42. № 1. P. 49-58.
Havas A., Schartinger D., Weber A. (2010) The Impact of Foresight on Innovation Policy-Making: Recent Experiences and Future Perspectives//Research Evaluation. Vol. 19. № 2. P. 91-104.
Havas A., Schartinger D., Weber K.M. (2007) Experiences and Practices of Technology Foresight in the European Region. Paper presented at the UNIDO Technology Foresight Summit 2007, 27-29 September, Budapest.
JICA (2004) JICA Guideline for Project Evaluation (September 2004). Japan International Cooperation Agency. P. 141-145. http://www.jica.go.jp/english/operations/evaluation/tech_and_grant/guides/guideline.html (accessed 15 August 2012).
Kováts F. (2000) The Hungarian Technology Foresight Programme. Report by the Steering Group. Budapest.
Kuwahara T., Kameoka A., Yokoo Y. (2004) A challenge of integrating technology foresight and assessment in industrial strategy development and policymaking//Technological Forecasting and Social Change. Vol. 71. № 6. P. 579-598.
Li S.S., Kang M.H., Lee L.C. (2009) Developing the Evaluation Framework of Technology Foresight Program: Lessons Learned from European Countries. Paper presented at the Atlanta Conference on Science and Innovation Policy, Atlanta, October 2-3.
Medina J. (2006) Colombian Program on Technological and Industrial Foresight: Achievements and Perspectives. Bogotá: Colciencias.
Meissner D. (2012) Results and impact of national Foresight-studies//Futures (forthcoming).
Meissner D., Cervantes M. (2008) Results and Impact of National Foresight Studies. Paper presented at the Third International Seville Seminar on Future Oriented Technology Analysis: Impacts and Implications for Policy and Decision-Making, Seville, October 16-17.
Miles I. (2002) Appraisal of Alternative Methods and Procedures for Producing Regional Foresight. Report prepared by CRIC for the European Commission's DG Research funded STRATA -ETAN Expert Group Action. Manchester.
Miles I. (2003) Ten Years of Foresight in the UK. Paper presented at NISTEP Second International Conference on Technology Foresight, Tokyo, February 27-28.
Mishra S. (2009) Engineering Economics and Costing. New Delhi: PHI Learning Private Limited.
Owen J., Lambert F. (1998) Evaluation and the Information Needs of Organizational Leaders//American Journal of Evaluation. Vol. 19. № 3. P. 355-365.
PMI (1996) A Guide to the Project Management Body of Knowledge. Newton Square, PA: Project Management Institute.
Phillips J.J., Bothell T.W., Snead G.L. (2002) The Project Management Scorecard: Measuring the Success of Project Management Solutions. Woburn: Butterworth-Heinemann.
Popper R. (2009) Mapping Foresight: Revealing How Europe and Other World Regions Navigate into the Future, EFMN. Luxembourg: Publications Office of the European Union, European Commission.
Popper R., Georghiou L., Keenan M., Miles I. (2010) Evaluating Foresight: Fully-Fledged Evaluation of Colombian Technology Foresight Programme. Santiago de Cali: Universidad del Valle.
Rader M. (2003) Hungary -Hungarian Foresight Programme (TEP) 1997/99. FISTERA report WP 1 -Review and Analysis of National Foresight D1.1. HU -Case Study.
Remer D.S., Nieto A.P. (1995) A Compendium and Comparison of 25 Project Evaluation Techniques. Part 1: Net Present Value and Rate of Return Methods//International Journal of Production Economics. Vol. 42. № 1. P. 79-96.
Remer D.S., Nieto A.P. (1995) A Compendium and Comparison of 25 Project Evaluation Techniques. Part 2: Ratio, Payback and Accounting Methods//International Journal of Production Economics. Vol. 42. № 2. P. 101-129.
Remer D.S., Stokdyk S.B., Driel M.V. (1993) Survey of Project Evaluation Techniques Currently Used in Industry//International Journal of Production Economics. Vol. 32. № 1. P. 103-115.
Ricker K.T., Brown L.C., Leeds C.F., Leeds R.P., Bonar Bouton E.K., Volgstadt C.E. (1998) Water Quality Project Evaluation: A Handbook for Objectives-Based Evaluation of Water Quality Projects. http://ohioline.osu.edu/b868 (accessed 13 July 2012).
Rollwagen I., Hofmann J., Schneider S. (2008) Improving the Business Impact of Foresight//Analysis & Strategic Management. Vol. 20. № 3. P. 337-349.
Saritas O. (2006) Systems Thinking for Foresight. PhD Thesis. The University of Manchester.
Saritas O., Taymaz E., Tumer T. (2007) Vision 2023: Turkey's National Technology Foresight Program: A Contextualist Analysis and Discussion//Technological Forecasting and Social Change. Vol. 74. P. 1374-1393.
Tyler R. (1942) General Statement on Evaluation//Journal of Educational Research. Vol. 35. № 7. P. 492-501.
Westat J.F. (2002) The 2002 User-Friendly Handbook for Project Evaluation. Arlington: The National Science Foundation. http://www.nsf.gov/pubs/2002/nsf02057/start.htm (accessed 15 August 2012).
Wood J.V. (2001) UK Foresight Programme -A Panel Chairman's View. http://www.nistep.go.jp/achiev/ftx/eng/mat077e/html/mat0774e.html (accessed 07 June 2012).
Zarinpoush F. (2006) Project Evaluation Guide for Nonprofit Organizations: Fundamental Methods and Steps for Conducting Project Evaluation. Ottawa: Imagine Canada.
Майлс Й. (2008) Разработка сценариев и дорожных карт для ключевых технологий: предупреждение наводнений и защита береговых территорий в Великобритании//Форсайт. Т. 2. № 4. С. 50-59.
Майсснер Д., Сервантес М. (2010) Успешный Форсайт: дизайн, подготовка, инструментарий//Форсайт. Т. 4. № 1. С. 74-81.
This work is licensed under a Creative Commons Attribution 4.0 International License.