The Future of Employment: Evaluating the Impact of STI Foresight Exercises

The present paper addresses the impact of Science, Technology and Innovation (STI) Foresight and the resulting STI policy thereof on the critical issue of future industry studies, the demand for employment. The impact evaluation connecting STI Foresight to employment development proposed here brings together and integrates different scientific sectors such as the interdependence between employment and welfare systems; the role and importance of technological change for employment conditions; the prospects of emerging and future technologies impacting employment in future industry; the contribution of science, technology and innovation (STI) policies to promoting the generation and real application of new technologies. The paper follows the premise that STI Foresight, as well as Future-oriented Technology Analysis (FTA), offer a robust basis for the elaboration and monitoring of STI policies with anticipatory intelligence.

T he present paper addresses the impact of Science, Technology and Innovation (STI) Foresight and the resulting STI policy thereof on the critical issue of future industry studies, the demand for employment.
The impact evaluation connecting STI Foresight to employment development proposed here brings together and integrates different scientific sectors such as the interdependence between employment and welfare systems; the role and importance of technological change for employment conditions; the prospects of emerging and future technologies impacting employment in future industry; the contribution of science, technology and innovation (STI) policies to promoting the generation and real application of new technologies.
The paper follows the premise that STI Foresight, as well as Future-oriented Technology Analysis (FTA), offer a robust basis for the elaboration and monitoring of STI policies with anticipatory intelligence. Keywords: STI policy; foresight; new technologies; impact assessment; evaluation; employment; technology change; future industry; indicators; computerization The core of the paper is dedicated to address the main question of how to identify and choose variables and indicators able to reflect vectors towards the future of employment. The selected vectors are referred to cross effects, trends, and time scales. As far the relationship of technology and employment is concerned, the paper examines cross effect impacts resulting from an inputoutput analysis, trends indicated in the FTA studies and time scaling of the technology lifecycle. These parameters are meant to constitute the basic elements for impact evaluation algorithms. In this connection, the paper proposes concepts, measurement techniques, and methods for the evaluation of foresight exercises influencing future changes on employment.
Linking policy-making, Foresight, and specific futurelooking themes, the paper offers building blocks for constructing standards for the evaluation of foresight exercises.
Citation: Seidl da Fonseca R. (2017) The Future of Employment: Evaluating the Impact of STI Foresight Exercises. Foresight and STI Governance, vol. 11, no 4, pp. 9-22. DOI: 10.17323/1995-459X.2016 The Future of Employment: Evaluating the Impact of STI Foresight Exercises International Researcher and Advisor, Former Senior Professional Officer of United Nations Industrial Development Organization (UNIDO), r.seidl.fonseca@gmail.com, ricardo@seidldafonseca.at Labour Market in the Context of Technological Transformations T his paper is an output of the author's line of research dedicated to the theoretical and empirical conceptualization of evaluating the impact of foresight exercises. 1 It addresses the impact of Science, Technology and Innovation (STI) Foresight and the STI policy resulting thereof on a critical issue of future industry studies, the demand for employment. The importance of the consideration of the future of employment is illustrated by a key report presented at the 2017 World Economic Forum. The changing demand for employment and was discussed there as one major challenge faced as part of the Fourth Industrial Revolution [WEF, 2016]. The impact evaluation exercise on STI Foresight and employment proposed here brings together and integrates different aspects of future employment. Starting with the interdependence between employment and wellfare systems, it highlights the risks of changes in spending on salaries for the sake of converging with and sustaining social welfare models, as was examined by Zeitlin and Trubek [Zeitlin, Trubek, 2003]. Further, it considers the contribution of technological change to transformations in the structure of employment, citing Keynes [Keynes, 1936] and Aghion and Howitt [Aghion, Howitt, 1994] 2 . The present exercise considers the real prospects of emerging and future technologies impacting employment in the industry of the future as was discussed by Frey and Osborne [Frey, Osborne, 2015]. Another important background feature here is the contribution of STI policies in promoting the creation and real application of new technologies. 3 The concrete results of STI policies in developing countries have been researched and monitored since the United Nations Conference on Science and Technology for Development, in Vienna, Austria in August 1979 [UN, 1979]. The Organisation for Economic Co-operation and Development (OECD) dedicated a breakthrough conference in May 1997 to the exploration of the concepts, theoretical and empirical approaches to the alignment of STI and industry, as well as their mutual development (see report at [OECD, 1997]). These aforementioned reports are basic sources of reference for tackling the subject of the present article. Furthermore, the paper adheres to the premise that STI Foresight, as well as Future-oriented Technology Analysis (FTA), offer a robust foundation for the elaboration and monitoring of STI policies with anticipatory intelligence. The contribution of such forward-thinking perspectives to strategies and policy making have been emphasized by numerous works, which are summarized by Havas, Schartinger and Weber [Нavas et al., 2010]. Indeed, with a systemic view, the impact evaluation exercise can be considered essential for a successful STI Foresight exercise, which shall be embedded in the formulated STI policies from the start. This view is discussed by Seidl da Fonseca and Saritas [Seidl da Fonseca, Saritas, 2005]. Against the aforementioned background, the main purpose of the paper is to address the critical question of how to identify and choose variables and indicators capable of illustrating the trends that determine the future of employment. Given the connection between technological development and employment, this article analyses the cross-effect impacts using an input-output analysis, development trends for future technologies, and also the time scales for technologies' lifecycles. These parameters are meant to constitute the basic elements for impact evaluation algorithms. Following the basic concept that impact evaluation variables and indicators should be built into the design of foresight studies related to development policy frameworks, the present paper closes by proposing concepts, measurement techniques, and methods for the evaluation of foresight exercises and their impact upon future changes in employment. By linking policy-making, foresight, and the selected topic of strategic research, this paper offers building blocks for the construction of a set of standards for the evaluation of foresight exercises.

Methodology
As a general approach, the structure of an impact evaluation of foresight exercises begins with the examination of FTA studies, anticipatory simulation studies, and forecasting econometric analysis to identify variables and indicators characterizing a particular sector, topic or issue. Following our conceptualization of the impact evaluation of STI foresight exercises [Seidl da Fonseca, 2016], the selected variables and indicators can be used again in the formulation of the foresight project. This will permit a post-foresight evaluation of the outcomes of the policies and strategies implemented using the findings and recommendations found in the foresight exercises. Feedback from the impact evaluation shall be used to improve further STI foresight projects. Figure 1 below shows the flow of activities of such an approach.

Why is Employment Critical for the Future of Industry?
The last twenty years testify to a dramatic change in the conditions of employment, the demand for and supply of skills, as well as the balance between the two. A large number of jobs and career trajectories connected with certain types of activity have become a thing of the past or have been discontinued or 1 The first paper of this series was debated at the HSE Annual Conference on Foresight and STI Policy, Moscow, October 30-31, 2013 andpublished by [Seidl da Fonseca, 2016]. A draft of the present paper was presented at the HSE Annual Conference on Foresight and STI Policy, Moscow, November 6-7, 2014. 2 For a summary of the evolving concepts on this feature see [Calvino, Virgillito, 2016]. 3 For a summary review and main references on this feature see [Martin, 2010]. curtailed dramatically. Often, they move from one region to another, while completely new jobs and carriers have emerged.

STI
In that period, what constituted the pillars of the modern socioeconomic framework, where salaries and taxes of the formally employed population gave rise to a complex and interlinked economic formula combining consumption, credits, savings, and public goods as well as social services and basic infrastructure, such as education, health, mobility and shelter, all has came under risk. Indeed, the growing gap in the welfare system's finances and missing public funds for social facilities have accompanied rising unemployment rates in almost all countries. Aging populations only aggravate this problem. Supporting the intricate social welfare network, including pension funds and benefits, protection for the unemployed, retraining and reallocation of labor resources, has become unstoppable process, which generates deficits for public sector accounts and places excessive burdens on the private sector. Therefore, the stability and cohesiveness of modern society will greatly depend upon how employment behaves in the future. Many countries have been trying different strategies and policies, on the one hand to consider and counteract the negative effects of the changing employment structures and markets, and on the other, to support the positive trends toward job creation and expansion. Moreover, it is however recognized that both the future evolution of employment conditions and the corresponding strategies are bound to long-term and complex events deriving from the very essence of production processes and related technological changes. In this connection, FTA studies, anticipating simulation studies, as well as STI foresight exercises have been carried out and relevant policies formulated and implemented. The focus of the present paper is on how to form a body of variables and indicators for evaluating the impact of STI foresight exercises, through their generated policies and strategies, upon employment.

Concepts for Measuring Impact of Technology on Employment
Estimating the future of employment conditions in a broader sense must consider a number of factors affecting the future supply of and demand for jobs and skills. According to experts [Mahrum et al., 2005], the factors exerting a major influence upon employment markets are the following: • the changing size of the workforce • the international outsourcing of skills • an aging workforce • technological changes • the continuous development of the workforce. In this paper, the pivotal question on the future of employment is to anticipate how technological progress will impact labor market outcomes in the future. That means focusing on the future relationship between technological progress and employment. This nevertheless implies anticipating the future balance of power between state, society and economic forces influencing decisions regarding technological progress and job conservation or destruction. Since the industrial revolution, such issues have been recurrently examined and considered by the developers of economic policy, for example, recall the concept launched by Keynes on technological unemployment [Keynes, 1936]. The information technology (IT) age has opened a new chapter in the discussion on the impact of technologies upon employment, where myriads of new technologies have been used in the production of goods and services, either facilitating and creating new jobs or, on the contrary, substituting human labor. Such novelties could be designated as "new technologies affecting employment" (NTAE). The emerging new industrial revolution, known as Industry 4.0, points to the more widespread or even full automation or robotization of the production lines, with increasingly less direct participation from labor. Moreover, with the dramatic technological advances experienced by now, and even more challenges foreseen in the future, studies and methodologies focused on the future of employment are in high demand. 4 When considered in the narrow view, technological progress has two competing effects on employment [Aghion, Howitt, 1994]. First, as far as new technology substitutes labor, there is a destructive effect, which requires companies to reduce, close, or reallocate their labor supply; and second, there is a capitalization effect: as more companies expand or join sectors where productivity is relatively high, this leads to an increase in employment in those sectors. Another aspect to consider is the emerging demand for skills and jobs necessary both for developing new technologies and their practical application. The introduction of new technologies could increase the productivity of directly or indirectly related occupations, for example, engineers or architects using automated systems for design and calculation activities. Looking at all these aspects, measuring the impact of technology upon the labor force must take into consideration the different operational levels or planes, from micro-to macroeconomic ones.

Assessing Cross Effect Impacts upon Employment
Considering the conceptualization and application of strategies for stimulus programs aimed at improving the efficacy of the economic and industrial structure in a given country, the evaluation of the relative impact of such strategies could be estimated using an input-output analysis, in which specific indicators address changes in the structure and volume of employment. Two methods for measuring cross effect impacts are discussed below.

Methods for Measuring the Impulse Intensity of Intersectoral Transmission of Increased Labor Productivity 5
This impact criterion captures the impulse that an increase (or decrease) of labor productivity in one given sector is transmitted to other sectors. In industry, such an impulse has an effect on the man-machine system and if such changes results in changing the value added, physical output, or labor efforts, there is a shift in the production function. 6 Technological progress inevitably affects labor productivity. This change in technology results in shifts in work qualifications (skills) and/or work conditions (work methods and production equipment). One can measure the efficiency of labor's contribution to the production process using an indicator for labor productivity. Labor productivity (pt) shows the relationship between the physical results of the production process or business and related work efforts. The mathematical equation is as follows: where: production results are value added (e) or the physical output (v); work efforts involve the number of workers (l) or the working hours (sl). Accordingly, the labor productivity increases if the added ivalue ncreases or the work effort decreases.
To measure the impact upon employment of a technological change in a selected sector, two methods described below can be used.

a. Method of the Relative Employment Proportion of an Industrial Sector
The relative share of employment in a sector indicates the potential importance of this sector with respect to its ability to transmit increased labor productivity to the whole production structure. An indicator for this is the direct employment coefficient (b), which classifies the sectors as high, middle, and low labor intensive. The equation is the following: sl (or l) Table 1 below presents the calculation of the direct employment coefficient and relative grouping and ranking 7 for a set of industries in the USA (2015). By identifying and ranking the existing sectors in an economy, it is possible in the planning process, both during the implementation and during the evaluation of a stimulus program to examine or use the sectors with a higher potential of transmitting increased 4 An example of this can be found in the study conducted by the UK Commission for Employment and Skills "Future of Work: Jobs and Skills in 2030" [CES, 2014]. This study is dedicated to forecasting the future of labor markets, where according to different scenarios about future social, demographic, political, or economic perspectives, one can derive the possible or expected changes in the structure of employment and the job market, and indicate strategic actions. 5 Elaborated upon in [Seidl da Fonseca, 1981]. 6 The production function is a mathematical equation or graph that shows the relationship between the inputs and outputs of a production process or business. Inputs and outputs can be expressed in physical or monetary terms. 7 Ranking and grouping are done for the year 2004.

Таble 1. Grouping and Ranking of Industries by Direct Employment Coefficient
(or decreased) labor productivity to related sectors. These areas of high potential can be considered the drivers of development, which are of special interest to foresight studies.
Organizing the data provided in the Table 1 in a way to give indications on how the analyzed industrial sectors would behave in relation to their ability to provide direct employment, Tables 2 and 3 below present information on how those sectors change their capacity to create jobs over the years and their ranking of importance as employment development drivers. Foresight exercises may consider trends, derived from future scenarios, for the direct employment impact in selected industrial sectors when changes that might influence employment, such as productivity or technological progress, are expected in the future.

b. Method for Evaluating the Total Employment Efficiency
One basic research question that considers the impact of changes on the economy concerns how they affect employment opportunities and structure. For foresight exercises, future economic trends should be compared with the prospects for job creation or destruction. The identification of sectors more sensitive to such an influence and an analysis of the consequences of such changes may serve as an impetus for a more grounded and focused study of future trends in the relevant areas.
Labour Market in the Context of Technological Transformations

Таble 2. Capacity to Create Jobs by Industry and Year
In this connection, with the method of total employment efficiency, one can estimate the effect of a change in the production capacity of a selected sector upon the number of employees in other sectors. Employment efficiency helps one uncover the ability of a sector to transmit increased (or decreased) labor productivity to other related industries. With the application of the concepts and data processing of the Input-Output Matrix 8 , the employment efficiency of a sector can be estimated through the Leontief inverse [Leontief, 1986]. According to Bacon and Kojima [Bacon, Kojima, 2011] 9 , the use of Input-Output tables may answer two basic, relevant questions: 8 Defined by Bacon and Kojima [Bacon, Kojima, 2011]: "The standard IO model of an economy links the gross output of a sector to the final demand for that sector and to the intermediate demands made by other sectors for its output. This can be expressed as X = A X + F, where X is a vector of gross outputs of the N sectors of the economy, F is a vector of final demands for these sectors, and A is the N × N matrix of technical coefficients that indicate how much output from sector i is directly required to produce one unit in sector j. The gross output is then related to final demand by the equation X = (I -A) -1 F = CF, where the coefficient matrix C (Leontief inverse) measures the total amount of sector i that is required to be produced in order to satisfy the direct and indirect demands produced by a one-unit increase in the final demand for sector j. " 9 The study done by Bacon and Kojima [Bacon, Kojima, 2011] is an example of a work dedicated to the impact of a broad or pervasive technological field, where alternatives for its development would result in the expansion or reduction of job capacities in one or more areas. This paper was prepared for the World Bank and estimates the impact of labor upon the energy sector. Furniture and related product manufacturing -1 Clay product and refractory manufacturing -1 Furniture and related product manufacturing 5.
Ship and boat building -3 Metalworking machinery manufacturing -1 Clay product and refractory manufacturing 6.
Wood product manufacturing 9 Water, sewage and other systems 1 Wood product manufacturing 7.
Textile mills and textile product mills -1 Wood product manufacturing 1 Bakeries and tortilla manufacturing 8.
Construction 4 Bakeries and tortilla manufacturing 2 Ship and boat building 9.
Cutlery and handtool manufacturing -2 Textile mills and textile product mills → 0 Textile mills and textile product mills 10.
Computer and peripheral equipment manufacturing -5 Ship and boat building -9 Apparel, leather, and allied manufacturing 11. Electric

Таble 3. Ranking of Sectors by Importance as Drivers of Employment Development
Labour Market in the Context of Technological Transformations 1. How many jobs are currently supported by a given sector? 2. If demand for the output of the sector changes by a certain amount, how many extra jobs would be created? In addition, the total change in the induced employment at indirectly and directly affected sectors is considered, which can be influenced if the sector under consideration experiences an increase (or decrease) in its final demand. For an inter-sectoral comparison, the average column sum over the total average of all elements of the employment inverse can be used: A value p L .j > 1 indicates that the relevant sector j transmits an above average intensive impulse on employment in other industries. The total employment efficiency coefficient cl .j and the relative transmission intensity p L .j can be used for ranking the individual sectors. Table 4 below proposes a way of using the employment-related indicators presented above for ranking a group of selected sectors according to the strength of each sector expressed by cross impacts upon the employment capacity of all other sectors in the group. 11 The identification of systemic relationships between different sectors in the economy using this method is also very useful for designing foresight exercises, particularly for scenario building.

Measurement of Technological Change Trends
The indicators of trends can be identified by empirical research carried out in the framework of FTA. Understanding the critical question of employment in the future, FTA studies dedicated specifically to this issue have been conducted. These studies revealed not only the relevant parameters and drivers but also reliable indicators of variation in levels of employment, which could correspond to specific or generic technological changes. The present paper extracted some indicators from a FTA study conducted by Frey and Osborne [Frey, Osborne, 2013] upon the impact of a specific new technology, where its introduction would imply job creation or destruction.

Method for Measuring the Risk of job Destruction due to the Introduction of New Technologies
A constant consideration regarding technology progress is the possibility of new technologies being used in the production process, which in turn forces a reduction in jobs, a decline in workers' skills, substitution, displacement or disappearance. Frey and Osbornе [Frey, Osborne, 2013] developed a procedure for categorizing occupations according to their susceptibility to new technologies and therefore estimate the number of jobs at risk using the 10 The employment coefficient represents the absolute number of employees (l) per gross production value or capacity (v) of the individual sector: b = l/v. 11 Presented here simply for illustrative purposes, the numbers in the table are taken from the Input-Output Table of Brazil for a selected group of industries in 1970 [IBGE, 1974]. It is worth noting that the calculation and ranking of industrial sectors on the basis of the Leontief inverse [Leontief, 1986] depends on the group of selected sectors considered in each concrete study.
example of computerization. 12 The application of the method by the authors was used to estimate the impact of computerization on the US labor market. The exercise considers a set of defined occupations, using available occupation catalogues, which specify in detail the characteristics of each occupation, such as the US-based O*NET database. 13. On the basis of the descriptions of the occupations, the method examines how each one is susceptible to computerization and estimates their probability of automation. Thus, the degree of computerization of jobs and how widely this technological process spreads will depend upon the potential of the latest technologies to provide engineering solutions. Following this evaluation, the results will be applied to the study of the expected consequences for the labor market connected with the introduction a new technology, such as computerization. First, jobs at risk are analysed and identified and the relationship between an occupation's probability of automation by a new technology (computerization), wages, and educational advancement is considered. According to Frey and Osborne [Frey, Osborne, 2013], "from a technological capabilities point of view, we aim to determine which problems engineers need to solve for specific occupations to be automated. By highlighting these problems, their difficulty and to which occupations they relate, we categorise jobs according to their susceptibility to computerization. The characteristics of these problems were matched to different occupational characteristics, using O*NET data, allowing us to examine the future direction of technological change in terms of its impact on the occupational composition of the labour market, but also the number of jobs at risk should these technologies materialise. " In general terms, such a method could be used to estimate the impact of the introduction of new technologies upon occupations. Based on the description above, the method follows the steps:

Labour Market in the Context of Technological Transformations
In the case of the original method, according to the literature and expert opinions, the computerization bottlenecks are related to the performance of three critical tasks: perception and manipulation, creative intelligence, and social intelligence. Table 5 below presents the corresponding O*NET variables for each task category. For each occupation selected for analysis, the composition and importance of these variables in the occupation characteristics, would lead experts to determine that these labor inputs are not susceptible to computerization. According to these findings, non-susceptible labor inputs (L NS ) can be described as [Frey, Osborne, 2013, p. 24]: where L PM , L C and L SI are labor inputs into perception and manipulation tasks, creative intelligence tasks, and social intelligence tasks. The authors describe their estimation process as follows [Frey, Osborne, 2013, p. 30]: "We combine and build upon two approaches. First, together with a group of ML researchers, we subjectively hand-labelled 70 occupations, assigning 1 if automatable, and 0 if not. For our subjective assessments, we draw upon a workshop (probably in 2013) held at the Oxford University Engineering Sciences Department, examining the automatability (or probability of computerization) of a wide range of tasks. Our label assignments were based on eyeballing the O*NET tasks and job description of each occupation. This information is particular to each occupation, as opposed to standardised across different jobs. The hand-labelling of the occupations was made by answering the question "Can the tasks of this job be sufficiently specified, conditional on the availability of big data, to be performed by state of the art computer-controlled equipment". Thus, we only assigned a 1 to fully automatable occupations, where we considered all tasks to be automatable. To the best of our knowledge, we considered the possibility of task simplification, possibly allowing some currently non-automatable tasks to be automated. Labels were assigned only to the occupations about which we were most confident. Second, we use objective O*NET variables corresponding to the defined bottlenecks to computerisation. More specifically, we are interested in variables describing the level of perception and manipulation, creativity, and social intelligence required to perform it. As reported in Table I, we identified nine variables that describe these attributes. These variables were derived from the O*NET survey, where the respondents are given multiple scales, with "importance" and "level" as the predominant pair. We rely on the "level" rating, which corresponds to specific examples about the capabilities required of computer-controlled equipment to perform the tasks of an occupation. For instance, in relation to the attribute "Manual Dexterity", low (level) corresponds to "Screw a light bulb into a light socket"; medium (level) is exemplified by "Pack oranges in crates as quickly as possible"; high (level) is described as "Perform open-heart surgery with surgical instruments". This gives us an indication of the level of "Manual Dexterity" computer-controlled equipment would require to perform a specific occupation. " Other critical step is correlating employment/labor inputs versus the potential for technological change (computerization or computer capital), per occupation and per mix of jobs. Based on the Cobb-Douglas [Cobb, Douglas, 1928] production function, such a correlation can be formulated as:

Computerization Bottleneck O*NET Variable O*NET Description
Perception and Manipulation Finger Dexterity The ability to make precisely coordinated movements of the fingers of one or both hands to grasp, manipulate, or assemble very small objects.

Manual Dexterity
The ability to quickly move your hand, your hand together with your arm, or your two hands to grasp, manipulate, or assemble objects. Cramped Work Space, Awkward Positions How often does this job require working in cramped work spaces that requires getting into awkward positions? Creative Intelligence Originality The ability to come up with unusual or clever ideas about a given topic or situation, or to develop creative ways to solve a problem.

Fine Arts
Knowledge of theory and techniques required to compose, produce, and perform works of music, dance, visual arts, drama, and sculpture. Social Intelligence Social Perceptiveness Being aware of others' reactions and understanding why they react as they do. Negotiation Bringing others together and trying to reconcile differences. Persuasion Persuading others to change their minds or behaviour. Assisting and Caring for Others Providing personal assistance, medical attention, emotional support, or other personal care to others such as co-workers, customers, or patients.

Таble 5. O*NET Variables that Serve as Indicators of Bottlenecks or Barriers to Computerization
where: Y -production level; L S -susceptible labor inputs; L NS -non-susceptible labor inputs; Ctechnological change level or grade (computer capital) For estimating the volume of employment inputs at risk of being substituted by new technologies, the aforementioned function can be plotted in an isoquant for each production level Y = f (L, C), where Y: production level; L: labor inputs; C: technological change grade (computer capital); C (0,1). The study done on the impact of computerization in the USA indicated that about 47 percent of total US employment would be within the high-risk category [Frey, Osborne, 2013, p. 38]. Figure 3 below shows the correlation between the job mix categories and the potential for computerization.

Measurement of the Timeframe for Impact Assessment
In order to estimate the future impact of technological change upon employment, the concepts of technological life cycle, its expansion, and diffusion can be employed. Firstly, for a given technology, its growth and decline are represented by its life cycle. The history of technology and its industrial application is full of empirical examples. The velocity of advancement from innovation to mature technology and the velocity with which a technology moves from late maturity to obsolescence are critical for employment. These processes correspond corresponding respectively to the increase and stability of jobs and to the decline and substitution of jobs. The empirical or forecasted velocity of the advancement of a given technology indicates how rapidly that technology is available for industrial application in comparison with similar or competing technologies. The speed with which a technology evolves and the necessary timeframe for it to move through the different stages of its lifecycle are related to its industrial dynamics and commercial behavior, which can indicate the number of jobs in related industries. The speed with which a technology becomes obsolete determines those jobs at risk. Figure [Frey, Osborne, 2013].
Labour Market in the Context of Technological Transformations development over time through the different stages of its lifecycle: innovation, adoption, industrial application, maturity, and obsolescence. One curve represents a given actual technology and another -a future technology, which would substitute the first once it reaches obsolescence. In reality, the probability of the adoption and application of a new disruptive technology in the production process or the final product depends upon its acceptability on the market and the willingness or propensity of consumers to substitute the technology or product with a new one. 14 According to this concept, following parameters, affecting labor inputs, would be relevant for estimating the impact of technological advancement upon employment.
where: L -labor inputs; A -velocity of advancement; O -velocity of obsolescence; ΔT a m -period of time from adoption/application to maturity; ΔT m o -period of time from (late) maturity to obsolescence; ΔT m -period of maturity; P a -probability of adoption/application of a given technology. Secondly, one must consider the expansion and diffusion phenomena of a technology spreading over the various planes of industrial application: innovation cell, application case, plant, sector, and macroeconomy. The speed of the technological diffusion and expansion is relevant for the increase or decrease of employment. Figure 5 below illustrates the expansion and diffusion of technology across the different planes of application. The figure also presents an example of the advancement of a critical industrial technology related to those planes. One could say that each technological cluster evolves from a given critical technology diffused at one plane, which expands across subsequent planes. The example considers tone decoder technology a critical development in electronic sensor circuits, which allows for the construction of welding robots used at factories; at the sector level, the formation of automated welding lines emerges at different 14 If it is possible to represent the consumption of a technology or product by a bell curve, one would observe that despite available new alternative technologies, some even obsolete technologies or products might have a long tail, being kept on the market for a longer period of time, thus delaying its substitution.  Automated welding production lines Wider electronic controlled welding production production facilities; at the macro-economic level, is possible to identify a full range of industries using electronically controlled welding production. The actual introduction of those technologies would create and/or destroy jobs, depending on the of efficiency and productivity they achieve in the frame of by a particular occupation. To characterize and eventually measure the complex process of technological advancement, one could propose a series of parameters that might influence the potential and reach of a new technology's impact upon employment. In general terms, such parameters affecting labor inputs can be represented as follows: , L -labor inputs; D -velocity of diffusion; E -velocity of expansion; P a -probability of adoption/ application of a given technology in a corresponding plane; ψ -pervasiveness of a given technology on a corresponding plane; eff -gradient of efficiency due to industrial application; pt -gradient of productivity due to industrial application Foresight exercises' use of the parameters, variables, and indicators mentioned in this chapter could configure the reference models and algorithms capable of grasping the complex relationships between technological change, productivity, and efficiency. The present study considers technological change's relationship with employment at the micro level and the possible implications of it at the macro level. STI foresight basically deals with the processes and systems at the macro-level, eventually at the level of the sector or field of research.

Evaluation of STI Foresight Exercises' Impact upon Employment
In the case of Science, Technology and Innovation (STI) foresight exercises, the indirect effects of various policy and strategy measures upon labor markets are considered. The approach to evaluating the impact of foresight exercises proposed here means to focus on the highest possible level of influence that might be achieved by a project or program. For that, it is necessary to identify feasible performance indicators, which can be measured using available or collectable data directly or indirectly related to the sphere of activity or system under consideration by the foresight study. Obviously, such indicators are not exclusively affected by the exercise itself, many factors and events could lead to other results. Moreover, the selected indicators should be appropriately sensitive to the political measures or strategies proposed by the foresight exercise.
To improve the quality of the evaluation work, the application of a logical framework is necessary and should be used throughout the entire foresight-strategic exercise: design, implementation, and evaluation, as stated in [Seidl da Fonseca, 2016]. In the case of STI foresight, the present paper proposes a group of impacts achievable by corresponding strategies and identifies feasible and adherent indicators. Such careful selection applies equally to the indicators of impact on employment. In the previous section, some methods and indicators for evaluating impact on employment were worked out. Table 6 below compares the potential impacts of STI foresight exercises and their performance indicators with the possible impacts upon employment and its related indicators.

Таble 6. Elements for Evaluating of Impact of STI Foresight Exercises upon Employment
Although STI foresight exercises in general cover a broad range of issues, given the importance and overarching influence of technology upon the development of employment, the set of indicators presented above can elucidate such effects.

Conclusion
The present paper proposes a methodology and measurement techniques for evaluating STI foresight and policy derived from such studies with relation to job creation or destruction. The full range of parameters, variables, and indicators discussed in this paper could serve as an eventual standard for evaluating the results achieved by foresight exercises. The classification of the effects upon employment and the evaluation of their impact proved to be a challenging subject, requiring sources for theoretical concepts and empirical evidence. The work done here, by providing some new concepts and insights into the subject, is meant to serve as a research agenda on how to further establish effective and robust evaluation mechanisms for foresight in general.