The Synergy and Cycle Values in Regional Innovation Systems : The Case of Norway

The innovation capacity of a system can be measured as the synergy in interactions among its parts. Synergy can be considered as a consequence of negative entropies among three parts of the system. We analyze the development of synergy value in the Norwegian innovation system in terms of mutual information among geographical, sectorial, and size distributions of firms. We use three different techniques for the evaluation of the evolution of synergy over time: rescaled range analysis, DFT, and

T he cyclical behavior of economic variables has been a research topic since the time of Schumpeter [Schumpeter, 1939], Kuznets [Kuznets, 1930], and Kondratieff [Kondratieff, 1935].Recently, Lucraz [Lucraz, 2013] analyzed innovation cycles in a finite, discrete R&D game, concluding that strategic interactions among firms are sufficient for generating cycles.De Groot and Franses investigated cycles in basic innovations [ de Groot, Franses, 2008, 2009] and more generally socioeconomic cycles [ de Groot, Franses, 2012].These authors concluded that there seems to be a common set of cycles across a number of socioeconomic variables.The regional dimensions of business cycles were investigated by Dixon andShepherd [Dixon, Shepherd, 2001, 2013], who filtered the data in terms of trends, cycles, and noise, and thus were able to show that similarities in cycles can be explained by the regional industry structure and the sizes of regions.Various techniques, such as autoregressive growth-rated models [Hodrick-Prescott, 1997] and frequency filter models, have been used to analyze cyclic data [Dixon, Shepherd, 2013].From another perspective, fractal statistics and rescaled range (R/S) analysis were used to analyze cycles in various processes in nature [Feder, 1988;Frøyland, 1992].These techniques were developed to analyze regional economic fluctuations by considering a variety of factors that might explain the cyclical movements.From this viewpoint, it is interesting to explore whether synergies behave like business cycles and therefore one must ask whether business cycles may comprise a component of synergy.Previous studies did not account for the synergy of economic interactions.If synergy also evolves in cycles, then it can be considered an additional factor of economic fluctuations.The core research questions of the present paper regarding temporal synergy evolution are as follows: how do the synergies evolve?Can they be analyzed as trend-like, chaotic, oscillatory, or perhaps some other functional dependency?Do synergy values affect temporal evolution?In other words: is there a difference in synergy evolution between configurations with high and low synergy?Can numerical indicators of synergy evolution be provided?

Synergy in Innovation Systems
In a series of studies, we measured the synergy of a Triple Helix (TH) system as the reduction of uncertainty using mutual information among the three dimensions of firm sizes, the technological knowledge bases of firms, and geographical locations. 1In these studies, we obtained maps of synergy distributions across territories.However, having only static measurement results, one is unable to answer questions such as those concerning the evolution of synergy over time.Does the synergy value affect the temporal evolution of the synergy in a system?Note that a TH system cannot be static [Etzkowitz, Leydesdorff, 2000], rather it is an ever-evolving system.This evolution can generate uncertainty or the reduction of uncertainty.Does the synergy in a system also evolve over time?
The innovation capacity of a system, for example, can be measured as the synergy of interactions among its parts.Both social and biological ecosystems can be expected to flourish and even proliferate if uncertainty in the relations among the constituent parts of the system is reduced [Ulanowicz, 1986].From this perspective, the Triple Helix (TH) model of university-industry-government relations serves as a specific example of the innovation system.Synergy refers to the interactions among two or more parts, so that the combined effect of this interaction exceeds the sum of individual effects.Synergy is based upon the coherent actions of a system's parts, which means that these actions depend upon one another.In terms of statistical mechanics, this means that the system becomes more organized, or in other words, more ordered.The more ordered the system is, the more coherent are the interactions between the parts of the system.Entropy can be used as an indicator of order.Entropy measures the degree to which the system is ordered with respect to different possible system states.The system configuration is not limited by actual realized states.In addition to actual system states, the system can potentially have other states (which may be realized in the future).We refer to the latter as virtual states.The difference between the maximum possible entropy and the actual entropy realized by the system provides a measure of order.This measure can be increased either by reducing the actual entropy (leaving the maximal entropy unchanged) or by increasing the maximal entropy (i.e., increasing the number of virtual states) while actual entropy remains unchanged.For example, in the 19 th century one attributed approximately 20% of the available funds to developing mechanical engineering technology.When the same amount of funds is attributed to mechanical engineering technology in the 21 st century, the percentage is unchanged, but the number of virtual options (supported by the corresponding technologies) in the 21 st century exceeds that of the 19 th century.In addition to mechanical engineering, one has now access to computer technology, biotechnology, nanotechnology, and so on.Hence, the ordering in the economy of the 21 st century exceeds that of the 19 th century econ-omy due to additional options that provide additional flexibility, adaptability, and competitive advantages.Information-theoretical probabilistic entropy as described by Shannon's mathematical theory of communications follows the definition of Boltzman's entropy [Shannon, 1948].Communicating sub-systems can provide additional options that can be measured as mutual information and this may lead to a reduction in uncertainty at the system level.This reduction of uncertainty can be considered a measure of synergy2 , which can be expressed in negative bits of information using the Shannon formula [Abramson, 1963;Theil, 1972;Leydesdorff, 1995]. 3n this study, we use the entropy approach to measure yearly synergy in the Norwegian innovation system during the period 2002-2014.Longitudinal synergy data provide a picture of temporal synergy evolution.The choice of the Norwegian system is guided by the availability of data.However, the method is generic and can be applied to any system that meets the criterion of possessing three (or more) analytically independent parts.

Methodology
The interaction between two system parts can be numerically evaluated using the tenets of Shannon's information theory by measuring mutual information as the reduction of uncertainty at the system level.In the case of three interacting parts, the mutual information in a configuration can be defined by analogy to mutual information between two parts, as follows [Abramson, 1963;McGill, 1954) (1) Here H i , H ij , H ijk denote probabilistic entropy measures in one, two, and three dimensions: The values of p represent the probabilities, which can be defined as the ratio of the corresponding frequency distributions: N is the total number of events, and n i , n ij , n ijk denote the numbers of relevant events in subdivisions.For example, if N is the total number of firms, n ijk is the number of firms in the i-th county, the j-th organizational level (defined by the number of staff employed), and the k-th technology group.Then n i and n ij can be calculated as follows: A set of L mutual information values for a certain time period, considered a finite time signal, can be spectrally analyzed with the help of the discrete Fourier transformation [Analog Devices, 2000]: Here: (6) The Fourier transformation by itself cannot provide us with information regarding synergy evolution except the values of the spectral coefficients: A, B l and D l .Because the aggregate (country-related) synergy is determined by additive entropy measures (Equation 1), it can also be decomposed as a sum of partial (county-related) synergies Т 1 , …Т n :4 So that each partial synergy can be written in the same form as Equation 5: Here: After substituting Equations ( 5) and ( 8) into (7) and re-grouping the terms, one obtains: (9) Leydesdorff and Ivanova [Ivanova, Leydesdorff, 2014a] showed that mutual information in three dimensions is equal to mutual redundancy (T 123 = R 123 ).Aggregated redundancy can equally be decomposed as a sum of partial redundancies, corresponding to the geographical, structural, or technological dimensions of the innovation system under study.Mutual redundancy changes over time, so one can write: In another context, Ivanova and Leydesdorff [Ivanova, Leydesdorff, 2014b] expressed a redundancy that can be obtained as follows (i= 1, 2 … n): (11) The oscillating function in Equation ( 11) can be considered a natural frequency of the TH system.This natural frequency is far from fitting the observed redundancy values for R 123 .However, real data for the definite time interval can be fit with the help of the discrete Fourier transformation, comprising a finite set of frequencies.Each frequency in the set composing Equation ( 10) can be considered a natural frequency of the TH system: (12) Comparing Equations ( 12) and ( 11), one can approximate the empirical data for three-dimensional redundancy as a sum of partial redundancies corresponding to the frequencies that are multiples of the basic frequency: w, 2w, 3w … etc.
In other words, a TH system can be represented as a string resonating in a set of natural frequencies with different amplitudes.Frequency-related amplitudes, which can be defined as modules of the corresponding Fourier coefficients, can be considered the spectral structure of the TH system.Absolute values of the Fourier-series coefficients can be defined as follows: These coefficients determine the relative contributions of the harmonic functions with corresponding frequencies to the aggregate redundancy (R 123 in Equation ( 12)).

Transmission Power and Efficiency
Following Mêgnigbêto [Mêgnigbêto, 2014, p. 287], the transmission power of synergy can be calculated according to the following formula: (15) The transmission power is designed to measure the efficiency of mutual information.While the transmission defines the total amount of configurational information, the transmission power represents the share of synergy in the system relative to its size.For positive transmission values, it is simply the overlapping area-total area ratio in a corresponding Venn diagram.Mêgnigbêto [Mêgnigbêto, 2014, p. 290] ar-gued that "… with such indicators, a same system may be compared over time; different systems may also be compared."

Characteristics of Norwegian Regions
The regions in Norway are illustrated in Figure 1.
Norway is divided into 19 counties at the Nomenclature of Territorial Units (NUTS) level 3 and seven regions at NUTS 2. These regions are the geographical units of analysis in this study.

Data
In order to compare the industry structure in various regions, we use a firm-based version of the Krugman index of dissimilarity [Krugman, 1991[Krugman, , 1993;;Dixon, Shepherd, 2013].
For each industry sector i, data on the number of firms in region A; X Ai and X Bi are provided.The total number of firms in each region is: X A and X B .The dissimilarity between the industry sectors in the two regions can then be calculated as:    [Eurostat, 2002] and the data during the period 2009-2014 were organized according to the NACE Rev. 2 classification [Eurostat, 2008].Some of the criteria for the construction of the new classification were reviewed: there is no one-to-one correspondence between NACE Rev. 1.1 (with 17 sections and 62 divisions) and NACE Rev. 2 (with 21 sections and 88 divisions) [Eurostat, 2008].To correctly merge the NACE Rev. 1.1 and NACE Rev. 2 data, one has to turn to a higher level of aggregation (Table 2) containing 10 classes [Eurostat, 2007].

Descriptive Statistics
Regional synergy is calculated as a sum of the synergies at the county level in accordance with Equation (7).The results of the calculations during the period 2002-2014 (in bits of information) are shown in Figure 2 for the national level and Figure 3 for the regional level.The synergy at the national level follows a general lateral trend with alternating upwards and downwards sectors.More negative T(uig) is observed until 2004, then a decrease in synergy takes place until the economic crisis in 2008, after which a recovery is present where synergy shows a positive trend.As can be seen from Figure 3, the country synergy is in large part shaped by the synergy in the capital region OA5 .The other six Norwegian districts demonstrate relatively stable development.These regions are subdivided into two visually separated strands with respect to synergy values: HO, AR, TR, and WE, SE, NN.Fluctuations in synergy data can be interpreted as synergy cycles.Like economic cycles, synergy cycles indicate endogenous characteristics of an innovation system such as cyclic oscillations of the market system [Morgan, 1991].An alternative to considering the fluctuations as cycles would be to consider them the result of noise in the data; this will be clarified this in the next section.

Transmission Power and Efficiency
The transmission power at national and regional level are given in Figures 4 and 5.
As can be seen from Figure 4, transmission power shows stability with a shift in 2008.A linear trend line would have indicated a weak growing efficiency of the Norwegian innovation system at the national level.Figure 5 shows that the rate of efficiency growth is most accentuated in the NN and HO regions.The Source: authors' calculations.Ivanova I., Strand Ø.,Leydesdorff L., OA capital region with the highest synergy values possesses medium transmission power.By comparing the results for synergy and transmission power at the regional level, it is shown that the high synergy in the U-I-G interaction does not necessarily imply the most efficient innovation system construct.
Comparing transmission power at the national level in Figure 4 with the synergy in Figure 2 shows slowly increased transmission power and accordingly in-creasing synergy over time.The dip in 2008 is more pronounced for static synergy data than for the dynamic measure of transmission power.At the regional level, the same patterns are most pronounced in NN, HO, WE, and to some extent in SE.A decreasing value in transmission can be found in TR, whereas OA and NN show more fluctuating development.The percentage of the average efficiency deviation: (17) where is the efficiency for the i-th region averaged over the period 2002-2014; is the summary average efficiency averaged over all of the regions (Figure 6), and the percentage of average synergy deviation where is the synergy for i-th county averaged over the period 2002-2014; and is the summary average synergy averaged over all of regions 8).Efficiency is above the country average in OA, NN and AR.Synergy is above average in OA, NN, WE.When comparing the figures one can observe that the efficiency and synergy peaks do not coincide: regions with the highest synergy values are not always the most efficient.While for OA, the above-average synergy value may indicate that the increase in synergy was caused by increased transmission power, in NN, on the contrary, relatively low synergy is accompanied by the highest value for efficiency.Spearman rank correlation between the percentages of synergy and the efficiency values is 0.64 (n.s.).This value of the Spearman rank correlation indicates that there is a monotonic dependence between the two variables.This sheds light on the need for more in-depth research on the parameters influencing innovation systems with respect to synergy-efficiency ratios.
As a next step, a deeper look is taken into the structure of the fluctuating behavior of the aggregate redundancy time series.First, the discrete Fourier transformation is implemented in accordance with Equation ( 5).The inputs of different frequency modes for Norway's synergy (w, 2w, 3w, 4w, 5w, 6w, 7w), calculated according to Equation ( 14), are shown in Figure 8.Each of the regional synergies can be mapped as fluctuations around an average value.Thus, the average values can be taken as the first terms in the corresponding Fourier decomposition describing nonfluctuating terms (f 0i in Equation ( 8)).These average values form the synergy line specter.Having calculated the modules of the Fourier series coefficients, which are the measures of different frequency modes, as well as the line specter synergy values, modules versus synergy values can be mapped.Because the real-number data (during the period 2002-2014) are addressed, then, due to the symmetry of DFT coefficients, only half the number of input data with differing frequency components (the first six) can be specified.C1 corresponds to a 12-year cycle; C2 to a 6-year cycle, and similarly the seventh component (C7) corresponds to the 1-year cycle, which is the highest frequency that can be calculated with this method.
In Figure 9 synergies (in bits of information) are plotted versus frequency amplitudes for the seven regions.Ivanova I., Strand Ø.,Leydesdorff L., with high values for the component is also found for Sør-Østlandet (SE), Vestlander (WE), and Nord Norge (NN).Hedemark og Oppland (HO), Agder og Rogaland (AR), and Trøndelag (TR) have the least accentuated oscillation behavior.Nord-Norge (NN) in contrast with other six regions is the region with the most dominant second component.Nord-Norge, where fishing and related industries play a dominant role, is exposed to fluctuations in the high frequency component.
There is a monotone dependence between the modules of Fourier coefficients and the percentage of average synergy deviations for the Norwegian regions.
The results of the Spearman correlation between these two values are provided in Table 3.In other words, the more synergetic is the system, the more strongly are the fluctuations of synergy expressed.
Previous studies of business cycles have shown that the Krugman dissimilarity index may be used to explain cyclic variations in regions [Dixon, Shepherd, 2013].Regions with a high degree of similarity in the industry structure, which is indicated by a low Krugman index, show similar cyclic patterns.The Krugman index as defined in Equation ( 15) is calculated based on two-digit NACE codes and firm level data for 2015.The results are given in Table 4.As can be seen from this table, the capital region, Oslo and Akershus (OA), is most dissimilar compared to the other regions.The highest similarity (lowest index) is found between Vestlandet (WE) and Agder og Rogaland (AR), and between Sør-Østlandet (SE) and Agder og Rogaland (AR).
The degree of synergy fluctuation randomness can also be evaluated using R/S analysis [Hurst, 1951;Feder, 1988].

Summary and Conclusions
Having studied TH synergy evolution, the following conclusions can be made: first, TH synergy demonstrates non-chaotic oscillatory behavior.That is, one can study 'synergy cycles' as they do for economic and technological cycles.Second, TH systems can be considered to be composed of a set of oscillatory modes in terms of high and low frequency oscillations.From a theoretical perspective, TH systems are expected to have only a single oscillatory mode.The finding of a set of modes implies a complex TH structure, composed of many 'elementary' helices, which can be theorized in terms of a fractal TH structure [Carayannis, Campbell, 2009;Ivanova, Leydesdorff, 2014a;Leydesdorff, Ivanova, 2016].Third, oscillation amplitudes were found to be proportional to average synergy values.Thus, the synergy oscillations can be scaled with respect to the average synergies of TH constituent components.In summary, the TH structure (at the level of regions and nations) may be more complex than expected.Three different techniques for the numerical evaluation of temporal synergy evolution in a three-dimensional system are used: R/S analysis, DFT, and geographical synergy decomposition.Briefly summarizing the results obtained from the study of the Norwegian innovation system, we can conclude that the Ivanova I., Strand Ø.,Leydesdorff L., synergy time series exhibit cyclic structures of a nonrandom nature.This is important because synergy oscillations can be caused, in part, by system-inherent factors, and, in part, by external systemic factors.This feature should be taken into consideration by policy makers when developing related policies for innovation in areas under their sphere of competence, given that innovation efficiency is both locally and globally determined.It demonstrates how the various methods can be used for mapping the evolution of synergy.However, longer time series and shorter sampling intervals would be preferable, even though this involves large amounts of register data.It would then be possible to link the indicated synergy cycles to other and more well-established business cycles through the co-integration of the time series.This could shed new light on the synergy control mechanisms in a TH innovation system.From a conceptual perspective, the synergy in TH innovation systems can be analyzed as a set of harmonic partials at the system's level, while an analytically "pure" TH system can be expected to contain only a single harmonic [Ivanova, Leydesdorff, 2014b].
The appearance of many oscillatory modes indicates a more complex and self-organized TH structure than was traditionally thought.For example, Norway's national innovation system can be presented as a geographically distributed network with nodes relating to corresponding regions and one should account for innovation systems at scales other than the national [Strand et al., 2016].
The synergy value is a monotonic function of frequency.Given that the frequency values are also a proxy of the speed of change of the corresponding frequencyrelated transmission parts (and otherwise, a proxy of volatility), one can expect frequency-related synergy volatility growth proportional to the value of synergy.This is the case for both transmission increases and decreases.In other words, the synergy in more coherently interacting systems grows faster than that in less coherent ones.In the case of decline, however, initially more coherent systems degrade faster.In other words, synergy formation is self-reinforcing, but so is its decay.

Policy Implications
The relative contribution of long-term frequencies increases with the increase of synergy values leading to a frequency shift.In other words, one can expect the synergy volatility to increase with synergy growth.

Further Research
Another result refers to the distinction between the synergy of interactions within a TH system and the system's efficiency.One can conclude that these two measures are statistically correlated though they capture different kinds of information.The study of factors influencing these two important features of innovation systems is a topic of future research.This raises further research questions which are relevant to innovation studies.One can further focus the analysis from the region to the firm-size level.
Assuming that the results remain the same, this may raise further research questions with respect to firm dynamics.According to Gibrat's Law for all firms in a given sector, however, the growth of a firm (i.e., the proportional change in the firm's size) is independent of its size [Gibrat, 1931].The studies of a number of firms confirmed Gibrat's Law [Samuels, 1965].However, one can expect a dependence between the firm's growth and its innovation capacity.The latter is proportional to synergy in interactions among the constituent actors.The actual functional relationship between the firm's size and its innovation and growth capacities needs further investigation in order to complement what has already been found in the literature with respect to the economics of innovation.
zero indicates that the industry structures in the two regions are equal.The opposite, when the .nothing in common would give an index value of 2. Norwegian establishment data were retrieved from the database of Statistics Norway [Statistics Norway, 2015].The data include time series of Norwegian companies during the period 2002-2014 and encompass approximately 400,000 firms per year.The data include the number of establishments in the three relevant dimensions: geographical (G), organizational (O), and technological (T).As noted, seven regions are distinguished in the geographical dimension.In the organizational dimension, establishments are subdivided with reference to the different numbers of employees by eight groups: no-one employed; 1-4 employees; 5-9 employees; 10-19 employees; 20-49 employees; 50-99 employees; 100-249 employees; and 250 or more em-

Figure 2 .
Figure 2. Summary of the Development of TH Synergy at the National Level for Norway (in bits of information)

Figure
Figure 3. Partial Ternary Synergy for the Seven Regions of Norway (in bits of information)

Figure 4 .
Figure 4. Summary of Norway Transmission Power τ (in relative units) during the Period 2002-2014

Figure 7 .
Figure 7. Percentage of Average Synergy Deviation for Norwegian Regions during the Period 2002-2014 (in percentages) The characteristics of the seven regions are provided in Table1.Data on the population and the numbers of firms are provided by Statistics Norway[Statistics  Norway, 2015].The most populated area is the capital region Oslo og Akershus (OA), the sparsely populated and areas dominated by primary industries are found inland (Hedmark og Oppland (HO)) and in the north (Nord-Norge (NN)).The center of the oil and gas industry is in Agder and Rogaland (AR) in the southwest, with Stavanger as the most important city.The region of Trøndelag (TR) includes the city of Trondheim where the main technical university and several research institutes are located, as well as agricultural areas in the northern part of the region.
The region Sør-Østlandet (SE) is composed of several counties with a diverse industry structure.Vestlandet (WE) is the center for marine and maritime related industries in Norway.According to the Regional Innovation Scoreboard 2015[European Commission, 2015], OA, WE, TR, and NN are classified as innovation followers, whereas HO, SE, and AR are classified as moderate innovators.The results from an analysis of TH synergy based on register data from 2008 are also given in Table1.From this it can be observed that synergy is highest in the regions Vestlandet (WE) and Sør-Østlandet (SE).Low levels of synergy are found in Oslo and Akershus (OA), Hedemark og Oppland (HO) and Trøndelag (TR).Moderate levels are found in Agder and Rogaland (AR) and Nord-Norge (NN).

Correspondence of High-Level Aggregation (ISIC ver. 4) to NACE Rev 1.1 and NACE Rev. 2 Classifications
Manufacture of food products and beverages DA 16 Manufacture of tobacco DB 17 Manufacture of textiles DB 18 Manufacture of wearing apparel, dressing and dyeing of fur DC 19 Tanning and dressing of leather, manufacture of luggage, handbags, saddlery, harness and footwear DD 20 Manufacture of wood and of products of wood and cork, except furniture DE 21 Manufacture of pulp, paper and paper products DE 22 Publishing, printing and reproduction of recorded media DF 23 Manufacture of coke, refined petroleum products and nuclear fuel DG 24 Manufacture of chemicals and chemical products DH 25 Manufacture of rubber and plastic products DI 26 Manufacture of other non-metallic mineral products DJ 27 Manufacture of basic metals DJ 28 Manufacture of fabricated metal products, except machinery and equipment DK 29 Manufacture of machinery and equipment n.e.c.DL 30 Manufacture of office machinery and computers DL 31 Manufacture of electrical machinery and apparatus n.e.c.DL 32 Manufacture of radio, television and communication equipment and apparatus DL 33 Manufacture of medical, precision and optical instruments, watches and clocks DM 34 Manufacture of motor vehicles, trailers and semi-trailers DM 35 Manufacture of other transport equipment DN 36 Manufacture of furniture, manufacturing n.e.c.

of Fourier Series Coefficients C versus Frequency for Seven Norwegian Regions (in bits of information)
It can be seen from the figure that the various Fourier components have very high values in Oslo and Akershus (OA), indicating that synergy does not possess strong cyclic components at the frequencies observed.Vestlandet (WE) is the region with second largest amplitudes for Fourier components.A similar pattern × × Figure 9. Modules

Spearman Rank Correlation between the Percentage of Average Synergy Deviation and Modules of Fourier Coefficients Таble 4. Krugman Index of Dissimilarity in Industry Structure for Norwegian Regions
This means that regions with high synergy values are expected to exhibit more fluctuations in synergy than low-synergy regions, demonstrating strong range fluctuations in periods of boost or decline.Based on the various techniques used in this study, it would be possible to develop indicators to monitor the innovation systems' response to external shocks like the fall in oil prices in 2015 and the structural effect of various political measures like the Norwegian government's crisis interventions in the petroleum dependent region of Agder og Rogaland in 2016.Such indicators could guide the government towards carefully considering both the timing, the regional setting, and the time-scale of political measures.Government interventions at national level could amplify or dampen the synergy fluctuations depending on the actual region.Government interventions in regions dominated by one industry sector can have an undesired effect if applied nationally or to regions with high industry activity.Regarding time scales, political measures should be designed to create long term (low frequency) positive economic effects rather than short term (high frequency) political effects.
Inga Ivanova acknowledges support of the Basic Research Program at the National Research University Higher School of Economics (HSE) and the Russian AcademicExcellence Project '5-100'.The authors declare that they have no conflict of interest.