Measuring Technological Change – Concept, Methods, and Implications

Technological change can be understood in terms of technological evolution. Because of this evolutionary aspect, a number of scholars have made analogies between technological innovation and biological evolution (Ziman, 2000). As technological advancement has played a crucial role in industrial development, almost every nation is concerned with monitoring technological change. Technology policies such as research and development (R&D), technology planning, technology management, etc. are related to this concern.


Introduction
Technological change can be understood in terms of technological evolution.Because of this evolutionary aspect, a number of scholars have made analogies between technological innovation and biological evolution (Ziman, 2000).As technological advancement has played a crucial role in industrial development, almost every nation is concerned with monitoring technological change.Technology policies such as research and development (R&D), technology planning, technology management, etc. are related to this concern.
As accurately diagnosing the current situation is the first step in solving problems, measuring the current technological state is also an important stage in the making of technology policies.Indeed, there have been many measurement cases on the national level such as census surveys.As Porter argues, quantification is "a social technology" and "a crucial agency for managing people and nature" (Porter, 1995: 49-50).Quantitative measurement conducted by governments has as its aim the control of resources.Likewise, measuring the technological state on the national level aims to increase the efficiency of the national innovation system. 1here has been a long history of accurate measurement being valued by governments.Regarding systems of measurement driven by nation states, France tried to make an objective and accurate standard during the Enlightenment.The French government created the metric system as a result and this had complex effects on the political economy (Heilbron, 1990;Alder, 1995).The tradition of measurement was also important in Victorian Britain.Accurate measurement was a crucial value in such areas as imperial triumphs (Schaffer, 1995).
Besides national measurement cases, there has been a great deal of literature dealing with the analysis of technological states.For example, Merton (1978) 2 analysed the state of science and technology in seventeenth century England using available data.This kind of literature is an analysis of just a certain country.The measurement of technology in this chapter is focused on comparisons among nations rather than the absolute technological state of any given country.
In this chapter, I will deal with the theme of measuring technological change as measuring a technological state.Though there are various actors, areas and levels, I will focus on measuring activities relevant to the public sector.The importance of the private sector in terms of R&D is now greater than ever before, but governmental R&D policies still have great influence on technological development in the national innovation system.There is important context in the case of each nation.I will survey measuring methods and cases from a global perspective.However, I will also provide some specific empirical cases as needed.
More specifically, I will review the concept and context of measuring a technological state by using a historical and theoretical approach.Then, I will survey various methods of measurement with brief notes about different cases.The limitations of each measuring method will be illustrated as well.Finally, I will discuss the implications of measuring a technological state in terms of effectiveness and applicability.Feller (2003) explains the evaluation of science and technology programs by using the newspaper reporter's algorithm of what, where, when, who, why and how.This format can easily communicate a difficult concept to readers.I have adopted his approach to introduce basic concepts of measuring technological change in this section.

What is it?
Measuring technological change is literally to measure the state of a certain technology along a changing path.As technology changes dynamically at all times, it is difficult to follow and check the moving state.Instead, a technological state can be measured at each stage.It is more realistic to describe measuring technological change as measuring technological states.As a result, these two expressions are sometimes used interchangeably in this chapter.
Measurement has different but overlapped meaning with similar concepts such as evaluation, assessment, and appraisal.As there are not strict distinctions among them, even experts sometimes mix them.However, evaluation is generally used for programs.R&D program evaluation exemplified this kind of activity.
Assessment is frequently mixed with evaluation.However, assessment has a somewhat different and specific meaning in 'technology assessment.' Technology assessment has been generally used to estimate the impacts, influences, or consequences of a specific technology on society and nature (Seki, 1992).Nowadays, it is also defined as "a scientific, interactive and communicative process which aims to contribute to the formation of public and political opinion on societal aspects of science and technology" by European institutes for technology assessment (Decker & Ladikas, 2004: 14).
The Office of Technology Assessment in the U.S. Congress, the first official organization for technology assessment in the world, had conducted the role of assessing technology from 1973 to 1995.The role of this office can be expressed as attempting to "minimize the negative effects of new technologies and maximize the positive effects" (Bimber, 1996: 26).
Unlike evaluation or assessment, measurement is to analyse an object in detail, precisely, and objectively more so than with other similar terms in the context of this chapter.Measuring a technological state is to estimate the precise level, degree, or stage of the technological trajectory.The results of measuring activities can be communicated as numbers, probabilities, or any other numerical figures.

When does it happen?
A technological state can be measured at each stage of the technological trajectory.If a certain technology is introduced, it evolves along its trajectory.As emerging technologies are intrinsically uncertain as to whether they will develop or fail in the future, each government has tried to measure the state of emerging technologies in their early stages.Governments make policies based on the results of this measurement.
Measuring a technological state is conducted periodically or irregularly by agencies.The results of these periodical measurements are used for time-series analysis in terms of technological change.On the contrary, irregular or one-time measurements are conducted for a specific technology.For example, the South Korean government conducts the measurement of the national technological level biennially, and conducts irregular measurement of specific technologies.
International organizations, such as the Organization for Economic Co-operation and Development (OECD), have regularly published the results of measuring technological states in terms of R&D and innovation.As a result of these regular reports relating to technological states, policy makers can understand the changing trends of technological states by time-series analysis.

Who does it?
There are various actors that initiate the measuring of technological states: nations, international organizations, companies, and so on. 3In the public sector, governments and international organizations are the main actors.In particular, governments have been interested in measuring their resources.Much like current measurements, German-speaking states around 1800 already measured the "strength of the state" by calculating population and wealth (Nikolow, 2001: 23-24).
Though governments initiate measuring policies, the actual actors of the measurement are mainly agencies.This measurement is done by government agencies and private bodies as well.However, the government is ultimately the main actor as it is the contractor.Although companies do conduct technology measurement on their own, governments contract agencies for measuring work.Overall, there are more measurement results revealed by governments than companies.The latter are often unwilling to make public the results that they use for their private decision making.As a result of this context, I will focus on technology measurement in the public sphere, as I have mentioned.
As the importance of technology planning activities4 has increased, the sphere of technology planning has become specialized in recent decades.The competency of an agency engaged in this work depends on the capabilities of its practitioners.These practitioners usually conduct other technology planning activities, as well as technology measurement.They possess the necessary methodology, practice, and experience in the area of technology measurement.However, they sometimes need other experts' knowledge in specific technology areas.Thus, technology measurement is conducted by many actors such as government officials, practitioners in agencies, expert groups including professors or engineers, and so on at the micro level.

Why do they do it?
Historically, developing countries such as South Korea and China have tried to catch up technologically to advanced nations.During such a 'catching-up' phase of a national innovation system, any developing country is willing to compare the state of their technologies to that of other nations.At this stage, the results of technology measurement can be a useful reference for catching-up strategies.The results of the measurement are generally used as references for R&D policies and innovation strategies by policy makers.
Technology measurement utilized exclusively by developing countries.Indeed, the United States, Japan, and advanced European countries have continuously conducted technology measurement.Even technologically advanced countries need to compare themselves to their competitors' states.The U.S. has also compared its domestic technological capability to that of other nations such as Japan, the EU, and so on.For instance, U.S. agencies published Science Indicators 1972 to compare its level of science and technology in terms of R&D expenditure, R&D man power, technology transfer, etc. to that of other nations (National Science Board, 1973;Elkana et al., 1978).Sometimes the U.S. has officially compared its technological capability to a specific nation, such as Japan (Arrison et al., 1992).
The ultimate aim of technology measurement is to provide referential materials for decision making.Governments always need evidence for any decision making.As the trend of 'evidence-based policy' has deepened, the need for reference data has increased.Especially in the R&D area, in the late Twentieth Century, governments have invested their budget in various programs rather than evaluating the performance of their R&D.However, many governments try to evaluate the performance and results of their R&D programs these days.The 'science of science and innovation policy' initiated by the United States is an example of this policy change.Under the U.S. federal government's 'science of science policy' initiatives, agencies should develop tools to improve R&D portfolio management and better assess the impact and performance that results from their investments (Fealing et al., 2011).

How is it conducted?
Unlike other objects, technology is abstract and intangible.It is difficult for even experts to measure or estimate a technological state.Indeed, technology measurement is to estimate the technological state at a specific stage.There are some methods for estimating a technological state piece by piece.Along the spectrum of available methods, there is a range between complexity and convenience.The more complicated a method is, the less practically attainable it is.Technology measurement is different from other methods generally used in laboratories or workshops.As mentioned above, the sphere of technology measurement has become specialized.There are some peculiar methods to measure the abstract state of technology.Specific types and methods will be explained in the next section.

Types and methods
In this section, I will classify measuring methods into five types: scoring models, data analyses, surveys, growth models, and indicators.Though each type of measuring is independent, different methods are sometimes combined.

Scoring models
Scoring models have been used as a means of ranking or rating technology quantitatively.In scoring models, detailed technological properties should be quantitatively measurable.There are various scoring models, but I will introduce the generally used models in this section.According to Martino's model, the technological state, in terms of total score, can be calculated by using the following equation.Each capital letter is a factor that composes the technological state.A and B are overriding factors.(C, D, E), (F, G), and (I, J) are exchangeable factors within brackets.I, J, and K are costs or undesirable factors (Martino, 1992).Gordon and Munson (1981) introduced a convention for measuring the state of the art.They suggested that different experts in the same technological area should estimate the state of the art at the same level.In the following convention, P n is the value of the n th parameter and K n is the weight.
The advantage of a scoring model is its quantitative measuring of results.As the results are calculated from detailed various factors or parameters, this method can well reflect technological properties.In spite of this advantage, the intrinsic aim of the scoring model cannot be easily realized.A scoring model should meet several requirements to fulfil its aim.The factors should be measurable and representative of the state of the art and data for measurement should be available.However, the concept of technology is too abstract to classify easily.It is also a difficult job to collect available data.
Recently, scoring models have not been used as a main method for measuring technological state.Instead, models have been complementarily used with other methods.For example,

94
the Korea Institute of S&T Evaluation and Planning (KISTEP) has regularly measured the level of major technologies on behalf of the South Korean government since 1999.The KISTEP has also used Gordon's model to aggregate weighted values gained from data or surveys.

Data analysis
Data analysis is a measuring method using data from published papers and patents.This method is useful for measuring specific technologies that can be classified according to given standards.Patents are directly linked to technological performance.As papers are a category of the results from technological output, papers are also important resources for measuring a technological state in addition to patents.
There are a number of standard tools used to calculate the output of papers and patents.Data analysis has emerged as an independent disciplinary area in other fields such as scientometrics, bibliometrics, informetrics, webometrics, netometrics, cybermetrics, and so on (De Bellis, 2009).Data analysis is more popular than other measuring methods.As the results of data analysis can be illustrated by quantified numbers, policy makers usually regard them as more objective than other figures.However, as many pieces of literature have shown, there are several issues such as language bias, timeliness of the analysis, comparability of the different research systems, statistical credibility, comparability of peer review judgement, and so on (Geisler, 2000;van Rann, 2004).
As papers and patents are just a portion of R&D outputs, there are some limitations in measuring the overall state of technology.Thus, data analysis should be used complementarily (Kim, 2010b).Output-centered measurement is not always appropriate for technology.In the case of a technology in the early or middle stages of its trajectory, outputs in these stages cannot reflect the overall potential of the technology.For example, five of the most important communications technologies such as telegraph, telephone, phonograph, personal computer, and cable television were not attractive to the market in the early stages of their development (Nye, 2006).
Nowadays, there are many cases of network analysis and mapping used in many agencies.Though using the same data, network analysis and mapping are different from data analysis.The most different aspect of the former is visualization.For instance, citation networks and distribution can be visualized as a map.Then, we can estimate technological change through comparing maps representing different times in terms of paper (or patent) citation.There are also limitations that apply to network analysis and mapping.Networking and mapping work are based on keywords, as in other forms of data analysis.Keyword setting is a timedemanding work and should be iterated until plausible results are achieved.

Surveys
The survey method is relatively easier than other methods.The basic assumption of the survey method is that experts are very well acquainted with the state of technology.In other words, the survey method utilizes the tacit knowledge of experts.Among survey methods, the Delphi survey has recently been used for measuring technology by many agencies.The Delphi survey was originally one of the foresight methods.In the early 1970s, the Japanese government initiated large-scale foresight surveys using the Delphi method, and the surveys have been repeated approximately every five years.In the Delphi survey, numerous experts are repeatedly surveyed with identical questions, with the results of previous rounds fed back to the respondents in order to revise their answers and draw out a consensus.
The U.S. had also used the Delphi method to solicit and synthesize the opinions and judgments of expert communities.For instance, the U.S. National Science Board conducted a Delphi survey in 1972.The topics of the survey were the future role of science and technology, the impacts of R&D funding, technological innovation, basic research, financial resource allocation, and graduate education (National Science Board, 1973).Though the Delphi survey was not directly conducted for measuring the technological state, the methodology was similar to the current measuring style used for this purpose.Moreover, the topics of the survey could be related to the technological state.
Since the late 1980s, the Delphi surveys have been popularized as a useful form of technology forecasting and measuring methods in many other countries.Particularly, many agencies in South Korea have applied the Delphi surveys to technology measurement since the turn of the century.Questionnaires in Delphi surveys usually include questions relating to the technological state such as the present level, impact, capability, competitiveness, and so on.The Delphi survey is a useful method in the case of a large number of technology areas, within a short time frame.Because of this strong point, the KISTEP has applied the Delphi survey for technology level evaluation since 1999.The agency has developed this method combined with a growth model in 2008 and 2010.
Survey methods are intrinsically subjective and qualitative.Though the answers of participants in the Delphi survey can be fed back to one another in repeated rounds, their decisions cannot be objective and quantifiable.Thus, there have been controversies whether the survey results are reliable.Some results from data analysis or indicators such as patent citation level can be objective and quantitative, but subjective methods can be employed in setting and drawing up policy.As Woudenberg (1991) reviewed, the Delphi method is not a science but an art of creating a consensus.The survey results are best understood as synthesized tacit knowledge and experts' intuition (Kim, 2010b).

Growth models
Growth models are not general methods for measuring something.Though they are more appropriate for forecasting rather than measuring, growth models are sometimes used for technology measuring.If other measuring methods are static, growth models are relatively dynamic.As growth models describe the changing trends of a technological state, policymakers can not only understand technological change dynamically, but also forecast the future state from growth curves.
In this section, S-shaped curve is used in the same sense as growth model.There are various S-shaped curves such as Bass, Pearl, Gompertz, and so on.The Bass model is an S-shaped curve model developed in the early stage.Nowadays, Pearl (3) and Gompertz (4) are generally used as S-shaped curve models.In both equations, L means the upper limit of technological development.
While S-shaped curves provide us with a model of the dynamic change of technology, they require several assumptions (Porter et al., 1991;Martino, 1993;Kim, 2010a).These assumptions include a correct equation, a definite upper limit, proper fitting, and so on.Among them, the definite upper limit is not easily defined because the upper limit is a rather theoretical concept.
Recently, growth models have been applied for measuring technological level (Bark, 2007;Kim, 2010a, Kim, 2010b, Ryu & Byeon, 2011).For instance, the KISTEP applied the model to evaluate national core technologies in 2008 and 2010.Though there were limitations such as the problem of the upper limit, they are appraised as advanced cases in terms of measuring dynamic technologies.
Policymakers may prefer growth models because of their forecasting aspect.Technology forecasting is not also an easy job as there are numerous variables relating to technological change.The future is intrinsically uncertain, thus technology forecasting from an S-shaped curve is uncertain as well.Because the growth model is a form of trend analysis as well as an extrapolation method, there are intrinsic limitations of the unpredictability and fallibility.
Technologies do not always follow forecasted trajectories due to unexpected factors such as 'disruptive innovation' (Christensen, 1997).

Indicators
There are many indicators and indices that can be used to measure a technological state.Though many indicators and indices are being made by various agencies, they are sometimes contrived as there are not enough objective criteria used for setting them.Sometimes, in the process of selection and rejection of indicators, political interests can be involved. 5To give an example from among the globally accepted indicators, it is useful to review those used by the OECD.The OECD has also published reports relating to indicators in terms of innovation.Though technology and innovation are conceptually different, measuring indicators are sometimes mingled with each other, as the technological state usually results from innovation.6Among them, the Frascati and Oslo Manuals are guidelines for the measurement of scientific and technological indicators.
The Frascati Manual is a set of guidelines for R&D statistics.The manual has been revised six times since it was issued in 1963.As R&D statistics are some of the most important indicators of economic growth, in terms of technological change, the Frascati guidelines have played a crucial role for many statistics and scoreboards made by the OECD and various countries.As a result, the Frascati guidelines have become a global standard for R&D surveys and statistics not only in OECD member countries, but also other organizations such as the UNESCO, the European Union and so on.
The Oslo Manual has been issued three times since 1992 and is more focused on innovation activities.The first edition, in 1992, included the results of surveys to develop and collect data on the process of innovation.The second edition, in 1997, updated its framework to enlarge the concept of innovation and the range of industries studied, and also improved innovation indicators to make them comparable among OECD countries.The most recent edition, issued in 2005, included a large amount of data and information from various surveys.It expanded its innovation measurement framework to relevant firms, services, etc. and types of innovation to organizational and marketing innovation.
Neither manual is a directly citable indicator for technological change.However, they provide relatively objective criteria for R&D and innovation statistics and indicators.Actually, many reliable technological indicators are based on data and information guided by those manuals.They are useful for comparing nations in terms of standardized indicators, but they have limitations regarding measuring specific areas of technology.As the guidelines deal with a relatively broad range of criteria for science and technology, more specific data and information would be needed in the case of measuring a specific technology such as nanotechnology.
There are also other indicators globally accepted such as IMD World Competitiveness.
Though the IMD indicators have some sub-indicators such as scientific infrastructure, technological infrastructure, etc., the indicators are not sufficient to decide the technological state in each nation.Locally, there are some indicators such as Composite Science and Technology Innovation Index (COSTII, South Korea), Japanese Science and Technology Indicators (Japan), and so on.Though these annual indicators are published by each nation, they include information pertaining to other competitive nations such as the U.S., Germany, the U. K., China, etc.Thus, those indicators can be understood as criteria for comparative analysis among major nations.

Discussions
As I have briefly reviewed above, measuring a technological state is not easy but is instead a complicated activity.There are still controversies over measurement matters both theoretically and practically.Indeed, even the object of measurement is different in each case.As the concept of technology is abstract, a specific technology is sometimes regarded as an end product or the process of realizing the product as well.However, there is no universal consensus or standard that can be applied to measuring activities.Thus, a measurement designer should define meticulously the concept and scope of technologies as objects in the early stage of the activities.
In the stage of defining technology, the classification of detailed technology is also important, as well as the concept and scope of the object.Although there are some standardized classifications that apply to the global or national level, these standards cannot be applied to specific measurement cases.The standards are usually too broad to apply to more specific measurement.For example, the OECD standards based on the Frascati Manual do not include detailed technologies.For an example on the national level, although the National S&T Standards Classification System South Korea is somewhat more detailed than global ones, it is sometimes not compatible with the aim of specific practical measurement.General classification standards are based on normal disciplinary areas, but practical cases need specific classifications for their own aims.Each measurement project usually involves a separate process to classify the object technology, and thus there are often different criteria used to measure, even in similar technology areas.
Once the scope and classification of the object has been defined, practitioners should decide which method is proper to be applied.As explained above, there are various methods such as scoring models, data analysis, surveys, growth models, indicators, etc. for measuring a technological state.However, the present trend of measurement has tended to become heterogeneous and complex.For example, the KISTEP has continuously used a heterogeneous method in which the Delphi survey, growth model, scoring model, and data analysis are mixed for its technology level evaluation.As part of this effort, in 2008, the KISTEP developed a method that combines the Delphi survey with a growth model (Kim, 2010a;Kim, 2010b;Kim & Kim, 2010;Ryu & Byeon, 2011).Gordon and Munson's model was applied to calculate the results of the Delphi survey.Thus, data analysis was conducted complementally with the survey.
Though practitioners control the whole process of measurement, they need the expertise of specialists in specific technology areas.As mentioned above, experts' judgement is a kind of tacit knowledge.Their expertise can be converged in some ways including seminars, interviews, surveys, and so on.A greater number of experts can participate in surveys than in the other methods.However, the results of surveys are not always reliable.According to the results of the Delphi survey for the Marine S&T evaluation in South Korea, conducted by the KISTEP in 2010, the expertise of the participants in the survey was sometimes unstable, indeterminate and uncertain.
Measurement results are different according to the context of each country.For example, some countries prefer the ranking or grading of the results.In case of South Korea, the government prefers measurement results in terms of percentages.In this case, 100% represents the level of the most advanced country, as well as the criterion of measuring the level of South Korea.In other countries, like the United States or Japan, agencies usually publish their measurement results in terms of grading.Their grading is generally ranked according to five categories in terms of Likert scales.
As Godin (2005) argues, statistics reflect values, interests, and ideologies, so the results of measuring a technological state reflect such social factors that are involved in selecting indicators and publishing the results as well.Like any other scientific results, the results of measuring a technological state are also constructed by various human and non-human actors. 7The measurement results are constructed by interacting relations among practitioners, technology specialists, measurement tools, figures, numbers, graphics, texts, and so on.
Once the results are published, the results in terms of numbers and figures have their own trajectory like a living thing.Published figures and numbers are sometimes appropriated by users.Policy makers use them to assess R&D programs, to compare R&D capabilities among countries, to make technology roadmaps, and so on.They are frequently cited as evidence in various official documents.Even if some mistakes are found in the figures and numbers, it is usually difficult to revise them after they have been made official.Though they are well aware of the limitations of each measurement results, policy makers usually use the results in order to legitimize their policies in terms of evidence.
Technology measurement sometimes plays an important role in visualizing previously vague technologies as concrete ones.Intangible technology can be visualized with the help of texts, numbers, graphs or maps.In other words, such figures and numbers represent intangible technology.The results represent experts' judgement on technology in terms of tacit and explicit knowledge.Though limitations mentioned above exist, the results of measurement have generally been used to help shape relevant policies.The necessity and applicability of measuring technological states has been generally accepted as well.Thus, more sophisticated methodologies are needed in the field of technology measurement.
Kim (2011)nalysis is a representative index used to measure a technological level qualitatively.As methodologies have developed, various indices such as the RCI (Relative Citation Index), CII (Current Impact Index), TII (Technology Impact Index), etc. are generally applied by practitioners.For example, recent cases such as Choi &Kim (2011)andKim (2011)analyzed papers and patents in terms of publication numbers, citation numbers, citation index, specialization index, etc.Though they measured the state of special t e c h n o l o g i e s i n S o u t h K o r e a , t h e y u s e d d a t a f r o m S C I p u b l i c a t i o n s a n d U .S .p a t e n t publications to enhance the objectivity of the result.