Category Archives: Research evaluation

South-south collaboration on HIV/AIDS prevention and treatment research: when birds of a feather rarely flock together (full-text)

South-south collaboration on health and development research is a critical mechanism for social and economic progress. It allows sharing and replicating experiences to find a “southern solution” to meet shared health challenges, such as access to adequate HIV/AIDS prevention and treatment. This study aimed to generate evidence on the dynamics of south-south collaboration in HIV/AIDS research, which could ultimately inform stakeholders on the progress and nature of collaboration towards increased research capacities in low- and middle-income countries (LMIC).

Bibliometric and social network analysis methods were used to assess the 10-year (2006–2015) scientific contribution of LMIC, through the analysis of scientific publications on HIV/AIDS prevention and/or treatment. Five dimensions oriented the study: knowledge production, co-authorship analysis, research themes mapping, research types classification and funding sources.

Publications involving LMIC have substantially increased overtime, despite small expression of south-south collaboration. Research themes mapping revealed that publication focus varied according to collaborating countries’ income categories, from diagnosis, opportunistic infections and laboratory-based research (LMIC single or LMIC-LMIC) to human behavior and healthcare, drug therapy and mother to child transmission (LMIC-HIC). The analysis of research types showed that south-south collaborations frequently targeted social sciences issues. Funding agencies acknowledged in south-south collaboration also showed diverse focus: LMIC-based funders tended to support basic biomedical research whereas international/HIC-based funders seem to cover predominantly social sciences-oriented research.

Although the global environment has fostered an increasing participation of LMIC in collaborative learning models, south-south collaboration on HIV/AIDS prevention and/or treatment research seemed to be lower than expected, stressing the need for strategies to foster these partnerships. The evidence presented in this study can be used to strengthen a knowledge platform to inform future policy, planning and funding decisions, contributing to the development of enhanced collaboration and a priority research agenda for LMICs.

For full-text see

Author(s): Bruna de Paula Fonseca e Fonseca, Priscila Costa Albuquerque, Ed Noyons, Fabio Zicker
Organization(s): Oswaldo Cruz Foundation (Fiocruz),  Leiden University
Source: Globalization and Health
Year: 2018

Effects of innovation management system standardization on firms: evidence from text mining annual reports

Using a management formula to standardize innovation management can be thought of as deeply contradictory, however, several successful firms in Spain have been certified under the pioneer innovation management standard UNE 166002. This paper analyzes the effects that standardization has in the attitudes and values as regard to innovation for a sample of firms by text-mining their corporate disclosures. Changes in the relevance of the concepts, co-word networks and emotion analysis have been employed to conclude that the effects of certification on the corporate behavior about innovation are coincident with the open innovation and transversalization concepts that UNE 166002 promotes.

Author(s): Gaizka Garechana, Rosa Río-Belver, Iñaki Bildosola, Marisela Rodríguez Salvador
Organization(s): University of the Basque Country UPV/EHU, Tecnológico de Monterrey
Source: Scientometrics
Year: 2017

Annual reports have been text-mined using the NLP tools provided by Vantage Point software to capture the concepts occurring in the vicinity of SI terms and the changes in concepts and their relationships, in addition to emotions, have been analyzed.

The Diffusion of Military Technology

The impact of national defense research and development spending on overall innovation depends on the extent to which the knowledge and technologies generated by defense funding diffuse. This article uses an original data-set of patents assigned to defense-servicing organizations to investigate the diffusion of military technologies. Contrary to the predictions of the prevailing scholarship, I find no difference in the rate of diffusion between civilian and military technologies. Neither do military technologies assigned to government agencies diffuse at different rates than those assigned to firms. The overall technological experience of the patent assignee is found to be a positive predictor of the diffusion of military technologies. The effect of the prevailing intellectual property rights regime is ambivalent: when US patents are included in the sample, the effect of patent protection is positive, when the US is excluded, the effect is either non-significant or negative depending on the model specification that is utilized.

Author(s): Jon Schmid
Organization: Georgia Institute of Technology
Source: Defence and Peace Economics
Year: 2017

Lessons from Ten Years of Nanotechnology Bibliometric Analysis

This paper summarizes the 10-year experiences of the Program in Science, Technology, and Innovation Policy (STIP) at Georgia Institute of Technology (Georgia Tech) in support of the Center for Nanotechnology in Society at Arizona State University (CNS-ASU) in understanding, characterizing, and conveying the development of nanotechnology research and application. This work was labeled “Research and Innovation Systems Assessment” or (RISA) by CNS-ASU.

RISA concentrates on identifying and documenting quantifiable aspects of nanotechnology, including academic, commercial/industrial, and government nanoscience and nanotechnology (nanotechnologies) activity, research, and projects. RISA at CNS-ASU engaged in the first systematic attempt of its kind to define, characterize, and track a field of science and technology. A key element to RISA was the creation of a replicable approach to bibliometrically defining nanotechnology. Researchers in STIP, and beyond, could then query the resulting datasets to address topical areas ranging from basic country and regional concentrations of publications and patents, to findings about social science literature, environmental, health, and safety research and usage, to study corporate entry into nanotechnology, and to explore application areas as special interests arose. Key features of the success of the program include:

  • Having access to “large-scale” R&D abstract datasets
  • Analytical software
  • A portfolio that balances innovative long-term projects, such as webscraping to understand nanotechnology developments in small and medium-sized companies, with research characterizing the emergence of nanotechnology that more readily produces articles
  • Relationships with diverse networks of scholars and companies working in the nanotechnology science and social science domains
  • An influx of visiting researchers
  • A strong core of students with social science, as well as some programming background
  • A well-equipped facility and management by the principals through weekly problem-solving meetings, mini-deadlines, and the production journal articles rather than thick final reports.

Author(s): Jan Youtie, Alan Porter, Philip Shapira, Nils Newman
Organization: Georgia Institute of Technology
Source: OECD Blue Sky Forum on Science and Innovation Indicators
Year: 2016

Collaboration and change in the research networks of five Energy Frontier Research Centers

Emphasizing the university research center model, from 2009 to 2014 the US Department of Energy (DOE) funded a first round of over 40 Energy Frontier Research Centers (EFRCs) spread out among 100 institutions. Early in its implementation, however, the EFRC model received criticism from scholars warning that the arrangements of the EFRCs did not provide adequate governance structures for coordinating research efforts. In this article, we seek to begin answering a call for ‘systematic and rigorous study of the implementation of EFRCs’ by studying a sample of five EFRCs and their individual members. We find that despite lacking formal mechanisms for coordinating research, EFRCs increase coauthorships among EFRC members, especially new coauthorships. Moreover, EFRC members’ research quality increases after each EFRC is formed. Through negative-binomial regression analysis on individual researcher outcomes, we find that stronger preexisting networks increase coauthorship among EFRC members. This finding supports the idea that preexisting research collaboration networks are indicative of research coordination mechanisms that researchers have discovered or established for themselves prior to becoming members of a research center. We posit that new research centers may leverage research coordination mechanisms embedded in preexisting coauthorship relations, rather than imposing new research coordination mechanisms.

Author(s): Alexander M. Smith, Samson Yuxiu Lai, Jonah Bea-Taylor, Rebecca B. M. Hill and Nabil Kleinhenz
Organization: Georgia Institute of Technology
Source: Research Evaluation
Year: 2016

Science system path-dependencies and their influences: nanotechnology research in Russia

In this paper, we study the influence of path dependencies on the development of an emerging technology in a transitional economy. Our focus is the development of nanotechnology in Russia in the period between 1990 and 2012. By examining outputs, publication paths and collaboration patterns, we identify a series of factors that help to explain Russia’s limited success in leveraging its ambitious national nanotechnology initiative. The analysis highlights four path-dependent tendencies of Russian nanotechnology research: publication pathways and the gatekeeping role of the Russian Academy of Sciences; increasing geographical and institutional centralisation of nanotechnology research; limited institutional diffusion; and patterns associated with the internationalisation of Russian research. We discuss policy implications related to path dependence, nanotechnology research in Russia and to the broader reform of the Russian science system.

Full-text available

Author(s): Maria Karaulova, Abdullah Gök, Oliver Shackleton, Philip Shapira
Organization(s): National Research University Higher School of Economics, University of Manchester
Source: Scientometrics
Year: 2016

Scientometric cognitive and evaluation on smart city related construction and building journals data

In this paper, scientometrics cognitive and knowledge visualization technology were used to evaluate global scientific production and development trends in construction and building technology research of smart cities. All the data was collected from the Science Citation Index-Expanded (SCIE) database and Journal Citation Reports (JCR). The published papers from the subject of construction and building technology and their journals, authors, countries and keywords spanning over several aspects of research topics, proved that architecture/building research grew rapidly over the past 30 years, and the trend still continues in recent smart cities era. The purposed of this study were to identify the journals in the field of construction and building technology in smart city, make a comparative report on related researches, as well as propose a quality evaluation of those journals. Based on JCR and SCI paper data, the journals related to construction and building technology in smart city were assessed using ten metrics: languages, active degree, references, citation trends, main countries, leading institutes, cooperation trends, productive authors, author keywords and keywords plus. The results indicate that all the factors have great significance and are related to the impact of a journal. It also provides guidance to both evaluators and the study groups which assess journals during the process of judging or selecting research outlets, and future perspective on how to improve the impact of a paper or a journal.

Author(s): Liang-xing Su, Peng-hui Lyu , Zheng Yang, Shuai Ding, Kai-le Zhou
Organization(s): Wuhan University; Hefei University of Technology
Source: Scientometrics
Year: 2015

China’s global growth in social science research: Uncovering evidence from bibliometric analyses of SSCI publications (1978–2013)

The phenomenon of China’s rise as an emerging scientific power has been well documented, yet the development of its social science is less explored. Utilizing up-to-date Social Science Citation Index (SSCI) publication data (1978–2013), this paper probes the patterns and dynamics of China’s social science research via bibliometric analyses. Our research indicates that despite the national orientation of social science research and the linguistic obstacle of publishing for an international audience, China’s publications in the SSCI dataset have been rising in terms of volume, world share, and global ranking. But China is still not yet a major player in the arena of social sciences, as is evidenced by the number of Chinese journals indexed in SSCI and the lack of Olympic players. Team research features China’s international publishing in social science, but the research outputs are highly unbalanced at regional and institutional levels.

Author(s): Weishu Liu, Guangyuan Hu, Li Tang, and Yuandi Wang
Orgainzation(s): Shanghai Jiao Tong University, Shanghai University of Finance and Economics, and Sichuan University
Source: Journal of Informetrics
Year: 2015

Impacts of an interdisciplinary research center on participant publication and collaboration patterns: A case study of the National Institute for Mathematical and Biological Synthes

Interdisciplinary research centers are typically viewed as a vehicle for creating opportunities in science where the intricacy of the research problem calls for persistent collaboration across multiple disciplines. This case study analyzed the effects of an interdisciplinary research center on the publication and collaboration behaviors of faculty affiliated with the center. The study also sought to determine through faculty interviews what factors contributed to these effects for participants whose publication and collaboration behaviors were most changed after affiliation. Results of the study indicate that affiliation with the center has a significant positive effect on participant collaboration activities, and a moderate positive effect on publication activities (i.e. publishing in new fields). Factors contributing to success cited by interviewees included organized leadership, a positive atmosphere, breaking into sub-groups, and the ability to collaborate with researchers with whom they would not have interacted outside of the center. This case study may be useful in providing a framework for early evaluation of the effects of interdisciplinary research centers on affiliated participants.

Author(s): Pamela R Bishop, Schuyler W Huck, Bonnie H Ownley, Jennifer K Richards and Gary J Skolits
Organization(s): University of Tennessee
Source: Research Evaluation
Year: 2014

The Role of the National Science Foundation in the Origin and Evolution of Additive Manufacturing in the United States

Additive manufacturing (AM) is growing rapidly as a field of research, as well as an emerging technology with the potential to revolutionize manufacturing. Firms in the United States are a dominant player in the field, selling over 70% of the professional grade
machines to date. This is a fitting time to look at the evolution of the field with a critical eye toward determining the roles of various institutions—public funders, private entrepreneurs and inventors, universities, and others—in its development. Continue reading The Role of the National Science Foundation in the Origin and Evolution of Additive Manufacturing in the United States