Perspectives on Research Data Management
17 Research Data Management and the Open Science Movement: Positions and Challenges
Cynthia Lisée and Édith Robert
Learning Outcomes
By the end of this chapter you should be able to:
- Understand the schools of thought influencing open science practices.
- Categorize the main areas of activity of open science.
- Characterize the presence of research data management practices in open science.
- Challenge the predominant discourse concerning open science.
Pre-assessment
In your opinion, what place does RDM have in open science practices?
Introduction
The international movement in favour of open science is very familiar with the interest our policymakers are taking in research data management (RDM) practices. The open science movement is helping to develop new research practices, as the excitement around research data encourages researchers to maximize their research impact through sharing results and data. However, we’d like to clarify RDM’s place in the open science movement and consider a few issues along the way. To do this, we’ll first summarize the various schools of thought that shape open science, highlight the key points and principles for developing open science practices, and note connections with RDM. Second, we will present some of the benefits attributed to open science by these schools of thought, considering them in the context of RDM. Lastly, we will question the predominant and resolutely optimistic discourse surrounding the benefits of open science. To do so, we will take a step back to reflect on the following two issues: 1) what past experiences in the open access movement have taught us; and 2) what qualitative research reveals about the relevance of this positive discourse to the sharing of research data. We will conclude this chapter by inviting RDM practitioners to consider how the elements discussed in this chapter inform current professional practices and how these elements can offer new perspectives for re-examining these practices.
Positioning RDM in Open Science
Following a conceptual analysis of a compilation of 75 rigorously selected studies, Vicente-Saez and Martinez-Fuentes (2018, p. 434) offer the following definition for open science: “Open Science is transparent and accessible knowledge that is shared and developed through collaborative networks.”
Transparency refers to sharing research results in a way that promotes their reuse. It covers all phases of the scientific research process. It implies that knowledge creation should be carried out in a way that enables it to be verified, reproduced, and reviewed by fellow researchers.
Accessible knowledge outputs are ones that are rapidly disseminated to all audiences and free of charge, usually on the Internet . These outputs can include articles, scientific opinions, data, conference communications, manuals, and software code. Accessibility also means that these knowledge outputs are easy to find.
Sharing should be considered from a transparency and access perspective: sharing should include both the intermediate stages of scientific research and the final published outputs. While sharing supports both access and transparency, access refers to the technical aspects of sharing, such as who will have access to the content, according to which security model, and whether access may be through through on-site consultation or file transfer. Transparency, on the other hand, relates to making content available to the appropriate audience for the purposes of accountability, research validation (e.g., publication of the research protocol), and knowledge sharing (e.g., pre-publication, or publishing an evaluation report).
Finally, the collaborative aspect of open science mainly involves using technologies to facilitate collaboration between scientists, but it also encompasses enabling open dialogue between nations, disciplines, and roles.
Having established these clarifications, we have adopted the above definition of open science – which is similar to other definitions shared in this textbook – as a common basis for understanding how RDM fits into open science.
Open Science Schools of Thought
The term “open science” covers a wide spectrum of practices influenced by varying perspectives. Fecher and Friesike (2014) proposed several different schools of thought to help understand the perspectives of various groups: the research community, policy makers, funding agencies, publishers, and the public. Although their literature review dates back to 2014, their analysis is still topical considering how frequently it is still cited. They summarize the developments of open science in five schools of thought.
The Public School
The public school argues that science should be accessible to citizens and that those responsible for research should communicate, and even collaborate, with the public. There are two levels of citizen interaction: making the final product comprehensible so that everyone can understand it and making the research process accessible by including the citizen.
The Democratic School
The democratic school argues that research products, such as articles, books, research data, and software code, should be freely available to everyone.
The Pragmatic School
The pragmatic school wants science to be more efficient and focuses on the development of collaborative work among scientists.
The Infrastructure School
The infrastructure school focuses its efforts on developing better, non-proprietary (where feasible) technologies and improving their interoperability to better support research. The idea is that these technologies will allow science to progress in a different way.
The Measurement School
Finally, the measurement school seeks to assess the impact of research using alternative standards that move away from more problematic bibliometric indicators (Gingras, 2014) and that take into account the digital context in which research is now conducted and published.
Schools of Thought | Examples of RDM Activities |
Public School
|
|
Democratic School
|
|
Pragmatic School
|
|
Infrastructure School
|
|
Measurement School
|
|
Each school of thought offers its own theory about developments in science, leading to activities that support a number of distinct objectives. These combined activities lead to changed practices in the conduct and administration of research and form what is called the open science movement. The following section categorizes these different areas of activity.
Open Science Areas of Activity
The Foster Open Science portal is an online learning platform covering all topics related to open science. It is intended for people who want to integrate open science practices into their work processes. It is the result of the European project, Fostering the Practical Implementation of Open Science in Horizon 2020 and Beyond, which was funded by Horizon 2020 between 2017 and 2019. In its section, “What is Open Science? Introduction,” there is a representation of the open science facets designed by Gema Bueno de la Fuente (n.d.). Open science extends its principles of openness, transparency, sharing, and collaboration into the areas of activity covering the entire research process, from its conception to its dissemination. The table below summarizes significant open science developments of these facets; we’ve added the “Open Research Protocols” facet early in the design phase to better reflect recent developments. For each facet, we’ve proposed the school of thought that seems to guide that area of activity. We’ve also provided some RDM actions to illustrate that RDM is present in all of these aspects of open science.
Research Phases |
PRACTICE |
SCHOOL OF THOUGHT |
SUMMARY | EXAMPLE OF RDM ACTION |
Conception
Dissemination
|
Open Protocols
|
Pragmatic
|
Publication of the methodology in a registry, such as OSF Registries, before starting data collection.
|
Manage data manipulation more effectively within a team with the help of transparent methodologies.
|
Open Notebooks
|
Pragmatic
|
Management of all data files to ensure the reproducibility of a research process.
|
Ensure the management of secure access to data in the active phase.
|
|
Open Data
|
Democratic
|
Sharing data in accordance with FAIR principles.
|
Choose a FAIR repository.
|
|
Open Peer Review
|
Pragmatic
|
Completely or partially waiving the anonymity of the people who do the evaluation and the writing.
|
Make the dataset available with appropriate documentation for reviewers.
|
|
Open Access
|
Democratic
|
Immediate, free access, without technical barriers, and allocation of user licenses.
|
Introduce a data availability statement in the publication.
|
|
Open Source Code
|
Infrastructure
|
Publicly funded research software and software used for research purposes should promote the technological autonomy of the scientific enterprise by using and producing open source code.
|
When sharing data, include processing and analysis codes.
|
|
Scientific Social Networks
|
Pragmatic |
Encourage networking and promoting research results.
|
Promote published datasets as research results in social networks.
|
|
Citizen Science
|
Public
|
Collaboration between those in charge of research and the public by involving the latter, possibly at all stages of the research process.
|
Train citizens in RDM practices. Their contribution will become a new source of data to be taken into account while managing the data.
|
|
Open Educational Resources (OER)
|
Public
|
Open access to scientific knowledge also involves educational practices that provide content to which everyone has access.
|
Use open data as an OER when in an educational context (Atenas & Havemann, 2015).
|
As we now understand from previous chapters, RDM practices are useful throughout all phases of a research project. It is interesting to note that RDM practices are also found throughout the different open science areas of activity. We also note that none of the emerging open science practices are directly associated with the measurement school of thought, whereas most of them are strongly influenced by the pragmatic school (four out of nine categories). Remember that the pragmatic school aims to make science more efficient, particularly by promoting collaboration.
The Benefits of RDM in Context
Open science practices are believed to have many benefits, including those illustrated in Canada’s Roadmap for Open Science. The table below includes questions about some of the open science benefits put forward by promoters of open science. Take this opportunity to reflect upon each of these questions.
Opening Science… | Predominant School of Thought | Question on the RDM Context |
Makes accountability easier | Pragmatic | Does the governing body behind the Tri-Agency Research Data Management Policy have the necessary monitoring system to enable this accountability? |
Increases the reproducibility of results | Pragmatic | How is the reproducibility of results understood in the case of qualitative research? |
Increases public trust in science | Public | How can we contribute to data literacy for citizens? |
Reduces duplication of effort | Pragmatic | How can we promote reproducibility? |
Accelerates innovation | Pragmatic | What types of data are used for what types of innovation? |
Values the diversity of knowledge systems | Public | How can marginalized knowledge be meaningfully incorporated? (e.g., OCAP® principles) |
Creates international and domestic synergies | Pragmatic | How can local elements be preserved despite the need for national or international harmonization? |
We must avoid seeing open science practices as a panacea for issues that have always existed. Even if these practices and related RDM activities help to redefine ways of doing research and pave the way for new solutions, these issues cannot truly be curtailed without taking into account the structural realities that underlie them — which open science does not do. Here are some reflections prompted by the questions in Table 3.
- Improving accountability: According to Canada’s Roadmap for Open Science, “Open Access to scientific research outputs provides greater accountability to taxpayers and research funders” (Office of the Chief Science Advisor of Canada, 2020). However, accountability requires effective RDM policy implementation from all levels of government, which seems unlikely given how little follow-up there has been by the three federal research funding agencies concerning their open access policy (Paquet et al., 2022).
- Increasing public trust in science: Trust can’t be established by simply making more data (and more articles) available. We must also work to improve the public’s data (and information) literacy. Implementing RDM practices in a vacuum, without aligning them with open science objectives and literacy issues, is unlikely to fulfill the potential to improve public trust.
- Accelerating innovation: Open science promotes the practice of sharing and reusing data that can support innovation. This is a laudable goal, especially if it includes social innovation, which we believe is both most needed by society and would most benefit from evidence-based data to inform decision-makers. However, there are methodological and epistemological challenges in producing evidence-based data in the humanities and social sciences and in developing infrastructure to enable their use by decision-makers. Canada, along with a dozen other countries, is working to establish mechanisms and information flows that would make evidence-based data accessible to decision-makers (Global Commission on Evidence to Address Societal Challenges, 2023).
Beyond the Optimistic Discourse on Opening Science
Mirowski (2018) believes that open science has its roots in a neoliberal ideology that underpins present-day science. He posits that the open science movement’s conceptualization of scientific institutions and of the nature of knowledge is driven by market imperatives rather than actual new problems in the conduct of research. For those who are less familiar with this political current, we suggest reading the article by McKeown (2022), which lists some characteristics of the neoliberal university.
In this section, we invite you to take a more critical look at these new developments by considering two issues. The first discussion seeks to draw lessons from the historical evolution of open access publishing; the second addresses the challenges related to data sharing in the context of qualitative research.
What Open Access Publishing Teaches Us
Commercial publishers have played a significant role in the evolution of scholarly communication practices in recent decades. The context of the recent COVID-19 pandemic has made it possible to demonstrate the role they could play in open access to knowledge. In 2020, there was an impressive increase in accessibility to scientific publications on coronaviruses compared to the previous two decades, and it was thanks to the cooperation of commercial publishers (Belli et al., 2020). However, it remains to be seen whether this openness will be maintained, since the strong growth was made possible through a bronze open access model, meaning many of the published articles do not have licenses guaranteeing their continued free access. Their availability is dependent on the goodwill of commercial publishers.
Types of Open Access | Definition | Free for Readers | Free for Authors |
Diamond
|
Publication in journals that offer immediate open access. Sometimes academia-controlled, immediate open access publishing initiatives supported by public funds and donations.
|
X X
|
X X
|
Hybrid
|
Some articles are released for open access upon payment of an article processing charge (APC); others require a subscription. Journals fully funded by APCs are classified as gold OA.
|
X
|
Depends on whether the author chooses to publish openly by paying an APC.
|
Bronze
|
Article made freely accessible by the publisher, but without a license guaranteeing perpetual open access.
|
X Possibly temporary
|
X
|
Green
|
Self-archiving of one of the manuscript versions in a repository.
|
X
|
X
|
Several funding bodies have come together to exert pressure on commercial journal publishers to transition their business models to open access. At the 14th Berlin Open Access Conference in 2018 (Max Planck Digital Library), organizations from 37 nations across five continents issued a joint statement of support for Plan S.[2] This is a strategy supported by a consortium of funding bodies, together named cOAlition S, which aims to make open access to publications a reality. The Fonds de recherche du Québec (FRQ) was one of the first North American funding organizations to join cOAlition S in 2021. This extensive and evolving movement in the scholarly information economy ecosystem means that libraries will eventually manage very few subscriptions. In all likelihood, subscriptions will be replaced by financial agreements with commercial publishers which will include the payment of researchers’ article processing charges by their respective institutions.
Suffice it to say that commercial publishers of scholarly journals will continue to thrive; according to a 2017 study reported by Zhang and her colleagues (2022), article processing charges are increasing at a higher rate than the consumer price index. This is a familiar echo of how increasing journal subscription costs previously put a stranglehold on academic libraries around the world, with public funds being used to pay for unsustainable price increases. Funds provided for these new financial agreements will mostly end up in the pockets of commercial publishers who control APC price increases — to the detriment of the development of diamond-type open access models. These types of open access models allow scientists to publish open access and at no cost and are more consistent with the principles of open science, because the journals are financed by public funds, university funds, or by foundations (Institut Pasteur, 2021). Diamond-type models are also in perfect harmony with the original motives of the first open access initiatives, like the Bethesda Statement and the Berlin Declaration in 2003: to give the power of the dissemination of knowledge products back to research communities.
Many people and groups participate in the scholarly information economy; some have corporate interests that are more focused on profit than on supporting actual research. Considering that data sharing as a formal part of scholarly communication (i.e., with its own publishing standards and practices) is still in its infancy, one wonders if similar economic forces aren’t seeking to shape these standards and practices and to control the underlying data infrastructures. Can proponents of new data sharing practices learn from the experience of open access?
Data Sharing and Qualitative Research
Researchers working in the field of qualitative research wonder about the impact that open science will have on publishing requirements in their discipline. This questioning stems from both the definition often attributed to research data and the trend towards open data observed in several countries. For example, the OECD Principles and Guidelines for Access to Research Data from Public Funding state that
Sharing and open access to publicly funded research data not only helps to maximize the research potential of new digital technologies and networks, but provides greater returns from the public investment in research (OECD 2007, p. 10).
Funding agencies that encourage data sharing often have a cursory definition of research data. The three federal research funding agencies (the agencies) define research data as “facts, measurements, records, or observations collected by researchers and others, with a minimum of contextual interpretation” (Government of Canada, 2023).
We will demonstrate below how data sharing and the definition of research data raise concerns for qualitative researchers.
Question of Context and Reproducibility
As shown in Table 3, one of the benefits attributed to open science is that it increases reproducibility. However, members of the qualitative research community maintain that research context is vital and should be considered before a project’s research results can be reproduced. From the positivist perspective of the natural sciences or biomedical sector, data is generally considered to be context-free, as the definition from the three federal research funding agencies (above) states. On the other hand, in qualitative research, which often uses a constructivist perspective, the context is inseparable from the research question (Hesse, 2018, p. 566). As such, the issue of the reproducibility of results cannot be addressed in the same way as it is in the pure sciences. With these considerations in mind, how can data from qualitative research projects be shared and reused? In practical terms, will it be possible, with shared data, to take the context of data production into account?
Myth of Raw Data and Neutral Data
In the context of qualitative research, it is important to be aware that shared data will have previously been the subject of interpretation. Regardless of the discipline, a dataset is a construction that cannot be abstracted from the people who created it. Before being deposited in a repository, the dataset was the subject of deliberations, negotiations, and decisions of inclusion and exclusion that are well anchored in predominant discourses as well as historical and socioeconomic realities. Therefore, it is impossible to claim that shared data is neutral (Neff et al., 2017). The importance of dataset documentation is therefore clear, but documentation nonetheless does not capture tacit knowledge that is invaluable for understanding a dataset. From this perspective, data sharing appears to be an eminently complex exercise.
Promotion of Particular Types of Research and Prioritization of Methodologies
According to the OECD, for a dataset to be shareable, it must meet certain criteria, including, ideally, being available digitally (OECD, 2007). The digital nature of data can lead to promoting the use of big data, since these large datasets, produced quickly and in a variety of formats, are increasingly available and easy to access. The emphasis being put on research involving massive datasets raises the risk that qualitative methodologies will be subordinated to quantitative ones. In addition, a shift could occur so that techniques typically used to analyze qualitative data will only serve to confirm the results provided through quantitative methods (Hesse et al., 2018). Finally, Hesse et al. also report the fear that research using small datasets will receive less recognition than research involving large data collections (2018).
Conclusion: Being Open About Open Science
We have seen how RDM activities permeate open science practices, and we’ve discussed how the predominant, enthusiastic, and resolutely optimistic discourse on the adoption of these open science proposals overlooks the complexity of the real world issues they purport to solve. The limited space for this chapter and the overall educational purpose of this textbook prevent an in-depth treatment of the ideological foundations of this call to open science. However, it is interesting to note that the concerns raised by these new practices have produced a new field of study: critical data studies. This recent area of research offers possible solutions for RDM practices that take different disciplinary norms into account. More specifically, since qualitative research is being pushed to change before important disciplinary consensuses have emerged, we believe that using critical data studies to analyze professional practices relating to RDM will help prevent qualitative research communities from being subsumed by more dominant research cultures.
A critical or socio-political approach to interpreting open science developments would make it easier to step back and shed new light on the enthusiastic discourse surrounding the open science movement and its practices. We are delighted to conclude this chapter by inviting RDM practitioners to take accountability in their professional practices by digging into the discourse of this vast open science movement to try to develop potential answers to the following questions: Which economic and political systems are producing social structures, values, norms, ideologies, goods, and financial products? For whom? With what technologies and why those technologies? Where in all of this are the open science infrastructures situated? And who benefits from the opening of science?
Reflective Questions
- Compare the definition of open science in the Foster Open Science portal with the one proposed in this chapter by Vicente-Saez and Martinez-Fuente. What differences and similarities can you identify?Foster Open Science definition: Open Science is the practice of science in such a way that others can collaborate and contribute, where research data, lab notes and other research processes are freely available, under terms that enable reuse, redistribution and reproduction of the research and its underlying data and methods.
- By which school(s) of thought do you think RDM is mainly influenced ?
- True or false: Considering how open access publishing has developed, there is no reason to worry that a few companies with commercial interests will build an oligopoly on products that facilitate the exploitation of research data.
- Why should the question of research reproducibility be addressed differently in qualitative research than in the pure sciences?
- What new area of research would enable you to gain a more critical perspective on RDM practices?
View Solutions for answers.
Key Takeaways
- Five schools of thought shape open science practices: the public school, the democratic school, the pragmatic school, the infrastructure school, and the measurement school.
- Open science practices can be categorized into nine major sectors of activity affecting all stages of a research project, from its conception to its dissemination: openness of research protocols, use of electronic notebooks, open data, open peer review processes, open access, open source code, scientific social networks, citizen science, and open educational resources.
- In the current structure of open access publishing, public funds are still largely allocated to commercial publishers, and the question remains as to whether current and future open science infrastructures could be subject to the same oligopolistic risk.
- The practices of opening and sharing research data present epistemological challenges in the fields of humanities and social sciences and in qualitative research methodologies. These include the complexity of sharing qualitative research data, the prioritization of certain research methodologies, and the impossibility of neutral data.
Additional Readings and Resources
- Foster Open Science portal, an online learning platform covering all open science topics
- Iwasiński, Łukasz. (2020). Theoretical Bases of Critical Data Studies. Teoretyczne podstawy critical data studies., 115A(1A), 96-109.
Reference List
Belli, S., Mugnaini, R., Baltà, J., & Abadal, E. (2020). Coronavirus mapping in scientific publications: When science advances rapidly and collectively, is access to this knowledge open to society? Scientometrics, 124(3), 2661-2685. https://doi.org/10.1007/s11192-020-03590-7
Bueno de la Fuente, G. (n.d.). What is open science? Introduction. Foster Open Science. https://web.archive.org/web/20181229190240/https:/www.fosteropenscience.eu/content/what-open-science-introduction
Coalition S. (n.d.) About Plan S. https://www.coalition-s.org/
Global Commission on Evidence to Address Societal Challenges (2023). Strengthen domestic evidence-support systems. https://www.mcmasterforum.org/networks/evidence-commission/domestic-evidence-support-systems
Fecher, B., & Friesike, S. (2014). Open science: One term, five schools of thought. In Bartling S. and Friesike S. (Eds.) Opening science: The evolving guide on how the internet is changing research, collaboration and scholarly publishing (pp. 17-47). Springer. https://doi.org/10.1007/978-3-319-00026-8_2
Gingras, Y. (2014). Les dérives de l’évaluation de la recherche. Du bon usage de la bibliométrie. Raisons d’agir.
Government of Canada. (2023, April). Tri-Agency research data management policy – Frequently asked questions. https://science.gc.ca/site/science/fr/financement-interorganismes-recherche/politiques-lignes-directrices/gestion-donnees-recherche/politique-trois-organismes-gestion-donnees-recherche-foire-aux-questions#1a
Hesse, A., Glenna, L., Hinrichs, C., Chiles, R., & Sachs, C. (2019). Qualitative research ethics in the big data era. American Behavioral Scientist, 63(5), 560–583. https://doi.org/10.1177/0002764218805806
Institut Pasteur. (2021, April 23). La voie diamant de l’Open Access. Open science: évolutions, enjeux et pratiques. https://openscience.pasteur.fr/2021/04/23/la-voie-diamant-de-lopen-access/
McKeown, M. (2022). The view from below: How the neoliberal academy is shaping contemporary political theory. Society, 59(2), 99-109. https://doi.org/10.1007/s12115-022-00705-z
Mirowski, P. (2018). The future(s) of open science. Social Studies of Science, 48(2), 171203. https://doi.org/10.1177/0306312718772086
Neff G., Tanweer A., Fiore-Gartland B., & Osburn L. (2017). Critique and Contribute: A Practice-Based Framework for Improving Critical Data Studies and Data Science. Big Data. 5(2), 85-97. https://doi.org/10.1089/big.2016.0050
OECD – The Organization for Economic Cooperation and Development. (2007). OECD principles and guidelines for access to research data from public funding. https://www.oecd.org/sti/inno/38500813.pdf
Office of the Chief Science Advisor of Canada. (2020). Roadmap for Open Science. Government of Canada. https://science.gc.ca/site/science/en/office-chief-science-advisor/open-science/roadmap-open-science
Paquet, V., van Bellen, S., & Larivière, V. (2022). Measuring the prevalence of open access in Canada: A national comparison. The Canadian Journal of Information and Library Science / La Revue canadienne des sciences de l’information et de bibliothéconomie, 45(1), 1–21. https://doi.org/10.5206/cjilsrcsib.v45i1.14149
SPOR Evidence Alliance. (2021). Vaccine Effectiveness Over Time in Vaccinated Individuals: A Living Review.
https://sporevidencealliance.ca/wp-content/uploads/2021/10/COVIDEND_MBMC_rapidreview_VE_infographic_final.pdf
Vicente-Saez, R., & Martinez-Fuentes, C. (2018). Open science now: A systematic literature review for an integrated definition. Journal of Business Research, 88, 428-436. https://doi.org/10.1016/j.jbusres.2017.12.043
Zhang, L., Wei, Y., Huang, Y., & Sivertsen, G. (2022). Should open access lead to closed research? The trends towards paying to perform research. Scientometrics, 127, 7653–7679. https://doi.org/10.1007/s11192-022-04407-5
- Other infographics are available on the COVID-END site, Scan evidence products, https://www.mcmasterforum.org/networks/covid-end/covid-end-evidence-syntheses/scan-evidence-products ↵
- cOAlition S defines Plan S as follows: “Plan S is an initiative for Open Access publishing that was launched in September 2018. The plan is supported by cOAlition S, an international consortium of research funding and performing organizations. Plan S requires that, from 2021, scientific publications that result from research funded by public grants must be published in compliant Open Access journals or platforms” (Coalition S, n.d.). ↵
the free, immediate, online availability of information coupled with the rights to use this information fully in the digital environment.
not owned by a company.
the ability of data or tools from non-cooperating resources to work with or communicate with each other with minimal effort using a common language.
online, free of cost, accessible data that can be used, reused, and distributed provided that the data source is attributed.
guiding principles to ensure that machines and humans can easily discover, access, interoperate, and properly reuse information. They ensure that information is findable, accessible, interoperable, and reusable.
a policy applying to data collected with research funding from one of Canada's three federal funding agencies. The policy is intended to encourage better research by requiring researchers to create data management plans and preserve their data.
data about data; data that define and describe the characteristics of other data.
online tools built off the design and use of paper lab notebooks
when software is open source, users are permitted to inspect, use, modify, improve, and redistribute the underlying code. Many programmers use the MIT License when publishing their code, which includes the requirement that all subsequent iterations of the software include the MIT license as well.
evidence-based data comes in a variety of forms and is the result of some form of research activity, including data analysis, modeling, literature syntheses, and evaluations that produce guidelines and assessments of the implementation of a process or technology and its cost-effectiveness.
a publication fee charged to authors or their institutions for making their work open access.
the Natural Sciences and Engineering Research Council of Canada (NSERC), the Social Sciences and Humanities Research Council of Canada (SSHRC), and the Canadian Institutes of Health Research (CIHR) (the agencies) are Canada’s three federal research funding agencies and the source of a large share of the government money available to fund research in Canada.