Frequently Asked Questions

How does Conservation Effectiveness work?

The ConservationEffectiveness.org platform is an evolving project. It started in 2016 with the publication of a single literature review on the effectiveness of tropical forest certification as a conservation strategy, and over time became an interactive, growing, community-sourced platform of evidence on the effectiveness of several conservation strategies.

To be able to use the evidence presented on the platform or to contribute to the platform, it’s important to understand which types of studies the platform includes, how they get there, and what information from each study appears. Please see below for details specific to individual strategies, like Reforestation and Forest Restoration or Payments for Ecosystem Services.

What evidence belongs on the platform?

To be included on ConservationEffectiveness.org, a scientific publication needs to be:

  1. Peer-reviewed. This means that the publication went through an editorial process during which it was critically reviewed by one or more scientists who were not involved in the study. Each scientific journal or organization has its own rules about the peer-review process.
  2. Outcome-based. The study has to measure or otherwise describe the outcome of an actual conservation intervention. For example, in the case of Reforestation and Forest Restoration, studies need to measure the outcomes of actual projects that already planted or attempted to plant trees. Generally, we exclude modeling studies that predict what the result of a planned conservation intervention might be in the future. However, we do include modeling studies that predict outcomes that cannot easily be measured directly, but that are based on measured variables from an already implemented reforestation effort. We also include studies that use modeling to predict a counterfactual scenario: what would have happened if a conservation strategy had not been implemented.
  3. Empirical. We include empirical studies, reviews and meta-analyses that yield qualitative or quantitative evidence on the effectiveness of a conservation strategy. Excluded are expert opinions, purely theoretical studies, comments, perspectives, and op-eds.
  4. Complete. The study must present a complete suite of standard information required by most scientific journals: authors, year of publication, title, where the study was carried out, what the methods were, what it measured and how, what the outcome was. The methods have to be clear and not inherently biased.
  5. Comparative. The study has to compare the outcome of a conservation project either to a control (e.g. a reforestation project could be compared to a similar area where no active reforestation happened, which would include natural regeneration, crop field, or degraded land) or to the state of the outcome as measured before the conservation was implemented, even if this comparison is only implicit (e.g. the number of jobs generated since a community forest management project started). Please see details in “Types of evidence” below. We typically exclude studies that compare one type of conservation strategy to another. For example, we would exclude a study that compared reforestation with native species to reforestation with exotic species, unless it also provided a comparison to a control. Similarly, we exclude studies that compare the effectiveness of, for example, a protected area and a payments for ecosystem services program. Although such studies are very important, we currently do not have the framework to include them.
  6. Free from conflict of interest. We exclude studies that have a conflict of interest that is not well explained, especially in cases where methods or analyses are unclear, or where data appear to have been cherry picked. An example of a study we would exclude: An NGO report that specifies that it has been peer-reviewed but does not describe the peer-review process, and the NGO itself carried out the project being evaluated. The report mentions briefly that parts of the project failed, but the quantitative results focus on a successful subset of the project area. The NGO’s future funding may depend on showing high levels of success, and this may present a substantial conflict of interest.

How does evidence get onto the platform?

In mid-2020 we began using a new, community-sourced approach to adding data to the ConservationEffectiveness.org platform. Anyone (a “participant”) can create an account and add evidence to an existing conservation strategy. From each publication, the participant extracts the following information and enters it onto the platform: 

  1. first author
  2. study title
  3. year of publication
  4. type of evidence (see below for details)
  5. country of study site
  6. thematic group of the assessed variable (typically environmental, social, or economic)
  7. variable that is being assessed (for example, animal diversity, empowerment, carbon storage)
  8. outcome (positive, neutral, negative; see below for details)
  9. detailed outcome (brief, non-technical description of the main finding, including information on what the outcome is being compared to)

One study can yield multiple findings. For example, a publication might report that equality worsened between participants in a community forest management project, and also that the project resulted in improvements in forest cover. In such a case, two separate findings should be entered, one on equality (a negative outcome), and one on reducing deforestation (a positive outcome).

Once a participant enters a new finding onto the platform, the editor for the conservation strategy in question receives an alert and checks the entry. The editor verifies whether the publication meets the inclusion criteria (above) and whether the participant entered the information correctly. One of several possible scenarios then follows: a) The editor agrees with everything, or makes minor stylistic changes to the detailed outcome that do not change its meaning, and approves the finding for publication. The finding appears immediately on the strategy’s visualization. b) The editor sees a need to change the outcome or another information field more substantially. The editor explains the concern in the platform’s back-end comments section, and the participant receives a notification. When the participant addresses the concern, the editor checks the revised entry, and either asks for further changes or approves the finding for publication. Alternatively, c) the editor finds that the publication or finding does not meet the inclusion criteria. The editor writes a comment to the participant suggesting that the finding be removed. Once the participant has seen the comment and agrees, the editor or participant deletes the finding. In cases where the study meets all inclusion criteria, but does not fall under any existing conservation strategy currently available on the ConservationEffectiveness.org platform, the editor can “shelve” the finding until an appropriate strategy is added. Annually, the editor reevaluates the overarching “verdicts” that appear at the top of the visualization (e.g. “mostly positive change”) to make sure they reflect the accumulated studies.

Some common reasons why the editor sends findings back to participants for review:

Prior to adopting this crowd-sourced approach in mid-2020, we used a different approach to identify publications to include in the visualizations. A small team of researchers used a systematic but non-exhaustive search based on keywords relevant to each conservation strategy. You can see the keywords used for each strategy that were generated before we shifted to the crowd-sourced system. The criteria for inclusion and the process for adding findings were essentially the same as they are today.

Currently, most visualizations on ConservationEffectiveness.org represent a mix of findings added using our pre-2020 approach and our current crowd-sourced approach. Typically, when a group of researchers proposes adding a new strategy, they seed the visualization with a number of studies. For example, if a group of researchers has carried out a literature review on a conservation strategy, they might propose adding the strategy to ConservationEffectiveness.org to make their results widely available and keep them up-to-date. They could enter all of their initial findings into the database, and then let additional findings accumulate through community participation. This means that the visualization for each new strategy on the platform may initially reflect its own distinct literature search protocol, but this gradually loses influence as community-contributed studies accumulate. If you wish to add a new strategy, please contact us.

What are the caveats and potential biases of our platform?

  1. Not an exhaustive or systematic collection of evidence. This is the first time a community-sourced or crowd-sourced approach has been used to populate an evidence platform and we believe this approach has many advantages. For example, the database can stay up-to-date because findings can be added continuously. We can tap into the multi-lingual skills of the broad community of participants to add research published in different languages. However, it could also have some blind spots. For example, whole groups of publications could be missing; if, say, nobody was interested in adding findings on the economic outcomes of reforestation, these findings would be missing. We are currently evaluating such potential biases by comparing this crowd-sourced approach with a systematic search. Other platforms provide exhaustive searches: If you are interested in an exhaustive appraisal of the literature on a specific question (e.g. What is the effect of prescribed burning in temperate and boreal forest on biodiversity?”), we encourage you to check whether your question has been answered in a systematic review on the Collaboration for Environmental Evidence website. Similarly, if you are interested in more specific conservation actions (e.g. “add gypsum to soil when planting”), we encourage you to consult the Conservation Evidence platform.
  2. Not all evidence is equal. The evidence in the database has to include information on whether implementing the conservation strategy was better than, the same as, or worse than not implementing it, but not necessarily by how much. Wherever quantitative information on the outcomes is available, we encourage participants to include it in the conclusion of each finding, which appears when a reader clicks on the square representing the finding in the visualization. It is important to note that individual findings are not often comparable to one another. In other words, one red square, representing a negative finding does not cancel out one green square, representing a positive finding, and two green squares do not have “equal” value. Statements about the findings that appear on the platform such as “50% of evidence was positive” are not valid.
  3. Not all evidence is independent. Similarly, not all findings are independent from each other, as some studies yield multiple findings. For example, one study could measure both animal diversity and seedling survival. To see all findings from a particular publication, click on the publication in the list under the visualization, and the relevant squares will appear highlighted.
  4. Different types of evidence. The individual studies were carried out with different degrees of rigor and different experimental designs (see “Types of evidence,” below). This is partially captured by the evidence types we use to categorize the findings. (You can filter the findings by evidence type using a drop-down menu at the top of the visualization.)
  5. Subjectivity. When extracting information from studies, participants and editors necessarily introduce a certain amount of bias and error. For example, while it might be possible to unequivocally say which country the study was performed in, there may be several possible interpretations of the results (positive, neutral, or negative), depending on how one defines conservation success. Wherever possible, the editor encourages the participants to provide additional information so that readers can judge whether they agree with the interpretation. Each finding provides a link to its source study for readers to refer to. Moreover, if a reader disagrees with the interpretation of a certain publication, they can comment on a finding. This will notify both the participant and the editor, who can then respond and discuss potential changes with the reader via the platform. See here how to comment on a finding.

What are the different types of evidence on the platform?

One of the pieces of information that participants enter for each finding is the type of evidence, which refers to the kind of study reporting the finding. The visualizations each have a drop-down menu enabling readers to filter findings by evidence type: Case report, Study I, Study II, Study III (quasi-experimental and randomized controlled trials), Meta-analysis, or Systematic review. Additionally, Study III and Systematic reviews appear as brighter squares in the visualization because in some respects, they can be considered stronger evidence than the others. This is because they can show whether a conservation strategy actually caused an outcome, rather than just being associated, or correlated with an outcome.

Please see the details of our typology in this peer-reviewed publication. We note, however, that there is no single universally accepted typology of evidence, and no typology (including ours) can capture all the nuances of different studies.

How can I add a new finding?

By adding new findings responsibly (please consider whether you might be biased for or against a particular conservation strategy in any way), you are helping conservation practitioners access existing research and make better-informed decisions, companies assess which conservation initiatives might make sense for them, researchers discover new publications, and funders understand where knowledge gaps lie. You are also helping the publications you add gain visibility – it’s OK to add your own publications! To add a finding:

  1. Go to the "Get Involved" page, create an account, and click on "Admin" to sign in to the platform
  2. On the New Finding page:
    • In the Publications box, search for the title or author name(s) of the article. If the publication is not yet in the database, click on “Add new publication.” On this new page, enter the title, author(s), publication year, description (optional), and URL of the publication. Click on "Save Changes" when done editing and on "Go back to Add new finding." You should now be able to select the publication just added.
  3. Enter the following features using the drop-down menu:
    • Strategy – which of the strategies (e.g. Protected Areas, Payments for Ecosystem Services) is your new finding most relevant to?
    • Theme – this will determine which column your new finding shows up in (Environmental, Social, Economic).
    • Metric (variable) – this will determine under which detailed category your finding shows up (e.g. Animal diversity, Profit).
    • Valence – (1) – conservation had a positive impact on your metric, (0) – conservation had no or insignificant effect, (-1) – conservation had a negative impact.
    • Evidence Type – please see detailed description of evidence categories here.
    • Conclusion - Summarize the main finding of the publication in a maximum of 60 words using short, non-technical language that a user who is unfamiliar with the publication can understand. This text will appear when users click on your finding’s square in the visualization. If you are not sure how to phrase your conclusion, try clicking on a few existing squares in the visualization.
  4. Once you are done editing, click on "Save Changes." An editor will review your new finding and then either return it to you for further modification, or add it to the visualization. You will receive an email notification either way. If your finding needs further changes, the editor will explain these in the “Comments” section, where you can also reply to the editor.

How can I make or reply to a comment?

Should you disagree with the way a publication’s result is represented or find a mistake, you can raise the issue in the "Comments" section, which allows researchers to exchange observations and suggestions about the findings. If you want to make a comment on a finding, click on "Admin" to sign in to the platform, go to the "All Findings" tab and select the finding you want to comment on. Next, write your comment in the "Comments" section and click on "Save Changes." You can reply to comments too.

By adding new findings responsibly (please consider whether you might be biased in any way), you are helping conservation practitioners access existing research and make better informed decisions, companies assess which conservation initiatives might make sense for them, researchers discover new publications, your publication to be more visible, funders understand where knowledge gaps lie.

Are there differences in the process for the various conservation strategies?

Reforestation and Forest Restoration

To be included, studies have to investigate some form of active reforestation, typically including at least some tree planting (it doesn’t matter if the trees are planted as seedlings, directly sown as seeds or transplanted via soil bearing seeds). We exclude studies that focus on silvicultural measures, such as thinning, liana cutting, etc. even if they may led to faster forest regeneration. We exclude studies that focus on natural regeneration, as most studies we included use natural regeneration as a control. The studies can cover initiatives to replant both natural, multi-species forests and monoculture tree plantations of native and exotic species alike. We include both studies where the researchers themselves planted the trees in question and studies of reforestation projects carried out by other parties, such as NGOs, communities, or governments.

To initially seed the database, several scientists added findings according to their own snowballing techniques in which they would read a relevant publication and follow the references in the publication to find more relevant publications. We also had an entire class of undergraduate students at the University of Wisconsin Madison each add one publication of their choice as a part of their final assignment. While seeding the database in this way, we also opened it to the entire conservation community to contribute additional research on an ongoing basis. 

Forest Certification, Payments for Ecosystem Services, Community-based Forest Management, Strict Protected Areas

To find relevant publications during initial seeding of the databases, we used specific search terms on the literature search engine Google Scholar. These terms included variations on the name of the conservation strategy (for example, forest certification, payments for ecosystem services, community forest management) AND tropical forest OR Africa OR Asia OR South America AND impact OR effect* AND social OR economic OR environment. This search typically returned between 30,000 and 500,000 results, sorted by relevance. We then scanned the 1,000 most relevant titles, after which the relevance of search results became too low to justify further processing.

Marine Protected Areas

To find relevant publications during initial seeding of the database, we used specific search terms on the literature search engine Google Scholar. Our search terms included synonyms for the focal conservation strategy (“Marine Protected Area” OR “MPA” OR “Marine Nature Reserve” OR “Marine Conservation Zone” OR “Marine” AND “Protected area” OR “National Park” OR “reserve” OR “Special Area of Conservation” OR “Special Areas of Conservation” OR “Special Protection Area” OR “Nature Conservation Zone” OR “conservation zone”) as well as terms related to environmental (“Biodiversity” OR “species” OR “population” AND “abundance” OR “richness” OR “diversity” OR “assemblage” OR “threatened” OR “red list” OR “rare” OR “fish stocks” OR “habitat”), social (“community” OR “indigenous” OR “local” OR “social” OR “government” AND “participation” OR “wellbeing” OR “health” OR “conflict” OR “cohesion” OR “benefit” OR “education” OR “collaboration”) and economic (“wealth” OR “jobs” OR “economy” OR “livelihood” OR “Direct economic benefit” OR “income” OR “financial” OR “opportunity costs”). This search returned ~300,000 results, sorted by relevance. We then went through the 1,000 most relevant Google Scholar search results and found ~200 potentially relevant article titles.

Policies to combat illegal timber production and trade

Studies included in this visualization were collected during the implementation of the systematic review "Effects of EU illegal logging policy on timber-supplying countries: A systematic review", Polo Villanueva et al. 2023, Journal of Environmental Management.

Where can I find more information about the conservation strategies on ConservationEffectiveness.org?

Mongabay.com, the environmental news outlet leading ConservationEffectiveness.org with tropical forest ecologist and conservation scientist Zuzana Burivalova, has produced in-depth stories related to several of the strategies on this platform. The stories interpret the findings for the strategy in question, as they stood at the time of publication, drawing on interviews and insights from leading scientists and real-world examples. You can find them here.

Thank you for being an active member of the global conservation community.

Do get in touch if you have further questions.