The science of the search

In health economics and outcomes research, evidence is everything. Behind every literature review, economic model, and value communication material lies a comprehensive and crucial process: the search for information, and at the heart of that process is the information scientist. We explore this process led by HEOR’s principal information scientist, Shelley de Kock, which is centred around designing and executing bespoke search strategies that uncover the critical evidence underpinning all our client work.

A custom search strategy for every evidence journey

Every project is different. A health technology assessment (HTA) submission, for example, requires a comprehensive and sensitive strategy that finds all possible and relevant evidence, searches a range of databases, and uses tested search filters, if available. Whereas a targeted literature review for a global value dossier (GVD), as the name suggests, will be more focused. Nonetheless, even if an approach is targeted or focused, Shelley needs to be confident that the most important studies are identified, which is why it is pivotal to understand the purpose for evidence. Thus, Shelley works closely with clients from the outset, ensuring that the search strategy is aligned with the project’s objectives, as well as timelines, and budget.

Beyond keywords: finding the evidence that hides

Sometimes identifying the right evidence for a particular need isn’t straightforward. Some concepts—like “burden” or “unmet need”—aren’t always explicitly defined in the literature. These qualitative constructs often require creative approaches to uncover. In such cases, Shelley may recommend citation analysis to trace the web of references and citations around key papers. This method has proven invaluable in global systematic literature reviews (SLRs), especially when seeking country-specific data not indexed in mainstream databases. To support this, Shelley has tested and validated several citation tools, selecting those that are industry-accepted, user-friendly, and time-efficient.

Collaboration for a clear and comprehensive strategy

Search strategy development is a collaborative process. Shelley works closely with the wider evidence review and synthesis (ERS) team—particularly at the outset of each project—to ensure that all information needs are fully captured in the search design. This collaboration involves ongoing discussion and feedback, fostering continuous refinement of methods and building confidence that the final strategy is precisely aligned with the client’s objectives.

While Shelley leads the initial design, the broader ERS team plays a critical role in reviewing and refining the strategy. The validation process includes two key steps:

1. 1. The search strategy must be able to capture relevant papers selected by the client or team member
   2. Peer review of the strategy by a knowledgeable team member who checks for errors, ensures alignment with client needs, and suggests improvements

In addition, Shelley develops training sessions and resources to ensure that all search strategies meet the highest standards of reliability and validity.

Real world challenges, real world solutions

Some projects, for example, a HTA submission, require the ERS team to work under pressurised deadlines. In these situations, to save time and costs, searches need to be specific. However, a specific search, as opposed to a sensitive search, means there is a danger of missing relevant studies. For example, a search strategy looking for a specific gene mutation cancer may miss relevant studies because the title and abstracts refer only to the broader disease population, and the gene mutation details can only be identified in the full text. When resources are pressurised, undertaking a comprehensive search which increases costs and time is hard to justify, but, at the same time, the risk of missing something critical is too risky. To ensure nothing is missed but resources are not put under pressure, Shelley conducts a supplementary search for named clinical trials identified at first pass screening, ensuring no key publications are missed. Shelley has also discovered through these supplementary searches that there can be additional author reporting inconsistencies and misspellings, which cause papers to be missed, issues that standard search strategies can’t always catch.

AI assisted, expert led

Effective search strategies require deep subject knowledge to enable recognition of nuanced indicators of relevance, such as diagnosis terms implying a disease, beyond explicit keywords. AI currently lacks the ability to fully incorporate such contextual understanding.

On the other hand, whilst AI tools like Semantic Scholar can’t yet replace the transparent, reproducible methodology required for systematic reviews, they are powerful assistants to identify key papers that can be used to inform search strategy development. Shelley uses them to:

• • Quickly gather background information
  • Identify synonyms and related terms
  • Spot errors or gaps in draft strategies

When the evidence advances, so do we

Some areas of research evolve rapidly, making it easy for systematic reviews to become outdated. That’s why Shelley implements literature surveillance throughout the project life, which enables the integration of new findings as they emerge, ensuring conclusions remain current and credible.