Sources of error in scientific studies
Despite the effort to rationally assess all available information and derive logical conclusions from it, even in scientific reasoning, distortions occur as a result of systematic errors.
Especially thanks to progress in EBM it is possible to better work with potential sources of distortion (bias) in individual studies, to be healthy skeptic even about your beliefs and thus limit possible distortion of results[1].
Bias[edit | edit source]
Bias is defined as "the deviation of results or conclusions from the truth, or the sum of the processes leading to such a deviation" [2][3].
However, bias should not be confused with inaccuracy. Bias refers to systematic errors, meaning that even doing the same study multiple times would still lead to the wrong answer. In contrast, imprecision refers to an error that is introduced into the study by chance. In this case, repeating the same study multiple times would show variance in the results due to random errors, but would give the correct answer on average. [4]
Avoiding methodological bias during the creation of a scientific study is very difficult - the vast majority of studies (from laboratory experiments to cohort studies to meta-analyses) contain a certain element of bias [5].
Bias can influence the results of studies in two directions - different bias can lead to an underestimation or, conversely, an overestimation of the effect of a real intervention. Bias can also vary in magnitude: some biases are small (and trivial compared to the observed effect) and some are large (ie a particular result may be entirely due to bias). It is usually impossible to know to what extent a given bias affected the results of a particular study (even if there is good empirical evidence that specific flaws in the design, conduct and analysis of scientific studies lead to bias). The results of some studies may actually be unbiased despite methodological error, thus it is worth considering the risk of bias. This applies, among other things, especially if we are analyzing or otherwise working with data from primary studies in larger quantities (e.g. creating a systematic review or meta-analysis). [6]
Types of bias[edit | edit source]
The primary source of bias is our own beliefs and ingrained traditions of experience that distort the objectivity of the content we create. Probably the most appropriate and ubiquitous example is the so-called confirmation bias. This type of bias occurs as a result of human bias, where we tend to typically unconsciously support conclusions that match our own initial beliefs [7][8]. Another bias, which comes primarily from the individual approach of the author of the study, is the so-called reputational bias. It means that in our references we prefer to mention a person who is respected in the field or whom we know personally, and then we tend to mention him in the final summary of the bibliography at the expense of other publications [9].
Bias can then arise in one (or more) of the subsequent steps:
- initial selection of study participants (selection bias),
- study implementation
- result analysis [5].
Selection bias[edit | edit source]
Selection bias occurs when the study population (participants) is not a representative sample of the target population. Thus, the external validity is questionable and the conclusion made in the study should not be extended to other patients. An even more serious bias arises when selection bias is differential, that is, when the way patients are selected differs between two or more groups. The reason for selection bias is only a small sample of participants, the results of such a study are more random. [10][11]
Each type of study has its own "typical" risks of bias. As an example of selection decline, we will mention the assessment of the diagnostic test accuarcy (DTA), in which two indicators are important – specificity and sensitivity. Their values are directly dependent on the choice of tested participants. If only a part of the participants is included in the study (e.g. with a severe course of a certain disease), a distorted result of the effectiveness of the given diagnostic test will arise. [12][13][14]
Bias during study implementation[edit | edit source]
There are several biasesthat can arise in this part of the study creation, we will give three examples.
- Interviewer bias arises from the way in which specific information is obtained, recorded or interpreted. We use it to indicate the bias that the author (interviewer) commits if he knows in advance the course of the investigated issue (e.g. illness). If we would like to minimize this risk of bias, we will arrange for the author to be independent and impartial in the evaluation of individual results (e.g. a prospectively maintained database, or the evaluation of the results will be performed by another specialist who is not familiar with the treatment results, etc.) [15][16]
- Chronological bias arises when applying already published ("historical") studies and their participants as a comparison group for current participants undergoing a certain intervention. Bias lies in the imbalance of diagnosis, treatment, and obtaining preferred outcomes. An example can be surgical retrospective studies - if we compare the current surgical results with those 10 years ago, a bias will arise because the treatment conditions are not the same in these two groups of patients. [17]
- Transfer bias is present in the vast majority of cases in cohort studies. If the study monitors the outcome of treatment in a specific group of patients, it often happens that a part of them is not available for follow-up - i.e. that it is not possible to check the long-term result with them. The question for the authors of the studies is whether the number of these "lost" patients is too large and threatens to distort the conclusions of their study or not. [18]
Bias after the data analysis[edit | edit source]
The most common distortion we encounter in scientific studies is the so-called citation bias, also known as publication bias. This refers to the tendency where authors of scientific papers may not always be willing to publish unfavorable results of their publications. Thus, it often happens that a study with positive results is more likely to be published compared to a citation with negative results. In order to prevent this distortion, it is the duty of the authors, especially of randomized clinical and prospective cohort studies, to register their citation in a publicly available register. [19][20][21]
Assessing bias in systematic reviews and meta-analyses[edit | edit source]
As mentioned above, if we are working with multiple primary studies (and their data), it is necessary to assess the risk of bias in order to be retrospectively verifiable that these are relevant and empirically supported results. For this purpose, specific tools are used to assess methodological bias. There is a whole spectrum of them, here are a few examples of the most commonly used ones:
- Cochrane Risk of Bias Tool – a very high-quality and professionally developed tool, there are several modalities that can be applied to a given type of studies;
- QUADAS-2 – the best-known tool for evaluating the quality of diagnostic tests;
- AMSTAR – a simple tool, it is mainly used for systematic reviews containing a smaller number of analyzed studies;
- Newcastle-Ottawa Scale – it can only be used for case reports and cohort studies.
In addition to the individual methodological quality of the primary studies, the above-mentioned potential bias is also evaluated - how the literature was identified, whether the selection of relevant studies took place independently, and whether the data were accurately extracted and analyzed. [22]
Video[edit | edit source]
Summarizing video about the types of bias in medical research, the basic reasons for its occurrence and suggestions for what can be done to eliminate it.
Links[edit | edit source]
Related articles[edit | edit source]
External links[edit | edit source]
- Úvod do problematiky zkreslení od Cochrane
- Podrobný článek o zkresleních ve vědeckých studiích a jak se ho vyvarovat
- Cochrane Risk of Bias Tool
- QUADAS-2
- AMSTAR
- Newcastle-Ottawa Scale
