Kappa values varied more strongly under the 32 conditions than PABAK values. As a general rule, PABAK values should not be interpreted as measuring the same consistency as kappa in administrative data, in particular with regard to the state of low prevalence. There is no single statistical measurement agreement that collects the information requested for the validity of administrative data. Researchers should report kappa, prevalence, positive convergence, negative concordance, and relative frequency of each cell (i.e., b, c) and d), so that the reader can assess the validity of administrative data in several aspects. Our study has at least two limitations. First, we cannot record the “true” value of Kappa for conditions in administrative data in our study. It is therefore not possible to assess the difference between the “actual” value and the estimated value of kappa and its variations due to the variation in prevalence. Second, we used diagram data that was extracted by experts as a gold standard to assess the validity of the ICD-10 data. Such a standard depends on the quality of the diagrams.
This article describes how to create an R-compliant diagram. The diagrams of the 4008 randomly selected patients were located based on the unique identifier and the date of personal registration. Two professional evaluators conducted an in-depth review of the charts of 4008 patients using the graphical cover sheet, discharge summaries, narrative summaries, pathology reports (including autopsy reports), trauma and resuscitation records, admission notes, consultation reports, operation/operative reports, anesthesia reports, the doctor`s daily progress notes, medical orders, diagnostic reports and transfer notes for the detection of one of the 32 diseases studied. The process took about an hour for each diagram. Thomsen PT, Baadsgaard NP: intra- and inter-observer agreement of a protocol for the clinical examination of dairy cows. Preventive veterinary medicine. 2006, 75 (1-2): 133-139. 10.1016/j.prevetmed.2006.02.004. Our study shows that the prevalence of diseases varies depending on the sampling method used, and these changes affect cappa and PABAK.
Although PABAK theoretically adapts to prevalence, this statistic can be high for assessing agreements between two data sources and can lead to misleading conclusions….