- Bernstein RM, Hollingworth GR, Viner G, Miller P. "A Method of Assessment of Reliability of Coding Clinical terms to ICD-10 and ICPC Using ENCODE-FM©, a Primary Care Controlled Clinical Terminology". Journal of Informatics in Primary Care, Jan 2000.
ABSTRACT
Background: Data entry into electronic medical records remains a barrier to their use in primary care. One of the difficulties in data recording has been the use of terminologies unsuited to clinical data entry by physicians. Canada has chosen ICD-10 as its standard of classification of medical diagnoses and the World Organization of Family Doctors created and uses ICPC-2. In this study, we tested a clinical terminology for reliability of classification. ICD-10 is not intended to be used by clinicians as care is given, and ICPC is too small to be useful to follow patients in a clinical record. ENCODE-FM© is a clinical terminology specifically designed to overcome these limitations and provide both clinical specificity of health problems for patient care, and data aggregation for statistics and research. This study was intended to both test the reliability of data entry using ENCODE-FM and to serve as a model methodology for testing vocabularies in general.
Method: Terms for "reason for encounter" taken from a random selection of encounter forms in family practice were coded by 5 different physician coders using a computerized search engine for ENCODE-FM. Intraclass correlations were calculated to see how well clinical data grouped to ICD-10 and ICPC.
Results: Use of the ENCODE-FM clinical terminology resulted in highly reliable data aggregation to the standard international classifications ICD-10 and ICPC. Intraclass correlations were .87 (p<.001) and .85 (p<.001).
Interpretation: The study shows that the method of assessment is both simple and acceptable. ENCODE-FM can be used reliably for data entry into an electronic medical record, and analysis of coding errors suggests that direct data entry by care providers would be more reliable than third party coding. Physician coders prefer simple partial word searches.
ABSTRACT
The electronic medical record is an instrument to produce clinical classification in primary care. A controlled clinical terminology specifically designed for this purpose, ENCODE-FM©, was tested to determine the reliability of ICPC classification of clinical reasons for encounter. Results show a substantial concordance of ICPC coding of 83.9%. There were no encounter terms for which an adequate match in ENCODE could not be found. 91.7% of the matches between the encounter term and the ENCODE term were rated as excellent or good. A qualitative analysis of the terms with coding variability suggests that reliability of coding would be enhanced by point of service data entry as opposed to third party coding, and by specific training in the use of standardized terminologies and ICPC.
- Bernstein RM, Viner G, and Hollingworth GR. "Why Electronic Medical Records in Primary Care Fail". Canadian Organization for the Advancement of Computers in Health (COACH) Conference 22, April 1997. Scientific Program Proceedings, pp 27-35, 1996. COACH: Suite 216, 10458 Mayfield Rd., Edmonton Alberta, Canada, T5P 4P4
ABSTRACT
In spite of evidence for the effectiveness of PARTS of an electronic medical record, evidence for cost effectiveness of large integrated electronic records in primary care is lacking. There is suggestive evidence that such systems, although potentially providing better data to manage the health care system, are very difficult to implement in physicians offices. Evidence also suggests that few physicians use the computer for more than billing/accounting and that that the proportion of physicians that do is not growing. The reasons for this paradox include issues with: Data entry; Stability; Top down design; All or none phenomena; The less is more contradiction; Complexity; Flexibility vs. too many choices; Modularity; Data standards; Non verbal communication of the chart; Cost of networking; Cost of hardware and repeated upgrades; Conflicting needs of clinicians vs. those who analyse data; and Fear of failure and the cost of redundancy
- Bernstein RM, Hollingworth GR, Viner GS, "Evaluation of Controlled Medical Terminologies for use at the Point of Service in Primary Care Electronic Records". Canadian Organization for the Advancement of Computers in Health (COACH) Conference Scientific Proceedings 1996, Vol 21: 27-35.
- Bernstein RM, Hollingworth GR, and Viner GS. ENCODE-FM /CODE-MF: (Electronic Nomenclature and Classification Of Disorders and Encounters for Family Medicine/Codification Electronique pour la Medecine Familiale), Book., INSITE-Family Medicine Inc., Copyright 1997. (ISBN 0-88927-029-5).
-
Bernstein RM, Hollingworth GR and Viner G. ENCODE-FM: (Electronic Nomenclature and Classification Of Disorders and Encounters for Family Medicine): An ICPC-based Controlled Clinical Terminology for Use in Primary Care Electronic Records. in The Clinical Practice Management Network, Final Report, Feasibility Phase, PP 125-131. The College of Family Physicians of Canada, 2630 Skymark Ave., Mississauga, Ontario Canada, L4W 5A4, May, 1996. (ISBN# 1-896014-14-3)
-
Hollingworth GR, Bernstein RM, Viner GS, Remington JS, and Wood WE. "Prompting for Cost-Effective Test Ordering: A Randomized Controlled Trial". Journal of the American Medical Informatics Association, 1995, Vol 19: 635-639.
ABSTRACT
This randomized, controlled trial tests the efficacy of a computerized prompting system for test ordering. The system, makes use of the sensitivity, specificity, positive and negative predictive values of tests. It was tested using clinical vignettes in an academic family medicine center with first and second year residents. We found that there was a 38% decrease in the numbers of tests ordered (p<.01) and a 12% decrease in the costs of tests ordered by using the prompting system. We suggest that when used at the point of the patient encounter, this system has the potential for promoting more appropriate test ordering and for saving considerable health care dollars.
-
Viner G, Hollingworth GR, Bernstein R, "SIN-FM: A Short Indexed Nomenclature of Family Medicine", Journal of the American Medical Informatics Association, 1994, 18: 1032.
-
Bernstein R, Hollingworth GR, Viner G. "Prompting Physicians For Cost-Effective Test Ordering in the Low Prevalence Conditions of Family Medicine", Journal of the American Medical Informatics Association, 1994, 18: 824-828.
ABSTRACT
We have developed a computerized prompting system for test ordering which we feel will decrease the cost of investigations and at the same time promote an evidence based learning approach to test ordering. Prompting systems have been shown to be cost-effective but suffer from many disadvantages in the family practice setting. They tend to be difficult to modify by the user and contingent on an inflexible rule based structure. Many suggestions are ignored implying that they are not relevant. In family practice most conditions are of low prevalence. Prompting for test ordering where the pre-test likelihood of disease is small will result in a large number of false positives and many unnecessary repeat or confirmatory investigations and attendant anxiety unless the prompting system is specifically designed to be used in a low prevalence environment. PROMPTOR-FM (PRObabilistic Method of Prompting for Test ORdering in Family Medicine) was developed to overcome these perceived difficulties. It allows the physician to rapidly calculate the positive and negative predictive values of a test being considered based on the clinical index of suspicion. The physician is able to repeat the calculations and compare the results with previous calculations. By using PROMPTOR-FM repetitively, the clinician can learn to balance the risk of "missing" a rare but serious condition against the risk of falsely identifying disease with its downstream hazards and costs of further investigation. Prompting for test ordering is therefore uniquely tailored to each patient's situation.
-
Bernstein R, Hollingworth GR, Viner G. "Family Practice Informatics: Research Issues in Computerized Medical Records", Journal of the American Medical Informatics Association, 1993, 17: 93-97.