Issue: 2016 > May > original article

Guideline-related barriers to optimal prescription of oral anticoagulants in primary care

A.L. Beukenhorst, D.L. Arts, W. Lucassen, K.J. Jager, S.N. van der Veer
AbstractFull textPDF

Full text


Atrial fibrillation, anti-thrombotic treatment, cardiology/ standards, practice guidelines as topic, primary stroke prevention 


Prevalence of atrial fibrillation (AF), a common arrhythmia, has increased over the last 30 years.1 In 2006 its prevalence ranged from 0.7% (age group 55-59) to 17.8% (age group > 85). AF increases the risk and severity of stroke.2,3 Antithrombotic therapy with oral anticoagulation (OAC) decreases this stroke risk, but at the same time increases the risk of major bleeding.2
National and international clinical practice guidelines on AF management provide guidance on how to weigh these risks against expected benefits, and include recommendations on antithrombotic treatment.3,4 Yet, antithrombotic treatment is often not in line with these recommendations.5,6 For example, a study in the Netherlands estimated that less than 50% of AF patients received treatment according to national guidelines.6
Various types of barriers may thwart physicians in following guidelines in clinical practice, such as lack of familiarity with the guideline’s content, lack of skills or resources to change current practice or patients not reconciling with guideline recommendations.7 Suboptimal guideline features (i.e. format and content) may also hamper implementation.7,8 Lugtenberg et al. reported this as one of the barriers that hindered general practitioners (GPs) in following the Dutch College of General Practitioners (NHG) AF guideline.9
Improving these features may positively affect guideline use.8,10,11 Therefore, this study systematically appraised the format and content of the NHG AF guideline4 to identify suboptimal features that may hamper its use in Dutch primary care. We focused on the guideline section related to prescription of OACs for stroke prevention in AF patients. The results of this appraisal may contribute to improving features of future AF guideline versions, as well as to developing tools and strategies for AF guideline implementation.


The NHG AF guideline 
The NHG aims to promote evidence-based primary care by bridging the gap between theory and practice. With 12,000 members,12 they cover around 80% of all Dutch GPs13 and nurse practitioners.14 The NHG has developed over 100 guidelines covering the diagnosis and treatment of acute and chronic conditions, with ten guidelines related to cardiovascular diseases. In the current study, we reviewed the 2013-updated version of the NHG guideline on diagnosis and treatment of patients with atrial fibrillation.4 It consists of 34 pages of background information on AF, recommendations for practice, endnotes and references. We focused on three key recommendations: (I) Eligibility criteria for OACs; (II) Which type of OAC to prescribe: coumarin derivatives versus new OACs (NOACs); and (III) Type and dosage of coumarin derivatives. The recommendations are displayed in table 1.

Systematic appraisal of guideline features
To systematically appraise the features of the NHG AF guideline, we used the Appraisal of Guidelines, Research and Evaluation (AGREE) II tool15 and the GuideLine Implementability Appraisal (GLIA) tool.16 Both tools are publicly available, and have previously been used for guideline appraisals.17-19
The AGREE II tool focuses on assessing the methodological rigour and transparency with which a guideline has been developed. It contains 23 items grouped in six domains (table 2, first column). Each item reflects a statement that refers to the guideline as a whole (e.g., ‘Key recommendations are easily identifiable’), and is scored on a seven-point Likert scale, ranging from 1 (strongly disagree) to 7 (strongly agree).15
To identify obstacles to implementation of the guideline’s key recommendations on OAC prescription (table 1), we completed GLIA appraisals. GLIA consists of 21 items in eight dimensions that – in contrast to AGREE II — are scored at the level of individual recommendations (table 3, first column). Each item is formulated as a question (e.g., ‘Is justification for the recommendation stated explicitly?’) with response categories ‘yes’, ‘no’, ‘not applicable’, and ‘unsure’. We did not assess GLIA’s global dimension that appraises the guideline in its entirety and largely overlaps with the AGREE appraisal.

Data collection and analysis
Following the AGREE II and GLIA manuals,20,21 our appraisal panel consisted of four experts, representing a mix of clinical and methodological guideline expertise: one general practitioner (WL), one expert on antithrombotic treatment and stroke prevention in AF patients (DA), and two experts on guideline development and implementation (AB, SV). Panel members first individually performed the appraisals, using the online AGREE (www.agreetrust. org) and GLIA (eGLIA; tools. They also provided additional information in free text fields to explain their scores. The appraisal process was primarily informed by the guideline document itself, but when necessary, extra information was collected from: i) the NHG website; ii) the booklet on NHG guideline development procedures;22 and iii) a structured interview with two members of the NHG AF guideline development group. The appraisal coordinator (AB) then summarised the results as input for a group discussion based on which panel members could alter their scores when they considered this appropriate (e.g., to correct for available data that were overlooked during the initial appraisal). We discussed every item for which scores differed by more than one point, and every item for which the NHG development group members provided additional information.

AGREE II domain scores were calculated by summing up the individual appraisers’ scores for each item within a domain (i.e., obtained score), and then standardising this as a percentage of the possible maximum score for that domain,20 as follows:

As a result of the consensus procedure for GLIA scores, features were categorised as optimal (‘Y’ in table 3) or suboptimal (‘N’ in table 3). Per recommendation, we calculated the percentage of suboptimal features as follows:


AGREE II appraisal of overall guideline features
Appendix 1 presents the individual appraisers’ scores before and after group discussion. The group discussion resulted in 24 out of 92 (26%) scores being changed. The main reasons for appraisers to change their scores were: correction for available data that were overlooked during the initial appraisal (8 of 24 changes; 33%); a change of opinion following clarification of other appraisers’ opinion (7 of 24; 29%); correction for additional information provided by the NHG guideline development group members (6 of 24; 25%). After the group discussion, standard deviations of item scores ranged between 0 and 1.6, with the majority (75%) being 1 or lower.
Table 2 presents the final item and domain scores assigned during the AGREE appraisal of the guideline. Domain scores ranged from 52.8% for ‘Stakeholder involvement’ to 87.5% for ‘Editorial independence’. 

GLIA appraisal of key recommendation features
Table 3 displays suboptimal features at the recommendation level, which may hinder the guideline’s implementation and applicability in practice. The percentage of suboptimal features for the three key recommendations on OAC prescription (R1-3) ranged from 24% to 45%. The panel considered the domains ‘Effect on process of care’ and ‘Measurability’ optimal across all recommendations.
We found that all recommendations suffered from suboptimal decidability due to ambiguous or unclear conditions for when to apply a recommendation. For example, not all GPs may agree on what could be considered ‘high’ (R2) levels of medication adherence, or ‘relative contraindications’ for prescribing (R3 - referenced table). This, together with the lack of detail on how to prescribe NOACs (i.e., suboptimal executability of R2), led the panel to score many of the computability features as problematic. This reflects the panel’s expectation that operationalising the recommendations in an electronic information system may be difficult.
The strength of recommendations, and thus the degree to which they applied to all patients, was often unclear, resulting in a suboptimal flexibility score. This stemmed from a lack of standardisation of how recommendations were formulated. Terminology ranged from ‘absolute contraindications of NOACs are…’ (R2) to ‘Coumarin derivative dosing […] should aim’ (R3), and from ‘Recommend oral anticoagulants …’ (R1) to ‘… consider NOACs only…’ (R2). In some cases, actions were only suggested indirectly: ‘…coumarin derivatives are preferred’ (R2); ‘…a loading dose is given for the first days’ (R3). 


In this study we systematically appraised features of the NHG AF guideline to identify guideline-related barriers that may hamper optimal prescription of oral anticoagulants in Dutch primary care. The editorial independence of the guideline development group scored highly; scores for all other aspects of the guideline’s methodological quality were acceptable. At the recommendation level, the main implementation obstacles were the lack of explicit statements on the quality of the underlying evidence, lack of clarity around the strength of recommendations, and suboptimal computability hampering operationalisation of recommendations in electronic systems.
The scores for the NHG AF guideline were high in comparison with those assigned in other, similar guideline appraisal studies. For example, the systematic review of guideline appraisal studies by Alonso-Coello et al.23 summarised the methodological quality of 626 guidelines, and reported mean AGREE II scores that were lower for all six domains. The study by Sabharwal et al.17 appraised 101 cardiac clinical practice guidelines. Compared with the NHG AF guideline, they found higher mean scores for the ‘Scope and Purpose’ domain (64% and 85%, respectively) and for ‘Clarity of Presentation’ (67% versus 82%), but lower scores for the four remaining domains.

Suggestions to improve the NHG AF guideline 
1. Extend stakeholder involvement in the guideline development process
The AGREE II domain ‘Stakeholder involvement’ obtained the lowest score of all domains (53%). This was partly due to the lack of representation of a wide range of stakeholders in the guideline development group, which consisted of general practitioners and an epidemiologist. Other relevant disciplines, such as neurologists and representatives of anticoagulation clinics, were consulted at the external review stage. Yet, AGREE II and other accepted guideline standards advocate multidisciplinary development groups because they tend to generate more balanced views than single-speciality groups.24,25 Inviting representatives of other disciplines as members of the NHG guideline development group would be one way to increase multidisciplinary involvement. An alternative approach may be to consult the current panel of external reviewers earlier on in the process, for example when setting the guideline scope, selecting or rating the evidence, or when formulating the recommendations.
The low ‘Stakeholder involvement’ score also stemmed from the apparent limited extent to which patient experiences and expectations informed the NHG AF guideline. The national patient organisation was invited for external review, but it was unclear if and how their suggestions were addressed. Similar to the approaches for increased stakeholder involvement described above, patient involvement may be facilitated by having patient representation in the guideline development group, or by formal patient consultation at earlier stages of guideline development.26 Alternatively, a literature review or patient interviews could inform a guideline section summarising patient views on and experiences with antithrombotic treatment for stroke prevention.

2. Standardise layout and language of key recommendations
Although the key recommendations are clearly presented in the online summary of the guideline (https://www.nhg. org/standaarden/samenvatting/atriumfibrilleren), they remain hidden in the full text version. Using a specific font, framing them in a box, or (if possible) presenting them as a flowchart would support distinguishing key recommendations from other types of information in the guideline. Additionally, applying standardised language may help guideline users to recognise recommendations as such, whilst aligning interpretations of whether recommendations should be considered relevant for all patients. The Grading of Recommendations, Assessment, Development and Evaluation (GRADE) framework27 proposes standardised guideline language: recommendations can be either strong (level ‘1’) or weak (level ‘2’), which translate into the phrases ‘we recommend’ or ‘we suggest’, respectively. Adopting GRADE as part of the guideline development methodology also ensures systematic assessment of the quality of the underlying evidence, with the letters ‘A’ (indicating high quality) to ‘D’ (very low quality) explicitly communicating evidence quality at the recommendation level. For future versions of the AF guideline, the NHG’s updated procedure booklet (published January 2015) includes guidance on how to use the GRADE framework for assessing and summarising quality of the underlying evidence.

3. Suggest criteria for monitoring the guideline’s use in practice
Facilitating local or regional monitoring of the guideline’s use in practice requires clearly defined criteria derived from the guideline’s key recommendations. For the NHG AF guideline, examples of criteria may be ‘the percentage of female patients who are prescribed oral anticoagulants and are aged 65 years or older’, or ‘the percentage of patients on coumarin derivatives with an INR between 2.0 and 3.0’. Suggesting monitoring criteria as part of the guideline would provide a suitable starting point for developing audit and feedback, which Dutch GPs considered an encouraging strategy to improve guideline adherence.28

4. Prepare electronic implementation in parallel with the guideline development process
Recent studies have focused on providing GPs with clinical decision support systems (CDSS) to improve primary care stroke prevention in AF patients.29-31 These systems use decision rules to evaluate the current treatment of the patient and, if necessary, recommend the GP to modify it. Creating these decision rules involves translating guidelines into a format that is interpretable by a computer. The GLIA dimensions ‘Decidability’ and ‘Computability’ relate to obstacles for electronic implementation, i.e. translating guideline recommendations into actionable, computable decision rules. In the current study, the NHG AF guideline scored poorly for these dimensions, indicating presence of ambiguous terms. Although this guideline was included in CDSSs for Dutch GPs,26,29 the suboptimal computability hampered interpretation and translation of individual guideline statements into electronic decision rules. Especially the lack of clear definitions for certain contraindications and the unavailability of structured data to identify contraindications in electronic health records required input from an expert group of clinicians to fully convert the guideline into unambiguous decision rules.26 Based on this finding we suggest involving a CDSS specialist when formulating recommendations for future updates of the NHG AF guideline. By preparing electronic implementation in parallel with the guideline development process, vague and inconsistent recommendations can be identified and resolved before publication.30 This may not only improve overall implementability of the guideline in practice, but also facilitate the development of effective CDSS interventions.

In conclusion, this study provides pointers for optimising future versions of the NHG AF guideline. Future research should investigate whether applying these suggestions indeed positively affects implementation of the guideline in primary care, which in turn may improve the adequacy of antithrombotic treatment and stroke prevention in patients with atrial fibrillation.


We would like to thank Maureen van der Donk and Wim Opstelten for providing additional information on the NHG guideline development process.


Sabine van der Veer is funded by the European Renal Association – European Dialysis Transplant Association (ERA-EDTA) for a research fellowship on guideline development and implementation. Wim Lucassen is appointed member of the Authorisation Committee of the NHG.


  1. Heeringa J, van der Kuip DA, Hofman A, et al. Prevalence, incidence and lifetime risk of atrial fibrillation: the Rotterdam study. Eur Heart J. 2006;27:949-53. 
  2. Hart RG, Pearce LA, Aguilar MI. Meta-analysis: Antithrombotic therapy to prevent stroke in patients who have nonvalvular atrial fibrillation. Ann Intern Med. 2007;146:857-67. 
  3. Camm AJ, Lip GYH, De Caterina R, et al. 2012 focused update of the ESC Guidelines for the management of atrial fibrillation. Eur Heart J. 2012;33:2719-47. 
  4. NHG werkgroep atriumfibrilleren. M79 NHG-Standaard Atriumfibrilleren. 2013;56:392-401. 
  5. Nieuwlaat R, Olsson SB, Lip GYH, et al. Guideline-adherent antithrombotic treatment is associated with improved outcomes compared with undertreatment in high-risk patients with atrial fibrillation. The Euro Heart Survey on Atrial Fibrillation. Am Heart J. 2007;153:1006-12. 
  6. Arts DL, Visscher S, Opstelten W, Korevaar JC, Abu-Hanna A, van Weert HC. Frequency and risk factors for under- and over-treatment in stroke prevention for patients with non-valvular atrial fibrillation in general practice. PLoS One. 2013;8:e67806. 
  7. Cabana MD, Rand CS, Powe NR, et al. Why don’t physicians follow clinical practice guidelines? A framework for improvement. JAMA. 1999;282:1458-65. 
  8. Shekelle PG, Kravitz RL, Beart J, Marger M, Wang M, Lee M. Are nonspecific practice guidelines potentially harmful? A randomized comparison of the effect of nonspecific versus specific guidelines on physician decision making. Heal Serv Res. 2000;34:1429-48. 
  9. Lugtenberg M, Zegers-van Schaick JM, Westert GP, Burgers JS. Why don’t physicians adhere to guideline recommendations in practice? An analysis of barriers among Dutch general practitioners. Implement Sci. 2009;4:54. 
  10. Woolf S, Schünemann HJ, Eccles MP, Grimshaw JM, Shekelle P. Developing clinical practice guidelines: types of evidence and outcomes; values and economics, synthesis, grading, and presentation and deriving recommendations. Implement Sci. 2012;7:61. 
  11. Gagliardi AR, Brouwers MC, Palda VA, Lemieux-Charles L, Grimshaw JM. How can we improve guideline use? A conceptual framework of implementability. Implement Sci. 2011;6:26. 
  12. NHG. Het NHG ter introductie [Internet]. 2016 [cited 2016 Mar 16]. Available from: 
  13. Van Hassel DTP, Kaseleijn A, Kenens RJ. Cijfers uit de registratie van huisartsen. Utrecht, Nivel: 2014. 
  14. Heiligers PJM, Noordman J, Korevaar JC, et al. Kennisvraag: praktijkondersteuners in de huisartspraktijk (POH’s) klaar voor de toekomst? Utrecht: Nivel, 2012. 
  15. Brouwers MC, Kho ME, Browman GP, et al. AGREE II: advancing guideline development, reporting and evaluation in health care. Can Med Assoc J. 2010;182:E839-42. 
  16. Shiffman RN, Dixon J, Brandt C, et al. The GuideLine Implementability Appraisal (GLIA): development of an instrument to identify obstacles to guideline implementation. BMC Med Inform Decis Mak. 2005;5:23. 
  17. Sabharwal S, Patel V, Nijjer SS, et al. Guidelines in cardiac clinical practice: evaluation of their methodological quality using the AGREE II instrument. J R Soc Med. 2013;106:315-22. 
  18. Van Dijk LJ, Nelen WL, D’Hooghe TM, et al. The European Society of Human Reproduction and Embryology guideline for the diagnosis and treatment of endometriosis: an electronic guideline implementability appraisal. Implement Sci. 2011;6:7. 
  19. Nagler EV, Vanmassenhove J, van der Veer SN, et al. Diagnosis and treatment of hyponatremia: a systematic review of clinical practice guidelines and consensus statements. BMC Med. 2014;12:1. 
  20. Brouwers MEA. Appraisal of Guidelines for Research & Evaluation II. Agree Next Steps Consort. 2009;(May):1-56. 
  21. Workgroup eGLIA. eGLIA 2 instructions [Internet]. 2011. Available from: 
  22. NHG. NHG Procedure booklet [Internet]. 2010 [cited 2015 Jan 20]. Available from: uploads/procedureboek_versie_090810.pdf 
  23. Alonso-Coello P, Irfan A, Solà I, et al. The quality of clinical practice guidelines over the last two decades: a systematic review of guideline appraisal studies. Qual Saf Health Care. 2010;19:e58. 
  24. Graham R, Mancher M, Wolman DM, Greenfiled S, Steinberg E. Institute of Medicine: Clinical Practice Guidelines We Can Trust. Natl Acad Press. 2011;1-13. 
  25. Eccles MP, Grimshaw JM, Shekelle P, Schünemann HJ, Woolf S. Developing clinical practice guidelines: target audiences, identifying topics for guidelines, guideline group composition and functioning and conflicts of interest. Implement Sci. 2012;7:60. 
  26. (G-I-N) GIN. G-I-N PUBLIC Toolkit : Patient and public involvement in guidelines. [Internet]. 2012. Available from: working-groups/gin-public/toolkit 
  27. Guyatt GH, Oxman AD, Vist GE, et al. GRADE: An emerging consensus on rating quality of evidence and strength of recommendations. Chinese J Evidence-Based Med. 2009;9:8-11. 
  28. Lugtenberg M, Burgers JS, Han D, Westert GP. General practitioners’ preferences for interventions to improve guideline adherence. J Eval Clin Pract. 2014;20:820-6. 
  29. Arts DL, Abu-Hanna A, Büller HR, Peters RJG, Eslami S, van Weert HCPM. Improving stroke prevention in patients with atrial fibrillation. Trials. 2013;14:193. 
  30. Chen R, Valladares C, Corbal I, Anani N, Koch S. Early Experiences from a guideline-based computerized clinical decision support for stroke prevention in atrial fibrillation. Stud Health Technol Inform. 2013;192:244-7. 
  31. Bajorek B, Magin P, Hilmer S, Krass I. A cluster-randomized controlled trial of a computerized antithrombotic risk assessment tool to optimize stroke prevention in general practice: a study protocol. BMC Health Serv Res. 2014;14:55. 
  32. Goud R, Hasman A, Strijbis A-M, Peek N. A parallel guideline development and formalization strategy to improve the quality of clinical practice guidelines. Int J Med Inform. 2009;78:513-20.