Visual Abstract
Abstract
Background The four-variable kidney failure risk equation (KFRE) is a well-validated tool for patients with GFR <60 ml/min per 1.73 m2 and incorporates age, sex, GFR, and urine albumin-creatinine ratio (ACR) to forecast individual risk of kidney failure. Implementing the KFRE in electronic medical records is challenging, however, due to low ACR testing in clinical practice. The aim of this study was to determine, when ACR is missing, whether to impute ACR from protein-to-creatinine ratio (PCR) or dipstick protein for use in the four-variable KFRE, or to use the three-variable KFRE, which does not require ACR.
Methods Using electronic health records from OptumLabs Data Warehouse, patients with eGFR <60 ml/min per 1.73 m2 were categorized on the basis of the availability of ACR testing within the previous 3 years. For patients missing ACR, we extracted urine PCR and dipstick protein results, comparing the discrimination of the three-variable KFRE (age, sex, GFR) with the four-variable KFRE estimated using imputed ACR from PCR and dipstick protein levels.
Results There were 976,299 patients in 39 health care organizations; 59% were women, the mean age was 72 years, and mean eGFR was 47 ml/min per 1.73 m2. The proportion with ACR testing was 19% within the previous 3 years. An additional 2% had an available PCR and 36% had a dipstick protein; the remaining 43% had no form of albuminuria testing. The four-variable KFRE had significantly better discrimination than the three-variable KFRE among patients with ACR testing, PCR testing, and urine dipstick protein levels, even with imputed ACR for the latter two groups. Calibration of the four-variable KFRE was acceptable in each group, but the three-variable equation showed systematic bias in the groups that lacked ACR or PCR testing.
Conclusions Implementation of the KFRE in electronic medical records should incorporate ACR, even if only imputed from PCR or urine dipstick protein levels.
Introduction
The kidney failure risk equation (KFRE) is a widely validated tool for estimating the absolute risk of kidney failure over 2- and 5-year time horizons in patients with GFR <60 ml/min per 1.73 m2. The four-variable KFRE incorporates age, sex, GFR, and albuminuria as assessed by the urine albumin-creatinine ratio (ACR) (1⇓⇓–4). These estimates of kidney failure can help guide clinical actions, such as referral for nephrology or vascular access (5⇓–7). Indeed, several health systems are working to implement automatic reporting of the four-variable KFRE in electronic health records (EHR).
Unfortunately, the inputs necessary for the four-variable KFRE are not always present in the EHR. eGFR is commonly assessed and usually available, along with age and sex, but albuminuria is often not. In a study from the Cleveland Clinic, 36% of patients with eGFR 30–59 ml/min per 1.73 m2 did not have any measurement of albuminuria within a year after CKD was confirmed (8). In addition, the type of albuminuria testing can vary, with urine protein-creatinine ratio (PCR), and urine dipstick protein level assessment frequently used to assess albuminuria instead of ACR. We recently developed equations to convert PCR and urine dipstick to ACR, providing a tool to screen for and stage CKD in the absence of ACR measurements (9). However, how best to estimate kidney failure risk in the setting of missing data inputs and different measures of albuminuria is uncertain (4).
Using a database of health information on more than 120 million patients, the major objectives of this study were two-fold. First, we aimed to obtain a “real-world” assessment of frequency and type of albuminuria testing available to health care organizations (HCOs) in the clinical care of patients with eGFR <60 ml/min per 1.73 m2. Second, we aimed to evaluate strategies of kidney failure risk estimation in patients missing ACR, comparing the four-variable KFRE with imputed ACR to the three-variable KFRE that does not require ACR measures, and different “look-back” periods in which to capture albuminuria measures.
Materials and Methods
Study Population
This study used deidentified data from EHRs from OptumLabs Data Warehouse (OLDW). The database contains longitudinal health information on patients, representing a diverse mixture of ages, ethnicities, and geographic regions across the United States. The EHR-derived data include a subset of EHR-derived data that has been normalized and standardized into a single database (10). For the purposes of this study, we selected data from all HCOs meeting our inclusion criteria in the EHR-derived data. The index date was considered the first date in 2013 on which eGFR was <60 ml/min per 1.73 m2, and patients were followed until 5 years after the index date. At the patient level, all patients with eGFR <60 ml/min per 1.73 m2 measured in the outpatient setting in 2013 were included. Each HCO had to have at least 50 ESKD events after 2013 in the group with an available ACR test; with these criteria, 39 HCOs were included (N=976,299) (Supplemental Appendix 1). In a sensitivity analysis, each person had to have a confirmed eGFR <60 ml/min per 1.73 m2 with at least one preceding eGFR <60 ml/min per 1.73 m2 between 90 and 730 days before the index date, to identify a population with potentially higher rates of albuminuria testing. eGFR was calculated using serum creatinine and the CKD Epidemiology creatinine equation (11). Specific information on standardization of assays for serum creatinine was not available, but standardization was routine in laboratories by 2011, before the beginning of the study period (12).
Clinical Variables and Outcomes
Demographic and clinical covariates were determined using the EHR and defined as present on or before the index date. Demographic covariates included age and sex, and clinical variables included diabetes mellitus and hypertension as defined by International Classification of Diseases ninth revision (ICD-9) clinical modification and ICD-10 codes (hypertension codes 40×, I1× and diabetes mellitus codes 250×, E10, E11, and E13). The absence of an encounter was considered the absence of disease; as such, there were no missing values for comorbidities defined by encounters. All laboratory values for ACR, PCR, and urine dipstick protein levels measurement were abstracted within 3 years of the index date.
ESKD was defined as having a current procedural terminology code (90,919–90,999) or health care common procedure coding system code (G0308–G0327, G0257, Q4081) or ICD-9 and ICD-10 code (39.95, 5A1D, 54.98, 3E1M39Z, V45.1, Z91.15, Z99.2, V56, Z49, 585.6, N18.5, N18.6, V42.0, Z94.0, T86.1, 55.69, 585.5, 0TY, 996.81) occurring within 2 or 5 years after the index date in 2013. These ICD codes include kidney transplant, stage 5 CKD, or ESKD. Also included were procedure codes for dialysis, provided there were at least two codes separated by at least 1 month and within a 3-month period. Previous studies have suggested this method is both sensitive (95%) and specific (100%) (13).
Statistical Analyses
Patient characteristics were evaluated within each OLDW HCO. We evaluated the availability of ACR tests on or before the index date in the previous 1, 2, and 3 years. We categorized patients on the basis of the presence or absence of albuminuria testing within each time period, using the guideline-recommended hierarchy of testing (14,15). Group 1 was defined as those patients with an available ACR test; group 2 included patients who had no available ACR but a PCR test; group 3 included patients with no available ACR or PCR but who had a urine dipstick protein level measurement; and group 4 had no ACR, PCR, or urine dipstick protein level measurement. Patient characteristics across these groups were examined, with age described as mean (SD) on the index date in 2013 and sex, diabetes, and hypertension summarized as proportions. Urine ACR and PCR were summarized using median and interquartile range because of skewed distributions. ESKD incidence was calculated within each center as the number of events over person-time at risk, with time at risk accruing from the index date until ESKD event, last active date, defined as the last encounter date in the EHR (defined by OptumLabs), or death.
We considered the groups formed on the availability of albuminuria testing in the 3 years before the index date as our primary analysis. We used the KFRE to estimate 5-year risk of ESKD and determined discrimination using C-statistics (3). For groups 1, 2, and 3, we compared the three-variable KFRE (which incorporates age, sex, and eGFR) with the four-variable KFRE (which incorporates age, sex, eGFR, and ACR), calculating the difference in C-statistic within each HCO and then meta-analyzing using random effects meta-analysis. Because ACR was not available in group 2 and group 3, we used the PCR and urine dipstick protein levels to estimate ACR using the following equations: for PCR (exp [5.2562+0.2445× log (min [PCR/50, 1]) +1.5531× log (max [min (PCR/500, 1) 0.1]) +1.1057× log (max [PCR/500, 1])–0.0793× (if female) +0.0802× (if diabetic) +0.1339× (if hypertensive)]), and for urine dipstick protein (exp [1.9911+0.7370× (if trace) +1.6936× (if +) +3.2783× (if ++) +4.5732× (if >++) +0.0907× (if female) +0.2789× (if diabetic) +0.3307× (if hypertensive)]) (9). For group 4, we reported only the three-variable equation. We repeated these procedures for the 5-year KFRE. Calibration was tested by plotting deciles of expected 2- and 5-year risk against observed risk and calculating the Greenwood-Nam-D’Agostino statistic within each HCO. Deciles with < five events were combined. A smaller Greenwood-Nam-D’Agostino statistic was considered an improvement in calibration and was compared between three-variable KFRE and four-variable KFRE using Wilcoxon matched-pairs signed rank test. Only HCOs with at least 50 ESKD events were included. We also evaluated the net reclassification index using clinically relevant risk categories (<10%, 10%–19%, 20%–39%, 40%+ for 2-year time horizon, corresponding risk categories for the 5-year horizon) comparing three-variable and four-variable KFRE for groups 1, 2, and 3 individually across HCOs and then meta-analyzed. All analyses were performed using Stata 15 (StataCorp 2017; StataCorp LLC, College Station, TX).
Results
Study Population
There were 976,299 patients included from 39 different HCOs (Table 1). Overall, the average age was 72 years (SD, 11 years), and the percentage of women was 59%. The mean eGFR in 2013 was 47 ml/min per 1.73 m2; 32% had diabetes, and 71% had hypertension. There was some variation across HCOs, with the average age ranging from 62 to 75 years and average eGFR ranging from 44 to 49 ml/min per 1.73 m2.
Baseline characteristics of patients with eGFR <60 ml/min per 1.73 m2, by health care organization
In the 634,259 participants with confirmed eGFR <60 ml/min per 1.73 m2, the average age was 73 years old, 60% were women, the mean eGFR was 46 ml/min per 1.73 m2, 35% had diabetes, and 76% had hypertension (Supplemental Table 1).
ESKD Incidence
There were 29,653 ESKD events over a mean follow-up of 4.3 years (Supplemental Table 2). The overall incidence of ESKD was 7.05 events per 1000 person-years. The range of ESKD incidence by HCO was 3.67 events per 1000 person-years to 30.3 events per 1000 person-years.
Frequency of ACR Testing within the Previous 3 Years
Overall, only 15% of patients had ACR testing within 1 year before the index date, 18% had testing within 2 years, and 19% had testing within 3 years (Table 2). There was variability in testing by HCOs, with the range of patients who had ACR testing within 1 year between 5% and 68%, within 2 years from 6% to 73%, and within 3 years from 6% to 74%.
Proportion with albumin-creatinine ratio testing according to “look-back” window
In the population with confirmed eGFR <60 ml/min per 1.73 m2, the proportion with ACR testing was slightly greater than that of the population with nonconfirmed eGFR <60 ml/min per 1.73 m2 (Supplemental Table 3). Of the 634,259 patients, 18% had ACR testing within 1 year, 22% had testing within 2 years, and 23% had testing within 3 years before the index date.
Characteristics of Patients On the Basis of Frequency and Type of Albuminuria Testing
Of the 976,299 participants, 188,048 (19%) had ACR testing (group 1), 16,130 (2%) had PCR testing (group 2), 354,627 (36%) had urine dipstick protein levels (group 3), and 417,494 (43%) did not have any type of albuminuria testing (group 4) in the 3 years before index date (Table 3). Mean age was similar across the groups; however, the proportion of patients with diabetes was much higher in group 1, and eGFR was much lower in group 2. Hypertension was less common in groups 3 and 4.
Characteristics of patients, on the basis frequency and type of albuminuria testing within 3 yr
Performance of the KFRE in Group 1 (ACR Testing Available)
In group 1, where ACR was available, the incidence of ESKD was 10.49 patients per 1000 person-years and median C-statistic of the four-variable KFRE across 39 HCOs was 0.895. As expected, it outperformed the three-variable KFRE in 36 HCOs, for a meta-analyzed difference in C-statistic of 0.039 (0.036–0.043) (Table 4, columns 2–3). The calibration was improved for both the 2- and 5-year KFRE using the four-variable risk equation compared with the three-variable risk equation (Supplemental Figure 1). The Greenwood-Nam-D’Agostino statistic was smaller in 16 out of 20 HCOs (P=0.001) for the 2-year and 25 out of 36 HCOs (P=0.02) for the 5-year risk equation. The meta-analyzed net reclassification index was 0.139 (95% confidence interval [95% CI], 0.108 to 0.171) and 0.159 (95% CI, 0.137 to 0.180) at 2 and 5 years, respectively (Supplemental Table 4).
C-statistics in groups with albumin-creatinine ratio, protein-to-creatinine ratio/dipstick, and no urine testing within 3 yr
Performance of the KFRE in Group 2 and 3 (PCR or Dipstick Protein Testing Available)
In group 2, where PCR was available, the incidence of ESKD was 32.0 patients per 1000 person-years and the median C-statistic of the four-variable KFRE using an imputed value for ACR across ten HCOs was 0.847. This outperformed the three-variable KFRE in nine HCOs, for a meta-analyzed difference in C-statistic of 0.022 (0.017–0.026) (Table 4, columns 4–5). Calibration was adequate for both the three-variable and four-variable KFRE (Supplemental Figure 2). The meta-analyzed net reclassification index was 0.099 (95% CI, 0.057 to 0.140) and 0.136 (95% CI, 0.105 to 0.0168) at 2 and 5 years, respectively (Supplemental Table 4).
In group 3, where only dipstick protein was available, the incidence of ESKD was 5.31 patients per 1000 person-years and the four-variable KFRE using an imputed value for ACR again outperformed the three-variable equation (difference in C-statistic, 0.012; 95% CI, 0.010 to 0.015) (Table 4, columns 6–7). Observed risk of kidney failure was similar to predicted risk for most HCOs for the four-variable KFRE but not the three-variable KFRE, which showed systematic bias, with predicted risk greater than observed risk (Supplemental Figure 3). Greenwood-Nam-D’Agostino statistic was smaller in 17 out of 21 HCOs (P=0.001) for 2 years and 28 out of 32 HCOs (P<0.0001) for 5 years. If the three-variable equation were used to classify 2-year risk, 0.4% of those with <10% predicted risk developed kidney failure, 6% of those with 10%–19% predicted risk, 14% of those with 20%–39% predicted risk, and 26% of those with 40% predicted risk. In crossclassification, using imputed ACR from dipstick and the four-variable equation resulted in lower predicted risk than the three-variable equation for most patients. Thus, the net reclassification index for events (N=3682) resulted in a negative value, with a positive value only for the net reclassification for the nonevents (N=350,945).
For group 4, where no measure of albuminuria was available, the incidence of ESKD was relatively rare, at 5.98 patients per 1000 person-years, similar to group 3. The three-variable KFRE performed fairly well, with a median C-statistic of 0.887, although predicted risk was greater than observed risks in most of HCOs, demonstrating systematic bias (Supplemental Figure 4, Tables 4 and 5, column 8).
When differences in C-statistics were meta-analyzed separately within CKD stages G3 and G4–5, the four-variable equation performed better than the three-variable equations for all groups (Supplemental Table 5).
Discussion
Efforts to incorporate the KFRE in EHRs are underway, with the goal of better informing referral to nephrology, multidisciplinary care, and transplant evaluation or vascular access creation (16⇓⇓⇓⇓–21). However, in this study of more than one million patients with eGFR <60 ml/min per 1.73 m2, only 19% of participants had the necessary ACR testing to calculate the four-variable KFRE. An additional 38% had PCR or dipstick protein available, which, with our recently published equation for the conversion of PCR or dipstick protein to ACR, may now be used to impute ACR and calculate the four-variable KFRE. We demonstrate that, for those patients with urine PCR or dipstick protein testing only, using the four-variable KFRE with imputed ACR—despite the known inaccuracy in conversion—was a better option than using the three-variable KFRE without ACR in most patients.
Although the overall ESKD risk in participants with no measures or only dipstick protein measures of albuminuria was low, the three-variable KFRE was not well calibrated in this group. Incorporating measures of dipstick protein tended to lower the KFRE risk prediction, which was useful for the large number of patients without ESKD events. In contrast, patients with events also had lower risk estimates. This may be in part due to the crude conversion: the imputed ACR value from urine dipstick protein has a maximum value of 1000 mg/g of creatinine. Our results suggest individuals with high-risk estimates using the three-variable KFRE may benefit from a more quantitative measure of albuminuria for risk assessment. In general, albuminuria testing remains underutilized, yet vitally important for prognostication.
Previous smaller studies have also reported low rates of albuminuria testing in patients with CKD. The Cleveland Clinic reported 36% of patients with CKD had no proteinuria assessed (8). A British cohort of 12,988 patients showed 36% had ACR testing over a 7-year period, and only 17% had ACR testing within the first year of registration of CKD. Of note, after ACR testing was included in the United Kingdom’s National Institute for Health and Care Excellence Quality and Outcomes Framework CKD targets and there were changes in incentives, ACR testing increased nearly 10% compared with the prior year (22,23). The reason why ACR is not tested more frequently in patients with CKD is uncertain (24⇓–26). Some suggest an unclear rationale for testing and confusion as to the appropriate method for measuring albuminuria (27,28). Although Kidney Disease: Improving Global Outcomes (KDIGO) has consistently recommended measuring ACR if possible, other guidelines have stressed ACR measurement primarily in patients with diabetes (27,29).
Our study builds on the existing evidence that ascertainment of albuminuria is central to ESKD prognosis (3,4). We found that the four-variable KFRE (with albuminuria) consistently performed better that the three-variable KFRE (without albuminuria), even when ACR was imputed from PCR or urine dipstick protein levels. Others have evaluated the contribution of albuminuria in different ways. In a study from Australia, spot PCR, spot ACR, and 24-hour urine protein assessment were all strongly associated with the composite outcome of death, ESKD, or >30% decline in eGFR (30). In a study estimating the time to ESKD, incorporating age, sex, proteinuria, and eGFR provided a better estimate compared with eGFR alone, supporting the assertion that kidney failure risk should be used in consideration of referral for a nephrology consultation or referral to transplant evaluation rather than eGFR alone (31). In a large meta-analysis of cohorts from around the world, change in albuminuria was strongly associated with risk of ESKD (32,33).
The study has several limitations that should be considered when interpreting the results. There is variability of the laboratory methods across HCOs; serum creatinine and albuminuria tests were not analyzed in a central laboratory. Laboratory data available in the EHR-derived data from OLDW are received from participating provider networks, so there may have been additional laboratory tests performed that were not present in the data. ESKD was captured by diagnostic code and not linkage to the US Renal Data System (USRDS) registry, a method that may miss patients treated outside the provider network. That said, previous studies have demonstrated high accuracy for using ICD codes to identify patients with ESKD (13). A strength of the study is the large number of participants—nearly 5% of the nation’s estimated population with eGFR <60 ml/min per 1.73 m2—from diverse age, sex, ethnic groups, and geographic locations. In this study of over a million patients with eGFR <60 ml/min per 1.73 m2, incorporating albuminuria in ESKD risk prediction resulted in consistent improvement in risk discrimination, even when ACR was imputed from PCR or urine dipstick protein levels. However, very few patients with GFR <60 ml/min per 1.73 m2 received the necessary albuminuria testing. Additional policies or incentives should be instituted to encourage albuminuria testing for all patients with CKD.
Disclosures
A. Levey reports receiving research funding from grants and contracts paid to Tufts Medical Center from the National Institutes of Health (NIH) and the National Kidney Foundation (NKF); contracts to paid from AstraZeneca (DSMB); and reports receiving honoraria from academic medical centers for visiting professorships. J. Coresh reports consultancy agreements with Healthy.io, Kaleido, and Ultragenyx; having an ownership interest in Healthy.io; reports receiving research funding from the NKF and the NIH; and reports being a scientific advisor or member of Healthy.io and NKF. M. Grams reports receiving honoraria from academic institutions for giving grand rounds and American Society of Nephrology for Young Investigator Award; reports being a scientific advisor or member of AJKD, CJASN, and the JASN Editorial Fellowship Committee, NKF Scientific Advisory Board, KDIGO Executive Committee, USRDS Scientific Advisory Board; reports having other interests/relationships with the NKF, which receives funding from Abbvie, Relypsa, and Thrasos; reports receiving travel support from Dialysis Clinics, Inc. to speak at the annual meeting and KDIGO for participation in scientific meetings and the executive committee. S. Dunning reports having an ownership interest in UnitedHealth Group. All remaining authors have nothing to disclose.
Funding
Funding for the project included support from the US National Kidney Foundation and the National Institute of Diabetes and Digestive and Kidney Diseases (R01DK100446-01 and R01DK115534).
Supplemental Material
This article contains supplemental material online at http://kidney360.asnjournals.org/lookup/suppl/doi:10.34067/KID.0005592020/-/DCSupplemental.
Supplemental Appendix 1. Selection of health care organizations for participation in this study.
Supplemental Table 1. Sensitivity analysis of baseline characteristics of patients with eGFR <60 ml/min/1.73 m2, by health system center, in confirmed eGFR <60 ml/min/1.73 m2.
Supplemental Table 2. Incidence of ESKD and death, by center.
Supplemental Table 3. Sensitivity analysis, proportion with ACR testing according to “look-back” window, in confirmed eGFR <60 ml/min/1.73 m2.
Supplemental Table 4. Cross classification of risk categories by the 3-variable KFRE and the 4-variable KFRE, by event status, in groups 1, 2, and 3.
Supplemental Table 5. Meta-analyzed difference in c-statistic between 3- and 4-variable equation in groups 1, 2, and 3 within stage G3 and stage G4-5 CKD.
Supplemental Figure 1. Calibration plot. Patients with ACR testing (group 1) within 3 years, 2-year and 5-year 3-variable and 4-variable KFRE.
Supplemental Figure 2. Calibration plot. Patients with PCR testing (group 2) within 3 years, 2-year and 5-year 3-variable and 4-variable KFRE.
Supplemental Figure 3. Calibration plot. Patients with dipstick protein testing (group 3) within 3 years, 2-year and 5-year 3-variable and 4-variable KFRE.
Supplemental Figure 4. Calibration plot. Patients without albuminuria testing (group 4) within 3 years, 2-year and 5-year 3-variable KFRE.
Author Contributions
F.S. Naranjo wrote the original draft; J. Coresh and M.E. Grams were responsible for funding acquisition; J. Coresh, M.E. Grams, and Y. Sang were responsible for methodology; M.E. Grams was responsible for conceptualization and supervision; M.E. Grams, N. Stempniewicz, and S.C. Dunning were responsible for data curation; N. Stempniewicz was responsible for resources, and software; S.C. Dunning and S.H. Ballew were responsible for project administration; Y. Sang was responsible for formal analysis; and all authors reviewed and edited the manuscript, and approved the final version of the manuscript.
- Received September 16, 2020.
- Accepted December 22, 2020.
- Copyright © 2021 by the American Society of Nephrology
References
Jump to section
More in this TOC Section
Original Investigations
Chronic Kidney Disease
Cited By...
- No citing articles found.