Introduction

Systemic inflammatory response syndrome (SIRS) is defined as an acute host reaction to various different stimuli, including both infectious and non-infectious causes. The definition of SIRS is based on physiological parameters including body temperature, heart beat rate, respiration rate (or oxygen saturation), as well as abnormalities in leukocyte counts (leukocytosis, an elevation of immature neutrophils or leukopenia) [1]. These criteria are easily applicable but also imply patients without major inflammatory disorders and are therefore not specific. In clinical routine it is of crucial importance to rapidly identify patients with SIRS due to infection (sepsis), as these patients require prompt appropriate management, as well as immediate antimicrobial therapy [2]. On the other hand, improper use of antibiotics in the hospital setting may favor the emergence of multi-resistant bacteria and may be associated with adverse drug reactions resulting in prolonged hospitalization and decreased cost efficiency [3,4,5].

On the basis of clinical criteria alone it is impossible to discriminate between septic patients and patients with SIRS due to other causes. Today, physicians often rely on classical microbiological methods, e.g. blood cultures, to identify possible infection sources. These methods, however, may need several days before results are gained. In contrast, molecular microbiological methods may provide results within hours, but require high amounts of financial as well as laboratory resources. Further, only a limited spectrum of pathogens can be detected by some of these methods. Regardless of the method used, even negative results do not exclude severe infection. In the literature, the true positive rate of blood cultures is ranked between 5–10% and a further five percent are false positives due to contamination [6,7,8]. The costs of unnecessary blood culture requests, especially when false positive are included, are substantial [9,10].

To identify infection in patients with SIRS, various studies have been performed evaluating different assessment scores or laboratory parameters. Among assessment scores, the infection probability score (IPS, range: 0–26 points) represents a prospectively evaluated score with a high negative predictive value (NPV) with which to exclude infection in severely ill patients [11]. This score is calculated using six parameters, namely heart beat rate, respiration rate, body temperature, white blood cell count (WBC), C-reactive protein (CRP), and the sequential organ failure assessment (SOFA) score [12]. Laboratory parameters in use for the rapid identification of infection include procalcitonin (PCT), interleukin 6 (IL-6), lipopolysaccharide binding protein (LBP), and CRP [13,14,15,16]. However, the clinical use of these parameters might be limited, since in literature reports on the diagnostic value of the discrimination of sepsis and SIRS vary. Additionally, assessment scores as well as sepsis parameters have been mainly evaluated in patients requiring intensive care or at emergency departments [15,16,17,18]. Data on the utility of such scores or sepsis parameters in standard care patients presenting with SIRS are rare or not available.

Thus, the present study was set out to assess the utility of the IPS and several sepsis parameters for identifying infections in standard care patients with SIRS.

Materials and Methods

Results

During the study period, a total of 2,384 patients were screened. Of these, 1,940 patients (81%) presented with less than two criteria for SIRS and 140 patients were excluded due to exclusion criteria prior to participation. After inclusion, six participants were removed from analysis since it was not possible to classify the patient correctly. These patients had a single blood culture positive for CNS without any infectious focus or suffered from invasive mycosis or parasitic infection. A total of 298 study participants showing at least two SIRS criteria were finally analyzed. Figure 1 presents information regarding the recruitment process of study participants.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Figure 1. Recruiting of the study population.

CNS = coagulase-negative staphylococci, after initial recruitment, six patients were excluded from further analysis; * patients had a single blood culture positive for CNS without any infectious focus.

https://doi.org/10.1371/journal.pone.0082946.g001

Bacterial infection was found in 216 patients (72%). Among those patients, the most common infections were blood stream infections (35%) and pneumonia (25%), followed by gastrointestinal system infections (13%) and urinary tract infections (11%). Details on the distribution of ECDC classes of patients with infections are presented in table 1. The most common pathogens isolated from blood cultures were E. coli (23%), S. aureus (17%), and K. pneumoniae (10%). Prior to analysis, five patients with CNS isolated from blood cultures and a focal infection were considered as non-bacteremic infection. SIRS due to causes other than bacterial infection was found in 82 out of the 298 patients (28%). The most frequent causes of SIRS without infection were hematological malignancies (disease or treatment-related, 31%), solid organ malignancies (16%), auto-immune diseases (10%), bleeding or embolism (10%), and cardiomyopathy (9%). The clinical characteristics of the study population are summarized in table 2. Concerning demographic parameters, no significant differences were found between SIRS patients with or without infection or between SIRS patients with or without positive blood cultures. However, SIRS patients with negative blood culture results had a higher rate of antimicrobial treatment prior to the sampling of the blood.

Prediction of infection

The median IPS among SIRS patients with infection was 16 and was thus not different to the median IPS observed in SIRS patients without infection (p = 0.769). The area under the ROC curve was 0.51 (see: Figure 2A), yielding 30.1% sensitivity, 64.6% specificity, 26.0% NPV, and 69.2% PPV. After application of the Bonferroni-Holm method, none of the individual parameters forming the IPS demonstrated a significant difference between both groups. The most discriminatory marker was CRP (p = 0.017) with an ROC-AUC of 0.59. Information on the capacity of the individual IPS parameters and sepsis biomarker to discriminate between infection and inflammation in SIRS patients is provided in tables 3 and 4.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Figure 2. ROC-curves of various parameters.

A: prediction of infection; ROC-AUCs of LBP: 0.63, PCT: 0.59, CRP: 0.59, IPS: 0.51;. B: prediction of bacteremia; ROC-AUCs of PCT: 0.78, Bili (bilirubin): 0.65, LBP: 0.62, Temp (body temperature): 0.61, IPS: 0.58;.

https://doi.org/10.1371/journal.pone.0082946.g002

Parameter	Overall	Inflammation	Infection	p-value	ROC-AUC
LBP	24.9 (15.6-37.2)	19.1 (13.6-29.3)	26.2 (17.5-26.5)	0.001*	0.63 (0.57-0.68)
PCT	0.4 (0.1-1.8)	0.3 (0.1-1.2)	0.5 (0.2-2.2)	0.014	0.59 (0.53-0.65)
CRP	14.5 (9.3-21.6)	12.8 (7.2-20.1)	15.4 (10.5-21.8)	0.017	0.59 (0.53-0.65)
SOFA	1.0 (0.0-3.0)	2.0 (0.8-3.0)	1.0 (0.0-3.0)	0.053	0.57 (0.51-0.63)
Temp1	38.5 (38.0-38.9)	38.3 (37.5-38.8)	38.5(38.1-39.0)	0.093	0.56 (0.50-0.62)
IL-6	48.5 (28.4-105.3)	39.9 (22.0-105.2)	52.3 (30.6-107.3)	0.116	0.56 (0.50-0.62)
RR2	21.0 (16.0-24.0)	21.0(16.0-24.0)	21.0 (16.0-24.0)	0.274	0.54 (0.48-0.60)
HBR3	98.0 (91.0-107.0)	97.5 (91.0-104.0)	100 (90.0-109.0)	0.392	0.53 (0.47-0.59)
WBC	9.6 (5.4-13.5)	8.8 (2.8-14.5)	9.7 (5.8-13.3)	0.435	0.53 (0.47-0.59)
IPS	16.0 (11.0-17.0)	16.0 (11.0-18.3)	16.0 (11.0-17.0)	0.769	0.51 (0.45-0.57)
Bilirubin	0.7 (0.5-1.1)	0.7 (0.5-1.0)	0.7 (0.5-1.1)	0.848	0.51 (0.45-0.57)

Table 3. Discriminatory capacities of parameters for infection.

numbers represent the median (Q₁-Q₃); 95% confidence interval of the ROC-AUC is given in parentheses; parameters are ranked in the order of their p-values; ¹body temperature, ²respiration rate, ³heart beat rate.

CSV

Download CSV

Outcome	Parameter	Sensitivity	Specificity	NPV	PPV
Sepsis	IPS	30.1 (24.0-36.7)	64.6 (53.3-74.9)	26.0 (20.1-32.6)	69.2 (58.8-78.3)
	LBP	57.5 (50.6-64.2)	67.1 (55.6-77.3)	36.8 (28.9-44.2)	82.6 (75.5-88.3)
Bacteremia	IPS	21.3 (12.7-32.3)	65.0 (58.4-71.3)	71.1 (64.3-77.2)	17.0 (10.1-26.2)
	Temp1	48.0 (36.3-59.9)	72.2 (65.8-78.0)	80.5 (74.3-85.8)	36.7 (27.2-47.1)
	PCT	86.3 (76.3-93.2)	59.6 (52.7-66.1)	92.9 (87.3-96.6)	41.5 (33.5-49.7)
	Bilirubin	79.7 (68.8-88.2)	46.2 (39.5-53.0)	87.3 (79.9-92.7)	33.0 (26.1-40.4)
	LBP	61.6 (49.5-72.8)	62.3 (55.5-68.7)	83.0 (76.4-88.4)	35.2 (26.9-44.1)

Table 4. Performance measures of the IPS and sepsis biomarkers with statistical significance.

¹ body temperature, 95% confidence interval is given in parentheses.

CSV

Download CSV

Among the biomarkers evaluated in the present study, the best parameter was LBP with a median level of 26.2 pg/ml in patients with infections and a median level of 19.1 pg/ml in those without infections. Thus, LBP was significantly higher in SIRS patients with infection when compared with the LPS of SIRS patients without infection (p = 0.001). Although the ROC-AUC was in a moderate range (0.63), it was significantly higher compared to the ROC-AUC of the IPS (DeLong test, p = 0.043). Using a cut-off value of 24.35 pg/ml (Youden-index method), LBP demonstrated 57.5% sensitivity, 67.1% specificity, 36.8% NPV, and 82.6% PPV. Concerning the other sepsis biomarkers evaluated, no significant differences were observed after correcting for errors related to multiple testing.

Discussion

In patients with SIRS, detection of infection is crucial for proper management. Since there is a lack of accurate, rapid and cost efficient diagnostic tools for the identification of septic patients, physicians are regularly faced with resulting uncertainties [22]. Moreover, there is major variation in the host´s immune response. The spectrum of the host´s immune response ranges from immunoparalysis to hyperinflammation, partly independent of the expansion of the infectious focus. Therefore, the robustness of biomarkers is pivotal for their applicability in the everyday routine [23,24].

The IPS and various sepsis biomarkers have been shown to be beneficial in the identification of infection, although the data on its clinical utility is controversial. Most studies have been conducted in critical care patients with severe disease or at emergency departments, but evaluation in standard care patients with an appropriate pre-selection in order to focus on relevant patients has rarely been performed. In addition, in the majority of studies outcome parameters were based on discharge diagnosis rather than on well evaluated and reproducible criteria.

Due to the absence of a real gold standard and a lack of an applicable SIRS classification system, surveys on patients with suspected infection are challenging. In the present study, 2,384 standard care patients with clinical suspicion of infection were consecutively screened for the occurrence of SIRS. To obtain a relevant study population, only patients with SIRS were included. IPS and sepsis biomarkers were evaluated regarding their potency to differ between SIRS patients with infection and those with SIRS due to other causes. Furthermore, the capacity to identify SIRS patients with bacteremia was assessed. Infection as the main outcome parameter was defined according to an established and robust protocol [21]. In order to minimize false positive blood culture results, patients with a possible contaminant in their blood culture and an unclear infectious focus were excluded [19,20].

Regarding the differentiation of SIRS patients with infection from those with systemic inflammation due to other reasons, the diagnostic ability of the IPS and sepsis biomarkers was poor in the present study. In fact, the IPS was developed as an infection score in severely ill patients, for which Bota et al. have shown a high NPV (89.5%) to exclude infections [11]. Likewise, other initial evaluations in severely ill patients as well as in hemato-oncological patients were promising [25,26]. In contrast to these studies, only patients at risk of infection were included in the current study, as described above. This pre-selection step led to an increased prevalence of infection and subsequently to an increased pre-test probability. The poor outcome of the IPS might be related to this alteration of the prevalence of infection, indicating low robustness of the score.

For prediction of infection in SIRS patients, no parameter displayed persuasive discriminatory capacities. Of the sepsis biomarkers, in the present study LBP was the most reliable parameter with its ROC-AUC as well as sensitivity and specificity remaining in a moderate range. According to our data, its clinical relevance regarding this differentiation setting must be questioned. However, in literature, LBP presents a better predicting power to identify infection or sepsis compared to our study [27,28]. Those studies also included patients without SIRS and were conducted at intensive care units. PCT and CRP initially showed discriminatory capacities, but were not considered to differentiate significantly after applying the Bonferroni-Holm correction for multiple testing. Their ROC-AUC curves were also in a lower range. Likewise, in other studies PCT and CRP present a better diagnostic potency compared to our study [13,28].

Regarding the prediction of bacteremia, the IPS and most of the sepsis biomarkers applied demonstrated better diagnostic abilities compared to the prediction of infection. However, after applying the Bonferroni-Holm correction, the IPS was not found to reveal statistically different results. Among its individual clinical parameters, body temperature was the best predictor of bacteremia. Nevertheless, the relevance of the temperature difference (0.1° Celsius) in SIRS patients with and without bacteremia must be questioned.

Of interest, serum bilirubin, a parameter which was analyzed to compute the IPS, presented a significant difference between patients with and without bacteremia. This finding is described in literature [29,30,31]. Hyperbilirubinemia is a risk factor, as well as a recognized complication of sepsis, which is associated with a reduction of the bile flow in hepatocytes [32,33]. To our knowledge, a systemic analysis of bile acid flow in patients with severe infections has not yet been assessed, although in 1901 Osler already described toxaemic jaundice in patients with pneumonia [34].

Among the sepsis biomarkers evaluated in the present study, PCT was the best parameter for the prediction of bacteremia. Secondarily, LBP, which was also associated with bacteremia [35,36], presented a lower diagnostic performance compared to PCT, with a ROC-AUC in a moderate range. The superiority of PCT related to other parameters is in accordance with the literature [37,38]. In recent studies, conducted at an emergency department, similar ROC curve results for the prediction of bacteremia were assessed [39,40]. In a meta-analysis including 30 studies altogether involving 3,244 patients with suspected sepsis, a mean sensitivity of 77%, and specificity of 79% for the prediction of bacteremia was shown for PCT [41]. It is noteworthy that, of the 30 studies included, only two studies had been conducted in standard care wards [42,43]. In both studies, the ROC-AUC of PCT was in a comparable range (both: 0.75).

Interestingly, widely used parameters such as CRP or WBC showed low discriminatory capacity to differentiate between systemic inflammation based on infection or other causes or bacteremia and non-bacteremia in patients with SIRS. The usefulness of CRP is the subject of some controversy [13,39,44,45,46]. Accordingly, our data suggest the routine use of CRP as a standard infection marker should be reconsidered.

Due to the complexity of the inflammatory response elicited by infectious or non-infectious stimuli, it is most likely that a combination of biomarkers is needed to improve diagnostic abilities [47]. In our study, multivariate modeling performed with logistic regression did not improve the predictive value of several parameters (data not shown). This is in accordance with Tramp et al., who reported that a combination of various biomarkers or clinical signs did not improve the diagnostic ability of PCT regarding bacteremic patients. In this survey, PCT had a sensitivity of 89%, a specificity of 58%, and a ROC-AUC of 0.80 [39]. It is likely that non-linear prediction models, including support vector machines or artificial neural networks, are be better suited for this classification task and might improve the diagnostic ability of combined analysis of parameters [48,49,50].

Low robustness might explain the lower discriminatory power of biomarkers and especially of the IPS in contrast to previous results. In IPS studies, no pre-selection of cases was done, leading to a lower pre-test probability and subsequently to a higher NPV. Therefore, our findings emphasize the need for careful validation in different patient populations [51]. In this survey a cohort study design was chosen, including only patients fulfilling two or more SIRS criteria. This leads to a higher prevalence of infection and bacteremia and subsequently to a higher pre-test probability than described elsewhere.

Conclusions

In this patient cohort consisting of standard care ward patients with SIRS, the prevalence of infection was 72%. To correctly classify the remaining 28% SIRS patients, a conclusive definition of SIRS is needed to improve the quality of such studies. The IPS and widely used biomarkers, including CRP and WBC, showed a low diagnostic performance regarding the identification of infection and bacteremia. The diagnostic abilities of sepsis biomarkers performed in a similar way. Among the sepsis parameters tested, no parameter had persuasive diagnostic capacities to identify infections in patients with SIRS. For the prediction of bacteremia, PCT was the most promising parameter. Furthermore, a change in the bilirubin pattern could indicate ongoing infection. Thus, bilirubin might be useful as a cheap screening parameter for identification of patients at risk of suffering from bacteremia. A linear combination of several parameters did not improve the diagnostic ability of PCT. Non-linear models are probably more appropriate for this classification task.

Acknowledgments

We thank Dr. Wolfgang Pöppl for critically reading and commenting on the manuscript.