OUP user menu

Successful novice's training in obtaining accurate assessment of carotid IMT using an automated ultrasound system

Davide Vanoli , Urban Wiklund , Per Lindqvist , Michael Henein , Ulf Näslund
DOI: http://dx.doi.org/10.1093/ehjci/jet254 637-642 First published online: 30 December 2013

Abstract

Aims The aim of this study was to assess the feasibility and learning curve of training novice operators in using automated ultrasound to achieve satisfactory carotid intima-media thickness (CIMT) measurements.

Methods and results Four novices underwent 4 weeks carotid ultrasound training using a newly developed automated ultrasonograph. A longitudinal B-mode image of the distal right common carotid artery (CCA) was acquired in 96 patients. The interoperator CIMT reproducibility was analysed by the coefficient of variation (CV) and intraclass correlation coefficient (ICC) for every week and compared with that from an expert operator. The weekly mean CV of the measurements on the 24 patients made by all novices was consistently reduced: 0.06, 0.05, 0.03, and 0.02, respectively. For the expert, the mean CV was 0.02, 0.02, 0.03, and 0.02, respectively. The novices' standard deviation (SD) of CVs also reduced weekly from 0.04 in the first week to 0.01 in the last week (P < 0.05). The corresponding weekly variation in the SD for the expert was 0.02 for the first week to 0.01 in the last week (P = 0.27). The agreement between measurements made by the novices was expressed by the ICC being 0.97 (P < 0.001) in the first week and increased to 0.99 (P < 0.001) in the fourth week.

Conclusion CIMT assessment by novices using an automated ultrasound could be reliably achievable after a short training period. These results may have encouraging implications when designing screening programmes for primary prevention in community health service.

  • Carotid artery
  • Intima-media thickness
  • Health screening
  • Vascular ultrasound

Introduction

Carotid intima-media thickness (CIMT) measurements have been applied for the assessment of early atherosclerosis since the late 1980s, having started as a research tool but now been recommended for regular clinical use.13 A review of eight epidemiological studies4 showed that the CIMT of common artery or combined with internal had an independent predictive power for cardiovascular events, and hence the current guidelines recommendation for routine clinical application in asymptomatic patients.5 Despite that, such service is currently limited to research studies and selected academic institutions, because of the historical need for special expertise in image acquisition and analysis. On the other hand, CIMT is well recognized as an early manifestation of arterial wall disease, being the most superficial and the easiest accessible artery for disease monitoring. Therefore, optimizing the quality of CIMT acquisition in order to improve its reproducibility for early disease detection is highly desirable, particularly in the setting of community health service. A logistical strategy to achieve that is to avoid dependence on ultrasound department operators and to encourage novices to obtain a skill, limited to a specific target. We aimed, therefore, in this study to assess the feasibility and reproducibility of training a small group of novices in acquiring the skill of obtaining satisfactory results of CIMT measurements using an automated ultrasound system.

Methods

Population

This study included 96 patients (24 patients per week) consecutively recruited from those referred to the Clinical Physiology Department, Heart Centre, Umeå University Hospital, from other clinics or general practitioners for a carotid ultrasound examination. Five operators performed all measurements. Four of them were novices without any experience in performing carotid examinations, they were enrolled among the personnel of the department. The fifth operator was an expert with >250 self performed CIMT scans.6 The subjects were divided into groups of six individuals per novice per week. The study protocol and the written participant information were approved by the Ethics Committee of Umeå University. Before the carotid examination, the study protocol was explained to the patients, who all gave informed consent prior to participation in the study. Candidates' risk assessment for atherosclerosis was carefully assessed including weight, height, smoking habits, family history of heart/vascular disease, diabetes, and dyslipidaemia. Body mass index (BMI) was calculated as weight in kilograms divided by the square height in metres. Participants with BMI of >35 kg/m2 were excluded. Subjects with previously diagnosed carotid stenosis, carotid endo-arterectomy, carotid aneurysms, atrial fibrillation, or pacemaker treatment were excluded. Blood pressure was measured with the patient in the supine position after 5–10 min rest prior to the vascular ultrasound examination using a standard mercury sphygmomanometer.

Carotid ultrasound

The four novices initially received an hour of theoretical tutorial consisting of a presentation illustrating the anatomy of the neck with particular emphasis on the carotid vessels, the basic principles of ultrasound physics, and a short illustration of scanning techniques and protocols. This was followed by a practical 2-h training using a newly developed automated radio frequency (RF)-based ultrasonograph (CardioHealth Station®, Panasonic Healthcare Co. Ltd, Yokohama, Japan) equipped with a 9.0 MHz linear array transducer. The principles of RF signal-based ultrasonography and main features of the automated system have been previously described.7,8 During this short training, the novices acquired the basic skills necessary to operate the probe and to produce longitudinal and cross-sectional views of the CCA, carotid bulb, internal carotid artery, and external carotid artery.

The study was performed over the course of 4 weeks. In the first week, each novice performed an ultrasound scan of the right CCA (acquiring the CIMT on the vessel in two different scans taken 1 min apart from each other) in a group of six patients. The same protocol was repeated for the subsequent weeks. Each time the novice was not allowed to review the previous scan (Figure 1). The distal centimetre of the far wall of the right CCA was visualized from an anterior approach according to the current recommendations.911 The vessel motion detector system identified end-diastole based on the change in arterial diameter during the cardiac cycle. The auto-region of interest (ROI) tool automatically identified the interface lumen-intima of the posterior wall of the vessel where it was positioned. The mean and maximum CIMT values were automatically measured by the system and registered in an Excel-file by a secretary. The novices were not aware of which numerical information, among the many items that appear simultaneously on the screen, corresponds to the CIMT. The maximum value of CIMT was defined as ‘CIMT max’, and the average mean value as ‘CIMT mean’. All images were stored as JPEG images. At the end of each examination, the carotid scan was repeated by an experienced operator, blinded to the novices' results, using the same device, protocol, and system settings.

Figure 1

Longitudinal view of CCA and carotid bifurcation. The rectangular box corresponds to the ROI placed within the last centimetre of the CCA. The two dotted lines represent the automatically traced IMT on the far wall. The continuous green line under the ROI represents a visual indicator of signal quality. On the left of the screen shot, an arrow indicates the maximum, mean, and minimum values of the IMT. A double arrow indicates the angle of insonation that is visually and numerically represented.

Data analysis

Statistical analysis was carried out using the Matlab software (MathWorks, Inc., Natick, MA, USA). Numeric values are stated as mean ± standard deviation (SD). Inter- and intraoperator reproducibility was analysed by the intraclass correlation coefficient (ICC), calculated from a one-way random model with subjects as a random factor, where an ICC of >0.75 was used as an indicator of excellent agreement.12 An intraoperator ICC was calculated based on the two measurements made on each patient by each observer. The interoperator ICC was calculated as the mean of the two measurements on each patient, compared between the expert and the novice (based on 4 novices × 6 patients = 24 values every week). The coefficient of variation (CV) for the intraoperator variability was calculated as CV for each patient, based on the two measurements of CIMT mean made by each observer on each patient having taken these two measurements each time on different acquisitions. The average value of CV was also determined for each week. Differences were characterized by Bland–Altman plots constructed from the mean of the two measurements made by the expert and the novices (whole group) on each patient, then the average of the expert and novice was calculated and plotted on the x-axis, while the difference between the readings by the expert and the novices was calculated and plotted on the y-axis, where the non-parametric Wilcoxon test was used to test if the bias was different from zero. A P-value of <0.05 was considered as significant. Weekly averages of CVs were compared using the non-parametric Friedman test.

Results

Subjects' characteristics are summarized in Table 1. All four novices completed the 4-week training programme.

View this table:
Table 1

Subjects characteristics

Variable
Gender (male/female)61/35a
Age, years44 ± 14.9b
Weight, kg69 ± 12.2b
Height, cm169 ± 8.7b
Hypertension, %18
Diabetes, %1
Hypercholesterolaemia, %5
Smoker, %1
  • aAbsolute number.

  • bMean ± SD.

Feasibility

During the first week, Novice 1 was unable to achieve the CIMT measurement in 2 of 6 (33%) subjects, and Novice 4 in 1 of 6 (17%). During the second week, Novice 2 was unable to achieve the CIMT measurement in 3 of 6 (50%) subjects, Novices 3 and 4 in 1 (17%) subject each. During the fourth week, all novices were able to achieve the CIMT measurement (100%). For the expert, the feasibility was 83% in the first week and 100% in the fourth week.

For Novice 1, the maximum value of CIMT ranged between 1.150 and 0.420 (mean 0.697 ± 0.163) mm, and the mean value ranged from 1.000 to 0.390 (0.625 ± 0.161) mm. For Novice 2, the maximum value of CIMT ranged from 0.890 to 0.380 (0.596 ± 0.125) mm, and the mean value of CIMT ranged from 0.740 to 0.350 (0.524 ± 0.099) mm. For Novice 3, the maximum value of the CIMT ranged from 1.600 to 0.420 (0.703 ± 0.224) mm, and the mean value of CIMT ranged from 1.300 to 0.380 (0.608 ± 0.166) mm. For Novice 4, the maximum value of CIMT ranged from 0.850 to 0.460 (0.612 ± 0.113) mm, and the mean value of CIMT ranged from 0.740 to 0.420 (0.550 ± 0.106) mm.

Intraobserver variability

The mean CV of the novices' measurements of the 24 patients decreased over the course of 4 weeks: 0.06, 0.05, 0.03, and 0.02, respectively (P = 0.03) (Figure 2). The corresponding mean CV for the expert was 0.02, 0.02, 0.03, and 0.02, respectively (P = 0.68). The pattern of reduced weekly mean CV was seen in all four novices: for Novice 1, the weekly CV fell from 0.05 (SD 0.03) to 0.01 (SD 0.01); Novice 2, from 0.06 (SD 0.05) to 0.02 (SD 0.02); Novice 3, from 0.05 (SD 0.04) to 0.01 (SD 0.01); and Novice 4, from 0.06 (SD 0.05) to 0.02 (SD 0.01).

Figure 2

Weekly means of CVs of all novices compared with those from an expert operator.

The average of the novices' SD of CVs was 0.04 in the first and second weeks, and then reduced to 0.02 in the third and to 0.01 in the last week (P < 0.05). The corresponding weekly variation in the SD for the expert was 0.02, 0.01, 0.01, and 0.01 (P = 0.27), respectively (Figure 3). The agreement between novice's measurements was expressed by the ICC: it was 0.97 (P < 0.001) in the first week, which increased to 0.99 (P < 0.001) in the fourth week. For the expert, the ICC in the first week was 0.99 (P < 0.001) and remained unchanged (0.99, P < 0.001) at the fourth week.

Figure 3

Weekly averages of CV and SD of CVs for the different novices and experts. Novices (circles and solid lines) and experts (squares and dashed lines). Vertical lines indicates SD of the six CVs that were determined for each week.

Interobserver variability

The interobserver variability between novice and expert was expressed by the ICC, showing good correlation already at the end of the first week (0.98, P < 0.001), and the agreement remained throughout the 4 weeks (0.95, P < 0.001 at the end of the fourth week). As shown in Figure 4, there was no obvious relationship between the difference and the mean of the measurements made by the novices and the expert, in the fourth week. During the first week, the mean difference between the expert and the novices (whole group) was −0.004 mm with a limit of agreement (LOA) of −0.103 to +0.096 mm, and during the second, third, and fourth weeks, the mean differences were 0.029 (LOA −0.082 to +0.141) mm, −0.007 (LOA −0.126 to +0.112) mm, and −0.005 (LOA −0.123 to +0.113) mm, respectively. The mean difference between the expert and the novices was significantly different from zero in the second week (P < 0.01), but the bias was non-significantly different from zero in the other weeks (P > 0.40).

Figure 4

Bland–Altman analysis of the agreement between the measurements made by the novices and experts, shown as a plot of the average and the difference of the CIMT measurements made by the novice and the expert on the same patient (24 patients/week).

Discussion

CIMT is generally accepted as an early marker of atherosclerosis, before lumen narrowing plaques are formed, having been recommended for arterial screening by the current guidelines.13,14 However, it should not be forgotten that CIMT increases normally with age, from an average of 0.5 mm in early childhood to that of 0.8 mm in healthy octogenarians,15 thus accounting for an approximate annual increase in thickness between 0.03 and 0.15 mm.11,16 Such variations in CIMT and technical bias connected to the scanning technique make consideration of absolute measurements and their change overtime potentially difficult to collocate in a clinical scenario. A number of factors contribute to the reproducibility of CIMT measurements reported in the literature. This includes the type of ultrasound system used (manual, semi-automated, or fully automated with direct on screen results) and the techniques applied (B-mode imaging based or RF data based).7,1721 A number of algorithms have been developed for the fully automatic segmentation of the carotid intima-media.2224 Some of these have already found a commercial application.8,25 Among the systems currently available, a fully automatic RF data-based software seems to have a comparable or even higher reproducibility than off-line semi-automatic programmes or manual systems, with an ICC of 0.95, 0.91, and 0.73, respectively.8,17,19 The automatic on-line systems have also the advantage to be less time consuming than any off-line software or manual technique.8 From the 1980s, a tendency towards automation of carotid intima-media segmentation (measurement) seems to have taken hold promising to improve the reproducibility of results and to shorten the learning curve of sonographer. In fact, the experience of the operators using the ultrasound systems, above all if manual or semi-automatic, could multiply the potential variability of the measurements, which can compromise the clinical value of the CIMT. As a result, the use of CIMT measurements still confined to cardiovascular centres where only well-trained personnel undertake such examinations, particularly where a disease prevention service is established.26 Therefore, the current study aimed to address most of the above factors that contribute to CIMT limited reproducibility. Our results demonstrate that even novices who receive well-structured and designed brief training, preceded by an intensive theoretical background on carotid anatomy, ultrasound physiology, and optimum measurement acquisition succeed in achieving highly reproducible measurements not different from experts (ICC 0.97 at baseline increasing to 0.99 at the end of the 4-week training). Despite the variability of novices' CIMT measurements of the first week, these have regressed consistently, in all four novices, until they became dissimilar from those of the expert by the end of the training period. The second variable we succeeded to overcome was the conventionally used manual ultrasound systems; instead, we used an automated system which freezes the CIMT once an excellent quality is obtained and which also acquires the measurements automatically. Such consistency in the image acquisition must have contributed to the rapid improvement of the novice's measurement reproducibility. Finally, having a system measuring such small thickness fractions based on RF data rather than ECG-gated measurements with potential variations in the phases of the cardiac cycle must have added extra accuracy to the measurements we reported. We should not ignore the fact that the automated system we used also provided novices with continuous assistance including projections, angle of insonation, and gain settings of the imaging as well as storing. This comprehensive ultrasound system assistance was quite advanced compared with what has previously been reported, using an off-line semi-automated software.27 We believe that although the kind of ‘helping technology’ we used in this study resulted in a very successful fast training, it should be preceded by a comprehensive theoretical tutorials, irrespective of the background of the operators.

Our study has some limitations. First, our results are based on a small number of novices who underwent the training protocol; however, since they all achieved our set target without fail, we feel that a larger number of trainees would not have changed our conclusion. Secondly, it should also observed that different subjects were evaluated by novices, on one hand mimicking the ‘real patient population’ and on the other hand providing the encountered wide variation in the CIMT of our study. However, the parallel low CVs represent the accuracy of the method in assessing a structure that has a normal biological variation. Thirdly, in this paper, we have not tested the reproducibility of CIMT measurements provided by novices with a manual system that requires a far longer training to be operated. Finally, we do not propose using the above training programme or the ultrasound system in patients with well-developed carotid disease and plaque formation since the automated system we used was not designed for such purpose.

The findings of our study encourage opening CIMT assessment to novices, of no medical background, provided it is preceded by an intensive well-designed theoretical tutorial. With such fast improvement of the novices' accuracy in obtaining highly reproducible measurements, not different from experts, at the end of 4-week programme, they can be encouraged to undertake such clinical activity in community health screening and atherosclerosis prevention centres, in order to spare experts to serve in highly specialized centres.

Conclusion

CIMT can successfully be assessed in a highly reproducible fashion by training novices with no medical background and by using an advanced technology such as an automated carotid RF-based ultrasound system. Such development could have a significant clinical implication in health screening and early disease detection and prevention in areas remote from highly specialized centres; however, such statement should be supported by further studies in this perspective.

Funding

This work was supported by grants from The Heart Foundation of Northern Sweden, The Västerbotten County Council for Health Care, Umeå University, and a non-commercial support by Panasonic Healthcare Co. Ltd, Yokohama, Japan, made the prototype ultrasound equipment available for the study.

Acknowledgements

We gratefully acknowledge Mrs Eva Jonasson, Mrs Karin Olsson, Mrs Titti Tollefsen, and Mrs Eva Mäkitalo for their invaluable help in performing this study.

Conflict of interest: none declared.

References

View Abstract