Clinical application, BREC-plot and advanced modelling of sequential tumour marker data
Initially, the best achievable application of tumour markers was considered to be the prediction of non-durable benefit (NDB) in a selection of the NSCLC patients treated with immunotherapy. This was considered to allow early discontinuation of ineffective treatment (reducing side effects and costs) and timely initiation of a next, potentially more effective treatment. To enable a reliable discontinuation of immunotherapy, the accuracy of the predicted NDB had to be high and therefore the minimum test specificity of the to-be-developed test, was set at 95%. Optimization and training of tumour marker models were then primarily focused on this potential clinical application and set requirements. The clinical definition of NDB was defined as death, progressive disease objectified by imaging, or clinical progression determined by the treating physician within 6 months after treatment initiation. As a first, rather simplified approach, the biomarker response characteristic plot (BREC)-plot was developed to graphically represent the clinical significance of relative changes in tumour marker concentrations (Fig. 1) . Using this approach, some simple logical tumour mark cut-offs for CEA, CYFRA 21-1 and NSE were identified that could predict NDB in approximately 32.3 % of NSCLC patients .
However, the use of the rather simple rules was considered to limit the potential performance of such a test and, as a next step, more advanced modelling techniques were investigated. First, modelling strategies focused on individual tumour markers were investigated, ranging from simple logical models to machine learning and AI methods . Next, primarily AI-based methods (including neural networks, boosting, and random forest methods) were used to investigate their suitability for automatically optimizing models that interpret the changes in multiple tumour markers together . Finally, a model was selected, a random forest model based on CYFRA 21-1, CEA and NSE results (Fig. 2), which provided the best diagnostic performance, to be further validated in a new and independent cohort of NSCLC patients reflecting the current medical practice. In this independent cohort of 242 patients, the newly assigned ‘Serum Tumor Marker-based Outcome Prediction’ (STOP) model, enabled the detection of NDB with 38.1% sensitivity and 95.9% specificity, and was found to have a similar diagnostic performance to imaging performed at the same time (6 weeks after treatment initiation). Most importantly, only by combining imaging and the STOP test enabled identification of patients with NDB as early as 6 weeks after treatment initiation with 100% specificity and a positive predictive value of 100%; when both indicated progressive disease at 6 weeks none of the patients benefited from treatment based on the criteria used. Overall, using the latter criterion, 21% of the patients with NDB were accurately identified.
Summary and conclusions
The studies performed showed that the use of the more traditional tumour markers, such as CEA, CYFRA 21-1 and NSE, can provide relevant monitoring support for NSCLC patients. In addition, new methods were developed to allow more advanced modelling of sequential (tumour marker) data, which improved the diagnostic performance of the tumour marker models. Future research and development in this area should focus on further developing powerful mathematical methods for interpreting sequential data and to applying them to other longitudinal (tumour marker) data, such as those available for other types of cancer. For the developed STOP model, an additional blinded multicentre prospective clinical validation would be of added value to ultimately confirm the presented results and strengthen its potential place in NSCLC clinical care.