Artificial-intelligence-based end-to-end prediction of cancer immunotherapy response

TANGERINE aims to build deep learning models to predict response and toxicity to immune chekpoint blockade therapy in cancer from the analysis of histopathology and radiology images

TRANSCAN 3 project funded in the 2021 Joint Transnational Call for Proposals (JTC 2021) co-funded by the European Commission/DG Research and Innovation

Cancer immunotherapy with immune checkpoint inhibitors (ICIs) is widely used in multiple cancer types, with proven benefits. However, response is not guaranteed, difficult to predict, and serious toxicity may occur.

Project Aims

Primary: aim: To develop, validate and openly publish an AI system which can predict ICIs response based on routine histopathology slides, CT scan images and routine clinical data.
As co-primary aims, disease-free survival, overall survival and toxicity will be predicted.

Secondary aim: To validate the capacity of this system to identify cellular structures and image patterns associated with ICI response and toxicity that explain model predictions (explainability). This aim is exploratory and qualitative.


Digital images of tumour histopathology slides and CT scans will be retrieved, linked to clinical outcomes data and anonymized for analysis.
An initial retrospective (2017-21) data retrieval from 1800 patients at 6 centres will continue with a prospective recruitment of 600 more to validate models.
Patients that received ICls as first line for any tumour will be included and response recorded according to iRECIST.


Deep convolutional neural networks and radiomcs will be used to build end-to-end predictive models.
Spatial transcriptomics data on a subset of 30 patients wil help model explainability.


Retrospective study

Retrospective (2017-21) data retrieval from 1800 patients at 6 centres.

Model training

Deep convolutional neural networks for histopathology and radiology data.

Prospective study

Prospective recruitment of 600 patients treated with ICIs.

Model validation

Testing the accuracy of the model in the prospective study.


Spatial transcriptomics of selected tumors to understand why models predict.

Capacity Building

Students will train across institutions.


Retrospective study
Model training
Prospective study
Model validation
Capacity Building

Research team

Victor Moreno Catalan Institute of Oncology / IDIBELL (Barcelona, Spain)
Victor Moreno
Jakob N Kather Technical University Dresden / EKFZ (Dresden, Germany)
Jakob N Kather
Raquel Pérez-López Institut d’Investigació Oncológica de Vall d’Hebron (Barcelona, Spain)
Raquel Pérez-López
Julien Calderaro Henri-Mondor University Hospital / INSERM (Creteil, France)
Julien Calderaro
Nicoleta Antone The Oncology Institute 'Prof Dr. Ion Chiricuta' (Cluj-Napoca, Romania)
Nicoleta Antone
Gad Rennert Technion & Carmel Medical Center (Haifa, Israel)
Gad Rennert
Ovidiu Balacescu The Oncology Institute 'Prof Dr. Ion Chiricuta' (Cluj-Napoca, Romania)
Ovidiu Balacescu
Zunamys Carrero Technical University Dresden / EKFZ (Dresden, Germany)
Zunamys Carrero
Sabine Marschollek Technical University Dresden / EKFZ (Dresden, Germany)
Sabine Marschollek
Marta Ligero Institut d’Investigació Oncológica de Vall d’Hebron (Barcelona, Spain)
Marta Ligero
Silvia Murcia Catalan Institute of Oncology / IDIBELL (Barcelona, Spain)
Silvia Murcia
Miguel Socolovsky Catalan Institute of Oncology / IDIBELL (Barcelona, Spain)
Miguel Socolovsky
Lois Riobo Catalan Institute of Oncology / IDIBELL (Barcelona, Spain)
Lois Riobo


IDIBELL Bellvitge Biomedical Research Institute
ICO Catalan Institute of Oncology
INSERM Institut National de la Santé et de la Recherche Médicale
HUHM AP-HP Hôpitaux Universitaires Henri Mondor
TU Dresden Technische Universität Dresden
EKFZ Else Kröner Fresenius Center for Digital Health
Technion Technion Israel Institute of Technology
Carmel MC Carmel Medical Center
IOCN The Oncology Institute „Prof. Dr. Ion Chiricuţă”
VHIO Vall d'Hebron Institute of Oncology


Transcan3 Research Project
Funded by national agencies:

Instituto de Salud Carlos III , co-funded by FEDER funds –a way to build Europe– (Spain)

Scientific Foundation of the Spanish Association Against Cancer (Spain)

ARC French Foundation for Cancer Research (France)

Federal Ministry of Education and Research (Germany)

The Chief Scientist Office of the Ministry of Health (Israel)

Executive Agency for Higher Education, Research, Development and Innovation Funding (UEFISCDI) (Romania)