You are here

MICCAI Automatic Prostate Gleason Grading Challenge 2019

MICCAI Automatic Prostate Gleason Grading Challenge 2019

This challenge is part of the MICCAI 2019 Conference to be held from October 13 to 17 in Shenzhen, China. This challenge will be one of the three challenges under the MICCAI 2019 Grand Challenge for Pathology.

The challenge is also hosted on GrandChallenge.org here

Challenge Description 

Prostate Cancer (PCa) is the sixth most common and second deadliest cancer among men worldwide. There exist various techniques for PCa detection and staging. However, microscopic inspection of stained biopsy tissue by expert pathologists is the most accurate method. Based on the observable histological patterns, each region of the tissue is assigned a Gleason grade of 1 to 5. The final Gleason score is reported as the sum of the most prominent and second most prominent patterns; e.g., a tissue with the most prominent pattern of Gleason grade of 4 and the second most prominent pattern of Gleason grade of 3 will have a Gleason score of 4+3.

This challenge aims at the automatic Gleason grading of prostate cancer from H&E-stained histopathology images. This task is of critical importance because Gleason score is a strong prognostic predictor. On the other hand, it is very challenging because of the large degree of heterogeneity in the cellular and glandular patterns associated with each Gleason grade, leading to significant inter-observer variability, even among expert pathologists.

Gleason grading of prostate cancer is usually performed via visual inspection (with a microscope) of the prostate tissue by expert pathologists. However, this is a time-consuming task and suffers from very high inter-observer variability. Automatic computer-aided methods have the potential for improving the speed, accuracy, and reproducibility of the results.

Objectives

This challenge will provide a unique dataset and evaluation framework for the important and challenging task of prostate cancer Gleason grading. It will help establish a benchmark for assessing and comparing the state of the art image analysis and machine learning-based algorithms for this challenging task. It will also help evaluate the accuracy and robustness of these computerized methods against the opinion of multiple human experts. Given the critical importance of prostate cancer and extreme utility of Gleason grade system for detection and diagnosis of prostate cancer, the results of this challenge can be of utmost utility for medical community.

Tasks

The challenge involves two separate tasks:

Task 1: Pixel-level Gleason grade prediction

Task 2: Core-level Gleason score prediction

Challenge Dataset

Data used in this challenge consists of a set of tissue micro-array (TMA) images. Each TMA image is annotated in detail by several expert pathologists.

Study conditions: Prostate tissue microarray obtained from patients suspect of having prostate carcinoma. For each tissue micro-array core image in the training set, the participants are provided with the Gleason score in the form of the the most prevalent Gleason grade + the second most prevalent Gleason grade. For each tissue micro-array core image in the test set, no additional information other than the image is provided.

The histopathology data that will be used in this challenge have been acquired from tissue microarray blocks that have been constructed and processed at the Vancouver Prostate Centre. This study has been approved by the Clinical Research Ethics Board
of the University of British Columbia (CREB #H15-01064).

For each tissue micro-array core image in the training set, the participants are provided with a set of 4 to 6 images of the same size that indicate the detailed annotations provided by expert pathologists. For each tissue micro-array core image in the test set, no additional information other than the image is provided.

Annotation policy for training cases: The pathologists used an in-house developed application loaded on a tablet with a stylus to annotate TMA cores as benign, and Gleason grades 3, 4, and 5 tissue. They were instructed to draw regions (closed contours) on the cores and mark each region with a grade. To capture the true variability in expert annotations, we did not arrange for a consensus meeting between the pathologists prior to the annotation.

All six annotators were expert pathologists. Specifically, one of them was a clinical general pathologist, one was a research genitourinary pathologist, and the other four were clinical genitourinary pathologists. They had on average 15 years of experience.

Data can be downloaded here

Submission Format & Rules

1. Each individual or collaborating team agrees to create one single account for participating in this challenge.

2. Each participating individual or team is allowed a maximum of one submission prior to the date of the conference. After the date of the conference and evaluation of all participating teams, we will allow for a maximum of two more submissions by each team.

3. Participating teams are allowed to publish the methods that they develop using the data provided in this challenge, provided that they properly cite this challenge.

4. Participating teams are not allowed to share the data.

5. Only fully-automatic algorithms are allowed.

6. Shortly after the conference, a manuscript will be written in which the top 10-15 methods will be explained and their performance will be compared and discussed. Up to two authors from each of the top performing methods will be invited as co-authors. This manuscript will be submitted to a prestigious journal. The author ordering will be decided by the challenge organizers.

Submitting 

The participating teams will submit their pixel-wise predictions in the form of images (one for each test image). The image can have a single channel, where the value of each pixel indicates the predicted class, or 4 channels where each channel represents the probability of one of the four classes, in order.

Each participating team will submit their estimated Gleason scores as a plain text file such that one single score is reported for each TMA core in the test dataset. Specifically, the submitted text file should have one row for each of the test cores. Each row will have two fields separated with a comma (","). The first field should be the name of the corresponding TMA core image file and the second field the estimated Gleason score for that core. For example:

slide001_core015, 4
slide001_core019, 3
slide001_core020, 4
...
...

Labels 0, 1, and 6:     benign (no cancer)
Label 3:                Gleason grade 3
Label 4:                Gleason grade 4
Label 5:                Gleason grade 5
Note: On the test data, we obtain "ground-truth" estimation of pixel-level Gleason grades using probabilistic analysis of labels provided by multiple experts using the STAPLE algorithm (Warfield et al. IEEE TMI 2004).

We will not accept missing data. Only complete submissions are accepted.

Data can be downloaded here

Test Data Release: June 30, 2019

Submission deadline extended: September 27, 2019

Results will be announced September 30, 2019

Submit results here