Get Time
Search | Watch Thread  |  My Post History  |  My Watches  |  User Settings
View: Flat (newest first)  | Threaded  | Tree
Previous Thread  |  Next Thread
Inconsistency of crack probability data with ground truth data | Reply
According to the problem statement, all non-zero probability values should receive a non-zero label, while all zero-valued probabilities should receive a label of 0. However, the ground truth file does not meet this criteria:

There are a few voxels with zero probability, but receiving non-zero ground truth label. E.g., voxel (z, y, x) = (201, 84, 115) is labeled with 19 while having probability 0.

There are also several voxels with non-zero probablility, but receiving zero ground truth label. E. g., voxel (z, y, x) = (332, 153, 49) lies in the training area in a labeled slice, and its probability is 255, but its ground truth label is 0.

This is something, which should be either corrected, or at least announced in the problem statement, since it may cause some weird behavior in some solution implementations and also makes it harder in our offline testers to reproduce the same scores as getting from example submissions.
Re: Inconsistency of crack probability data with ground truth data (response to post by nofto) | Reply
There are indeed several discrepancies between the probability and ground truth files. The ground truth was made by a human expert, so several errors of the type you mention do exist in the file. The number of voxels with the errors you mention is small, but I am trying to fix these errors at least in the system region.

In your submissions, please continue to label all non-zero probability voxels with positive labels and all zero probability voxels with the label 0. Any submissions which do not label all voxels in the 3D volume in this way will receive a score of 0.
Re: Inconsistency of crack probability data with ground truth data (response to post by jonathanps) | Reply
An updated training ground truth file fixing these inconsistencies can be downloaded here.

This change is not yet reflected in the online test harness, but it will be shortly. I will make an additional post once it is updated.