JOIN
Get Time
forums   
Search | Watch Thread  |  My Post History  |  My Watches  |  User Settings
View: Flat (newest first)  | Threaded  | Tree
Previous Thread  |  Next Thread
Final testing question: will models be trained from scratch? | Reply
Hello, the problem statement specifies that in the final round each of the top 10 competitors would be subject to 3 tests:

1) validation on the available test set,
2) testing on a new set of images, and
3) validation and scoring of steps 1) and 2).

My question is whether the testing/scoring will be performed using the already-runnable model submitted on the docker file (already trained) OR the model trained from scratch (train.sh script) as per the requirements of the fMoW Final Testing Guide.

Thanks.
Re: Final testing question: will models be trained from scratch? (response to post by antor) | Reply
Your system must support both. Usually we test first with your home-made model to verify that we can reproduce your scores on the provisional test set, and to establish a temporary final ranking using the final test set. Then for the top few ranked contestants (as determined above) we make sure that the whole train/test workflow is repeatable, so we retrain and retest on the final test set. (This is a very time consuming step so we don't do this for everyone.)
RSS