Landmark Detection Progress

Model #1 trained on all the landmarks(51) with a patch size of 17 for 3 epochs(Each image is seen thrice by the network). Results on the weighted metric (All errors multiplied by their weights and normalized) are as follows

MEAN OF WEIGHTED ERROR is 3.002102020671362

VARIANCE OF WEIGHTED ERROR is 2.2291132515913104

MAX OF WEIGHTED ERROR is 12.617353498206596

Time taken is around 6-7 seconds per image

What I noticed is that training all landmarks greatly improves detection of other more important landmarks. For instance, these are the results of the same model for AC point

MEAN DISTANCE OF AGENT 0 is 0.9526148835843511

VARIANCE DISTANCE OF AGENT 0 is 0.4608614282106958

MAX DISTANCE OF AGENT 0 is 4.242640687119285

which is really good for just 3 epochs on unseen data. The model is using only 4-5 GB of GPU memory.

However, the training is taking very long (Around one day for a whole pass through the data) and ubuntu sometimes randomly kills the process making it even harder to train.