Small thing, but costed me several hours to find :)
In the documentation example of Siamese mnist .

We see a code for contrastive loss, based on a paper. But the labels in this function are reversed from
the paper. Meaning in the paper Y=0 if X1,X2 are from same domain, Y=1 otherwise. In the code example in the documentation it is reversed.

The code example quote the paper, so I thought it might be confusing :)

My suggestion is to change the function to match the paper, and also change the code to use the reverse labels (reverse the 0 and 1).