I realize this may be intended behavior for the use case that the authors have in mind, but I just wasted ~30 CPU-days because I didn't realize this was the default. I was training on a dataset that couldn't fit on disk uncompressed, so I did what seemed reasonable -- extract a batch, read the NN from a file, run the batch, write the NN back to the file, repeat for the next batch. But since this option is true by default, it deleted anything from those rounds once a later round started. I have fixed the problem now, but I think this is an unreasonable default.