Join GitHub today
GitHub is home to over 50 million developers working together to host and review code, manage projects, and build software together.
Sign uptrain with different learning_rate_constant, but got similar train & dev loss, also similar test bleu #1799
Comments
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
I download Arabic-Chinese parallel corpus from opus as train data and xnli as dev & test data, apply sentencepiece to encode raw text into sentence pieces/ids, then train arabic-to-chinese translation model with learning_rate_constant=2 and learning_rate_constant=3, and keep other parameters default. However, I got similar results, including train loss, dev loss, and test bleu. Any idea or suggestion for this? Thanks!