Hello All,

My name is Sean Kelley and I am an undergrad at UMass Amherst studying CS. For our course, CS 585 Intro to NLP, my team experimented with additional token and sequence embedding techniques on the CodeSearchNet challenge. Our team was wondering what the process is for having our results officially submitted to the leaderboard?

One particularly interesting finding was that our Continuous Bag of Words (CBoW) implementation was able to beat the Self Attention baseline MRR scores, but was not able to beat the Neural Bag of Words baseline on the NDCG leaderboard.

Our CBoW submission can be viewed here, our repository here, and our final report here. Unfortunately, to turn in our work we had to merge some not finished code into master, but each method we implemented still has its own branch.

Also, thank you for your work on this challenge. It was quite interesting to work on and the code was very well-written.

Best,
Sean

Hi @seangtkelley

Thanks for participating in the CodeSearchNet challenge! I hope that you enjoyed the task :)

I am not entirely sure what you're asking here: I can see your submission as # 9 in the leaderboard (the pagination is a bit odd/confusing, and you need to move to the next page):

Maybe the results hadn't propagated at the time you looked at the leaderboard? Or maybe different people are seeing different versions of the leaderboard? Let me know and I can look into getting this fixed.