Submit your results

We have set up a CodaLab competition based on this evaluation framework at https://competitions.codalab.org/competitions/15984 . In this competition you can upload the output of your system in the concatenation of all datasets (you can also upload results on individual datasets following the gold-standard "ids" included in the unified dataset). The output file should follow the same format of the gold key files included in the evaluation data.

We encourage researchers to test their models on this evaluation framework. To enable a fair comparison, supervised systems should report results using the same training data. If you would like to use different training data, you can share your data to be included in the evaluation framework so we can recompute the results of all comparison systems using the same data. We additionally encourage the release of the code and the inclusion of details about the algorithms and parameters used.

If you have inquiries about the CodaLab competition, please post your questions on the Google Group. Alternatively, you can always contact and email us directly.