NLPContributionGraph

Directions for participating in the SEMEVAL-2021 NLPContributionGraph Shared task via CodaLab (https://competitions.codalab.org/competitions/25680)

Steps:

1. Create an account in CodaLab (https://competitions.codalab.org/). Sign in.
2. Edit your profile appropriately. Make sure to add a team name, and enter names of team members. (Go to "Settings", and look under "Competition settings".)
3. Proceed to task webpage on CodaLab. A direct link to the NLPContributionGraph CodaLab competition is here. Read information on all the pages (including Terms and Conditions). Click on the Participate tab and Register.
4. Download data: training, development, and test sets (when released from their respective Github links)
5. The Shared Task will be organized in three main evaluation phases, viz. Evaluation Phase 1: End-to-end evaluation phase, Evaluation Phase 2, Part 1: Phrase Extraction Testing, and Evaluation Phase 2, Part 2: Triples Extraction Testing. Please refer to our Codalab competition site for detailed information. Our evaluation scripts are additionally released on Github: https://github.com/ncg-task/scoring-program.

FAQ:

Q. If I am part of a team, can I make submissions to the task with my personal user account?
A. Yes and no. Yes, if as an individual person you are submitting your own system results that is different from the team system. No, if you are merely wishing to evaluate the team system with your individual account. We have a cap on the maximum allowed tries per system in each evaluation phase and doing this will violate that term.

Q. How do I/we include more than one score on the leaderboard?
A. CodaLab allows only one score on the leaderboard per user.