github.com/webis-de/wat, where you will find more detailed explanations of how to set everything up. For this demonstration, you will see annotations of remote workers (here anonymized) for a real argumentative segment labeling task, the result of which was published in Coling 2016. We just removed their progress for some of the tasks. For the purpose of this demonstration, we made some of the internal WAT-SL files accessible by browser.
The annotation interface is available via a web browser at
wat/annotate. Please log in as "Charlie" (password: "password"). You will see the list of available tasks with Charlie's progress on them (scroll upwards to see the finished tasks). Click on one of them to see the annotation interface. All progress is saved automatically.
To login page
Annotations are immediately sent to the server. The annotation log contains the time, the user's time offset, whether the annotation was performed by an annotator or admin, and the label assignment. You can find Charlie's log for task 001 here.
Now, please log in as "Edwin" (who has curation tasks, password: "password") and select the first task. Whenever a majority of annotators agreed on the label for a segment, this label is pre-selected for that segment. Moreover, when one opens the drop-down menu of a segment, numbers show how often each label was selected.
To login page
After the curation, you can create result files like this one using the administration page (password: "password").
Tasks correspond to directories with the segments in plain text files. You can find the data for the first task (001) here. For curation, add the annotation logs to the directory (like here).
Annotator credentials and task assignments are managed in the
Available labels are defined in a project-wide
segment-labeling.conf. Just add an image with corresponding name to the
The interface can then be customized using Cascading Style Sheets.