WebAnno User Guide

Alternatively, you can start WebAnno from the command line, in particular if you wish to provide it with additional memory (here 1 GB) or if you want it to store its data in a different folder.

Mind to replace /my/webanno/home with path of a folder where WebAnno can store its data.

By default the server starts on port 8080 and you can access it via a browser at http://localhost:8080 after you started it. You can add the parameter --httpPort=9999 at the end of the command line to start the server on port 9999 (or choose any other port).

A full list of the command line parameters can be found in the Winstone documentation.

Before any upgrade, make a copy of your WebAnno home folder.

When navigating to the Annotation page, a dialogue opens that allows you to select a project, and a document within the project. If you want to open a different project or document later, click on Open to open the dialog.

Projects appear as folders, and contain the documents of the project. Double-click on a document to open it for annotation. Document names written in black show that the document has not been opened by the current user, blue font means that it has already been opened, whereas red font indicates that the document has already been marked as done.

Sentence numbers on the left side of the annotation page show the exact sentence numbers in the document.

The arrow buttons first page, next page, previous page, last page, and go to page allow you to navigate accordingly. The Prev. and Next buttons in the Document frame allow you to go to the previous or next document on your project list. You can also use the following keyboard assignments in order to navigate only using your keyboard.

A click on the Help button displays the Guidelines for the tool and The Annotator’s Guide to NER-Annotation. When you are finished with annotating or curating a document, please click on the Done button, so that the document may be further processed. If the button above the Done is a cross symbol, it means the documents have already been finished. If the symbol has a tick, it is still open.

Annotation of spans works by selecting the span, or double-clicking on a word. This activates the Actions-box on the right, where you can choose a layer. One can also type in the initial letters and chose the needed layer. After having chosen a layer, the drop-down menu inside the Features-box displays the features you can use during the annotation. The tag can be selected out of the drop-down menu inside the Features-box which contains the tags of the chosen layer.

To change or delete an annotation, double-click on the annotation (span or link annotations). The Actions-box is now activated. Changes and Deletions are possible via the respective buttons.

Link annotations (between POS tags) are created by selecting the starting POS-tag, then dragging the arrow to connect it to its target POS tag. All possible targets are highlighted.

To create annotations in WebAnno, you have mainly two options:

The definition of layers is covered in section Layers.

Once the document is opened, a default of 5 sentences is loaded on the annotation page. The Settings button will allow you to specify the settings of the annotation layer.

Next to Annotation layers, you to select the annotation layer which is displayed during annotation. This is useful to reduce clutter if there are many annotation layers. Mind that hiding a layer which has relations attached to it will also hide the respective relations. E.g. if you disable POS, then no dependency relations will be visible anymore.

The Remember layer checkbox controls if the annotation layer selected in the Actions box. It will work as main layer during the annotation process. Only instances of this layer will be created, even if an annotation in another layer is selected. If necessary, it is possible to change active instances. Still, if a new instance is selected, the main layer is automatically activated.

The Sidebar size controls the width of the sidebar containing the annotation detail edtior and actions box. In particular on small screens, increasing this can be useful. The sidebar can be configured to take between 10% and 50% of the screen.

The Number of sentences controls how many sentences are visible in the annotation area. The more sentences are visible, the slower WebAnno will react.

The Auto-scroll setting controls if the annotation view is centered on the sentence in which the last annotation was made. This can be useful to avoid manual navigation.

If Use the same color for all tags in a layer is chosen, annotations are colored per layer. If this option is off, then annotations are colored by their labels (all annotations with the same label also have the same color). Mind that there is a limited number of colors such that eventually colors will be reused.

Annotations are always immediately persistent in the backend database. Thus, it is not necessary to save the annotations explicitly. Also, losing the connection through network issues or timeouts does not cause data loss. To obtain a local copy of the current document, click on export button. The following frame will appear:

Choose your preferred format. Please take note of the facts that the plain text format does not contain any annotations and that the files in the binary format need to be unpacked before further usage. For further information on the formats provided in WebAnno, please consult the corresponding chapter Formats.

The document will be saved to your local disk, and can be re-imported via adding the document to a project by a project administrator. Please export your data periodically, at least when finishing a document or not continuing annotations for an extended period of time.

After clicking on Create Project on the Projects page, select automation as your project type. The detailed description may be found in Projects.

The documents, that are to be annotated, have to be uploaded in the frame Documents. Please make sure that the chosen format corresponds to the format of the files you are uploading.

To manage the automation process, choose the Automation frame. The following frame will appear:

First choose your target layer in the Select automation layer frame. If you want to train a non-custom layer, please make sure you created or imported it in the Layer frame (for instructions to do so, see Projects).

Here you may choose the format of the target layer and optionally add some feature layers on which you want to train.

In the tab Target layer you may upload training files containing the target layer in WebAnno Export formats (WebAnno CPH TEI reader, plain text, binary format, XMI format, old WebAnno Format, WebAnno Format, Weblicht TCF Format, for more information on these formats, see [https://code.google.com/p/webanno/wiki/Format?ts=1407336468&updated=Format Format]).

In the next tab TAB-SEP target, you may upload training files containing the target layer in a tab-separated format, which is structured by writing each single word in a line together with its target tag, separated by a tab. Sentences are separated by blank lines.

The same goes analogically for the feature layers. The Other layers tab gives the possibility to upload WebAnno Export formats and choosing the layers that are to be used in training in the format window. The TAB-SEP feature tab gives the possibility to upload files in the above described tab-separated format, containing the feature tags in the second column. Every file will be regarded as one separate feature.

After choosing the training files, uploading them in the right format and importing them (by clicking on Import), every file will be displayed in the corresponding tab in the frame Documents. Click on the button Start Automation on the left, when you have uploaded your training data. Be prepared to wait for some time, as automation is a non-trivial process.

You can see that the automation has finished either by the fact that the Start Automation button is enabled again, or on the Monitoring page, by choosing the project in Monitoring and looking at the progress shown in the Training results /status frame.

To see the tags that were automatically created during the previously described, go to Home and choose the Automation page. Then select a project and a file, analogically to Annotation. The page, which is demonstrated below will be displayed. The navigation, export and the marking of finished documents is the same as in Annotation.

In the lower part, you see two horizontal frames, the lower one showing the automatically created annotation. By clicking on the tags, they are selected and therefore appear in the upper frame Annotation. You may see that selected tags turn grey in the Automation frame and blue in the Annotation frame. You may also add new tags to the Annotation, just like on the Annotation page.

The following table will explain the different symbols which explain the status of a document for a user and the described task.

You can also alter the document status of annotators. By clicking on the symbols you can change between Done and In Progress. You can also alter between New and Locked status. The second column of the document status frame displays the status of the curation.

As there is only one curator for one document, curation is not divided into individual curators.

Scrolling down, two further frames become visible. The left one, named Layer, allows you to chose a layer in which pairwise kappa agreement between annotators will be calculated.

Agreement can be inspected on a per-feature basis and is calculated pair-wise between all annotators across all documents.

The first time a feature is selected for agreement inspection, it takes a moment to calculate the differences between the annotated documents. Switching between different features subsequently is much faster.

Agreement is calculated in two steps:

The lower part of the agreement matrix displays how many configuration sets were used to calculate agreement and how many were found in total. The upper part of the agreement matrix displays the pairwise Cohen’s kappa scores.

The agreement calculations considers an unset feature (with a null value) to be equivalent to a feature with the value of an empty string. Empty strings are considered valid labels and are not excluded from agreement calculation.

Annotations for a given position are considered complete when both annotators have made an annotation. Unless the agreement measure supports null values (i.e. missing annotations), incomplete annotations are implicitly excluded from the agreement calculation. If the agreement measure does support incomplete annotations, then excluding them or not is the users' choice.

After clicking on Users, you are displayed a new pane in which you can add new users by clicking on the button Add User. After doing so, you get a list of users in the system which can be added to the project. By making a tick in front of the login, you can chose a new user.

Please do not forget to save after choosing all members of the project. Close the pane by clicking on Cancel. The rights of users created like this are that of an annotator. If you want to expand the user’s status, you can do so by clicking on the user and then on Change Permission. The following frame will pop up.

After ticking the wished permissions, click on Update. To remove a user, click on the login and then Remove User.

To add or delete documents, you have to click on the tab Documents in the project pane. Two frames will be displayed. In the first frame you can import new documents.

Choose a document by clicking on Choose Files. Please mind the format, which you have to choose above. Then click on Import Document. The imported documents can be seen in the frame below. To delete a document from the project, you have to click on it and then click on Delete in the right lower corner.

In WebAnno, all annotations belong to an annotation layer. Each layer has a structural type that defines if it is a span, a relation, or a chain. It also defines how the annotations behave and what kind of features it carries.

To administer the tagsets, click on the tab Tagsets in the project pane.

To administer one of the existing tagsets, select it by a click. Then, the tagset characteristics are displayed.

In the Frame Tagset details, you can change them, export a tagset, save the changes you made on it or delete it by clicking on Delete tagset. To change an individual tag, you select one in the list displayed in the frame Tags. You can then change its description or name or delete it by clicking Delete tag in Tag details. Please do not forget to save your changes by clicking on Save tag. To add a new tag, you have to click on Create tag in Tag details. Then you add the name and the description, which is optional. Again, do not forget to click Save tag or the new tag will not be created.

To create an own tagset, click on Create tagset and fill in the fields that will be displayed in the new frame. Only the first field is obligatory. Adding new tags works the same way as described for already existing tagsets. If you want to have a free annotation, as it could be used for lemma or meta information annotation, do not add any tags.

To export a tagset, choose the format of the export at the bottom of the frame and click Export tagset.

To import a constraints file, go to Project and click on the particular project name. On the left side of the screen, a tab bar opens. Choose Constraints. You can now choose a constraint file by clicking on Choose Files. Then, click on Import. Upon import, WebAnno checks if the constraints file is well formed. If they conform to the rules of writing constraints, the constraints are applied.

To add or delete guidelines, which will be accessible by users in the project, you have to select the tab Guidelines. Two new frames will be displayed. To upload guidelines, click on Choose files in the first frame – Add guideline document, select a file from your local disc and then click Import guidelines.

Uploaded guidelines are displayed in the second frame – Guideline documents. To delete a guideline document, click on it and then on Delete in the right lower corner of the frame.

Projects are associated with the accounts of users that act as project managers, annotators, or curators. When importing a previously exported project, you can choose to automatically generate missing users (enabled by default). If this option is disabled, projects still maintain their association to users by name. If the respective user accounts are created manually after the import, the users will start showing up in the projects.

WebAnno offers two modes of exporting projects:

The format of the exported annotations is selected using the Format drop-down field. When AUTO is selected, the file format corresponds to the format of the source document. If WebAnno has not write support for the source format, the file is exported in the WebAnno TSV format instead.

When exporting a whole project, the structure of the exported ZIP file is as follows:

Currently, WebAnno does not allow the user to choose a specific format for bulk-exporting annotations. However, this mailing list post describes how DKPro Core can be used to transform the UIMA CAS formats into alternative formats.

Constraints can be applied to the roles of slot features. This is useful, e.g. when annotating predicate/argument structures where specific predicates can only have certain arguments.

Consider having a span layer SemPred resembling a semantic predicate and bearing a slot feature arguments and a string feature senseId. We want to restrict the possible argument roles based on the lemma associated with the predicate. The first rule in the following example restricts the senseId depending on the value of a Lemma annotation at the same position as the SemPred annotation. The second rule then restricts the choice of roles for the arguments based on the senseId.

Note that some role labels are marked with the flag (!). This is a special flag for slot features and indicates that slots with these role labels should be automatically displayed in the UI ready to be filled. This should be used for mandatory or common slots and saves time as the annotator does not have to manually create the slots before filling them.

WebAnno TSV 3 file starts with the following header marker

Layers are marked by the # character followed by T_SP= for span types (including slot features), T_CH= for chain layers, and T_RL= for relation layers. Every layer is written in new line, followed by the features in the layer. If all layer type exists, first, all the span layers will be written, then the chain layer, and finally the relation layers. Features are separated by the | character and only the short name of the feature is provided.

Here the layer name is webanno.custom.Pred and the features are named bestSense, lemmaMapped, senseId, senseMapped. Slot features start with a prefix ROLE_`followed by the name of the role and the link. The role feature name and the link feature name are separated by the `_ character.

The target of the slot feature always follows the role/link name

Here the name of the role is webanno.custom.SemPred:RoleSet and the name of the role link is webanno.custom.SemPredRoleSetLink and the target type is uima.tcas.Annotation.

Chain layers will have always two features, referenceType and referenceRelation.

Relation layers will come at last in the list and the very last entry in the features will be the type of the base (governor or dependent) annotations with a prefix BT_.

Here, the relation type de.tudarmstadt.ukp.dkpro.core.api.syntax.type.dependency.Dependency has a feature DependencyType and the relation is between a base type of de.tudarmstadt.ukp.dkpro.core.api.lexmorph.type.pos.POS.

In this section we discuss the different representations of texts and annotation in WebAnno TSV3format