CoBaLD semantics

The semantic pattern of CoBaLD annotation is based on the Compreno model. The reason of choosing the Compreno formalism is that it is very simple from the structural point of view, on the one hand, and suggests full semantic markup both for the word meanings and the relations between words, on the other hand.

The core of the model is the Semantic Hierarchy — a thesaurus-like semantic tree, which consists of the semantic classes (SCs). SCs correspond to semantic fields universal for all languages included in the hierarchy and denote semantic senses common for all languages. The SCs are filled with lexical contents (lexical classes, or LCs) special for each language of the hierarchy.

All semantic links between words are expressed through the semantic roles, or deep slots (DSs) corresponding to actant valencies (Agent, Experiencer, Addressee, etc.), adjuncts (Locative, Distance, Time, Condition, Concession, and so on),
characteristics (for instance, evaluation, speed, price, form, or size), specifications and others. It allows one to annotate the semantical meanings of all words and to define all semantic relations of each word, both actant and circumstantial.

However, the model suggests a heavily detailed description: namely, it contains more than 200,000 SCs (which seems too much for a machine learning based parser trained on the dataset of our volume) and more than 330 SSs, which, in turn, does not seem necessary for most application tasks.

Therefore, we decided to reduce the number of categories.
First, we have used not the terminal SCs, which denote the exact word meanings, but the hyperonym classes. That is, all words with motion semantics would now belong to the hyperonym class MOTION.
Second, we have reduced the number of the DSs and used the DSs with more general senses. For example, full Compreno markup suggests different roles for different characteristic dependencies, such as form, taste, sound, appearance, importance, genuineness, and so on — more than 60 characteristics in total. In the generalized variant, all such characteristics correspond to one characteristical slot. Or, full model contains several Instrument slots, which differ by the SCs each slot can include (see fig.1) — in the simplified variant, they are joined in one Instrument slot.

As a result, the number of hyperonym SCs used in the markup was reduced to 1085 classes, and the number of the SSs — to 143 slots.

The semantic hierarchy of the hyperonym classes can be found on the Compreno Semantics Github:
GitHub repository.

The hierarchy is presented in the form a .csv table which contains the information needed to visually re-create the Semantic Hierarchy as a tree for exploration. We also provide a link to the GUI program for browsing the hierarchy (Windows only).

The list and the description of the semantic roles are also available on the corpus page:
simplified version of the semantic slot list.

ABBYY decided to publish this technology for the needs of the scientific society in the area of corpus linguistics under terms of the CC-BY-NC 4.0 License.

The Compreno model includes not only the semantical pattern, but morpho-syntax and parser as well. Nevertheless, we preferred not to take the whole model but to integrate its semantic level into another formalism — into UD.
The reasons are the following.
The Compreno markup can include the boundaries of the constituents, the SCs (their labels are marked with green below), the DSs (their labels are marked with brown below) — see fig. 2:

The markup can also be provided with surface, or syntactic, roles (marked with $ sign — see Fig. 3 below), coreference and non-tree links. However, this kind of markup representation does not contain morphological and other grammatical information. Nevertheless, after a sentence is annotated, the parser can build its parsing tree (see (Anisimovich et al., 2012)), where each token is provided with full grammatical and semantic data.
Fig. 3 is an illustration of the Compreno parsing tree for the above given sentence,
and fig. 4 being the fragment of the tree shows the morphological grammemes for the node «готовить:готовить:PREPAREDNESS»:

The «bracket» format presented on fig. 2 is the one that the annotators work with to point out the information necessary for building the correct structure of a sentence, whereas the parsing tree is where full information about the sentence is stored (its syntactic and semantical structure, syntactic and semantic slots, SCs, grammatical features and information about coreference and non-tree links).
Unlike Compreno, UD stores all relevant information in the markup itself, presented in a table-view, which makes it much more convenient for practical usage. Most important is that storing morphological information only in the parsing tree instead of the markup itself implies the necessity to build a correct tree for every sentence, which can evidently be a problem, while indicating all the information in the markup provides the correct data immediately with no risks.

Nevertheless, combining to annotation formats togather evokes anothr problem — the necessity to convert the Compreno morpho-syntax scheme into the UD one.

Of course, there is a number of differences between the Compreno and the UD formats in this respect. Most significant distinctions concern POS-tagging, tokenization, lemmatization, asymmetry of mapping some grammatical features, ellipsis and copula description, coordination and dealing with punctuation.
Besides, the UD format marks the tokens up with so called dependency heads (each token gets the index of its head as a label) whereas the Compreno model operates with the boundaries of the constituents. The conversion process is thoroughly discussed in (Ivoylova et al., 2023).

After the conversion, the markup looks as in fig. 5 and includes morphology, syntax, and semantics:

KV Anisimovich, K Yu Druzhkin, KA Zuev, FR Minlos, MA Petrova, and VP Selegei. 2012. Syntactic and semantic parser based on ABBYY Compreno linguistic technologies. // Proc Dialogue, Russian International Conference on Computational Linguistics, P 91–103.

MA Petrova. 2014. The compreno semantic model: the universality problem. International Journal of Lexicography, 27(2):105–129.

Ekaterina Manicheva, Maria Petrova, Elena Kozlova, and Tatiana Popova. 2012. The compreno semantic model as integral framework for multilingual lexical database. In Proceedings of the 3rd Workshop on Cognitive Aspects of the Lexicon, pages 215–230.

Yudina M. V., Garaschuk R. V., Petrova M. A., Druzhkina A. A. 2018. Semi-Automatic Integration Of A New Language Into A Multilingual NLP Model: The Case Of Japanese // Computational Linguistics and Intellectual Technologies.

J Detkova, V Novitskiy, M Petrova, and V Selegey. 2020. Differential semantic sketches for russian internet corpora. // Computational Linguistics and Intellectual Technologies, P 211–227.