Semantic subcategorisation for creative generation of light verb constructions
Details
Lin de Huybrecht (Add me on LinkedIn) E-mail:lin.de.huybrecht@vub.be Promotor: Prof. Dr. Dr. Geraint Wiggins Affiliation: Computational Creativity Lab, AI Lab Vrije Universiteit Brussel (VUB)
Event: Flanders Artificial Intelligence Program (FLAIR) Research Day 14/10/2024 FLAIR info: GC2 (Situated AI), WP3 (Interaction & Cognitive Systems), task 3.1 (Cognitive Architecture and Knowledge Representation), use case: Society: Education & training
Current language models are not able to capture nuance in linguistic expressions in a transparent way. To model this nuance, we need transparent language models that are grounded in linguistics and cognitive science. A possible answer to this problem is the DisCoCat (Distributional Compositional Categorical) framework, that explicitly models both the syntax and semantics of language. It has been successfully used for modelling the meaning of existing text. This research focuses on extending the DisCoCat framework for creative natural language generation. More specifically, we focus on generating text in the light verb construction domain. This research will give us a deeper understanding of how semantic spaces interact to convey meaning in language.
"the philosophy, science and engineering of computational systems which, by taking on particular responsibilities, exhibit behaviours that unbiased observers would deem to be creative”.
If you want to know more about Computational Creativity, here are some important papers from the field:
Wiggins, G. A. (2006). Searching for computational creativity. New Generation Computing, 24(3), 209–222. DOI: 10.1007/BF03037332
Wiggins, G. A. (2006). A preliminary framework for description, analysis and comparison of creative systems. Knowledge-Based Systems, 19(7), 449–458. DOI: 10.1016/j.knosys.2006.04.009
Ritchie, G. (2007). Some Empirical Criteria for Attributing Creativity to a Computer Program. Minds and Machines, 17(1), 67–99. DOI: 10.1007/s11023-007-9066-2
Wiggins, G. A., Tyack, P., Scharff, C., & Rohrmeier, M. (2015). The evolutionary roots of creativity: Mechanisms and motivations. Philosophical Transactions of the Royal Society B: Biological Sciences, 370(1664). DOI: 10.1098/rstb.2014.0099
Ventura, D. (2017). How to Build a CC System Paper type: System Description Paper. Proceedings of the Eleventh International Conference on Computational Creativity, 262–269. Download
What are light verb constructions?
Light verb constructions are complex predicates that have a semantically bleached verb.
Here are some publications about light verb constructions:
Wittenberg, E. (2016). With light verb constructions from syntax to concepts (Vol. 7). Universitätsverlag Potsdam. Download
Wittenberg, E., & Levy, R. (2017). If you want a quick kiss, make it count: How choice of syntactic construction affects event construal. Journal of memory and language, 94, 254-271. Download
Gilquin, G. (2019). Light verb constructions in spoken L2 English: An exploratory cross-sectional study. International Journal of Learner Corpus Research, 5(2), 181-206. Download
What is the difference between light verbs, auxiliary verbs and full verbs?
Full verbs carry semantic information in a sense that light verbs do not. Light verbs need (an) additional word(s), usually a noun, to convey meaning. Light verbs are somewhat similar to auxiliary verbs, but do not satisfy the tests for whether or not a verb is an auxiliary verb or not.
What are some examples of light verb constructions in Dutch?
een afspraak maken (afspreken)
een douche nemen (douchen)
een speech geven (speechen)
een pauze nemen (pauzeren)
een dutje doen (dutten)
een presentatie geven (presenteren)
een knuffel geven (knuffelen)
What is the DisCoCat framework?
The DisCoCat (Distributional Compositional Categorical) framework uses category theory (Eilenberg & MacLane, 1945) to represent both grammar and semantics and the relation between them. In this framework, sentences are modelled by using one category for the semantics of language (e.g., FVect, the category of finite‐dimensional vector spaces) and one for the grammar (e.g., a pregroup, Preg, or any other categorial grammar).
The meanings contained in the semantics space are obtained using distributional methods. The types of the categorial grammar for the grammar space can be obtained using a part‐of‐speech‐tagger.
The system can compute whether or not consecutive words make up a well‐formed sequence, according to pre‐specified types, which may or may not result in grammatical reductions that reduce to the sentence type. The key here is that the two categories share a common structure, or in other words: they belong to a weaker category (FVect and Preg are both compact closed). This will ensure that there exists a mapping from the grammar category to the semantics category so that we end up with a new category, i.e., the combination of the grammar and semantics categories. This results in a truly compositional model.
You can read more about DisCoCat here:
Coecke, B., Sadrzadeh, M., & Clark, S. J. (2010). Mathematical foundations for a compositional distributional model of meaning. Linguistic Analysis, 36(1–4), 345–384. https://arxiv.org/pdf/1003.4394
Coecke, B., Grefenstette, E., & Sadrzadeh, M. (2013). Lambek vs. Lambek: Functorial vector space semantics and string diagrams for Lambek calculus. Annals of Pure and Applied Logic, 164(11), 1079–1100. DOI: 10.1016/j.apal.2013.05.009
Sadrzadeh, M., Clark, S., & Coecke, B. (2013). The Frobenius anatomy of word meanings I: subject and object relative pronouns. Journal of Logic and Computation, 23(6), 1293-1317. DOI: 10.1093/logcom/ext044
Sadrzadeh, M., Clark, S., & Coecke, B. (2014). The Frobenius anatomy of word meanings II: possessive relative pronouns. Journal of Logic and Computation, 26(2), 785-815. DOI: 10.1093/logcom/exu027
Sadrzadeh, M., Kartsaklis, D., & Balkır, E. (2018). Sentence Entailment in Compositional Distributional Semantics. Annals of Mathematics and Artificial Intelligence, 82(4), 189–218. DOI: 10.1007/s10472-017-9570-x
What does DisCoCat offer that most large language models do not?
First of all, it is transparent and meaning‐aware (Coecke et al., 2022). Once the grammar and semantics categories are instantiated, every step in the process of constructing meaning representations is interpretable, which is not the case in most deep learning methods. The relations between word meanings are explicitly modelled in DisCoCat (Coecke, 2017) and the grammaticality of sentences is verified via type‐reductions. Furthermore, DisCoCat has been demonstrated to outperform noncompositional models and n‐gram models (Grefenstette, 2013) on word‐sense disambiguation tasks and verb disambiguation tasks (Wijnholds, 2020).
Anything you want to discuss? Get in touch with me through LinkedIn or via e-mail.