- Title
- Augmenting encoder-decoder networks for first-order logic formula parsing using attention pointer mechanisms
- Creator
- Tissink, Kade
- Subject
- Translators (Computer programs)
- Subject
- Computational linguistics
- Subject
- Computer science
- Date Issued
- 2024-04
- Date
- 2024-04
- Type
- Master's theses
- Type
- text
- Identifier
- http://hdl.handle.net/10948/64390
- Identifier
- vital:73692
- Description
- Semantic parsing is the task of extracting a structured machine-interpretable representation from natural language utterance. This representation can be used for various applications such as question answering, information extraction, and dialogue systems. However, semantic parsing is a challenging problem that requires dealing with the ambiguity, variability, and complexity of natural language. This dissertation investigates neural parsing of natural language (NL) sentences to first-order logic (FOL) formulas. FOL is a widely used formal language for expressing logical statements and reasoning. FOL formulas can capture the meaning and structure of natural language sentences in a precise and unambiguous way. The problem is initially approached as a sequence-to-sequence mapping task using both LSTM-based and transformer encoder-decoder architectures for character-, subword-, and wordlevel text tokenisation. These models are trained on NL-FOL datasets using supervised learning and evaluated on various metrics such as exact match accuracy, syntactic validity, formula structure accuracy, and predicate/constant similarity. A novel augmented model is then introduced that decomposes the task of neural FOL parsing into four inter-dependent subtasks: template decoding, predicate and constant recognition, predicate set pointing, and object set pointing. The components for the four subtasks are jointly trained using multi-task learning and evaluated using the same metrics from the sequence-tosequence models. The results indicate improved performance over the sequence-to-sequence models and the modular design allows for more interpretability and flexibility. Additionally, to compensate for the scarcity of open-source, labelled NL-FOL datasets, a new benchmark is constructed from publicly accessible data. The data consists of NL sentences paired with corresponding FOL formulas in a standardised notation. The data is split into training, validation, and test sets. The main contributions of this dissertation are: an in-depth literature review covering decades of research presented with a consistent notation, the formation of a complex NL-FOL benchmark that includes algorithmically generated and human-annotated FOL formulas, proposal of a novel transformer encoder-decoder architecture that is shown to successfully train at significant depths, evaluation of twenty sequence-to-sequence models on the task of neural FOL parsing for different text representations and encoder-decoder architectures, the proposal of a novel augmented FOL parsing architecture, and an in-depth analysis of the strengths and weaknesses of these models.
- Description
- Thesis (MSc) -- Faculty of Science, School of Computer Science, Mathematics, Physics and Statistics , 2024
- Format
- computer
- Format
- online resource
- Format
- application/pdf
- Format
- 1 online resource (xii, 198 pages)
- Format
- Publisher
- Nelson Mandela University
- Publisher
- Faculty of Science
- Language
- English
- Rights
- Nelson Mandela University
- Rights
- All Rights Reserved
- Rights
- Open Access
- Hits: 236
- Visitors: 244
- Downloads: 16
Collections
NMU School of Computer Science, Mathematics, Physics and Statistics
| Thumbnail | File | Description | Size | Format | |||
|---|---|---|---|---|---|---|---|
| View Details Download | SOURCE1 | Tissink, KD.pdf | 2 MB | Adobe Acrobat PDF | View Details Download |