Reference Object Choice in Spatial Language: Machine and Human Models

DSpace/Manakin Repository

Open Research Exeter (ORE)

Reference Object Choice in Spatial Language: Machine and Human Models

Show simple item record

dc.contributor.author Barclay, Michael John en_US
dc.date.accessioned 2011-06-29T14:40:55Z en_US
dc.date.accessioned 2013-03-21T10:49:35Z
dc.date.issued 2010-08-17 en_US
dc.description.abstract The thesis underpinning this study is as follows; it is possible to build machine models that are indistinguishable from the mental models used by humans to generate language to describe their environment. This is to say that the machine model should perform in such a way that a human listener could not discern whether a description of a scene was generated by a human or by the machine model. Many linguistic processes are used to generate even simple scene descriptions and developing machine models of all of them is beyond the scope of this study. The goal of this study is, therefore, to model a sufficient part of the scene description process, operating in a sufficiently realistic environment, so that the likelihood of being able to build machine models of the remaining processes, operating in the real world, can be established. The relatively under-researched process of reference object selection is chosen as the focus of this study. A reference object is, for instance, the `table' in the phrase ``The flowers are on the table''. This study demonstrates that the reference selection process is of similar complexity to others involved in generating scene descriptions which include: assigning prepositions, selecting reference frames and disambiguating objects (usually termed `generating referring expressions'). The secondary thesis of this study is therefore; it is possible to build a machine model that is indistinguishable from the mental models used by humans in selecting reference objects. Most of the practical work in the study is aimed at establishing this. An environment sufficiently near to the real-world for the machine models to operate on is developed as part of this study. It consists of a series of 3-dimensional scenes containing multiple objects that are recognisable to humans and `readable' by the machine models. The rationale for this approach is discussed. The performance of human subjects in describing this environment is evaluated, and measures by which the human performance can be compared to the performance of the machine models are discussed. The machine models used in the study are variants on Bayesian networks. A new approach to learning the structure of a subset of Bayesian networks is presented. Simple existing Bayesian classifiers such as naive or tree augmented naive networks did not perform sufficiently well. A significant result of this study is that useful machine models for reference object choice are of such complexity that a machine learning approach is required. Earlier proposals based on sum-of weighted-factors or similar constructions will not produce satisfactory models. Two differently derived sets of variables are used and compared in this study. Firstly variables derived from the basic geometry of the scene and the properties of objects are used. Models built from these variables match the choice of reference of a group of humans some 73\% of the time, as compared with 90\% for the median human subject. Secondly variables derived from `ray casting' the scene are used. Ray cast variables performed much worse than anticipated, suggesting that humans use object knowledge as well as immediate perception in the reference choice task. Models combining geometric and ray-cast variables match the choice of reference of the group of humans some 76\% of the time. Although niether of these machine models are likely to be indistinguishable from a human, the reference choices are rarely, if ever, entirely ridiculous. A secondary goal of the study is to contribute to the understanding of the process by which humans select reference objects. Several statistically significant results concerning the necessary complexity of the human models and the nature of the variables within them are established. Problems that remain with both the representation of the near-real-world environment and the Bayesian models and variables used within them are detailed. While these problems cast some doubt on the results it is argued that solving these problems is possible and would, on balance, lead to improved performance of the machine models. This further supports the assertion that machine models producing reference choices indistinguishable from those of humans are possible. en_GB
dc.identifier.uri http://hdl.handle.net/10036/3163 en_US
dc.language.iso en en_GB
dc.publisher University of Exeter en_GB
dc.rights.embargoreason To allow time for papers arising from the Thesis to be published en_GB
dc.subject Spatial Language en_GB
dc.subject Reference Object Choice en_GB
dc.subject Machine Learning en_GB
dc.subject Bayesian Networks en_GB
dc.title Reference Object Choice in Spatial Language: Machine and Human Models en_GB
dc.type Thesis or dissertation en_GB
dc.date.available 2012-12-31T05:00:05Z en_US
dc.date.available 2013-03-21T10:49:35Z
dc.contributor.advisor Galton, Antony en_US
dc.publisher.department Computer Science en_GB
dc.type.degreetitle PhD in Computer Science en_GB
dc.type.qualificationlevel Doctoral en_GB
dc.type.qualificationname PhD en_GB


Files in this item

Files Size Format View Description
BarclayM_fm.pdf 40.59Kb PDF View/Open Thesis front matter
BarclayM.pdf 14.70Mb PDF View/Open Full Thesis

This item appears in the following Collection(s)

Show simple item record

Browse

My Account

Local Links