Eliciting and combining expert opinion : an overview and comparison of methods
- Authors: Chinyamakobvu, Mutsa Carole
- Date: 2015
- Subjects: Decision making -- Statistical methods , Expertise , Bayesian statistical decision theory , Statistical decision , Delphi method , Paired comparisons (Statistics)
- Language: English
- Type: Thesis , Masters , MSc
- Identifier: vital:5579 , http://hdl.handle.net/10962/d1017827
- Description: Decision makers have long relied on experts to inform their decision making. Expert judgment analysis is a way to elicit and combine the opinions of a group of experts to facilitate decision making. The use of expert judgment is most appropriate when there is a lack of data for obtaining reasonable statistical results. The experts are asked for advice by one or more decision makers who face a specific real decision problem. The decision makers are outside the group of experts and are jointly responsible and accountable for the decision and committed to finding solutions that everyone can live with. The emphasis is on the decision makers learning from the experts. The focus of this thesis is an overview and comparison of the various elicitation and combination methods available. These include the traditional committee method, the Delphi method, the paired comparisons method, the negative exponential model, Cooke’s classical model, the histogram technique, using the Dirichlet distribution in the case of a set of uncertain proportions which must sum to one, and the employment of overfitting. The supra Bayes approach, the determination of weights for the experts, and combining the opinions of experts where each opinion is associated with a confidence level that represents the expert’s conviction of his own judgment are also considered.
- Full Text:
- Date Issued: 2015
- Authors: Chinyamakobvu, Mutsa Carole
- Date: 2015
- Subjects: Decision making -- Statistical methods , Expertise , Bayesian statistical decision theory , Statistical decision , Delphi method , Paired comparisons (Statistics)
- Language: English
- Type: Thesis , Masters , MSc
- Identifier: vital:5579 , http://hdl.handle.net/10962/d1017827
- Description: Decision makers have long relied on experts to inform their decision making. Expert judgment analysis is a way to elicit and combine the opinions of a group of experts to facilitate decision making. The use of expert judgment is most appropriate when there is a lack of data for obtaining reasonable statistical results. The experts are asked for advice by one or more decision makers who face a specific real decision problem. The decision makers are outside the group of experts and are jointly responsible and accountable for the decision and committed to finding solutions that everyone can live with. The emphasis is on the decision makers learning from the experts. The focus of this thesis is an overview and comparison of the various elicitation and combination methods available. These include the traditional committee method, the Delphi method, the paired comparisons method, the negative exponential model, Cooke’s classical model, the histogram technique, using the Dirichlet distribution in the case of a set of uncertain proportions which must sum to one, and the employment of overfitting. The supra Bayes approach, the determination of weights for the experts, and combining the opinions of experts where each opinion is associated with a confidence level that represents the expert’s conviction of his own judgment are also considered.
- Full Text:
- Date Issued: 2015
Models for forecasting residential property prices using paired comparisons
- Authors: Mpapela, Sinazo
- Date: 2014
- Subjects: Paired comparisons (Statistics) , Statistics
- Language: English
- Type: Thesis , Masters , MSc
- Identifier: http://hdl.handle.net/10948/5839 , vital:21006
- Description: Residential real estate forecasting has become a part of the larger process of business planning and strategic management. Several studies of housing price trends recommend confining statistical analysis to repeated sales of residential property. This study presents an alternate methodology which combines information only on repeated residential sales regardless of the changes that has been made in the house in-between the sales. Additive and multiplicative models were used to forecast the residential property prices in Nelson Mandela Metropole. Data was collected from various sources and was reconciled into one data set for analysis through a process of data screening.
- Full Text:
- Date Issued: 2014
- Authors: Mpapela, Sinazo
- Date: 2014
- Subjects: Paired comparisons (Statistics) , Statistics
- Language: English
- Type: Thesis , Masters , MSc
- Identifier: http://hdl.handle.net/10948/5839 , vital:21006
- Description: Residential real estate forecasting has become a part of the larger process of business planning and strategic management. Several studies of housing price trends recommend confining statistical analysis to repeated sales of residential property. This study presents an alternate methodology which combines information only on repeated residential sales regardless of the changes that has been made in the house in-between the sales. Additive and multiplicative models were used to forecast the residential property prices in Nelson Mandela Metropole. Data was collected from various sources and was reconciled into one data set for analysis through a process of data screening.
- Full Text:
- Date Issued: 2014
Time series models for paired comparisons
- Authors: Sjolander, Morne Rowan
- Date: 2011
- Subjects: Paired comparisons (Statistics)
- Language: English
- Type: Thesis , Doctoral , PhD
- Identifier: vital:10577 , http://hdl.handle.net/10948/d1012858
- Description: The method of paired comparisons is seen as a technique used to rank a set of objects with respect to an abstract or immeasurable property. To do this, the objects get to be compared two at a time. The results are input into a model, resulting in numbers known as weights being assigned to the objects. The weights are then used to rank the objects. The method of paired comparisons was first used for psychometric investigations. Various other applications of the method are also present, for example economic applications, and applications in sports statistics. This study involves taking paired comparison models and making them time-dependent. Not much research has been done in this area. Three new time series models for paired comparisons are created. Simulations are done to support the evidence obtained, and theoretical as well as practical examples are given to illustrate the results and to verify the efficiency of the new models. A literature study is given on the method of paired comparisons, as well as on the areas in which we apply our models. Our first two time series models for paired comparisons are the Linear-Trend Bradley- Terry Model and the Sinusoidal Bradley-Terry Model. We use the maximum likelihood approach to solve these models. We test our models using exact and randomly simulated data for various time periods and various numbers of objects. We adapt the Linear-Trend Bradley-Terry Model and received our third time series model for paired comparisons, the Log Linear-Trend Bradley-Terry Model. The daily maximum and minimum temperatures were received for Port Elizabeth, Uitenhage and Coega for 2005 until 2009. To evaluate the performance of the Linear-Trend Bradley-Terry Model and the Sinusoidal Bradley-Terry Model on estimating missing temperature data, we artificially remove observations of temperature from Coega’s temperature dataset for 2006 until 2008, and use various forms of these models to estimate the missing data points. The exchange rates for 2005 until 2008 between the following currencies: the Rand, Dollar, Euro, Pound and Yen, were obtained and various forms of our Log Linear-Trend Bradley-Terry Model are used to forecast the exchange rate for one day ahead for each month in 2006 until 2008. One of the features of this study is that we apply our time series models for paired comparisons to areas which comprise non-standard paired comparisons; and we want to encourage the use of the method of paired comparisons in a broader sense than what it is traditionally used for. The results of this study can be used in various other areas, like for example, in sports statistics, to rank the strength of sports players and predict their future scores; in Physics, to calculate weather risks of electricity generation, particularly risks related to nuclear power plants, and so forth, as well as in many other areas. It is hoped that this research will open the door to much more research in combining time series analysis with the method of paired comparisons.
- Full Text:
- Date Issued: 2011
- Authors: Sjolander, Morne Rowan
- Date: 2011
- Subjects: Paired comparisons (Statistics)
- Language: English
- Type: Thesis , Doctoral , PhD
- Identifier: vital:10577 , http://hdl.handle.net/10948/d1012858
- Description: The method of paired comparisons is seen as a technique used to rank a set of objects with respect to an abstract or immeasurable property. To do this, the objects get to be compared two at a time. The results are input into a model, resulting in numbers known as weights being assigned to the objects. The weights are then used to rank the objects. The method of paired comparisons was first used for psychometric investigations. Various other applications of the method are also present, for example economic applications, and applications in sports statistics. This study involves taking paired comparison models and making them time-dependent. Not much research has been done in this area. Three new time series models for paired comparisons are created. Simulations are done to support the evidence obtained, and theoretical as well as practical examples are given to illustrate the results and to verify the efficiency of the new models. A literature study is given on the method of paired comparisons, as well as on the areas in which we apply our models. Our first two time series models for paired comparisons are the Linear-Trend Bradley- Terry Model and the Sinusoidal Bradley-Terry Model. We use the maximum likelihood approach to solve these models. We test our models using exact and randomly simulated data for various time periods and various numbers of objects. We adapt the Linear-Trend Bradley-Terry Model and received our third time series model for paired comparisons, the Log Linear-Trend Bradley-Terry Model. The daily maximum and minimum temperatures were received for Port Elizabeth, Uitenhage and Coega for 2005 until 2009. To evaluate the performance of the Linear-Trend Bradley-Terry Model and the Sinusoidal Bradley-Terry Model on estimating missing temperature data, we artificially remove observations of temperature from Coega’s temperature dataset for 2006 until 2008, and use various forms of these models to estimate the missing data points. The exchange rates for 2005 until 2008 between the following currencies: the Rand, Dollar, Euro, Pound and Yen, were obtained and various forms of our Log Linear-Trend Bradley-Terry Model are used to forecast the exchange rate for one day ahead for each month in 2006 until 2008. One of the features of this study is that we apply our time series models for paired comparisons to areas which comprise non-standard paired comparisons; and we want to encourage the use of the method of paired comparisons in a broader sense than what it is traditionally used for. The results of this study can be used in various other areas, like for example, in sports statistics, to rank the strength of sports players and predict their future scores; in Physics, to calculate weather risks of electricity generation, particularly risks related to nuclear power plants, and so forth, as well as in many other areas. It is hoped that this research will open the door to much more research in combining time series analysis with the method of paired comparisons.
- Full Text:
- Date Issued: 2011
SL-model for paired comparisons
- Authors: Sjölander, Morné Rowan
- Date: 2006
- Subjects: Paired comparisons (Statistics) , Mathematical statistics
- Language: English
- Type: Thesis , Masters , MSc
- Identifier: vital:10574 , http://hdl.handle.net/10948/605 , Paired comparisons (Statistics) , Mathematical statistics
- Description: The method of paired comparisons can be found all the way back to 1860, where Fechner made the first publication in this method, using it for his psychometric investigations [4]. Thurstone formalised the method by providing a mathematical background to it [9-11] and in 1927 the method’s birth took place with his psychometric publications, one being “a law of comparative judgment” [12-14]. The law of comparative judgment is a set of equations relating the proportion of times any stimulus k is judged greater on a given attribute than any other stimulus j to the scales and discriminal dispersions of the two stimuli on the psychological continuum. The amount of research done for discrete models of paired comparisons is not a lot. This study develops a new discrete model, the SL-model for paired comparisons. Paired comparisons data processing in which objects have an upper limit to their scores was also not yet developed, and making such a model is one of the aims of this report. The SLmodel is thus developed in this context; however, the model easily generalises to not necessarily having an upper limit on scores.
- Full Text:
- Date Issued: 2006
- Authors: Sjölander, Morné Rowan
- Date: 2006
- Subjects: Paired comparisons (Statistics) , Mathematical statistics
- Language: English
- Type: Thesis , Masters , MSc
- Identifier: vital:10574 , http://hdl.handle.net/10948/605 , Paired comparisons (Statistics) , Mathematical statistics
- Description: The method of paired comparisons can be found all the way back to 1860, where Fechner made the first publication in this method, using it for his psychometric investigations [4]. Thurstone formalised the method by providing a mathematical background to it [9-11] and in 1927 the method’s birth took place with his psychometric publications, one being “a law of comparative judgment” [12-14]. The law of comparative judgment is a set of equations relating the proportion of times any stimulus k is judged greater on a given attribute than any other stimulus j to the scales and discriminal dispersions of the two stimuli on the psychological continuum. The amount of research done for discrete models of paired comparisons is not a lot. This study develops a new discrete model, the SL-model for paired comparisons. Paired comparisons data processing in which objects have an upper limit to their scores was also not yet developed, and making such a model is one of the aims of this report. The SLmodel is thus developed in this context; however, the model easily generalises to not necessarily having an upper limit on scores.
- Full Text:
- Date Issued: 2006
An evaluation of paired comparison models
- Venter, Daniel Jacobus Lodewyk
- Authors: Venter, Daniel Jacobus Lodewyk
- Date: 2004
- Subjects: Paired comparisons (Statistics) , Mathematical statistics
- Language: English
- Type: Thesis , Masters , MSc
- Identifier: vital:11087 , http://hdl.handle.net/10948/364 , Paired comparisons (Statistics) , Mathematical statistics
- Description: Introduction: A typical task in quantitative data analysis is to derive estimates of population parameters based on sample statistics. For manifest variables this is usually a straightforward process utilising suitable measurement instruments and standard statistics such the mean, median and standard deviation. Latent variables on the other hand are typically more elusive, making it difficult to obtain valid and reliable measurements. One of the most widely used methods of estimating the parameter value of a latent variable is to use a summated score derived from a set of individual scores for each of the various attributes of the latent variable. A serious limitation of this method and other similar methods is that the validity and reliability of measurements depend on whether the statements included in the questionnaire cover all characteristics of the variable being measured and also on respondents’ ability to correctly indicate their perceived assessment of the characteristics on the scale provided. Methods without this limitation and that are especially useful where a set of objects/entities must be ranked based on the parameter values of one or more latent variables, are methods of paired comparisons. Although the underlying assumptions and algorithms of these methods often differ dramatically, they all rely on data derived from a series of comparisons, each consisting of a pair of specimens selected from the set of objects/entities being investigated. Typical examples of the comparison process are: subjects (judges) who have to indicate for each pair of objects which of the two they prefer; sport teams that compete against each other in matches that involve two teams at a time. The resultant data of each comparison range from a simple dichotomy to indicate which of the two objects are preferred/better, to an interval or ratio scale score for e d Bradley-Terry models, and were based on statistical theory assuming that the variable(s) being measured is either normally (Thurstone-Mosteller) or exponentially (Bradley-Terry) distributed. For many years researchers had to rely on these PCM’s when analysing paired comparison data without any idea about the implications if the distribution of the data from which their sample were obtained differed from the assumed distribution for the applicable PCM being utilised. To address this problem, PCM’s were subsequently developed to cater for discrete variables and variables with distributions that are neither normal or exponential. A question that remained unanswered is how the performance, as measured by the accuracy of parameter estimates, of PCM's are affected if they are applied to data from a range of discrete and continuous distribution that violates the assumptions on which the applicable paired comparison algorithm is based. This study is an attempt to answer this question by applying the most popular PCM's to a range of randomly derived data sets that spans typical continuous and discrete data distributions. It is hoped that the results of this study will assist researchers when selecting the most appropriate PCM to obtain accurate estimates of the parameters of the variables in their data sets.
- Full Text:
- Date Issued: 2004
- Authors: Venter, Daniel Jacobus Lodewyk
- Date: 2004
- Subjects: Paired comparisons (Statistics) , Mathematical statistics
- Language: English
- Type: Thesis , Masters , MSc
- Identifier: vital:11087 , http://hdl.handle.net/10948/364 , Paired comparisons (Statistics) , Mathematical statistics
- Description: Introduction: A typical task in quantitative data analysis is to derive estimates of population parameters based on sample statistics. For manifest variables this is usually a straightforward process utilising suitable measurement instruments and standard statistics such the mean, median and standard deviation. Latent variables on the other hand are typically more elusive, making it difficult to obtain valid and reliable measurements. One of the most widely used methods of estimating the parameter value of a latent variable is to use a summated score derived from a set of individual scores for each of the various attributes of the latent variable. A serious limitation of this method and other similar methods is that the validity and reliability of measurements depend on whether the statements included in the questionnaire cover all characteristics of the variable being measured and also on respondents’ ability to correctly indicate their perceived assessment of the characteristics on the scale provided. Methods without this limitation and that are especially useful where a set of objects/entities must be ranked based on the parameter values of one or more latent variables, are methods of paired comparisons. Although the underlying assumptions and algorithms of these methods often differ dramatically, they all rely on data derived from a series of comparisons, each consisting of a pair of specimens selected from the set of objects/entities being investigated. Typical examples of the comparison process are: subjects (judges) who have to indicate for each pair of objects which of the two they prefer; sport teams that compete against each other in matches that involve two teams at a time. The resultant data of each comparison range from a simple dichotomy to indicate which of the two objects are preferred/better, to an interval or ratio scale score for e d Bradley-Terry models, and were based on statistical theory assuming that the variable(s) being measured is either normally (Thurstone-Mosteller) or exponentially (Bradley-Terry) distributed. For many years researchers had to rely on these PCM’s when analysing paired comparison data without any idea about the implications if the distribution of the data from which their sample were obtained differed from the assumed distribution for the applicable PCM being utilised. To address this problem, PCM’s were subsequently developed to cater for discrete variables and variables with distributions that are neither normal or exponential. A question that remained unanswered is how the performance, as measured by the accuracy of parameter estimates, of PCM's are affected if they are applied to data from a range of discrete and continuous distribution that violates the assumptions on which the applicable paired comparison algorithm is based. This study is an attempt to answer this question by applying the most popular PCM's to a range of randomly derived data sets that spans typical continuous and discrete data distributions. It is hoped that the results of this study will assist researchers when selecting the most appropriate PCM to obtain accurate estimates of the parameters of the variables in their data sets.
- Full Text:
- Date Issued: 2004
- «
- ‹
- 1
- ›
- »