Determining the level of agreement of expert estimates Online publication date: Fri, 27-Apr-2007
by Valentinas Podvezko
International Journal of Management and Decision Making (IJMDM), Vol. 8, No. 5/6, 2007
Abstract: This paper considers the techniques of determining the concordance of expert estimates. Expert methods of evaluating particular criteria may be practically used, for instance, in multi-criteria decision support systems, if judgements are in good agreement. This is expressed by the concordance coefficient calculated by using expert-provided data. The concordance coefficient can be calculated only if the considered criteria are ranked. If other evaluation methods are used by experts, the data should be transformed to rankings before the calculation of the concordance coefficient is made. There is the dependence of the agreement of expert estimates on a particular method used. The highest degree of agreement has been obtained by using a direct ranking method. The effect of equally assessed criteria, that is, the tied ranks, on the concordance coefficient and thereby on the level of expert judgements agreement is usually insignificant and cannot change the results of rating.
Existing subscribers:
Go to Inderscience Online Journals to access the Full Text of this article.
If you are not a subscriber and you just want to read the full contents of this article, buy online access here.Complimentary Subscribers, Editors or Members of the Editorial Board of the International Journal of Management and Decision Making (IJMDM):
Login with your Inderscience username and password:
Want to subscribe?
A subscription gives you complete access to all articles in the current issue, as well as to all articles in the previous three years (where applicable). See our Orders page to subscribe.
If you still need assistance, please email subs@inderscience.com
Our Blog 
Follow us on Twitter 
Visit us on Facebook