Title: Is the log score in line with forecasters' preferences?

Authors: Peter Schanbacher

Addresses: Department of Economics, University of Konstanz, Universitätsstraße 10, D-78464, Konstanz, Germany

Abstract: Scoring rules are the main tool to evaluate the quality of probabilistic forecasts. The most popular scoring rule is the log score. Under the log score interesting results have been derived. We present two counterintuitive properties of the log score. Further we show that the probabilistic predictions of the ECB's survey of professional forecasters and a large sample of economics students are in sharp contrast to the implications of the log score. Is the advice drawn by the log score really sensible to forecasters having another loss function? We conclude that some surprising results are solely driven by the special form of the log score and are not shared by other scoring rules or real world forecasters.

Keywords: density forecasting; forecast combination; scoring rules; log score; probabilistic forecasts; quality evaluation; loss function; professional forecasters; economics students.

DOI: 10.1504/IJADS.2013.056882

International Journal of Applied Decision Sciences, 2013 Vol.6 No.4, pp.406 - 430

Received: 01 Mar 2013
Accepted: 17 May 2013
Published online: 28 Nov 2013 *

Full-text access for editors Full-text access for subscribers Purchase this article Comment on this article