**Working Papers, Konjunkturinstitutet - National Institute of Economic Research**
# No 61:

Model Evaluation Using Stochastic Simulations: The Case of the Econometric Model KOSMOS

*Jan B. Gajda and Aleksander Markowski *()

**Abstract:** One aspect of model behaviour that is of interest to the
model builder is sensitivity to different forms of errors. This can be
investigated using stochastic simulations, as shown by Gajda [1995]. The
method involves generating random numbers from a given (usually normal)
distribution and introducing them as shocks to the model. In particular,
stochastic simulations can be employed to make an empirical investigation
into the following aspects of the model:

- the effects on the
forecast of random disturbances,
- the effects on the forecast of
random variation in equation parameters (sampling errors),
- error
propagation and accumulation patterns in the model,
- the effects on
the forecast of (random) errors in the exogenous variables.

The results of stochastic simulations can provide information on - inter
alia - the sampling distribution of the model forecast. In particular, the
model builder may be interested in the shape of this distribution. If it is
not symmetric, the mean (stochastic) forecast will be different from the
median forecast (which under certain conditions is equal to the
deterministic forecast). Furthermore, if the distribution is skewed, a
typical stochastic forecast (represented by the mode) will systematically
underestimate (or overestimate) both the mean and the median forecasts.

The purpose of the present paper is to investigate KOSMOS, the
econometric model of the National Institute of Economic Research in
Stockholm, from the point of view of the first three aspects mentioned
above. The main aim of this exercise is to look for ”weak links” in the
model, i.e. to find out which equations introduce most uncertainty and at
the same time are crucial for the forecast because of their strong
influence on it. Thus, our interest is not only in assessing the forecast
error variance as a descriptive statistic, but also - and primarily - in
finding those equations that are important for error propagation and those
coefficients whose values are crucial for the model.

The outline
of the paper is as follows. Section 2 discusses the analysis of expected
forecast errors (for linear models) based on analytical formulae. In
Section 3, model simulations and stochastic simulations are defined. The
two subsequent sections discuss the purpose of our experiments and their
design, respectively. Section 6 gives a brief description of the
econometric model KOSMOS, whose equations are subject to our investigation.
Section 7 describes the results of stochastic simulations with additive
equation disturbances. Section 8 presents the results of stochastic
simulations with both equation disturbances and disturbances to the
estimated coefficients. Sampling distributions of forecasts are discussed
in Section 9. Section 10 concludes.

41 pages, August 1, 1998

Before downloading any of the electronic versions below
you should read our statement on
copyright.

Download GhostScript
for viewing Postscript files and the
Acrobat Reader for viewing and printing pdf files.

**Full text versions of the paper:**

WP_61.pdf

Download Statistics

Questions (including download problems) about the papers in this series should be directed to Sarah Hegardt Grant ()

Report other problems with accessing this service to Sune Karlsson ()
or Helena Lundin ().

Programing by

Design by Joachim Ekebom