Does the Bic Estimate and Forecast Better than the Aic?*

Carlos A. Medel; Sergio C. Salgado

doi:10.4067/S0718-88702013000100003

Autores/as

Carlos A. Medel Central Bank of Chile
Sergio C. Salgado University of Minnesota

DOI:

https://doi.org/10.4067/S0718-88702013000100003

Palabras clave:

AIC, BIC, criterios de información, modelos de series de tiempo, sobreajuste, evaluación de proyecciones, prueba de hipótesis conjunta

Resumen

Contestamos dos preguntas: (i) ¿Es el criterio de información bayesiano (BIC) más parsimonioso que el criterio de información de Akaike (AIC)? y (ii) ¿Es el BIC mejor que el AIC para fines predictivos? Mediante el uso de datos simulados, proporcionamos inferencia estadística con respecto a ambas hipótesis de manera individual y luego conjunta con un procedimiento de pruebas de hipótesis múltiples para controlar mejor el error tipo I. Ambas pruebas entregan el mismo resultado: el BIC muestra una superioridad dentro y fuera de la muestra sobre el AIC sólo en un contexto de muestra larga.

Descargas

Los datos de descargas todavía no están disponibles.

Citas

Akaike, H. (1974). A new look at the statistical model identification. IEEE Transactions on Automatic Control, 19(6), 716–723. https://doi.org/10.1109/tac.1974.1100705

Bhansali, R. J., & Downham, D. Y. (1977). Some properties of the order of an autoregressive model selected by a generalization of Akaike's EPF criterion. Biometrika, 64(3), 547–551. https://doi.org/10.1093/biomet/64.3.547

Bickel, P., & Zhang, P. (1992). Variable selection in nonparametric regression with categorical covariates. Journal of the American Statistical Association, 87(417), 90–97. https://doi.org/10.2307/2290456

Bozdogan, H. (1987). Model selection and Akaike's information criterion (AIC): The general theory and its analytical extensions. Psychometrika, 52(3), 345–370. https://doi.org/10.1007/bf02294361

Burnham, K. P., & Anderson, D. R. (1998). Model selection and inference: A practical information theoretic approach. Springer.

Canova, F. (2007). Methods for applied macroeconomic research. Princeton University Press.

Chen, L., Giannakouros, P., & Yang, Y. (2007). Model combining in factorial data analysis. Journal of Statistical Planning and Inference, 137(9), 2920–2934. https://doi.org/10.1016/j.jspi.2006.10.005

Clark, T. E. (2004). Can out-of-sample forecast comparisons help to prevent overfitting? Journal of Forecasting, 23(2), 115–139.

Clark, T. E., & McCracken, M. (2001). Tests of equal forecast accuracy and encompassing for nested models. Journal of Econometrics, 105(1), 85–110. https://doi.org/10.1016/s0304-4076(01)00071-9

Corradi, V., & Distaso, W. (2011). Multiple forecast model evaluation. In M. P. Clements & D. F. Hendry (Eds.), The Oxford handbook of economic forecasting (pp. 391–414). Oxford University Press. https://doi.org/10.1093/oxfordhb/9780195398649.013.0014

Diebold, F. X., & Mariano, R. (1995). Comparing predictive accuracy. Journal of Business and Economic Statistics, 13(3), 253–263.

Fischer, S. (1981). Relative shocks, relative price volatility, and inflation. Brookings Papers on Economic Activity, 2, 381–431.

Geweke, J., & Meese, R. (1981). Estimating regression models of finite but unknown order. International Economic Review, 22(1), 55–70. https://doi.org/10.2307/2526135

Gordon, R. J., & King, S. R. (1982). The output cost of disinflation in traditional and vector autoregressive models. Brookings Papers on Economic Activity, 13(1), 205–244. https://doi.org/10.2307/2534320

Granger, C. W. J., & Jeon, Y. (2004). Forecasting performance of information criteria with many macro series. Journal of Applied Statistics, 31(10), 1227–1240. https://doi.org/10.1080/0266476042000285495

Hansen, P. R. (2005). A test of superior predictive ability. Journal of Business and Economic Statistics, 23, 365–380.

Hansen, P. R. (2009). In-sample fit and out-of-sample fit: Their joint distribution and its implications for model selection [Unpublished manuscript]. Stanford University.

Holm, S. (1979). A simple sequentially rejective multiple test procedure. Scandinavian Journal of Statistics, 6, 65–70.

Hurvich, C. M., & Tsai, C.-L. (1993). A corrected Akaike information criterion for vector autoregressive model selection. Journal of Time Series Analysis, 14(3), 271–279. https://doi.org/10.1111/j.1467-9892.1993.tb00144.x

Karagrigoriou, A., Mattheou, K., & Vonta, I. (2011). On asymptotic properties of AIC variants with applications. American Open Journal of Statistics, 1(02), 105–109. https://doi.org/10.4236/ojs.2011.12012

Kilian, L. (2001). Impulse response analysis in vector autoregressions with unknown lag order. Journal of Forecasting, 20(3), 161–179. https://doi.org/10.1002/1099-131x(200104)20:3<161::aid-for770>3.0.co;2-x

Koehler, A. B., & Murphree, E. S. (1988). A comparison of the Akaike and Schwarz criteria for selecting model order. Journal of the Royal Statistical Society, Series C (Applied Statistics), 37(2), 187–195.

Kuha, J. (2004). AIC and BIC: Comparison of assumptions and performance. Sociological Methods and Research, 33(2), 188–229. https://doi.org/10.1177/0049124103262065

Lütkepohl, H. (1985). Comparison of criteria for estimating the order of a vector autoregressive process. Journal of Time Series Analysis, 6(1), 35–52. https://doi.org/10.1111/j.1467-9892.1985.tb00396.x

Markon, K. E., & Krueger, R. F. (2004). An empirical comparison of information-theoretic selection criteria for multivariate behavior genetic models. Behavior Genetics, 34(6), 593–609. https://doi.org/10.1007/s10519-004-5587-0

Newey, W., & West, K. (1987). A simple, positive semi-definite, heteroskedasticity and autocorrelation consistent covariance matrix. Econometrica, 55(3), 703–708. https://doi.org/10.2307/1913610

Nickelsburg, G. (1985). Small-sample properties of dimensionality statistics for fitting VAR models to aggregate economic data: A Monte Carlo study. Journal of Econometrics, 28(2), 183–192. https://doi.org/10.1016/0304-4076(85)90118-6

Nishii, R. (1984). Asymptotic properties of criteria for selection of variables in multiple regression. Annals of Statistics, 12(2), 758–765. https://doi.org/10.1214/aos/1176346522

Pincheira, P. (2011a). A joint test of superior predictive ability for Chilean inflation (Working Paper 620) [In Spanish]. Central Bank of Chile.

Pincheira, P. (2011b). A bunch of models, a bunch of nulls and inference about predictive ability (Working Paper 607). Central Bank of Chile.

Pincheira, P. (2012). Un test conjunto de superioridad predictiva para los pronósticos de inflación chilena [A joint test of superior predictive ability for Chilean inflation forecasts]. The Chilean Economy, 15(3), 4–39.

Pötscher, B. M., & Srinivasan, S. (1991). A comparison of order estimation procedures for ARMA models. Statistica Sinica, 4, 29–50.

Raffalovich, L. E., Deane, G. D., Armstrong, D., & Tsao, H.-S. (2008). Model selection procedures in social research: Monte-Carlo simulation results. Journal of Applied Statistics, 35(10), 1094–1114. https://doi.org/10.1080/03081070802203959

Rissanen, J. (1978). Modeling by shortest data description. Automatica, 14(5), 465–471. https://doi.org/10.1016/0005-1098(78)90005-5

Sargent, T., & Sims, C. (1977). Business cycle modeling without pretending to have too much a priori economic theory (Working Paper 55). Federal Reserve Bank of Minneapolis.

Sawa, T. (1978). Information criteria for discriminating among alternative regression models. Econometrica, 46(6), 1273–1282. https://doi.org/10.2307/1913828

Schwarz, G. E. (1978). Estimating the dimension of a model. Annals of Statistics, 6(2), 461–464. https://doi.org/10.1214/aos/1176344136

Shibata, R. (1976). Selection of the order of an autoregressive model by Akaike information criterion. Biometrika, 63(1), 117–126. https://doi.org/10.2307/2335091

Shibata, R. (1980). Asymptotically efficient selection of the order of the model for estimating parameters of a linear process. Annals of Statistics, 8(1), 147–164. https://doi.org/10.1214/aos/1176344897

Shibata, R. (1981). An optimal selection of regression variables. Biometrika, 68(1), 45–54. https://doi.org/10.1093/biomet/68.1.45

Shittu, O. I., & Asemota, M. J. (2009). Comparison of criteria for estimating the order of autoregressive process: A Monte Carlo approach. European Journal of Scientific Research, 30(3), 409–416.

Sims, C. (1980). Macroeconomics and reality. Econometrica, 48(1), 1–48. https://doi.org/10.2307/1912017

Stone, M. (1979). Comments on model selection criteria of Akaike and Schwarz. Journal of the Royal Statistical Society, Series B (Methodological), 41(2), 276–278.

Sullivan, R., Timmermann, A., & White, H. (1999). Data-snooping, technical trading rule performance, and the bootstrap. Journal of Finance, 54(5), 1647–1691. https://doi.org/10.1111/0022-1082.00163

Wasserman, L. (2000). Bayesian model selection and model averaging. Journal of Mathematical Psychology, 44(1), 92–107. https://doi.org/10.1006/jmps.1999.1278

Weakliem, L. D. (2004). Introduction to the special issue on model selection. Sociological Methods and Research, 33(2), 167–186. https://doi.org/10.1177/0049124104268642

West, K. (1996). Asymptotic inference about predictive ability. Econometrica, 64(5), 1067–1084.

White, H. (2000). A reality check for data snooping. Econometrica, 68(5), 1097–1126. https://doi.org/10.1111/1468-0262.00152

Wolak, F. A. (1987). An exact test for multiple inequality and equality constraints in the linear regression model. Journal of the American Statistical Association, 82(399), 782–793. https://doi.org/10.1080/01621459.1987.10478499

Wolak, F. A. (1989). Testing inequality constraints in linear econometric models. Journal of Econometrics, 31(2), 205–235. https://doi.org/10.1016/0304-4076(89)90094-8

Xu, C. J., & McLeod, I. (2012). Further asymptotic properties of the generalized information criteria. Electronic Journal of Statistics, 6, 656–663.

Yang, Y. (2003). Regression with multiple candidate models: Selecting or mixing? Statistica Sinica, 13, 783–809.

Yi, G., & Judge, G. (1988). Statistical model selection criteria. Economics Letters, 28(1), 47–51. https://doi.org/10.1016/0165-1765(88)90070-5

Zucchini, W. (2000). An introduction to model selection. Journal of Mathematical Psychology, 44(1), 41–46. https://doi.org/10.1006/jmps.1999.1276