New Review

LINK: Decision Making Skill: From Intelligence to Numeracy and Expertise


Use the Berlin Numeracy Test

The Berlin Numeracy Tests are fast user-friendly psychometric assessment technologies (e.g., measurement instruments), validated for use with educated samples from diverse countries and cultures (e.g., college students, computer-literate adults, physicians). The simplest Berlin Numeracy Test is a traditional 4 question paper and pencil test that takes < 4 minutes to complete. The computer adaptive version of the Berlin Numeracy Test takes about 2 minutes to complete because it only requires 2-3 questions that are selected based on participant performance. If a test-taker answers the first question right or wrong then a harder or easier question is automatically presented. We have also validated multiple choice formats, parallel forms, extensive full-scale and sub-scale tests (e.g., numeracy for certainty v. uncertainty), as well as very fast single-item tests for use with general community or highly-educated samples (i.e., median-split).  All test formats are designed to address psychometric limitations of other numeracy and skilled decision making tests (e.g., negative skew, construct validity). A growing body of research indicates that the Berlin Numeracy Test tends to be the most efficient stand-alone assessment of numeracy, risk literacy, and general decision making skill currently available, more than doubling the predictive power of much longer numeracy and cognitive ability tests (e.g., intelligence, cognitive reflection, working memory; Cokely et al., 2012). We've also validated simple systems to combine our tests with other instruments for more extensive analyses, which can be valuable when working sub-samples like less-numerate patient groups. For help selecting the best test format for your needs please use our test recommendation tool.  

If you would like to use the Computer Adaptive Berlin Numeracy Test, we would be happy to program a unique link for you that can be added to or embedded in your experiment or survey. This link will offer features like automatic scoring and a simple online interface that can be accessed by any computer with internet access.  Results are then batched and emailed to you automatically when your experiment is complete (or upon your request).  To use this option, you will be asked to do the following:

(1) Supply a valid academic or institutional email**
(2) To indicate your professional affiliation**
(3) To confirm that your research will conform to high ethic standards (see NIH Bioethics)
(4) Answer a few basic questions (e.g., how many participants do you plan to collect)

Upon completion, you will receive an email containing a link to your personal Berlin Numeracy Research Site, which can be accessed  from any computer (and most mobile devices) with internet access or may otherwise be embedded into your experimental design. Please feel free to Contact Us with any questions, concerns, or ideas regarding how to make the test easier or more useful.  Use the Berlin Numeracy Test for Research: Click Here 

** We will never share your information without your consent.  We do not sell any information.

Test Validation References: 

Cokely, E.T., Galesic, M., Schulz, E., Ghazal, S., & Garcia-Retamero, R. (2012). Measuring risk literacy: The Berlin Numeracy Test.  Judgment and Decision Making, 7, 25-47. 

Description: A 1-4 item (ca. 3 minute) test of statistical numeracy and risk literacy that is well-suited for use with moderate-to-highly numerate individuals from diverse industrialized countries (e.g., college students, computer literate adults, physicians).  Scores from the 3 item Schwartz et al. (1997) test can be added to the Berlin Numeracy Test for a 5 minute assessment that provides additional discriminability at lower levels of numeracy (i.e., the BNT-S).

See also:

Cokely, E.T., Ghazal, S., Galesic, M., Garcia-Retamero, R., & Schulz, E. (2013). How to measure risk comprehension in educated samples. In R. Garcia-Retamero & M. Galesic (Ed.), Transparent communication of risks about health: Overcoming cultural differences (pp. 29-52). New York: Springer.

Cokely, E.T., Ghazal, S., & Garcia-Retamero, R. (in press). Measuring numeracy.  In B. L. Anderson & J. Schulkin (Eds.), Numerical Reasoning in Judgments and Decision Making about Health. Cambridge University Press, Cambridge, UK.

Garcia-Retamero, R., & Cokely, E.T. (2013).  Communicating health risks with visual aids. Current Directions in Psychological Science, 22, 392-399. DOI: 10.1177/0963721413491570

Garcia-Retamero, R., Wicki, B., Cokely, E.T., & Hanson, B. (in press). Factors predicting surgeons' preferred and actual roles in interactions with their patients. Health Psychology. 

Ghazal, S., Cokely, E.T., & Garcia-Retamero (2014). Predicting biases in very highly educated samples: Numeracy and metacognition. Judgment and Decision Making, 9, 15-34.

Find papers citing the Berlin Numeracy Test at Google Scholar Citations


Additional Resources:

Appelt, K. C., Milch, K. F., Handgraaf, M. J. J., & Weber, E. U. (2011). The Decision Making Individual Differences Inventory and guidelines for the study of individual differences in judgment and decision-making research. Judgment and Decision Making, 6, 252-262.
Description: An online database of psychometric instruments and inventories used in the decision sciences.

Blais, A.-R., & Weber, E. U. (2006). A Domain-Specific Risk-Taking (DOSPERT) scale for adult populations. Judgment and Decision Making, 1(1), 33-47.
Description: The updated, brief version of DOSPERT was designed to more quickly assess risk preferences through self-report in five domains.

Cokely, E.T., & Kelley, C.M. (2009). Cognitive abilities and superior decision making under risk: A protocol analysis and process model evaluation. Judgment and Decision Making, 4, 20-33.

Fagerlin, A., Zikmund-Fisher, B., Ubel, P., Jankovic, A., Derry, H., & Smith, D. (2007). Measuring numeracy without a math test: Development of the subjective numeracy scale. Medical Decision Making, 27, 672–680.
Description: A short instrument for collecting subjective numeracy self-estimates that is well suited for differentiating among low-to-moderately numerate individuals.

Galesic, M., y García-Retamero, R. (2011b). Graph literacy: A cross-cultural comparison. Medical Decision Making, 31, 444-457.
Description: A brief test that measures individual differences in Graph Literacy.

Gigerenzer, G. (2002). Calculated risks: How to know when numbers deceive you. New York: Simon & Schuster.

Gigerenzer, G. (2012). Risk literacy. In J. Brockman (Ed.), This will make you smarter: New scientific concepts to improve your thinking (pp. 259-261). New York: Harper Perennial.

Gigerenzer, G., Gaissmaier, W., Kurz-Milcke, E., Schwartz, L. M., & Woloshin, S. (2007). Helping doctors and patients to make sense of health statistics. Psychological Science in the Public Interest, 8, 53–96.

Kutner, M., Greenberg, E., Jin, Y., Boyle, B., Hsu, Y., & Dunleavy, E. (2007). Literacy in everyday life: Results from the 2003 National Assessment of Adult Literacy (NAAL). National Center for Education Statistics. Institute of Education Sciences. Retrieved from 

Lindskog, M., Kerimi, N., Winman A. & Juslin, P. (2015). A Swedish validation of the Berlin Numeracy Test. Scandinavian Journal of Psychology.

Lipkus, I. M., & Peters, E. (2009). Understanding the role of numeracy in health: Proposed theoretical framework and practical insights. Health Education and Behavior, 36(6), 1065-1081.

Lipkus, I. M., Samsa, G., & Rimer, B. K. (2001). General performance on a numeracy scale among highly-educated samples. Medical Decision Making, 21, 37-44.  
Description: 11 item test (5-10 minutes) that measures statistical numeracy and is suited for use with low-to-moderately numerate individuals.

Peters, E. (2012). Beyond comprehension: The role of numeracy in judgments and decisions. Current Directions in Psychological Science, 21(1), 31-35.

Peters, E., Västfjäll, D., Slovic, P., Mertz, C. K., Mazzocco, K., & Dickert, S. (2006). Numeracy and decision making. Psychological Science, 17(5), 407-413.

Reyna, V. F., Nelson, W. L., Han, P. K., & Dieckmann, N. F. (2009). How numeracy influences risk comprehension and medical decision making. Psychological Bulletin, 135, 943-973.

Shapira, M. M., Walker C. M., Cappaert, K. J., Ganschow, P. S., Fletcher, K. E., McGinley, E. L., Del Pozo, S., ... Jacobs, E. A. (2012). The Numeracy Understand in Medicine Instrument (NUMi): A measure of health numeracy developed using Item Response Theory. Medical Decision Making, 32, 851-865.

Schwartz, L. M. L., Woloshin, S. S., Black, W. C. W., & Welch, H. G. H. (1997). The role of numeracy in understanding the benefit of screening mammography. Annals of Internal Medicine, 127, 966-972.  
Description: 3 item test (1-2 minutes) test that is well suited for use with low-to-moderately numerate individuals.

Weber, E. U., Blais, A., & Betz, N. E. (2002). A domain-specific risk-attitude scale: Measuring risk perceptions and risk behaviors. Journal of Behavioral Decision Making, 15(4), 263-290. doi: 10.1002/bdm.414
Description: The DOSPERT was designed to assess risk preferences through self-report in five domains.

Weller, J., Dieckmann, N. F., Tusler, M., Mertz, C. K., Burns, W., & Peters, E. (2013). Development and testing of an abbreviated numeracy scale: A Rasch Analysis approach. Journal of Behavioral Decision Making, 26(2),198-212.
Description: A Rasch test (est. 10 minutes) that combines items from various tests (e.g., cognitive reflection test, Lipkus et al. 2011) and is designed for use with the general US population. NOTE: One item on the scale has been updated in 2013 see here