Modern Psychometrics: The Science of Psychological Assessment

This popular text introduces the reader to all aspects of psychometric assessment, including its history, the construction and administration of traditional tests, and the latest techniques for psychometric assessment online. Rust, Kosinski, and Stillwell begin with a comprehensive introduction to the increased sophistication in psychometric methods and regulation that took place during the 20th century, including the many benefits to governments, businesses, and customers. In this new edition, the authors explore the increasing influence of the internet, wherein everything we do on the internet is available for psychometric analysis, often by AI systems operating at scale and in real time. The intended and unintended consequences of this paradigm shift are examined in detail, and key controversies, such as privacy and the psychographic microtargeting of online messages, are addressed. Furthermore, this new edition includes brand-new chapters on item response theory, computer adaptive testing, and the psychometric analysis of the digital traces we all leave online. Modern Psychometrics combines an up-to-date scientific approach with full consideration of the political and ethical issues involved in the implementation of psychometric testing in today’s society. It will be invaluable to both undergraduate and postgraduate students, as well as practitioners who are seeking an introduction to modern psychometric methods.

Table of Contents:
1. The history and evolution of psychometric testing Introduction What is psychometrics? Psychometrics in the 21st century History of assessment Chinese origins The ability to learn The nineteenth century Beginnings of psychometrics as a science Intelligence testing Eugenics and the dark decades Psychometric testing of ability The dark ages come to an end An abundance of abilities Tests of other psychological constructs Personality Integrity Interests Motivation Values Temperament Attitude Belief Summary 2. Constructing your own psychometric questionnaire The purpose of the questionnaire Making a blueprint Writing items Alternate-choice items Multiple-choice items Rating-scale items All questionnaires Knowledge-based questionnaires Person-based questionnaires Designing the questionnaire Piloting the questionnaire Item analysis Facility Discrimination Distractors Obtaining the reliability Cronbach’s alpha Split-half reliability Assessing validity Face validity Content validity Standardization 3. The Psychometric principles Reliability Test-retest reliability Parallel-forms reliability Split-half reliability Interrater reliability Internal consistency The standard error of measurement (SEM) Comparing test reliabilities Restriction of range Validity Face validity Content validity Predictive validity Concurrent validity Construct validity Differential validity Standardization Norm referencing Criterion referencing Equivalence Differential item functioning Measurement invariance Adverse impact Summary 4. Psychometric measurement True-score theory Identification of latent traits with factor analysis Spearman’s two-factor theory Vector algebra and factor rotation Moving into more dimensions Multidimensional scaling Application of factor analysis to test construction Eigenvalues Identifying the number of factors to extract using the Kaiser criterion Identifying the number of factors to extract using the Cattell scree test Other techniques for identifying the number of factors to extract Factor rotation Rotation to simple structure Orthogonal rotation Oblique rotation Limitations of the classical factor-analystic approach Criticisms of psychometric measurement theory The Platonic true score Psychological vs. physical true scores Functional assessment and competency testing Machine learning and the black box Summary 5. Item response theory and computer adaptive testing Introduction Item banks The Rasch model Assessment of educational standards The Birnbaum model The evolution of modern psychometrics Computer adaptive testing Item equating Polytomous IRT An intuitive graphical description of item tesponse theory Limitations of classical test theory A graphical Introduction to item response theory The logistic curve 3PL-model: difficulty parameter 3PL model: discrimination parameter 3PL model: guessing parameter The Fisher information function The test information function and its relationship to the standard error of measurement How to score an IRT test Principles of computer adaptive testing Summary of item response theory Confirmatory factor analysis 6. Personality theory Theories of personality Psychoanalytic theory Humanistic theory Social learning theory Behavioral genetics Type and trait theories Different approaches to personality assessment Self-report techniques and personality profiles Reports by others Online digital footprints Situational assessments Projective measures Observations of behavior Task performance methods Polygraph methods Repertory grids Sources and management of bias Self-report techniques and personality profiles Reports by others Online digital footprints Situational assessments Projective measures Observations of behavior Task performance methods Polygraph methods Repertory grids Informal methods of personality assessment State versus trait measures Ipsative scaling Spurious validity and the Barnum Effect Summary 7. Personality assessment in the workplace Prediction of successful employment outcomes Validation of personality questionnaires previously used in employment Historical antecedents to the five-factor model Stability of the five-factor model Cross-cultural aspects of the five-factor model Scale independence and the role of facets Challenges to scale construction for the five-factor model Impression management Acquiescence Response bias and factor structure Development of the five OBPI personality scales Assessing counterproductive behavior at work The impact of behaviorism Prepsychological theories of integrity Modern integrity testing Psychiatry and the medical model The dysfunctional tendencies The dark triad Assessing integrity at work The OBPI integrity scales Conclusion 8. Employing digital footprints in psychometrics Introduction Types of digital footprint Usage logs Language data Mobile sensors Images and audiovisual data Typical applications of digital footprints in psychometrics Replacing and complimenting traditional measures New contexts and new constructs Predicting future behavior Studying human behavior Supporting the development of traditional measures Advantages and challenges of employing digital footprints in psychometrics High ecological validity Greater detail and longitude Less control over the assessment environment Greater speed and unobtrusiveness Less privacy and control No anonymity Bias Enrichment of existing constructs Developing digital-footprint-based psychometric measures Collecting digital footprints How much data is needed? Preparing digital footprints for analysis Respondent-footprint matrix Data sparsity Reducing the dimensionality of the respondent-footprint matrix Singular value decomposition Latent Dirichlet allocation Building prediction models 9. Psychometrics in the era of the intelligent machine History of computerization in psychometrics Computerized statistics Computerized item banks Computerized item generation Automated advice and report systems The evolution of AI in psychometrics Expert systems Neural networks (machine learning) Parallel processing Predicting with statistics and machine learning Explainability Psychometrics in cyberspace What and where is cyberspace? The medium is the message Moral development in AI Kohlberg’s theory of moral development Do machines have morals? The laws of robotics Artificial general intelligence Conclusion