Location: United States |  Change Location
Male flipping through Corwin book

Hands-on, Practical Guidance for Educators

From math, literacy, equity, multilingual learners, and SEL, to assessment, school counseling, and education leadership, our books are research-based and authored by experts on topics most relevant to what educators are facing today.


Assessing Teacher Competency  - Book Cover

Assessing Teacher Competency

Five Standards-Based Steps to Valid Measurement Using the CAATS Model
By: Judy R. Wilkerson, William Steve Lang

Foreword by Richard C. Kunkel

A complete, step-by-step guide to teacher assessments that meet national accreditation and accountability standards.

This comprehensive five-step model for measuring teacher knowledge and skills helps teacher educators and school administrators prepare fair, valid, and reliable assessments of teacher performance. The authors provide detailed guidance for the assessment process, creating an ideal resource for faculty, supervisors, and administrators preparing for accreditation reviews. Offering worksheets and activities illustrating the entire process, this handbook covers definitions, contextual factors, sampling, and:

  • Aligning performance tasks with standards defined by NCLB, NCATE, INTASC, and other groups
  • Designing and implementing data tracking and management systems
  • Ensuring psychometric integrity

Full description

Assessing Teacher Competency  - Book Cover
Product Details
  • Grade Level: PreK-12, Elementary, Secondary
  • ISBN: 9781412941204
  • Published By: Corwin
  • Year: 2007
  • Page Count: 408
  • Publication date: April 10, 2014

Price: $48.95

Price: $48.95
Volume Discounts applied in Shopping Cart

For Instructors

Request Review Copy

When you select 'request review copy', you will be redirected to Sage Publishing (our parent site) to process your request.



"I have not seen anything quite as systematic as this material in guiding the reader through a process for developing a valid and reliable assessment plan. Covers all the areas one would want in designing a system for accreditation or for other purposes."
—Martha Gage, Director, Teacher Education & Licensure
Kansas State Department of Education

"Realistically reveals the extent of the task of teacher certification and provides us with a structured learning experience that should improve our abilities with this task."
—Pearl Solomon, Associate Professor
St. Thomas Aquinas College

A complete, step-by-step guide to teacher assessments that meet national accreditation and accountability standards.

Written in a reader-friendly style for busy faculty members and school administrators with little or no prior knowledge of statistics, this comprehensive model is designed to create fair, valid, and reliable assessments of teacher knowledge and skills.

Evaluation experts Judy Wilkerson and Steve Lang provide detailed guidance for the complete five-step assessment process, making this an ideal resource both for preservice and inservice settings, including accreditation reviews and teacher induction programs. Offering worksheets and activities to illustrate every step of the process, this all-inclusive handbook covers:

  • Definitions, contextual factors, and sampling
  • Aligning performance tasks with standards defined by NCLB, NCATE, INTASC, and other groups
  • Designing and implementing data tracking and management systems
  • Ensuring psychometric integrity

Valid and reliable decisions about teacher competency are based on fair, valid, and reliable assessment systems. Assessing Teacher Competency is the book all teacher educators, supervisors, and mentors have been waiting for.

Key features

  • A comprehensive guide to designing and implementing accurate, reliable, and useful teacher assessments
  • Assessments are aligned with the standards and benchmarks defined by NCLB, NCATE, INTASC, and other key programs and groups in education
  • Provides background, context, guidelines for designing and implementing data tracking and management systems, and ensuring psychometric integrity 
  • Includes worksheets and activities to illustrate every step of the process
  • includes a Glossary, Index, and a Step-by-Step Guide to the CAATS Model 
  • Appropriate both for pre-service and in-service faculty, administrative staff, and supervisors




Judy R. Wilkerson photo

Judy R. Wilkerson

Judy R. Wilkerson is an Associate Professor of Research and Assessment at Florida Gulf Coast University, where she teaches graduate and undergraduate courses in measurement and evaluation. As in this book and all of her research, she focuses her efforts with students on providing a highly pragmatic approach, based in theory, with the goal of instilling a commitment in them to assess K-12 learning. Her Ph.D. is in Measurement and Research from the University of South Florida, where she served for 15 years as Director of Program Review, leading College and University efforts in accreditation. Her career has been dedicated to standards-based assessment of programs and teachers, beginning with the creation of an evaluation model for accreditation in 1987, which she implemented in several states. Beginning in 1990 for 15 years, she served as the primary consultant for higher education to the Florida Department of Education, where she drafted the standards for the initial approval of teacher education programs, designed the program approval process, and provided technical assistance to colleges of education in evaluation of teachers and programs statewide. She has consulted nationally on NCATE accreditation and worked with state associations of teacher educators on accreditation related issues. She has also consulted with school districts in Florida on assessment systems for teachers. She was lead designer of the assessment system for the Florida Alternative Certification Program, now used in over 40 of the 68 school districts in the State.
William Steve Lang photo

William Steve Lang

William Steve Lang is a Professor of Educational Measurement and Research at the University of South Florida St. Petersburg, where he teaches graduate courses in measurement, statistics, and research.   He, too, focuses his teaching on making meaningful and pragmatic uses of the disciplines he teaches.  He earned his Ph.D. from the University of Georgia in 1984.  He has taught as a public school teacher in South Carolina and Georgia and as a college faculty member in South Carolina, Georgia, and Florida.   He has published on a variety of applications in educational testing and works extensively with the Rasch Model of Item Response Theory.  He began working extensively with Judy when she joined the faculty of the St. Petersburg Campus in 2001.  Since that time, they have collaborated in all aspects of their research and service efforts with the Florida Department of Education, Florida school districts, and teacher education programs nationwide.   They are working together to build two teacher assessment scales – one on teacher competencies, the subject of this book, and another on dispositions.  Their work in both areas is standards-driven.
Table of Contents

Table of Contents

A Step-by-Step Guide to the CAATS Model

Foreword by Richard Kunkel



About the Authors

1. Expectations and Options for Accountability and Teacher Assessment

What This Chapter Is About

The Challenge from the National Commission on Teaching and America’s Future

Title II of the Higher Education Act Amendments of 1998

No Child Left Behind Legislation

National Research Council (2001) -- the Committee on Assessment and Teacher Quality

What A Few Others Have Said: A Brief Review of the Literature on Testing and Licensure

Standards: The Roadmap to Accountability and Scientifically-Based Performance Assessment

The Principal Sets of Standards Governing Our Work

National and State Pedagogical and Content Standards

Unit Accreditation and Operational Standards

Technical Standards for Measurement of Teacher Competency

Some Major Threats to Validity in Most Current Assessment Systems

Conceptual Frameworks: Pulling It All Together

NCATE Standards


INTASC Principles: Where NCATE and AERA, APA, & NCME Standards Converge

Making Sense of Conceptual Frameworks

Our Conceptual Framework: What We Value

Assessment Options

Records of Training Completed

Tests and Exam Scores

Observations of Performance

Portfolios of Assessable Artifacts

Job-Related Tasks and Work Sample Products

K-12 Student Work Samples


Activity #1: What’s Happening in Your State and School?

Activity #2: Questionnaire for Faculty Views on Competency Assessment

Activity #3: Assessment Belief Scale

Activity #4: Assessment Options

2. Portfolios – To Be or Not To Be?? That IS the Question!

What This Chapter Is About

The Portfolio: Panacea or Pandora’s Box

Portfolios as Certification “Tests”: Lessons from Standards and History

Assessment Illiteracy, Paradigms Shifts, and Conflicting Purposes

The Conflict of Formative vs. Summative Evaluation

The Conflict of Program Approval vs. Accreditation

The Conflict of Business or Legislative Personal/Professional Perspectives vs. Accountability Through Title II Requirements

The Conflict of Academic Freedom vs. Accountability

The Conflict of Constructivism vs. Positivism

Recommendations for Use of Portfolios in Accountability Contexts

Ten Recommendations for Assessment System Design

A Recommended, Standards-Based Model

Overview of “Competency Assessment Aligned with Teacher Standards” (CAATS) Model

CAATS Step 1: Define content, purpose, use, and other contextual

CAATS Step 2: Develop a valid sampling plan

CAATS Step 3: Create or update tasks aligned with standards and

CAATS Step 4: Design and implement data aggregation tracking and management systems

CAATS Step 5: Ensure credibility and utility of data


A Note about Chapters 8 and 9


Activity #1: Review Your Feelings

Activity #2: Thinking about Conflicting Paradigms

Activity #3: Getting Your Action Plan Started

3. CAATS Step 1: Assessment Design Inputs

Where We Have Been So Far

What This Chapter Is About

CAATS Step 1A: Define the Purposes and Uses of the System

The Importance of Purpose and Use

Different Strokes for Different Folks for Different Purposes

Purpose and Use in the Accountability Context

More on Accountability Based Systems: State Program Approval

CAATS Step 1B: Define the Propositions or Principles that Guide the System

CAATS Step 1C: Define the Conceptual Framework or Contents of the System

So What Is Assessment Content?

Standards as the Link between Purpose, Use, and Content

CAATS Step 1D: Review Local Factors That Impact the System

Wrap Up


Worksheet #1: Purpose, Use, Propositions, Content, & Context Checksheet

Worksheet #2: Purpose, Use, Content, Draft

Worksheet #3: Propositions

Worksheet #4: Contextual Analysis

4. CAATS Step 2: Planning with a Continuing Eye on Valid Assessment Decisions

Where We Have Been So Far

What This Chapter Is About

CAATS Step 2A: Organize Standards into Content Domains

All Those Standards Sets

Organizing for Alignment

“A Rose Is a Rose” Or More of the Same

Why Bother?

Crosswalks and Standards

CAATS Step 2B: Visualize the Competent Teacher Based on the Standards

CAATS Step 2C: Brainstorm a Set of Summative Tasks (Sampling Plan)

CAATS Step 2D: Sort Tasks into Formative and Summative Assessments

CAATS Step 2E: Build Assessment Frameworks

Framework Options

A Special Case: Aligning Tasks with NCATE Requirements



Worksheet #1: Organizing for Alignment (Version 1)

Worksheet #1: Organizing for Alignment (Version 2)

Worksheet #2: Our Critical Skills

Worksheet #3: Visualizing the Competent Teacher

Worksheet #4: Critical Task List

Worksheet #5: Sorting Formative and Summative Tasks

Worksheet #6: List of Summative Assessments by Competency Type

Worksheet #7: List of Summative Assessments by Levels of Inference

Worksheet #8: List of Summative Assessments by Points in Time

Worksheet #9: Matrix of Standard by Competency Type

Worksheet #10: Matrix of Critical Tasks by Competency Type and Benchmark

Worksheet #11: Aligning Tasks with NCATE Thematic Portfolios

5. CAATS Step 3 -- Writing Tasks Designed to Maximize Validity and Reliability

Where We Have Been So Far

What This Chapter Is About

CAATS Step 3A: Determine the Task Format for Data Aggregation

What Happens When There Is No Format?

Can I Use Percents and Total Points? No, They Don’t Cut It!

Elements of a Common Task Format

CAATS Step 3B: Create New Tasks or Modify Existing Tasks

Basic Concepts about Tasks

Hints and Advice about Writing Tasks

Rubric Examples: A Rose is a Rose

CAATS Step 3C: Conduct First Validity Study

CAATS Step 3D: Align Tasks with Instruction



Worksheet #1: Proficiency Level Descriptions

Worksheet #2: Task Design

Worksheet #3: Standards and Indicators Coverage Report

Worksheet #4: Individual Task Review for Job-Relatedness

Worksheet #5: Checklist for Reviewing Individual Tasks

Worksheet #6: Instructional Alignment

6. CAATS Step 4: Decision-Making and Data Management

Where We Have Been So Far

What This Chapter Is About

CAATS Step 4A: Determine How Data Will Be Aggregated

So What Is Data Aggregation?

The Relationship of Data Aggregation (Step 4A), Cut Scores, (Step 4B), and

An Approach to Decision-Making without Using Points and Percents

CAATS Step 4B: Set Standards for Minimal Competency

Different Strokes for Different Folks – Déjà Vu or Purpose Revisited

Is 100% Really Reasonable?

Criticality Yardstick Approach (CYA) to Cut Score Setting in Complex

Performance Assessments

First Cut

Second Cut

Working with Judges

An Example of What to Say to the Judges

CAATS Step 4C: Select and Develop a Tracking System

Sharing Information for Decision-Making: The Big Challenge

Data Storage Option #1: Course Grades or Records of Participation/Attendance

Data Storage Option #2: Teacher Folders

Data Storage Option #3: Portfolios

Data Storage Option #4: Electronic Data Management System

Reporting Aggregated Data

CAATS Step 4D: Develop Management System

Advising and Due Process

Scoring Procedures




Worksheet #1: Cut Score Decisions

Worksheet #2: Sample Format for Candidate/Teacher Tracking Form

Worksheet #3: Format for Data Aggregation

Worksheet #4: Management Plan

Worksheet #5: Rater Monitoring Record

7. CAATS Step 5: Credible Data

Where We Have Been So Far

What This Chapter Is About

What Is Psychometric Integrity and Why Do We Have to Worry About It?

CAATS Step 5A: Create a Plan to Provide Evidence of Validity, Reliability, Fairness, and Utility

Elements of a Plan

Element #1: Purpose and Use

Element #2: Construct Measured

Element #3: Interpretation and Reporting of Scores

Element #4: Assessment Specifications and Content Map

Element #5: Assessor/Rater Selection and Training Procedures

Element #6: Analysis Methodology

Element #7: External Review Personnel and Methodology

Element #8: Evidence of Validity, Reliability, and Fairness (VRF)

CAATS Step 5B: Implement the Plan Conscientiously

Wrap Up

Worksheets and Examples

Worksheet #1: Assessment Specifications

Worksheet #2: Analysis of Appropriateness of Decisions for Teacher Failures

Worksheet #3: Program Improvement Record

Worksheet #4: Expert Rescoring

Worksheet #5: Fairness Review

Worksheet #6: Analysis of Remediation Efforts and EO Impact

Worksheet #7: Psychometric Plan Format

Example #1 (Empirical Data): Logistic Ruler for Content Validity

Example #2 (Empirical Data): Computation of the Lawshe (1975) Content

Example #3 (Empirical Data): Disparate Impact Analysis

Example #4 (Empirical Data): Computation of Cohen’s Kappa (1960) for

Example #5 (Empirical Data): Spearman Correlation Coefficient and Scatterplot:

Example #6 (Empirical Data): Pearson Correlation Coefficient and Scatterplot:

Example #7 (Empirical Data): Correlation Matrix and Scatterplots Knowledge,

Example #8 (Empirical Data): T-Test Comparing Mathematics and Science

Example #9 (Empirical Data): Differential Item and Person Functioning

8. The Trouble with Tribbles: Standard Setting for Professional Certification

What This Chapter Is About

An Overview of Cut Score Setting

From Norm-Referenced Objective Tests to Criterion-Referenced Subjective Tasks

Controlling Human Judgment

Difficulty vs. Importance

Social Consequences

Tidbits to Remember

Cut Scores Setting in Traditional, Objective Tests

Holistic Impressions: Method One

Item Content: Method Two

Performance of Examinees: Method Three

Combination Approach: Method Four

Standard or Cut Score Setting in Performance Tests for Professionals: Is it the Same as Multiple Choice?

Quotes to Remember

The Extended Angoff Procedure for Performance Assessment

The Judgmental Policy Capturing Approach for Performance Assessment

The Dominant Profile Judgment for Performance Assessment

Standard Setting Using Item Response Theory

What’s Really Wrong with Current Approaches to Cut Score Setting

Berk’s Suggestions and Commentaries Over the Years

The Criticality Yardstick Approach

Wrap Up


Activity #1: Change the Difficulty of an Item

Activity #2: Change the Criticality of a Task

Activity #3: Replicate a Cut Score

Activity #4: Take Your Choice

9. Using Teacher Scores for Continuous Improvement

What This Chapter is About

Reasons Why We Use the Rasch Model

The Classical Approach

A Quick Overview of Where Rasch Fits Into the Grand Scheme of IRT Models

Rasch: The Basics

Getting Started

Differences that Item Writers Make

Guttman Scaling

A Sample Rasch Ruler

From Pictures to Numbers

The Fit Statistic

Gain Scores – Real or Imagined?

Ratings and Raters

Learning More about Rasch

Wrap Up


Activity #1: Decision Making Tool for Measurement

10. Legal Integrity

What IF?? A Legal Scenario

Psychometric Issues and Legal Challenges

Legal Issues and Precedents

End Note

Appendix: Tasks Developed for Florida Alternative Certification Program






Price: $48.95
Volume Discounts applied in Shopping Cart

For Instructors

Request Review Copy

When you select 'request review copy', you will be redirected to Sage Publishing (our parent site) to process your request.