1. What is the Pharmacy Curriculum Outcomes Assessment^® (PCOA^®)?
The PCOA is a 220 item multiple-choice paper-and-pencil assessment that has been developed by the National Association of Boards of Pharmacy (NABP). The PCOA is intended to be a valuable and integral component of the process that schools use to assess their curriculum. The data obtained from the PCOA will provide the individual colleges and schools of pharmacy with detailed feedback related to the subject matters covered throughout the professional pharmacy curriculum as well as provide detailed information to the individual students on their strengths and weaknesses in regard to the curriculum. The results are both formative and summative in measuring student progress and allow comparisons to a national reference group.

2. Why is the PCOA needed?
The colleges and schools of pharmacy have expressed a need for a national assessment that is psychometrically validated to assist with measuring curriculum development and student performance. The PCOA is very similar in concept to the in-service examinations in the postgraduate medical training programs for physicians, where those examinations are administered to interns and residents one time each year throughout their residency program to evaluate progress. The PCOA can be used:

• To measure the overall performance of pharmacy students and compare their scores to a national sample of students
• As a tool for faculty to provide constructive feedback on the strengths and weaknesses of the student and to track individuals’ scores from year to year and their individual growth over time
• As a component and outcome measure of the evaluation plan suggested by the Accreditation Council for Pharmacy Education (ACPE) Standards. The PCOA:
  • Has formative and summative measures
  • Provides for collective analyses of findings
  • Evaluates trends over time
  • Includes standardized instruments and data
  • Documents that the school is providing the scientific foundation necessary for achievement of the professional competencies
• To document improvement in student performance after the school’s curriculum has been modified or updated
• For research and correlational studies with existing assessments such as Pharmacy College Admission Test (PCAT) scores, current grades, and future North American Pharmacist Licensure Examination^™ (NAPLEX^®) administrations.

3.  How do I know the PCOA is a valid assessment of pharmacy knowledge?  How was the PCOA developed?
The PCOA has been developed using standard test development principles and psychometrics. A full report of its development can be found in the Validity and Reliability Evidence for the Pharmacy Curriculum Outcomes Assessment (PCOA) available through NABP.

The content of the assessment is based upon a blueprint that was derived from the ACPE Accreditation Standards and Guidelines for the Professional Program Leading to the Doctor of Pharmacy Degree. NABP convened a select committee of 11 stakeholders who collected curriculum data from 55 accredited United States pharmacy programs. The committee estimated the average number of curriculum hours that were allocated to the different areas of the new draft blueprint for each of the 55 schools in the sample, and used those averages to estimate the number of questions in a 220-item assessment that should be assigned to each topic in the proposed blueprint. These results were incorporated into a survey instrument sent to all US schools and colleges of pharmacy. Constructing the survey in this way utilized the number of credit hours allocated to the topics in the pharmacy schools’ curriculum and created a draft blueprint that would result in an assessment that reflected the relative emphasis of the various topics.

In March 2006, this survey instrument was sent to all deans of the schools and colleges of pharmacy asking for their recommendation on the number of questions that should be allocated to the delineated topics. Surveys were obtained from 41 accredited pharmacy programs. Since the survey had a high response rate and its results corresponded so closely with the overall assessment of curricular requirements across US schools and colleges of pharmacy, the results of the survey were adopted directly for the new blueprint and the 2007 pilot PCOA was created.

Analyses were completed on the 2007 pilot PCOA and the evidence demonstrated that there is high level of confidence in the reliability and validity of the assessment, in its scores, and in its inferences about the abilities of students and the effectiveness of a school’s curriculum. In summary, these analyses showed:

• The direct and precise linkage of the 2007 PCOA content to its blueprint
• High reliability indices, which show that the total assessment and the domains consistently and reliably measure pharmacy knowledge and skills
• High reliability index shows that the items measure a single dimensional construct
• That the exploration of the goodness-of-fit confirmed that the assessment scale supports construct validity
• Inter-scale correlations that indicated that there is some commonality between the domains, but that each domain also brings its own uniqueness to the assessment
• That the assessment behaves as expected as evidenced by examining the increase in pharmacy knowledge from professional year 1 to professional year 4
• Knowledge growth in the domains and differences between the domains over the years

4. When and where will the PCOA be administered?
The PCOA will be administered during a predetermined window of time in April 2008 Administering the assessment in the spring allows measurement of the knowledge and skills acquired during most of the program curriculum year, while still allowing enough time for the schools to receive the scores and provide feedback to students before the end of their program year. The assessment will be given at the participating schools of pharmacy.

5. How is registration for the assessment conducted?
Prior to the assessment, NABP will provide to the pharmacy schools the assessment date, registration materials, PCOA testing policies, and proctor requirements (to avoid conflicts of interest, assessments should not be proctored by pharmacy program faculty but other employees of the institution may proctor). Registration is to be completed by the pharmacy program, which includes a listing of the names of all students that will be taking the assessment, along with assessment fees ($75 per assessment) and assessment site administrator information. At the end of the assessment, the assessment site administrator will be responsible for securely returning all materials to NABP for scoring.

6. How is the PCOA scored?
NABP uses Rasch item response theory (IRT) to score the responses, which permits insight into the validity and reliability of an assessment. To ensure the assessment is as valid as possible, a key validation analysis of the data is also conducted.

Scoring is completed for the total assessment and scores on each of the major domains calculated, with the domain item difficulties directly equated to the total assessment. Assessments between annual administrations are also equated. This equating places all the scores on the same measurement continuum, permitting accurate comparisons to be made between the domain scores and between the scores from one year to the next. 

7. What information is provided in the score reports?
Two types of score reports are provided: one for each individual student and a school summary report for the pharmacy school.  Both of these score reports will be sent to the pharmacy school only, encouraging pharmacy school faculty to sit down with each student to provide and discuss his or her individual results and plan for the student’s next program year.

The student score report includes students’ scale scores, their national percentile score, and their national percentile for that student’s program year for the total assessment and for the four major domains. Additionally, the number of items correct on the 35 sub-topics is provided.

The school score report includes summaries of data for the particular school by program year, including the number of examinees, mean school scale scores, mean school percent correct, and mean school percentile. For information about the national reference group, the mean national scale score and mean national percent correct scores are provided. All these scores are provided for the total assessment and for each of the four major domains. Also, the average number of items correct on the 35 sub-topics is included, as well as the mean number of items correct for the national reference group. Finally, a score roster listing the schools students’ scores (in a condensed format) is provided.

Please refer to www.nabp.net for the latest information.

• Informational Brochure
• FAQ
• Benefits of the PCOA
• PCOA Content Areas