Senior Test Developer

09/12/2018 5:16 PM | Anonymous

American Nurses Association

Location: Silver Spring, MD

TO APPLY:  Apply online or email


Ensures validity, reliability, and psychometric soundness of certification programs, using established industry and best practices. Serves as a technical expert in test development and scoring. Maintains pool of exam items; leads test development activities; designs and conducts research, and develops software applications, programming, or syntax for statistical, analysis, graphical representations, and report generation. Conducts quality assurance on scores, item functioning, and psychometric properties of exams. Drafts technical reports.


  • Plan and lead test development activities (e.g., item writer training, item reviews and editing, test construction, standard setting, item and exam performance). Facilitate focus groups, prepare meeting materials, review materials created by colleagues, analyze data, and write technical reports and summaries. Monitor and evaluate item performance, differential item functioning, item drift, and exam psychometrics. Create automated reports. Coordinate with colleagues, subject matter experts, and vendors. Ensure compliance with industry best practice and accreditation requirements.
  • Manage development and maintenance of relational database of all exam items and associated statistics. Develop and maintain avenues for retrieving and housing statistical data; gather, import, and export data; and create databases of statistical information. Monitor status of item bank and advise colleagues and management.
  • Plan and conduct applied research (e.g., job analysis, validation, survey sampling, test equating). Program and perform statistical analyses using software programs (e.g., SAS, SPSS, R). Create and implement data collection and sampling plan, develop conceptual frameworks that address research topics, and adapt analytical and statistical methods as appropriate. Identify strategies to generate useful feedback for examinees. Draw conclusions and make recommendations. Provide results, conclusions, and visuals. Present to stakeholders.
  • Maintain and update written procedures and documentation pertinent to the position. Ensure all ANCC assessment development and psychometric practices are documented and consistently applied. Maintain all necessary documentation in support of accreditation requirements. Complete copyrighting of all MS proprietary information, working with OGC and copyright office, with special attention to maintaining exam security.


  • Perform other duties as assigned to support the department’s objectives and strategic goals.



Master’s degree (Ph.D. preferred) in industrial-organizational (I/O) psychology, testing and measurement, psychometrics, research and statistics, or related field, with emphasis on quantitative analyses.

Related Work Experience

Three years of professional experience. Two years of assessment development, psychometrics, data analyses, and technical report writing. Experience in test blueprint development, item specification, item pool development, item reviews, fairness reviews, constructed response scoring, and form development. Experience using classical or IRT statistical models to interpret item-level data and construction of test forms.


  • Research design and completion
  • Data collection tool development and administration
  • Data analysis
  • Choosing and applying appropriate analysis techniques
  • Identifying problems and conceptualizing solutions
  • Test construction, development of alternative items, and test validation
  • Detecting and correcting flaws in multiple choice and constructed response items
  • Meeting facilitation
  • Oral and written communication
  • Advanced use of office computer software (e.g., Microsoft Word, Excel, PowerPoint)
  • Intermediate use of statistical software (e.g., SAS, SPSS, R)
  • Analytical thinking
  • Writing technical reports
  • Producing concise, high-level summaries and briefings to non-technical audiences.

Additional Qualifications

Ability to teach others and give professional feedback; learn new technologies; organize, manage, and complete multiple ongoing projects and tasks; accurately attend to details; adapt to changing priorities and work demands; take initiative; work autonomously with limited supervision; and work collaboratively and effectively within a team.

Preferred Skills & Experience   

ABD or Ph.D. in I/O psychology, testing and measurement, psychometrics, research and statistics, or related field. Three years of experience in highly analytical test development or related roles, with responsibility for psychometrics and management of test development, administration, and maintenance. Experience with item banking software, form building, and job analysis.

2018 © PTCMW

Powered by Wild Apricot Membership Software