Association of American Medical Colleges (AAMC)
Location: 655 K St NW #100, Washington, DC 20001
The Association of American Medical Colleges (AAMC) is seeking a Manager, Admissions and Selection Research to join the program in conducting research and development on medical school admissions and residency selection, including evaluation of current processes and development of new tools and resources to aid the admissions and selection process. The work will include job analysis, needs analysis, development of new tools (e.g., SJTs, video interviews, biodata, etc.), big data analyses, and validation studies. This work will be conducted through partnerships with medical schools, residency programs, AAMC staff from multiple clusters, and contractors.
The Manager will lead some aspects of this work and will be expected to contribute intellectually to others. S/he will be responsible for conceptualizing new assessments (e.g., SJT, SVI, etc.), designing research to develop and evaluate new tools, managing implementation of research and development projects, and planning and conducting outreach efforts to support them. The Manager will also be responsible for directing the work of AAMC staff and contractors who are assigned to the project, as well as working with AAMC leadership and constituents to ensure understanding and support for the new assessments. The Manager will be responsible for leading planning and implementation of complex statistical and psychometric analyses, including traditional validity analyses, CTT, Generalizability Theory, IRT, and machine learning techniques.
Plan and execute research, development, and implementation of new tools for use in admissions and selection
- Lead research and development projects to create new assessments to measure non-cognitive attributes (e.g., SJT, SVI, etc.) and evaluate the psychometric properties and value of new tools developed for use in medical school admissions and residency selection
- Develop and facilitate job analysis and content-oriented data collection methods (interviews, focus group protocols, job observation, job analysis questionnaires, etc.)
- Design and conduct content-oriented, concurrent and predictive criterion, and transportability validation studies
- Design research to evaluate the psychometric properties and validity of new tools and evaluate the benefits and risks of new tools for use in medical school admissions and residency selection, including traditional validity analyses, CTT, Generalizability Theory, IRT, and machine learning techniques.
- Work with AAMC colleagues and vendors to address potential operational issues (e.g. number of administrations) that need to be considered during research and development efforts
Manage project team including vendors, AAMC colleagues, and constituent partners to on research and development efforts for new tools
- Plan and manage the AAMC staff and vendor’s work (e.g., tasks, staffing needs, etc.) to ensure that projects are completed and meet AAMC’s high standards
- Plan and monitor project timelines to ensure that vendors’ work is accurate and completed in a timely manner
- Plan meeting with the external Technical Advisory Committee (TAC) to ensure that new assessments adhere to technical best practices and appropriate research is being conducted to support their use in a high-stakes selection context
- Plan and monitor administrative and legal aspects of projects, such as the focus framework, non-disclosure agreements, data usage agreements, and Institutional Review Board (IRB) to ensure that all projects are in compliance
- Plan and monitor project budgets, including being responsible for tracking and paying invoices
Build relationships with key constituents and AAMC leadership
- Lead outreach efforts for the new assessments to gather constituent input and build support and buy-in for the new assessments
- Work with constituent advisory group to plan and execute research and communication efforts to support new tools
- Create and deliver presentations (PowerPoint, Webinar, Talks, etc.) at constituent meetings
- Deliver presentations and answer questions from AAMC leadership when called upon to explain the project
A Master's degree is required for this role; Industrial-Organizational Psychology, Psychometrics, Educational Measurement, Quantitative Psychology or related field with an emphasis on assessment and selection, psychometrics, and validation
A minimum of 8 years of related work experience is required for this role
- Intermediate to advanced knowledge of personnel selection and validation research.
- Advanced knowledge of research design, statistics, and psychometrics.
- Advanced skills in the full suite of Microsoft Office products, Windows, and database environments.
- Advanced SPSS or SAS or R.
- Experience managing large projects and complex business processes.
- Experience managing vendors.
- Able to work in a consensus-based decision making environment.
- Strictly observes AAMC policies regarding confidentiality of information and personally identifiable information.
- Strong verbal and writing skills necessary to communicate with a wide variety of stakeholders including AAMC staff and constituents.
- Demonstrates maturity, tact, and sound judgment in dealing with AAMC staff and constituents.
- Time-management skills with the ability to prioritize multiple, critical tasks while maintaining a high level of service and attention to detail and adhering to a strict schedule.
- Skill in managing project timelines. Escalates issues appropriately. Identify sources of problems and recommend solutions.
If a bachelor’s degree is required, related work experience may be substituted in some positions. One year of college course work at an accredited institution is equivalent to one year of related work experience.
The Association of American Medical Colleges (AAMC) is an Equal Opportunity/Affirmative Action Employer. The AAMC is committed to the policy of an equal employment opportunity in recruitment, hiring, career advancement, and all other personnel practices. The AAMC will not discriminate on the basis of race, color, sex, national origin, religion, age, marital status, personal appearance, sexual orientation, gender identity or expression, family responsibilities, matriculation, political affiliation, genetic information, disability, past or current military service, or any other legally protected characteristic.
Apply here: https://aamc.wd5.myworkdayjobs.com/en-US/AAMC/job/Washington/Manager--Admissions-and-Selection-Research_10002954-1