# Selection Decisions And Scoring > >- - **Type:** Skill - **Install:** `agentstack add skill-openmatter-network-agent-io-skills-selection-decisions-and-scoring` - **Verified:** Yes — security-reviewed for prompt injection and unsafe behavior - **Seller:** [OpenMatter-Network](https://agentstack.voostack.com/s/openmatter-network) - **Installs:** 0 - **Category:** [Agent Skills](https://agentstack.voostack.com/c/agent-skills) - **Latest version:** 0.1.0 - **License:** MIT - **Upstream author:** [OpenMatter-Network](https://github.com/OpenMatter-Network) - **Source:** https://github.com/OpenMatter-Network/agent-io-skills/tree/main/personnel-selection/skills/selection-decisions-and-scoring ## Install ```sh agentstack add skill-openmatter-network-agent-io-skills-selection-decisions-and-scoring ``` Requires the [AgentStack CLI](https://agentstack.voostack.com/docs/cli). Works with Claude Code, Cursor, and any MCP-compatible agent. ## About # Selection decisions and scoring How scores become hiring/promotion decisions. Every choice here affects validity, expected performance of those selected, and subgroup passing rates — so each requires a **documented rationale**. There are few absolutes; professional judgment driven by organizational goals governs. ## Combining procedures into a system When multiple procedures form the basis of a decision, **both the individual components and the combination** must be supported by validity evidence. Document the **method and rationale** for combining and sequencing. Organizations weight differently depending on whether they emphasize **maximizing validity**, **minimizing subgroup differences**, or balancing the two — state which. Recall from `criterion-related-validation`: **effective weights ≠ nominal weights** (they depend on component variances/covariances, and differential range restriction can distort them). ## Compensatory vs. multiple hurdle - **Compensatory** — candidates achieve a specified **total** across assessments; a high score on one can offset a low score on another. - **Multiple hurdle** — candidates must clear a score on **each** assessment (often sequential). - **Hybrids** are possible (a hurdle on one critical predictor, compensatory among the rest). No model is universally correct. The method of combining scores can affect the **overall reliability** of the process and **subgroup passing rates** (Sackett & Roth, 1996). Present the rationale and supporting evidence for the model recommended. ## Cutoff scores vs. rank order Two common strategies: a **cutoff score** (reject below a point) or **rank-order / top-down** selection. - There is **no single best method** for setting cutoffs. Options include **criterion-referenced** cutoffs (when the predictor links to a meaningful performance threshold) and others (Mueller, Norris, & Oppler, 2007). - With valid predictors showing **linearity**, cutoffs may be set **as high or low as needed** to meet organizational requirements. **Nonmonotonicity** in the predictor–criterion relation should inform how scores are used. - When local data can't establish linearity/monotonicity, consult **past research** and its implications (e.g., cognitive ability tends to relate linearly to performance; linearity for other predictors such as personality is less settled). - When setting any cutoff, consider the **conditional standard error of measurement** at the cutoff region; document the SEM model used. Consider reporting the percentage of applicants classified the same way (pass/fail) across replications at the cutoff (Haertel, 2006). ## Bands **Bands** are score ranges within which candidates are treated alike (a form of cutoff that defines ranges). Methods vary (Cascio, Outtz, Zedeck, & Goldstein, 1991; Campion et al., 2001). - Rationale can be **psychometric** (imprecision/SEM of scores) and/or **administrative/ organizational**. - Tradeoff: because banded candidates with different scores are treated alike, banding generally yields **lower expected criterion performance and utility** than top-down selection — but may be balanced by administrative ease and possibly **increased workforce diversity**, depending on how within-band selection is done. - Document the **basis for development** and the **decision rules** for administering the band. ## Top-down vs. cutoff vs. bands — the driving factors Decisions are typically driven by organizational goals and factors such as: estimated **cost–benefit ratio**, number of vacancies, **selection ratio**, labor market, expectancy of success vs. failure, **consequences of selection errors**, relative emphasis on performance vs. diversity goals, judgments about the level of KSAO/performance required, and the procedure's utility. Some organizations choose a **cutoff over rank order to increase diversity**, accepting possible reductions in performance and utility. Whatever the decision, **document the rationale**. ## Norms Present normative information for the **applicant pool and incumbent population** when appropriate. Describe the normative group's relevant demographic/occupational characteristics and the **time frame**. Note that large discrepancies between incumbents and the applicant pool can make incumbent-based cutoffs **too high** (or otherwise inappropriate). ## Communicating effectiveness - **Expectancy charts** relate score ranges to work performance; **Taylor–Russell tables** show the proportion of hires who will be successful under combinations of validity, selection ratio, and base rate. - **Utility estimates** project productivity gains (in dollars, output %, or reductions in accidents/person-hours). Utility values rest on **assumptions** and uncertain parameters — report them as **estimates**, and present **minimal and maximal** point estimates to reflect uncertainty. ## Appropriate use Use a procedure **only for purposes with validity evidence**. Changing the **mix or combination** of components (especially in a compensatory system) can fundamentally change the supported inference — the original validation evidence may no longer suffice. Don't repurpose a procedure (e.g., diagnostic use, or an education-designed test for employment) without supporting evidence. ## Pitfalls - Combining procedures without validating the **combination**, not just the parts. - Setting cutoffs without considering linearity/monotonicity or the conditional SEM. - Presenting utility as precise dollar truth rather than an assumption-laden estimate. - Using incumbent norms to set cutoffs when the applicant pool differs markedly. - Silently changing weights/combination and assuming prior validity carries over. ## Checklist - [ ] Combination & sequencing method documented with rationale - [ ] Compensatory/hurdle/hybrid choice justified; reliability & passing-rate effects considered - [ ] Cutoff/rank/band choice tied to org goals (selection ratio, vacancies, error costs, diversity) - [ ] Linearity/monotonicity and conditional SEM addressed for cutoffs - [ ] Band development basis and within-band decision rules documented - [ ] Norms described (group, demographics, time frame); incumbent-vs-applicant gap considered - [ ] Effectiveness communicated (expectancy/utility) with uncertainty ranges - [ ] Validity evidence covers the actual operational use ## See also `criterion-related-validation` (weights, composites, corrections) · `fairness-and-bias-analysis` (subgroup tradeoffs; analyze the operational composite) · `technical-validation-report` · `administration-documentation` *Source: Principles (5th ed., 2018), "Operational Considerations → Data Analyses (Combining procedures, Multiple hurdles vs. compensatory, Cutoff scores vs. rank orders, Bands, Norms) and Communicating the Effectiveness / Appropriate Use of Selection Procedures."* ## Source & license This open-source skill is cataloged on AgentStack and links to its original source — we do not rehost the code. - **Author:** [OpenMatter-Network](https://github.com/OpenMatter-Network) - **Source:** [OpenMatter-Network/agent-io-skills](https://github.com/OpenMatter-Network/agent-io-skills) - **License:** MIT Install and usage instructions live in the source repository linked above. ## Pricing - **Free** — Free ## Security capabilities Automated source analysis of v0.1.0 — what this tool can access: - **Network access:** no - **Filesystem access:** no - **Shell / process execution:** no - **Environment & secrets:** no - **Dynamic code execution:** no *"Yes" means the capability is present in the source — more access means more to trust, not that it is unsafe.* ## Versions - **0.1.0** — security scan: passed — Imported from the upstream source. ## Links - Listing page: https://agentstack.voostack.com/l/skill-openmatter-network-agent-io-skills-selection-decisions-and-scoring - Seller: https://agentstack.voostack.com/s/openmatter-network - Browse the marketplace: https://agentstack.voostack.com/browse --- Listed on AgentStack — the marketplace for AI agent skills and MCP servers. Every listing is security-reviewed. Creators keep 70%.