# Ai Job Analysis And Relevancy > >- - **Type:** Skill - **Install:** `agentstack add skill-openmatter-network-agent-io-skills-ai-job-analysis-and-relevancy` - **Verified:** Yes — security-reviewed for prompt injection and unsafe behavior - **Seller:** [OpenMatter-Network](https://agentstack.voostack.com/s/openmatter-network) - **Installs:** 0 - **Category:** [Agent Skills](https://agentstack.voostack.com/c/agent-skills) - **Latest version:** 0.1.0 - **License:** MIT - **Upstream author:** [OpenMatter-Network](https://github.com/OpenMatter-Network) - **Source:** https://github.com/OpenMatter-Network/agent-io-skills/tree/main/ai-selection-legal-ethical/skills/ai-job-analysis-and-relevancy ## Install ```sh agentstack add skill-openmatter-network-agent-io-skills-ai-job-analysis-and-relevancy ``` Requires the [AgentStack CLI](https://agentstack.voostack.com/docs/cli). Works with Claude Code, Cursor, and any MCP-compatible agent. ## About # AI job analysis & job relevancy (Concerns 2–3) Professional standards, legal guidelines, and case law all require selection systems to be **tied to job requirements**, established through an **analysis of work** (Morgeson et al., 2020). For AI tools built from convenient or scraped data, this is frequently the weakest link. (The article treats these two concerns together because requirements for a job analysis are hard to separate from job relevancy.) This skill is the AI-specific framing; for *how* to conduct a work analysis, use `work-analysis`. ## Why job analysis matters here - Traditionally, job analysis is the basis for identifying the **KSAOs** required and the appropriate variables for selection, and for **developing criteria / defining the performance domain.** - *Uniform Guidelines* §14B(2) requires a **review of job information** for criterion-related validity; the *Principles* state job analysis defines appropriate **predictors** and establishes the **relevance of the criteria.** - **In practice, many test users skip or abbreviate job analysis** — assuming, say, that all sales jobs are alike and a generic "sales test" transfers. Without a job analysis you cannot know (a) that the test's purported skills are the ones predicting in **this** organization, or (b) that the skills the job requires match those the test measures. Predictors and criteria may then be **wrong, inadequate, or unfair.** ### Is job analysis still needed when predictor–criterion relationships are strong? A live question: if a predictor set empirically predicts a criterion (locally or meta-analytically), does job relatedness still need a job analysis? **Risks of saying "no":** a **spurious** relationship can masquerade as job relatedness. *Example:* "leadership experience" correlates with managerial performance, but **gender also correlates** with both (men more often given leadership opportunities) — gender should be irrelevant, yet the relationship may be found. Like **ZIP-code-as-race** in credit scoring, AI's millions of correlations can rest decisions on hidden, illegitimate relationships. Without a job analysis to identify, define, and measure the KSAOs, predictors and criteria "may be used that are wrong, inadequate, or unfair." ## Forms and rigor of job analysis - Legal/professional guidelines allow **many acceptable methods** but are **silent on how comprehensive** the analysis must be. *Guardians* (2d Cir. 1980) holds job analysis should be **systematic and accurate regardless of methodology.** - **Competency models:** many test publishers apply a **standard competency model across organizations**, with loose ties to the actual tasks/behaviors of incumbents. Competencies may be specific KSAOs, broad KSAOs, or even organizational **aspirations** ("Demonstrates amazing customer service at all times"). Generic competency lists (e.g., Hunt, 1996) used for all jobs/levels for efficiency may not be **complete** relative to a specific job. Test publishers often omit a critical competency the test doesn't measure. - **O*NET** is a useful **starting point** for tasks/KSAOs but **should not substitute for a complete job analysis** (it has limitations given how the data were collected). - **Collecting task information** (frequency/importance ratings): UGESP requires measures of task importance/criticality and documents them as Essential (§15B(3)). This step is **often skipped**, yet collecting task ratings **orients SMEs to actual job requirements and away from stereotypes** — and reliability/validity of KSAO judgments rises when KSAOs are **concrete and tied to specific tasks** (Morgeson et al., 2020, p. 375). - **SME judgment quality:** interrater agreement (ICC) **assumes SMEs are similarly knowledgeable or trained** — an assumption **rarely evaluated** in practice. ## Job relevancy / job relatedness (the legal anchor) - **Job relevancy** = the core features of a job (KSAOs, work behaviors, environment), determined by job analysis. U.S. law **requires job relatedness** for tests with adverse impact (Title VII disparate impact → "job related for the position and consistent with business necessity"). - The *Uniform Guidelines* establish the explicit requirement to conduct a job analysis **whenever undertaking a criterion-related validation study** (§3.14(A)). - The *Principles* allow several routes to job relatedness: appropriate **criterion-related validity coefficients**, the **job relevance of the content**, or the **construct measured** (*Principles*, p. 90). Validity is a function of evidence supporting interpretation **for specific purposes.** - **The AI gap:** technologically enhanced tools may **predict outcomes** ("job related" in a statistical sense) yet **not be based in a job analysis** — leaving job relevance unestablished and legal defensibility weak. ## Questions to ask (from the article) - Is a job analysis necessary if predictors and criteria are strongly related in a criterion-related study? - Is a job analysis necessary to justify an operational performance measure (e.g., KPIs) used as the validation criterion? - To what extent is a **competency model** an adequate substitute for a job analysis? - Is it important to have a **complete** list of competencies? - How much **rigor** in the job-analysis methodology is necessary to establish job requirements? - Is **O*NET** an acceptable source for a complete list of KSAOs? - Is it important to collect **task information** (e.g., frequency and importance ratings)? - If a job analysis is not conducted, can job relevancy be demonstrated for the assessments or the criteria? Is predictor–criterion correlation **alone** sufficient to establish job relatedness under the *Uniform Guidelines*? ## Pitfalls - Deploying a generic "sales test" (or vendor competency model) without analyzing the local job. - Treating a strong predictor–criterion correlation as proof of job relatedness (spurious/proxy risk). - Using O*NET or an aspirational competency list as a complete job analysis. - Skipping task importance ratings, then over-relying on stereotype-tinged SME KSAO judgments. - Never checking whether SMEs are comparably knowledgeable before trusting interrater agreement. ## Checklist - [ ] A systematic, accurate job analysis underlies predictors and criteria (or its absence is flagged) - [ ] Spurious/proxy relationships ruled out via job-analytic grounding - [ ] Competency model's rigor and completeness for the specific job evaluated - [ ] O*NET used only as a starting point, not a substitute - [ ] Task importance/criticality information collected and documented - [ ] SME comparability assessed before relying on agreement statistics - [ ] Job-relatedness route (criterion / content / construct) identified for legal defensibility ## See also `work-analysis` (how-to) · `ai-predictor-theoretical-basis` · `ai-validity-evidence` · `ai-selection-legal-landscape` · `ai-input-data-and-design-audit` *Source: Tippins, Oswald & McPhail (2021), Concerns: "Job Analysis" and "Job Relevancy."* ## Source & license This open-source skill is cataloged on AgentStack and links to its original source — we do not rehost the code. - **Author:** [OpenMatter-Network](https://github.com/OpenMatter-Network) - **Source:** [OpenMatter-Network/agent-io-skills](https://github.com/OpenMatter-Network/agent-io-skills) - **License:** MIT Install and usage instructions live in the source repository linked above. ## Pricing - **Free** — Free ## Security capabilities Automated source analysis of v0.1.0 — what this tool can access: - **Network access:** no - **Filesystem access:** no - **Shell / process execution:** no - **Environment & secrets:** no - **Dynamic code execution:** no *"Yes" means the capability is present in the source — more access means more to trust, not that it is unsafe.* ## Versions - **0.1.0** — security scan: passed — Imported from the upstream source. ## Links - Listing page: https://agentstack.voostack.com/l/skill-openmatter-network-agent-io-skills-ai-job-analysis-and-relevancy - Seller: https://agentstack.voostack.com/s/openmatter-network - Browse the marketplace: https://agentstack.voostack.com/browse --- Listed on AgentStack — the marketplace for AI agent skills and MCP servers. Every listing is security-reviewed. Creators keep 70%.