AI-generated illustration. Confidential work can remain protected while a separate, invented exercise demonstrates relevant engineering skill.
Assess engineering candidates with confidential work through permitted evidence and job-relevant demonstrations. Record historical claims separately. An unavailable artifact is an evidence limitation, not proof of weak engineering. A synthetic exercise can show present reasoning without proving past delivery.
The question is whether a candidate meets the role’s requirements, not whether a previous employer allowed them to publish. Keep those decisions separate.
Separate skill, history, and permission
A portfolio review can blur three questions:
- Skill: Can this person perform the relevant engineering work?
- History: What did this person contribute, and what happened as a result?
- Permission: Is this material authorized for disclosure to this audience?
A public repository may help assess skill and history. But availability alone does not establish authorship, contribution, or business impact. Likewise, an inability to share employer-owned code does not establish a lack of competence.
Use three evidence labels:
- Authorized historical evidence: Past work the candidate is permitted to share with this audience. It may support delivery claims, subject to attribution and context.
- Synthetic demonstration: An exercise using invented code, data, requirements, or architecture. It demonstrates observed skills, not the history that inspired it.
- Unverified historical claim: An account of responsibilities or outcomes without sufficient permissible corroboration. Use it to guide questions, not as established history.
These categories are not rankings of candidate quality. A strong synthetic demonstration may reveal more about debugging skill than an authorized slide deck.
Use a claim-to-evidence matrix
Start with a role-relevant claim. Identify acceptable evidence, its limits, and a useful follow-up. Give each artifact credit only for what it shows.
| Hiring claim | Permissible material | What it can establish | What remains unknown | Follow-up |
|---|---|---|---|---|
| “I can diagnose concurrency failures.” | Invented code and synthetic traces | Observed hypothesis formation, test selection, and causal reasoning | Whether the candidate solved a similar production incident | Ask for a test that distinguishes two plausible causes |
| “I designed a service boundary.” | Authorized design excerpt with contribution context, or an invented design exercise | Historical design work within the excerpt’s scope, or current tradeoff reasoning in the exercise | Ownership, adoption, and operational consequences unless separately supported | Ask which constraints drove the boundary and when it would be inappropriate |
| “I led a difficult migration.” | Authorized plan and, where permitted, a reference familiar with the candidate’s role | Specific planning or coordination responsibilities | Ownership and outcomes beyond the evidence’s scope | Request a synthetic migration plan with failure checkpoints and explicit responsibilities |
| “My change reduced incident frequency.” | Authorized outcome records with a measurement window and contribution context | A reported change in the measured outcome | Causation, competing changes, and precise contribution | Ask what would distinguish the intervention’s effect from other explanations |
| “I mentor engineers effectively.” | Authorized account of responsibilities, plus an invented coaching exercise | Historical responsibilities within the account’s scope and observed feedback quality | Long-term mentee development or team effects | Ask the candidate to adapt feedback after a fictional learner reveals a misunderstanding |
Read each row narrowly. A reference confirming participation in a migration does not establish design ownership. A good design exercise does not prove that the candidate operated that design under real constraints.
Score skill and record evidence limits separately
Define the role’s competencies before reviewing candidates. These might include problem framing, diagnosis, design judgment, validation planning, and collaboration. Select only what the role requires.
Then define observable behaviors for each rating:
| Rating | Meaning |
|---|---|
| Not observed | Insufficient usable evidence. This is not a failing score. |
| Below requirement | Observed behavior misses a defined requirement, even after reasonable clarification. |
| Meets requirement | Observed behavior addresses the relevant constraints and supports a defensible decision. |
| Exceeds requirement | Observed behavior also handles consequential ambiguity or competing constraints beyond the required baseline. |
For diagnosis, “meets” might require targeted tests that distinguish plausible causes, rather than a list of guesses. Set that expectation before the interview.
Record the evidence limit separately:
Diagnosis: meets. Observed in an invented debugging exercise. Candidate selected a test that separated two plausible causes. Claimed historical incident reduction remains unverified.
Do not turn “cannot share” into “below requirement.” Nor should an articulate story become verified delivery.
If an essential dimension is not observed, use a preselected follow-up with a comparable time budget and difficulty. Consistency means assessing the same capability to the same standard, not demanding the same artifact from everyone. Record any uncertainty that remains.
AI-generated illustration. In the fictional debugging exercise, an atomic claim prevents two workers from claiming the same pending job. Retry safety remains a separate problem, and the demonstration does not verify past production impact.
Worked example: debugging skill is not proof of production impact
Hypothetical scenario: A candidate says they fixed duplicate job execution and reduced related incidents. Their former employer’s code, incident reports, and metrics are confidential. The interviewer supplies a fictional queue implementation and synthetic traces, unrelated to that employer’s system.
Exercise assumptions: Two workers can select the same pending job before either records ownership. Jobs may be retried, and the exercise permits changing the claim operation. All code and data are isolated test materials.
The candidate:
- Identifies the gap between selecting a job and claiming it.
- Constructs an interleaving in which both workers select the same job.
- Proposes an atomic claim operation.
- Explains why preventing simultaneous claims does not, by itself, make retries safe.
- Suggests tests for worker failure after an external effect but before completion is recorded.
This demonstrates relevant reasoning. The candidate identifies a race, explains its mechanism, distinguishes it from retry behavior, and proposes validation.
It does not verify that they implemented a similar fix at their former employer, that it reached production, or that it caused incident frequency to fall.
A useful assessment record would say:
Observed: Meets the debugging requirement in the synthetic exercise. Identified the ownership race and a separate retry risk. Proposed relevant tests but did not implement them during the session.
Historical claim: Production fix and incident reduction not verified. No protected material requested.
Follow-up: If prior operational ownership is essential, seek a permitted reference discussion focused on responsibilities. If unavailable, record the uncertainty rather than treating the exercise as historical proof.
The exercise’s value depends on preserving that boundary.
Treat permission as a gate, not a cleanup task
Redaction alone does not establish permission to share. Removing names, credentials, or customer identifiers may still leave protected code, architecture, operating details, or employer-owned material.
Before accepting historical evidence, ask:
- Is sharing authorized for this audience and purpose?
- Can the candidate describe their contribution within the permitted scope?
- Does the material include customer, colleague, or third-party information that also needs protection?
- If permission is unclear, can an invented exercise assess the skill instead?
Do not ask candidates to recreate a confidential system from memory or quietly display a private repository. Calling it a reconstruction does not make it safe. Synthetic work should use invented requirements and data, not a lightly disguised copy of protected work.
Give managers and candidates the same evidence card
Use this template for each important hiring claim:
Role requirement: What capability are we assessing?
Candidate claim: What exactly is being asserted?
Evidence type: Authorized history, synthetic demonstration, or unverified historical claim?
Permission boundary: What may be shared, and what is excluded?
Observation: What did the evidence actually show?
Limit: What does it not establish?
Assessment: Not observed, below, meets, or exceeds the defined requirement?
Follow-up: What bounded action would resolve the most important remaining uncertainty?
Candidates can use the card to prepare a confidentiality-safe portfolio. Managers can use it to compare assessments without silently rewarding access to publishable work. Mentors can identify where a clearer demonstration would help and where further disclosure is inappropriate.
The standard is not “show everything.” Make supported claims, demonstrate what can safely be assessed, and keep the unknowns visible.
To organize your own evidence, take this card to Skills Tech Network.