Attendly · AI Harness Onboarding
01 / 18
Attendly

AI Harness

Developer Onboarding · 10 min

Why this exists

Three problems that drove us to build the harness.

Inconsistent specs

Specs lived in Linear, Slack, Notion, heads — scattered across too many places.

Silent regressions

Features broke when adjacent code was touched. No one was looking.

Ad-hoc review

Code review happened when someone had time. Not every change was audited.

Three-Phase Flow

Verifiable end to end. Same playbook for every change.

PHASE 1

Plan

Source → proposal → validate → grill-me → alignment

PHASE 2

Implement

Ralph Loop per task · checkpoint commits

PHASE 3

Ship

Review + auto-fix + contract + changelog + deploy

Phase 1 · Plan

Source → proposal → 3 parallel validators → composite score

Source of Intent

  • Linear ticket
  • Video transcript
  • Verbal prompt
  • Slack thread
pinned in proposal.md
openspec/changes/<id>/
├─ proposal.md
├─ design.md
├─ tasks.md
└─ specs/

Requirement

  • completeness
  • traceability
  • clarity
score 0 – 10

NFR

  • coverage
  • baseline
  • testability
score 0 – 10

Impact

  • coverage
  • dep. depth
  • risk ID
score 0 – 10

Composite score

(requirement + nfr + impact) / 3

The proposal reads the raw source directly — no pre-refinement. Separating refinement from authoring creates anchoring bias that degrades the alignment check later.

Grill-me · Pre-mortem · Re-validation

Mandatory in Attendly. Exception scenarios are explicit, not implicit.

1. Grill-me

MANDATORY in Attendly
  • Actor & access
  • Happy path
  • Data model
  • Edge cases
  • Error handling
  • Multi-tenant
  • Existing behavior

2. Pre-mortem

MANDATORY — Critical/High
  • data
  • load
  • partial failure
  • rollout
  • external
  • security
  • adjacent

3. Re-validation

FINAL composite score
  • 3 validators re-run
  • Pre-grill vs post-grill
  • Delta shown to user
  • Both kept in history
  • Approval uses FINAL
  • Critical pre-mortem risks → task or acceptance

Phase 1 · Alignment Check

The last gate of Phase 1. Clean-context comparison on Sonnet.

Raw source

ticket · video · prompt

Unchanged since the proposal was written.

alignment-validator
clean context
model: sonnet
Covered
source → spec mapping
Gaps
source missing in spec
Scope creep
spec beyond source (classified)
Intent mismatch
behavior disagrees

🚨 Conditional scope modifiers sweep

Dedicated pass for stray-sentence qualifiers — the #1 source of post-deploy regressions. Modifier gaps default to Critical.

tenant segment time window feature flag geography tier device integration data state ordering

Example: "only for Sant'Anna, high school students only" — buried in paragraph 3 of a long ticket. Caught before production.

Verdict: Aligned → proceed · Misaligned → grill-me / edit spec, re-run alignment.

Phase 2 · Implement

Ralph Loop per task · checkpoint commits · 3-attempt convergence rule.

🔴 Red

Write failing test

🟢 Green

Make it pass

🔵 Refactor

Improve, stay green

Checkpoint commits

  • task 1 ✓
  • task 2 ✓
  • task 3 ✓
  • …
feat(advisor-ranking): normalize weights before scoring
Co-Authored-By: Claude <agent> · type(scope): description

Phase 3a + 3b · Review & Auto-Fix

Parallel sub-agent review. Critical/High auto-fixed. Drill on design decisions.

code-reviewer
  • architecture
  • data access
  • security
  • performance
  • testing
  • code quality
regression-analysis
  • contract violations
  • blast radius
  • exception paths
  • perf degradation

Auto-Fix — Exception Paths

  • Critical / Major (review) + Critical / High (regression) → attempt fix autonomously
  • Fix needs a design decision (hard-error vs warn, A vs B) → STOP + drill user via AskUserQuestion
  • 3-attempt convergence rule → if fix doesn't converge, revert and report
  • Fix breaks pre-existing tests → revert (never degrade test integrity)
  • Re-score after fix · both runs kept in daily score JSON history

Phase 3c · Contract Check

Binary verdict: PASS or FAIL. Breaking changes require explicit mitigation.

GraphQL Schema

  • field removal
  • required arg add
  • type rename
  • nullable → non-null

Prisma / DB

  • DROP COLUMN
  • NOT NULL without default
  • column rename
  • narrowing type

TypeScript Public Types

  • removed symbol
  • field removal
  • stricter generic
  • narrower return

REST API when applicable

  • route removal
  • required param add
  • response schema change
  • auth change

Phase 3d + 3e · Changelog & Deploy

Deliver the narrative. Verify ready to ship.

3d · Changelog

Auto-detects ONE delivery target:

· Linear ticket

stakeholder-friendly comment

· Open PR

technical description update

· CHANGELOG.md

Keep-a-Changelog format

3e · Deploy Checklist

Conditional — only on Critical/High or Contract FAIL

  • Migrations tested
  • Rollback plan
  • Feature flags
  • Secrets in vault
  • Runbook updated
  • Observability
  • Dependencies
  • Stakeholders

OpenSpec Structure

One source of truth for architecture, specs, and change history.

openspec/
├─ project.md— project overview, stack, conventions
├─ nfr.md— non-functional requirements baseline
├─ adr/— architecture decision records (MADR)
├─ specs/— stable capability descriptions
├─ changes/— active + archived proposals
└─ scores/— per-change evaluation history (JSON)

Six Sub-Agents

Each in an isolated context. Four in Phase 1, two in Phase 3.

requirement-validator
Phase 1
completeness · traceability · criteria_clarity
nfr-validator
Phase 1
coverage · baseline_alignment · testability
impact-analyzer
Phase 1
coverage · dependency_depth · risk_identification
alignment-validator
Phase 1 · Sonnet
covered · gaps · scope creep (classified) · intent mismatch · modifier sweep
code-reviewer
Phase 3
architecture · data · security · performance · testing · quality
regression-analysis
Phase 3
contract · blast radius · exception · performance

PR Workflow

Every change ships via pull request. No direct pushes.

Create Branch

type/ATT-XXXX-description

Draft PR

when Phase 1 approved · still in progress

Ready for Review

when Phase 2 completes · tests green

Phase 3 updates PR

changelog · scores · risk matrix in body

Squash Merge

single commit · --delete-branch cleanup

Commands Cheat Sheet

Tags in parentheses show whether a command auto-runs inside the 3-phase flow.

Setup — once per project
/update-skillsinstall / sync skills, rules, agents (manual)
/setup-permissionspre-authorize non-destructive commands (manual)
Diagnostic — anytime
/check-skills-versioncompare project vs latest tooling (manual)
/telemetry reportaggregated view of tool / skill usage (manual)
Workflow — inside the 3-phase flow
/openspec-proposalraw source → full proposal (auto — Phase 1)
/workflow-impact-analysisblast radius analysis (auto — Phase 1)
/workflow-grill-meinterview + pre-mortem (auto — Phase 1)
/workflow-alignment-checkclosing gate (spec vs source) (auto — Phase 1)
/workflow-contract-checkbreaking-change validation (auto — Phase 3)
/workflow-changelogfinal narrative delivery (auto — Phase 3)
/workflow-refine-ticketstructure a vague ticket into a pre-spec (manual, standalone)

A Typical Task

Mostly autonomous. A handful of human touchpoints.

Paste ticket link
→
openspec-proposal
→
3 validators
→
grill-me answers
→
pre-mortem story
→
re-validation
→
alignment check
→
approve Phase 1
→
Phase 2
→
Phase 3
→
approve merge
👤 human touchpoint autonomous

The 6 touchpoints

  • Start — paste the ticket link.
  • Grill-me — answer branch-by-branch questions.
  • Pre-mortem — tell the "two weeks live, something broke" story.
  • Phase 1 approval — confirm the final composite score + alignment verdict.
  • During auto-fix (only if prompted) — resolve a design-decision drill from the AI.
  • Merge — approve the squash merge.

Everything else — proposal drafting, validation, re-validation, implementation, review, regression analysis, auto-fix, contract check, changelog posting — runs on its own.

Getting Started — First Install

Three steps, ~2 minutes. Once per machine + once per project.

1

Clone the tooling repo

git clone <attendly-ai-tooling-url>

Source of truth for skills, agents, and rules. Do this once per machine.

2

Install global commands

cd ai-tooling && ./sync-global.sh

Writes update-skills, setup-permissions, telemetry, check-skills-version into ~/.claude/ so every Claude Code session finds them.

3

Install skills in each project

cd <attendly-project> && claude → /update-skills

Detects the stack, sets up .claude/ symlinks to .ai-tooling/, initializes OpenSpec, wires up CLAUDE.md.

Staying Updated

/update-skills auto-git-pulls the tooling repo and version-gates.

/update-skills

git pull + compute hash

hash matches?
abort — up to date
sync + show delta
new version installed

Diagnostic, read-only

/check-skills-version

See where you stand without syncing. Reports installed version + latest available + delta.

Zero wasted context — the version gate only syncs when there's something new.

Attendly
Welcome aboard.