# Website improvement experiment

Version: 2026-10-27.1. This is a blank protocol, not a completed experiment.

## Before implementation

Name the reader, page, hypothesis, primary outcome and observation window.
Record the baseline and source coverage. Freeze the question panel or page cohort
before the change. Separate branded and unbranded questions where relevant.
Choose a comparable holdout when feasible; random assignment can strengthen a
comparison. Do not create duplicate live pages just to compare generated drafts.

Record the same task, environment, tools, versions, instructions and resource
allowance for each arm. Identify exactly which variable changes. Keep factual
review independent of the arm where practical.

## Production and review

Capture the source packet, page decision, discarded alternative and final diff.
Record failures, retries, human review effort and actual cost; use unknown where
you lack a receipt. Review claims against the relevant source passages. Record
the tested revision, publication state and recovery path separately.

## Search observations

Record indexing state and reporting delays. Compare complete periods. Note other
site changes, seasonality and search-system changes. A before-and-after movement
alone does not establish that your edit caused it.

For AI answers, store exact wording, engine, exposed model/version, search setting,
language, market, date, capture method and source URLs. Repeat observations across
dates. Keep failures in the record. Distinguish mention, citation, recommendation
and referral. Report a numerator and denominator. API and consumer UI observations
are separate samples.

## Report

Publish the method, result, uncertainty and decision. Report negative and
inconclusive findings. Verify permission before redistributing raw provider data.
Disclose supplied access or sponsorship. Sponsors do not determine the result.

## Stop rule

Stop paid requests at the authorized budget. Stop publication for unsupported
material claims, exposed confidential data or a broken reader task. Record the
blocking evidence and the smallest next action that resolves it.