Business Automation | 2026-07-20 | 11 min read
Self-Improving Outbound: Let Codex Tune the Playbook
A practical way to use Codex as a weekly improvement loop for outbound scoring, plays, outcomes, evals, and human-approved pull requests.
Direct answer: A self-improving outbound system uses Codex to read last week’s outcomes, propose one change to scoring or messaging rules, run an eval, and open a pull request for human approval. It should never send messages, scrape real people, or merge its own changes.
Written by: Esmail Hanif, AI Visibility Strategist & Founder, Martecks
Short answer
A self-improving outbound system does not let AI send more messages. It lets AI improve the rules behind the messages.
The safe version is simple: log outcomes, keep scoring and plays in editable files, run an eval, let Codex propose one change, then require a human to approve the pull request before anything changes in production.
That is the difference between blind sales automation and a real improvement loop.
The core idea
Outbound already has cheap feedback. A lead replies, books, ignores, bounces, or turns out to be a bad fit. Most teams collect those outcomes in dashboards, but the actual judgment stays scattered across people, spreadsheets, and old campaign notes.
Codex becomes useful when those judgments are written as files. Scoring rules live in config. Plays live in prompt files. Outcomes live in a JSONL memory log. Evals decide whether a proposed change helped. Codex reads the system, edits one thing, runs the test, and opens a PR.
The repo shape
Start local and boring. Do not connect CRM, enrichment, or sending until the improvement loop proves it can behave on files.
| Folder or file | What it stores | Why it matters |
|---|---|---|
| AGENTS.md | Rules for what Codex can and cannot edit | Keeps the agent contained. |
| config/scoring.yaml | Signal weights, thresholds, and negative signals | Turns sales judgment into reviewable rules. |
| config/plays.yaml | Message plays and when to use them | Makes outreach strategy editable without hiding it in a tool. |
| memory/outcomes.jsonl | One row per touch once the outcome is known | Gives the system market feedback. |
| evals/fixtures.yaml | Known examples the system must route correctly | Prevents plausible changes from passing without proof. |
| evals/score.py | The scoring test | Turns improvement into a measurable gate. |
| prompts/*.md | Codex instructions for proposing changes and writing PR summaries | Makes the agent workflow repeatable. |
Write the law first
The most important file is not the prompt. It is the rule file. Codex needs a narrow job and hard boundaries before it edits anything.
The first version of AGENTS.md should say: never send messages, never scrape or enrich real people, never self-merge, change one concept at a time, cite outcome rows for every change, and stop if the eval does not improve.
| Rule | Why it exists |
|---|---|
| Never send messages | Improvement is allowed. Delivery stays human-approved. |
| Edit only scoring, plays, or prompt files | Prevents the agent from expanding scope. |
| Change one concept at a time | Makes the result reviewable. |
| Cite outcome rows | Forces the change to come from market feedback. |
| Run the eval before proposing a PR | Stops nice-sounding changes that do not improve the system. |
Move judgment into config
Most outbound systems fail because the judgment is invisible. Someone knows that a competitor-comparison signal is strong, a generic ebook download is weak, or a student research request is a bad fit, but the automation cannot see that judgment.
Put it in config. Five signals are enough to start. A simple scoring file is better than a complex black box because the team can argue with it, review it, and improve it from outcomes.
- Strong signal: competitor comparison, implementation page visit, active hiring pain.
- Weak signal: generic download, newsletter open, broad content interest.
- Negative signal: competitor, student research, vendor pitch, outside service area.
- Threshold: draft, human review, ignore.
Outcome memory is the compounding asset
A weekly dashboard can say reply rate dropped. A clean outcome log can say which signal, play, account type, or reason caused the change.
The reason field matters most. "No reply" is weak memory. "Content-only intent" is useful memory. "Asked about implementation timeline" is useful memory. Codex can only improve the playbook if the rows explain what the market actually did.
| Field | Example |
|---|---|
| date | 2026-07-20 |
| account | Northwind Finance |
| signal | implementation_page_visit |
| play | implementation_angle |
| score | 6 |
| outcome | reply, meeting, no_reply, bad_fit, bounced |
| reason | Asked about implementation timeline. |
The eval gate
Before Codex edits the playbook, build the test it must satisfy. The eval does not need to be huge. It needs enough ugly cases to catch bad behavior: weak intent, bad fit, stale signals, and accounts you wish the system had skipped.
A good first eval asks one thing: given these signals, should the account be ignored, drafted, or sent to human review? If a proposed scoring change does not improve or preserve the eval, it does not get merged.
The weekly Codex loop
The weekly job should be boring enough to trust.
| Step | What Codex does | What stays human |
|---|---|---|
| Read outcomes | Find one pattern from last week. | Confirm the pattern makes business sense. |
| Propose one change | Edit one scoring rule or one play. | Reject changes that overfit a tiny sample. |
| Run evals | Show before and after score. | Decide whether the eval is strong enough. |
| Open PR | Explain changed lines, evidence, and risk. | Approve, revise, or close the PR. |
| Deploy later | Nothing sends automatically. | Human controls merge and sending. |
What not to automate
The dangerous version is obvious: AI scrapes people, invents personalizations, sends at scale, and learns from noisy metrics without review. That is how outbound automation gets spammy and brittle.
The useful version is narrower. AI proposes changes to the system. Humans approve the changes. Sending, compliance, targeting, and brand judgment stay outside the loop until the process has earned trust.
- Do not let the loop send messages.
- Do not let it enrich private people data.
- Do not let it self-merge.
- Do not optimize only for replies if bad-fit replies increase.
- Do not change five rules in one PR.
Where this fits in a content system
The same pattern works for content automation. A topic calendar can log impressions, clicks, AI visibility checks, source gaps, and page updates. Codex can propose one change to a title, internal link, source block, or CTA, then run the site build and open a PR.
The principle is the same: version the playbook, log outcomes, build an eval, change one thing, and keep approval human.
Reference links
These references support the Codex and agent-loop patterns behind this workflow.
Sources: OpenAI: Codex and code generation, GitHub Docs: pull requests, Related source: Nicolas Finet on self-improving outbound
Final answer
Use Codex to improve the outbound system, not to blast the market.
Put scoring, plays, outcomes, and evals into files. Let Codex propose one evidence-backed change, run the test, and open a PR. Keep sending and merging outside the loop.