The Computomatix Times

Most "AI optimization" demos are fun to watch for the same reason benchmark tweets are fun to watch: they show you the win, not the search.
You see the final graph. You see the +12% or the "runs 2x faster now" claim. What you usually do not see is the graveyard of bad ideas behind it. The settings that looked promising but were just noise. The optimizations that made throughput better by quietly making the model worse. The fake wins that only happened because the benchmark got easier.
So I built a small repo called Auto-Inference-Optimiser (star the repository!) to study exactly that.
The idea is simple: lock the evaluation, open one file for experimentation, and let an AI coding agent hill-climb on inference speed forever on Apple Silicon.
The most interesting part was not that it achieved a speedup. It was the kind of speedup it achieved, what it failed to improve, and what that indicates about inference engineering on real hardware.
Let's get into it.
Why I Built This
I care a lot about inference right now.
Not in the abstract "LLMs are cool" sense. I mean the actual production questions: where latency comes from, what batching buys you, what prompt processing costs, how KV cache decisions change throughput, and where the hardware wall starts pushing back.
There is a lot of content online about training. There is also a lot of content online about agents. But there is still not enough content that combines both instincts: build a tight experimental harness, let the agent search inside it, and use that process to learn something real about inference.
This repo was my way of doing that.
It is inspired by Karpathy's Autoresearch, but pointed at a different layer of the stack. Instead of searching over training code on a GPU box, this one searches over an MLX inference pipeline on a Mac because I am GPU poor (please sponsor a GPU).
What The Repo Actually Does
At a high level, the repo turns "make inference faster" into a bounded optimization problem.
The structure is intentionally small:

That boundary is doing most of the work.
prepare.py is read-only. It fixes the benchmark model, the prompts, the warmup behavior, the averaging logic, and the quality gates. The agent cannot "win" by quietly changing the test.
inference.py is the search surface. That is where the agent is allowed to touch sampling, prefill step size, prompt formatting, and the general generation path.
program.md tells the agent how to behave:

That is the core harness.
And I like this design a lot because it bakes in three things that most autonomous coding demos hand-wave away:
Reversibility - bad ideas are cheap to discard.
Observability - every run leaves behind metrics and logs.
Constraints - the agent is not allowed to optimize by moving the goalposts.
The Evaluation Is The Real Product
The truth is that the most important file in this repo is not inference.py. It is prepare.py.
That file fixes the benchmark around a small Apple Silicon friendly model, runs warmups, averages across multiple runs, and evaluates five different prompt types:
explanation
long-context summarization
reasoning
creative generation
code generation
That already makes the benchmark better than a lot of speed demos, because decode-heavy and prefill-heavy cases behave differently.
But the more important choice is the quality gate.
This repo does not let the agent optimize only for tokens/sec. It requires two checks to pass:
avg_perplexity has to stay below a threshold
sanity_check has to stay above a threshold
That second gate matters a lot.
Perplexity is useful, but it is still a model-internal metric. It can tell you that outputs are becoming unstable or degenerate, but it does not fully tell you whether the answer is still usable. So the repo also checks for concrete task-level correctness: did the train speed answer contain 48? Did the transformer explanation mention the right ideas? Did the LCS prompt actually return something that looks like Python code?
This is one of my favorite design choices in the whole project.
Because if you do not defend quality explicitly, an optimization harness will absolutely "improve" your system by making it worse.
What Actually Worked
After the optimization runs, the pattern was surprisingly clear.
Here is the short version:

But the more interesting part is where they came from.
1. Argmax sampling was the biggest win
On the Qwen run, setting sampling to greedy decoding gave the largest gain: about +10.8% generation throughput.
On the Llama run, it was also the best keep: about +2.6%.
That tells you something important: sampling overhead is not free. Top-p decoding is doing real work every token, and if your objective is pure throughput, removing that work can matter more than a lot of fancier ideas.
Of course, there is a trade-off.
You get deterministic output and lose diversity. So this is not a universal recommendation for every product. But as an inference lesson, it is very clean: sometimes the fastest path is just doin… continue on X ↗

You've seen the X articles.
Maybe you've downloaded Claude Code but you weren't sure where to get started.
You saw @Karpathy's post about auto-research but you're not sure if you have anything interesting to research.
You saw a bunch of tweets about things to add to your CLAUDE.md, but then you saw another saying your CLAUDE.md was too long and should be cut down to basics.
You're lost and you don't know what to do.
This is the tweet for you.
I've read all those tweets for you. Or more accurately, I've read some of them and my Claude has read them all. And my Claude distilled all of that into a single, simplified workflow that you can copy for yourself.
Just clone this repository and you're good to go: github.com/griffinhilly/claude-code-synthesis
Alternatively you can have your Claude do it for you.
Every week I download all the bookmarks I've saved and have my Claude read them. We consider adding anything we don't have already in our workflow. Your Claude can do the same.
Here's what Claude and I have found:

The Operating Model (CLAUDE.md)
The single most important file. Copy it to ~/.claude/CLAUDE.md and it changes how Claude approaches every task. Here's what's in it:
Leverage Doctrine. You do the thinking. Claude does the doing. You ideate, decide, and steer. Claude researches, implements, and executes. When uncertain, it surfaces options with tradeoffs instead of deciding silently.
Plan-First Protocol. Every task starts with: what's the objective? How will we know it worked? What are the sub-tasks? Research agents plan, implementation agents execute. Never both.
Scope Discipline. Claude pushes back on ambitious plans. "This is a 3-session project. Want to start with just X?" A working smaller thing beats a half-finished grand vision.
Orchestrator-First. The session agent is a manager, not a worker. Before any task, it decides: handle directly, delegate to a subagent, or route to MCP? This is the single biggest lever for productivity.
Dialectic Reviews. For important decisions, don't ask "what should I do?" Spawn opposing agents — one argues FOR, one argues AGAINST — with a referee to synthesize. Dramatically better than asking one agent for pros and cons. (h/t @danpeguine and @systematicls for the Hunter/Skeptic/Referee pattern)
Anti-Sycophancy. If an approach has clear problems, Claude says so directly, proposes an alternative, and accepts override. Sycophancy is a failure mode.
Test-First Bug Fixing. When a bug is reported, write a test that reproduces it before trying to fix it. (h/t @tangming2005 — this was his "single biggest improvement to my CLAUDE.md")
Operationalize Every Fix. Don't just fix the bug. Write tests that catch the whole class of similar bugs. Check for other instances. If it reveals a gap in your instructions, update CLAUDE.md. Every bug is a learning opportunity. (h/t @doodlestein)
Evals Before Specs. Define how you'll evaluate success before writing the spec. The progression: evals → spec → plan → implement → verify. (h/t @synopsi, who now spends 90% of time on evals)
Prefer the Boring Solution. Can this be fewer lines? Are abstractions earning their complexity? Don't build 1,000 lines when 100 suffice. (h/t @karpathy — "agents bloat abstractions, have poor code aesthetics")
Progressive Disclosure. Don't dump everything into CLAUDE.md. Keep it lean with trigger rules ("when X happens, read guide Y"). Guides load on-demand. (h/t @toddsaunders and @mstockton)
Structured > Prose. For rules agents MUST follow, use XML tags and JSON, not markdown paragraphs. Claude processes tagged content differently. (h/t @ihtesham2005 and @Austen)
Workflow Evolution. The workflow is a living system. When a session reveals a new pattern, encode it. Use your tools to improve your tools. It's a flywheel, not a static config. (h/t @doodlestein's Agent Flywheel) ALSO SERIOUSLY, GO FOLLOW @doodlestein
Corrective Framing. "Remember to do X" doesn't work. Instead, present a possibly-wrong claim: "You should be doing X — are you still doing it?" Mismatches trigger natural correction. (h/t @yishan)

The COMP System
Every project gets 4 files:
- CLAUDE.md — how the AI should behave here
- ORIENT.md — what a human needs to know to work here
- MEMORY.md — accumulated decisions, gotchas, context
- PLAN.md — roadmap, progress, next steps
Separate behavioral instructions from human orientation from accumulated knowledge from direction. Each has a different audience and update frequency.

The Guides
Seven situational guides that Claude loads on-demand — delegation templates (7 agent types with prompt structures), shell rules, context efficiency, API-over-scraping, PostgreSQL batching, overnight autonomous runs, and a skills reference.
The delegation templates alone are worth the clone. Implementer, Researcher, Reviewer, Batch Worker, Session Reviewer, Explorer, Creative — each with a prompt template, model recommendation, and mandatory report format.

The Bookmark Pipeline
This is the meta-move. Every week… continue on X ↗

Google Stitch 2.0 + Claude Code via MCP is the workflow I’ve been testing… and the results are genuinely insane.
I've been saying this for a while now. If your app looks like AI slop, that's not an AI problem. That's a workflow problem.
Most builders are stuck in the same cycle. You build something functional with AI. The features work. The logic is solid. But the UI looks generic and you know it. Your users know it too. And the moment someone lands on your product, they make a judgment in about two seconds.
The old fix was to hire a designer. Spend $3,000 to $10,000. Wait weeks for Figma files. Then spend even more time getting someone to actually implement those designs into your codebase. By the time the design was live, you'd already lost momentum and burned through cash you didn't need to spend.
That entire process is now optional.
I was able to take a client project that looked like every other vibe coded app and completely redesign it in under an hour. Professional typography. Consistent color system. A design language that actually holds together across every screen.
This is the workflow I'm running now. And I think every builder shipping with AI needs to understand how it works.
Why Stitch 2.0 Is Actually Different
I've tested a lot of AI design tools over the past year. Most of them generate something that looks decent on the first screen and then falls apart the moment you try to build a second page. Consistency is the problem. It always has been.
Stitch 2.0 solves this in a way nothing else has.
It's an AI native canvas. You can feed it screenshots of your existing app, drop in inspiration images from places like Dribbble or 21st.dev, or even paste a URL and let it reverse engineer the design of any website you like. It takes those creative seeds and generates full UI designs with multiple variants you can pick from.
But the real unlock is not the designs themselves. It's what Stitch builds in the background while it's designing.
A complete design system.
Typography scales covering display, headline, label, title, and body fonts. A primary, secondary, and tertiary color palette that's auto generated to be complementary. Color scales for every shade. Component rules and patterns. Elevation and depth specs. Even dos and don'ts for your design language.
All of this gets documented automatically in a file called design.md. This is a plain markdown file that captures every single design rule in one place. And this is the file that changes everything when you bring Claude Code into the picture.
How I Actually Use This on Real Projects
Let me walk you through exactly what I do. No theory. Just the workflow.
I start by taking screenshots of whatever I'm redesigning. If it's a client project or something I've been building myself, I screenshot the main screens and drop them directly onto the Stitch canvas. If I'm starting fresh, I'll grab two or three inspiration images from Dribbble instead. You don't need more than that. You're not looking for something to copy. You're looking for a direction.
Then I write one focused prompt. I tell Stitch what the app does, which screens I want redesigned, and the design direction I'm going for. Dark mode, minimal, editorial, whatever fits the product. I also specify font preferences because fonts are honestly the single fastest way to elevate how an app feels. Serif for headings, clean sans serif for body is a combination that works really well for most SaaS products.
Stitch generates multiple variants from that one prompt. This is important. Don't just accept the first output. Look at each variant and pull the best elements from each. The typography from one, the layout from another, the color energy from a third. You're curating, not just accepting.
One thing most people miss about why Stitch produces such strong output is that it generates images first before any code. That means it's not constrained by what HTML and CSS can do. It can imagine anything visually. Then you work backwards from that reference to build it. That's why the designs feel so much more polished than what you get from prompting a coding tool directly.
You can also talk to Stitch via voice now instead of typing. It transcribes your words into prompts automatically. Small feature but genuinely useful when you're deep in a session and want to keep moving fast.
Design.md Is the Real Game Changer
Once you're happy with your designs, go to the right hand panel in Stitch and click on Design Systems. You'll see that Stitch has already created one for you automatically based on everything you've been designing.
Click into it and you'll find the full design system I described earlier. Typography, colors, components, rules, everything documented and organized.
Now click on design.md.
Copy the entire file. Go to your project. Create a new file called design.md in the root directory. Paste it in. Save it.
That file is now the single source of truth for your entire design language.
Here's why this matters … continue on X ↗