by Idan Shimon · notes from shipping AI tools
I test AI tools on real work, publish the numbers that hurt, and show every trial so you can check me.
One run made two models look unsafe. Three runs showed it was three, and how often.
The file got 34× smaller. The bill got 2–4× smaller. Both numbers go in the headline.
Claims about other people's tools get read in their code and run in their tests first.
Agent memory & context2026-10-1511 min read
Teaching an assistant a rule is easy. Teaching it when not to use the rule is harder. 216 generation trials and 96 calibration decisions, on a local 3B model and on GPT-5.6-sol.
Start here: visual guide and simulatorInteractive workflow mapRecorded data
Decision models2026-09-258 min read
When an application needs a choice, why generate a paragraph? A visual guide to Jev and Kev, a 60-passage diagnostic, and why better decision models need better questions.
GitHub & Azure2026-09-0210 min read
It looks like a five-minute settings change, until the consent screen refuses you and it turns out it was never a GitHub permissions problem. The exact menus, the real screens, and five things to set up the day after.
GitHub & Azure2026-07-137 min read
With usage-based billing, one heavy team can drain a credit pool everyone paid into. Cost centers fix that, but a cap is not a reservation. The verified version, checked against the REST API.
Agent memory & context2026-06-274 min read
A tiny, dependency-free retrieval layer that says “I don't have that” instead of guessing. 99.2% fewer tokens per query, measured, with a live demo running the real engine in your browser.
Agent memory & context2026-06-186 min read
Another engineer's open-source SDK, checked against its source and its 115 tests. It writes a project file by file and keeps only signatures of finished files, so file 1,000 costs about the same as file 1.
Agent memory & context2026-06-1711 min read
432 trials, 9 models, 3 runs. When the answer was in the brief, every model found it. When it wasn't, one decoy number caught three smaller models, and one run would have measured them wrong.
I help teams put AI coding tools and agents to work, and I build small open-source tools when the existing ones don’t hold up: workingset, groundedwork. My background is offensive security, so I tend to ask what breaks first.
Everything here is CC-BY 4.0 and built without analytics or trackers. If the writing helped you, you can sponsor my work on GitHub. 100% goes to me, no platform fees.