Testing & Quality

1+ hour, 3+ min ago   (1563+ words) On accuracy, Jev sits level with mid-price LLMs and 6.5 to 11.5 points behind the frontier in the cleanest comparison. Out of the box its probabilities are the best calibrated of the models measured on familiar English tasks and are wrong in…...

1+ hour, 3+ min ago   (769+ words) The same move as writing the test first, one level up: the checkable condition is written before the thing that satisfies it. Four levels - requirement, contract, spec, performance. The honest example is a coverage ratchet standing at 0 while real line…...

2+ hour, 6+ min ago   (695+ words) It's a campus tour guide system written in C. Console application, no GUI, no networking. A user logs in, picks from a menu, and queries information about 12 campus locations connected by a weighted undirected graph. Nothing exotic. Exactly the kind of…...

2+ hour, 7+ min ago   (500+ words) Look through a maintenance tool's test suite long enough and you'll run into a small puzzle: a function already has a test, so why does another file add a second one for what looks like the same behavior? Two tests…...

2+ hour, 2+ min ago   (931+ words) Same rule as last time: every incident below is one I personally caused, on 23 September 2026. Where I did not measure something, it says so. In an earlier post I listed the ways my own checks reported success while being unable…...

2+ hour, 16+ min ago   (1152+ words) Rule I followed: every incident below is one I personally caused. Nothing borrowed. Where I did not measure something, it says I did not measure this. I build tooling that checks other tooling. In the week of 7 September 2026 I kept…...

2+ hour, 30+ min ago   (29+ words) โดย Nokka (นก-กา)... Tagged with ai, programming, testing, productivity....

3+ hour, 4+ min ago   (319+ words) Jev is a new model format where instead of outputting text, it outputs certainties for a defined set of options. Due to this structure, it’s extremely fast! Naturally, a classification task that comes to mind is monitoring harmful thought traces....

3+ hour, 47+ min ago   (527+ words) Originally published at erikhill.dev. The numbers below are checked against the repository they come from. This is a finding about my own harness. The suspect is the probe, not the models it measures. I built a detector that decides…...

3+ hour, 47+ min ago   (723+ words) Originally published at erikhill.dev. The numbers below are checked against the repository they come from. This is a finding about a measurement rule, not about a model. The models did not change. I was comparing two models on a…...