Comparing one model call with separate vision and writing steps

This is a small tool I built for comparing ways to turn product images into descriptions. It has a React interface and a Node backend, with one combined model call or separate vision and writing calls. Each phase can use a different provider and model.

The useful difference is that the two-phase path returns the visual description as well as the finished copy. I can inspect what the first model said about the images before judging how the second model wrote about them.

Follow a detail through the two steps

Suppose a product is red but the finished description calls it blue. That is an illustrative example, not a recorded mistake from this project.

With only the final paragraph, it is hard to tell where the error entered. In the two-phase path, I can check the intermediate description. If that already says blue, the writing step received the wrong detail. If it says red and the final copy says blue, the detail changed during writing.

The implementation separates those requests:

Product images
  -> analyzeImages(): visual description
  -> formatDescription(): requested language, style and format
  -> final copy

analyzeImages() sends the images to a model that accepts visual input. formatDescription() receives the returned text; it doesn't inspect the images again. An error in the visual description can therefore carry into the final result. Splitting the calls makes that intermediate step visible, not automatically correct.

Keep the copy instructions out of the first request

The vision prompt leaves out free-form custom instructions and the requested output language. A controlled tone preset can still be included.

The writing prompt adds the language, tone, custom instructions and HTML or plain-text requirements. These are different jobs: describing the visible product and presenting that description in the requested form.

The single-call branch puts the images and those requirements into one request. Both routes remain available in the prototype. I haven't established that the extra call produces better copy, and leaving instructions out of one phase is not a guarantee against malicious instructions in the input.

Inspect the prompt as well as the paragraph

The newer two-phase API returns the system prompt, user prompt and response for both steps. It also returns provider/model choices, prompt-template versions, image count and each phase's response time and token usage. The newer single-call mode returns its combined prompts and totals; the older API has a smaller response without those prompt records.

That gives a comparison more context than a final paragraph alone. A changed template or model choice is part of the experiment, so it should travel with the result being compared.

Cost needs the same care. For OpenRouter, the code first checks for a returned usage cost:

if (result.provider === 'openrouter' && result.usage?.cost !== undefined) {
  return result.usage.cost;
}

When that isn't available, it estimates cost from input and output token counts using the configured price table. A provider-returned amount and an estimate have different origins. Current comparisons also need current prices; the saved configuration isn't a current price quote.

The local debug log retains input images, instructions and final descriptions for inspection. It is a simple JSON file, so it needs better handling before concurrent use. Its grouped provider statistics also repeat the whole request cost under each involved provider, so those groups should not be read as a phase-by-phase bill.

The comparison paths and measurements are implemented. What is still missing is a reviewed set of outputs from the same images, judged against the same product facts. Until that exists, this is a bench for asking which path works better, not evidence that one has won.