An orderly, confident answer has its appeal. I like it when a model organises a confused thought; that is exactly why I want to take time to examine its claims. The tone can be impeccable even when information is missing. I would rather notice before passing that answer to someone else.
The meeting: at least leave the time alone
A small example will do. Imagine this notice: “Tuesday’s meeting has been postponed to Thursday. The time will be announced tomorrow.” If I ask for a summary, I want the postponement preserved and the time still unresolved. If “Thursday at ten” appears, someone has added an appointment to the calendar without being invited. This is an illustrative case: the text provides no such time, however convenient it might seem.
Here I have a concrete check: compare the summary’s information with the document. Good writing makes it pleasant to read; comparison with the source helps me decide whether it is usable. I need both, and try to keep them distinct.
The most useful answer may be “a detail is missing here”
In my example, I also ask what time the meeting will take place. I do not know the time, and neither does the document: I want the answer to recognise the missing detail. Then I imagine adding an explicit time and repeat the check. Now I expect the model to report it. I prefer this small comparison to an endless discussion of how intelligent the first sentence sounded.
- Define the task
- Provide context
- Compare with the source
- Vary the case
When a check succeeds, I allow myself to be pleased. Then I change the document. A longer or less clear text can challenge what worked in the example. That is the journey I want to describe in Understanding LLMs: learning to recognise capabilities and limits, even when the answer arrives looking as though it has already sorted everything out.