if you point it to the web it got the point, but started treating everything like fiction. so it simply started making up possible scenarios and playing them off as real answers when asked for factual information.
i could not tell what the issue was or how to fix it because the reasoning is encrypted. the obfuscation model spat out something like: 'the user is asking for details about a fictional scenario in which the usa has assassinated the leader of iran'
i really don't like the way big ai companies are going. encrypted thinking, guardrails, adversarial personality, moralizing. it is creating something anti-human.
Which makes sense, because of you have looked into search and crawlers you notice that search is actual quite expensive (which is why e.g. Kagi charges a few bucks for search every month).
The model wasted over half the token budget, each time, on internal debates over the current date.
When generating a World Cup summary, for example, it refused to believe qualification rounds were over and refused to even call the web searching tool to collect the data.
I injected the current datetime at the very beginning of the system prompt, but Gemma refused to believe it!
The m-effer insisted the timestamp was fake and hypothesized it was being evaluated in a synthetic lab test with simulated future dates!
No amount of system prompting could convince it to trust the clock.
That was the most frustrating and bizarre "bug" I ever faced!
I've noticed Qwen3.6 struggles a bit with today/date based logic.
Only after some cajolling it finally went to check the history I asked it to (I had been testing KoboldCPP's web search).
I wonder how much that extends to using LLMs for programming. I assume most knowledge of programming language syntax still comes from training data.
Were you expecting your model to be updated on current events? Why?
Also the specific event you are referring to is a statistically very improbable event, prior to its actually happening.
>It only acquiesced when I specifically directed it to check Reuters.
Do all models do this? They check in with Reuters? Why would a model think that you asking about an extremely improbable event warranted reaching out to Reuters?
It’s clearly warranted because a model that trusts its weights on current events will give an outdated answer. Extremely improbable events happen all the time.
I can teach myself to play an instrument, and I'm not just building this huge lookup table that I have to access every time I play the instrument. I am updating the weights in my neurons.
Until AI can do this it's not AGI in my book.
The slop would multiply if we keep feeding it to new models in a loop
it must ship with some default old model if you didn't need to explicitly download one
Fortunately increased capabilities seem to make this a basic expectation with new releases.
To me, intelligence or an intelligent entity should be able to learn from its mistakes and learn new things on its own. Having to start from scratch to teach an AI new facts or new skills is not very intelligent IMO.
(with "AI" developing the technique of course)
You’re not going to get all that just by pointing the LLM to the recently released Java 27 documentation. That information is also potentially adding tons of content to your context, which is already filled with tons of other data (your code, other recently released libraries it has to get documentation for, etc).
I’m going to add this site to my agents files so that they’re explicitly aware of their own limitations.