Most people assume that when an AI assistant provides a link, it must be real. After all, the tool searched the web and found a source, so the citation should be trustworthy. The truth is far less reassuring. AI citations are a mixture of genuine references and fabricated URLs, and the difference between the two is not always obvious.
In this article, I explain what AI citations actually are, how AI assistants decide when to fetch outside sources, and why you should verify every link before trusting it.
How AI Assistants Choose Between Memory and Search
An AI assistant can respond in two fundamentally different ways. The first is answering from its training data, the vast body of text it was exposed to during training. When you ask something general, such as how to edit a PDF, the model often has enough stored knowledge to produce a useful answer without looking anything up. The second approach involves a retrieval step. The model searches the web or pulls documents from an index, then writes an answer grounded in those documents.
I have tested this myself many times. A question like “how do I edit a PDF” typically gets answered from memory. But a time-sensitive question like “what is the weather in Zurich today” forces the model to search, because its training data cannot possibly contain today’s forecast. The decision between these two paths is not random. It depends on whether the model judges the query to require fresh or external information.
What surprises many users is that the model does not always get this judgment right. Sometimes it answers from memory when a search would have been more accurate. Other times it searches unnecessarily. This inconsistency is part of why AI-generated citations can be unreliable, and it is something most providers are still working to improve.
What AI Citations Actually Are
Citations in AI assistants appear as clickable links or small reference boxes alongside the generated text. In tools like ChatGPT, they often show up as numbered grey boxes that you can click to visit the source. When the assistant performs a retrieval step, these citations point to the web pages or documents it consulted. They serve a similar purpose to footnotes in academic writing: they tell you where the information supposedly came from.
However, there is a critical distinction between citations produced after a genuine search and links that the model generates from memory. According to OpenAI’s documentation on browsing, ChatGPT uses a browsing tool to fetch real-time information. When the browsing tool is active, the citations are grounded in actual retrieved pages. When it is not, any URLs in the response come from the model’s training data, and those links may no longer exist or may never have existed at all.
This is the core problem. The visual presentation of a citation looks identical whether the link is real or invented. There is no label that says “this one was actually retrieved” versus “this one was generated from patterns in the training data.”
Why AI Citations Are Often Wrong
Here is where my view parts from the optimists. I believe the citation problem in AI assistants is more serious than most people acknowledge. The term for invented references is “hallucination,” and it affects URLs just as much as it affects factual claims. A model might generate a URL that looks plausible, follows the correct domain structure, and even includes a realistic page slug, yet leads to a 404 error when you actually click it.
I have seen this repeatedly in my own server logs. Hallucinated URLs from AI tools generate real HTTP requests that hit real websites and return 404 responses. If you run a website and notice a sudden increase in 404 errors with oddly specific paths, AI-generated links could be the cause. A study published on arXiv confirmed that large language models frequently produce non-existent references, especially when generating academic citations.
The risk is not just inconvenience. If you are researching a medical question, a legal issue, or a financial decision, a hallucinated citation can lend false authority to bad information. The link looks credible. The surrounding text reads confidently. But the source does not exist.
How to Verify AI Citations Before Trusting Them
The practical solution is straightforward: open every citation in a new tab before you rely on it. If the page loads and contains the information the AI referenced, you can have some confidence in that particular claim. If you get a 404 or the page content does not match the AI’s summary, discard it.
Beyond manual checking, look for signals that a retrieval step actually happened. In Google’s Gemini, for instance, you can sometimes see a “search” indicator that confirms the model queried the web. If no such indicator is present, treat any links with extra caution. I also recommend cross-referencing important claims with a traditional search engine. It takes an extra minute, but it can save you from citing a source that does not exist.
Some users assume that paid tiers or newer models are immune to this problem. They are not. While retrieval-augmented generation has improved citation accuracy, no current system guarantees that every link is valid. Trusting AI citations blindly is a habit worth breaking now, before it costs you credibility.
Where AI Citations Are Heading
The next generation of AI tools will likely separate retrieved citations from generated ones more clearly. I expect we will see explicit labelling, confidence scores, and perhaps even automated link-checking built into the response pipeline. Some early experiments with attributed question answering at Google Research point in this direction.
Until those safeguards arrive, the responsibility sits with the user. Every AI citation is a claim, not a fact. Treat it accordingly, verify before you share, and remember that a confident-sounding answer with a broken link is worse than no answer at all.
Leave a Reply