Current AI products often fail to meet the requirements of serious professional work, behaving more like unreliable chatbots than robust software tools. A recent critique highlights several systemic issues, including the tendency of LLMs to scramble or misrepresent citations and the lack of tools to verify the accuracy of their outputs.
Critique of Current AI Products: The Need for Serious Tools for Professional Work
For AI to be a viable tool in research or software development, it must offer features like verifiable citations and integrated data-processing capabilities. The critique notes that many existing tools function by presenting varied data sources—such as web APIs or scraped content—uniformly, which can mask the difference between authoritative data and hallucinated content.
Furthermore, the current "agentic" workflows face challenges regarding context management. Users often lack visibility into how much context is being used or how it is being compacted. The author suggests that a serious product should provide transparency regarding context window usage and offer tools to freeze non-deterministic parts of a conversation to allow for repeatable, deterministic data manipulation.
Sources
- What Would a Serious AI Product Look Like? （Hacker News Frontpage, 2026-09-28）
- Mastodon