Nach dem Token: Wie neue KI-Architekturen die nächste Entwicklungsphase prägen

The current generation of large language models has sparked a remarkable wave of innovation. But the next advances could come from entirely different directions. New AI architectures make decisions without complex text generation, generate content in parallel, or process information at more abstract levels. Some are already commercially available, while others are still in the research phase. What can these methods achieve, what are their limitations, and what implications do they have for the cost, speed, and quality of future AI applications?

The development of artificial intelligence (AI) is primarily discussed in the context of new large language models. OpenAI, Anthropic, Google, and other vendors are competing with increasingly powerful models to achieve the best results in programming, mathematics, research, and complex problem-solving. Standardized performance tests, known as benchmarks, are intended to make progress comparable. However, they are only one point of reference: their significance depends on which tasks were examined and how well these reflect real-world applications.

Those who measure progress solely by such comparisons may overlook the technological possibilities already emerging on the horizon. The well-known language models are not the only way to make artificial intelligence more powerful. Architectures have been and continue to be developed that solve specific tasks differently, thereby opening up new possibilities for speed, cost-effectiveness, and result quality.

The technical starting point lies in language processing. A language model breaks down an input—known as a prompt—into individual processing units called tokens. These correspond, for example, to words, parts of words, or punctuation marks. Most widely used large language models operate in an autoregressive manner: they calculate the next token of their response based on the input and tokens already generated. In classic generation, a response with a thousand tokens therefore requires a thousand consecutive generation steps. Additional acceleration techniques can optimize this processing.

Even complex tasks are often handled using additional linguistic intermediate steps. Such methods are referred to as reasoning, or inferential processing. This is powerful but incurs additional computational overhead. At the same time, the question arises as to whether free-form text generation is even the right approach for every task.

Five areas of development are particularly interesting:

These developments are at different stages of maturity and cannot be directly compared with one another. Their common goal can nevertheless be examined using three questions: What are the costs of processing? How quickly is a result available? And what level of quality does it achieve for the intended task?

Many tasks in businesses do not require a freely formulated response. Documents must be categorized, customer inquiries prioritized, or information evaluated based on defined criteria. Generative language models can handle these tasks, but they use a mechanism that is also designed to generate arbitrary new content.

Specialized decision models, on the other hand, focus on structured evaluations. They receive an input and a set of predefined response options. In a single pass, the model determines probability values for the possible outcomes without having to formulate a lengthy response token by token.

For example, a company could have an incoming customer inquiry analyzed. The possible categories are specified as “complaint,” “claim,” and “general inquiry.” The model evaluates the three options and returns its assessment in a structured data format. The following figure illustrates the difference between step-by-step text generation and a direct evaluation of defined options.

Jev from TypeSafe AI and Microsoft Decision-1, both of which were introduced in the fall of 2026, demonstrate that this approach is already practical. Microsoft’s model is based on a fine-tuned language model with nine billion parameters but is specifically tailored for structured decision-making tasks.

Time: A key advantage of specialized decision models lies in their low response latency. Since there is no need to generate a lengthy text sequence, structured evaluations can be performed much more quickly. TypeSafe AI reports response times for Jev ranging from 70 to 500 milliseconds. According to the manufacturer, Microsoft Decision-1 also achieves significant speed advantages over the generative language models examined. These figures are based on different test conditions and are therefore not directly comparable.

Costs: Both vendors quote a price of $0.042 per million input tokens. According to the published pricing models, there are no additional token costs for structured output. This means that decision models can be economically attractive, particularly for large processing volumes. However, the actual total cost of an application also depends on integration, infrastructure, operation, and potential post-processing.

Quality: Jev and Microsoft Decision-1 have different areas of focus. TypeSafe AI particularly emphasizes the calibration of the output probabilities and reliable adherence to specified output formats. Microsoft reports high accuracy across various classification and scoring tasks. The results are promising but are based primarily on vendor-conducted studies. It is not possible to conclude that one of the two models is generally superior based on this information. The decisive factor remains how reliably a model fulfills the specific business task.

While the predefined answer options prevent a model from inventing arbitrary new categories, an incorrect assessment is still possible. Likewise, the output probabilities must be verified. For example, with a well-calibrated model, out of one hundred decisions rated with 90 percent certainty, approximately ninety should be correct in the long term. Whether this applies to a specific use case must be determined using appropriate test data.

Decision models are therefore particularly useful for standardized processes involving large numbers of cases, such as the pre-sorting of documents or the selection of appropriate tools within an AI agent. Such an agent can process tasks across multiple processing steps, utilizing databases or external applications in the process. Before implementation, however, it should be determined whether the task actually requires AI. Clear-cut rules can often be implemented more reliably and cost-effectively using traditional software.

The real innovation does not lie in the fact that AI can make decisions. It lies in the ability to handle certain evaluation tasks without the effort involved in generating free-form text.

In decision-making models, free-form text generation is largely eliminated. Other areas of research, however, seek to modify the text generation process itself. One approach involves diffusion language models. The principle is familiar from image generation: instead of building a result exclusively element by element, an initially incomplete representation is refined over several steps. In the case of language, different positions in a text sequence can be predicted in parallel and subsequently revised.

Unlike traditional autoregressive generation, this means that each position does not have to be calculated only after all preceding positions have been definitively determined. Depending on the method, content that has already been generated can also be modified in later steps.

With Mercury 3, Inception Labs offers a commercial diffusion language model. The manufacturer specifies a generation speed of more than a thousand tokens per second. By comparison, even fast autoregressive models achieve high output speeds, though these values can vary significantly depending on hardware, model, and task. An isolated speed metric therefore does not yet allow for a fair product comparison.

Inception reports that Mercury 3 can compete with other speed-optimized language models in selected tests. These figures come from the manufacturer. Whether a comparable level of quality is achieved for demanding enterprise tasks must be verified in the specific use case.

A second example is Google’s DiffusionGemma. This research model, published in 2026, demonstrates that an originally autoregressive language model can be converted to diffusion-based generation through targeted retraining. This capability is technologically significant because it means that the two approaches do not necessarily have to remain completely separate model ecosystems.

Next-Concept Modeling takes a different approach. The method works with a hierarchical representation of text. First, a coarse structure is generated, followed by increasingly detailed levels. Within each level, multiple components can be predicted in parallel.

This distinction is important: diffusion models improve a text representation iteratively. Next-Concept Modeling, on the other hand, builds up the result across multiple hierarchical levels of representation. Both methods enable parallel computations but use different technical mechanisms to do so. The following figure contrasts classical sequential generation with these two alternatives.

Time: This is where the most clearly discernible potential lies so far. According to the manufacturer, Mercury 3 achieves very high output speeds. The researchers behind Next-Concept Modeling also report significant speedups compared to the experimental comparison methods they examined. However, these results are not readily transferable between products and research studies.

Cost: Faster processing can reduce operating costs, but this is not guaranteed. The decisive factor is how much computing power, memory, and hardware capacity are actually required. For diffusion methods, this depends, among other things, on the number of refinement steps. For a robust cost-effectiveness comparison, the costs per successfully completed task must be considered.

Quality: Results vary considerably depending on the task and model. Quality issues can arise particularly with complex tasks or when accurately reproducing names, amounts, and identifiers. Hierarchical methods also introduce their own potential sources of error during text reconstruction. Research shows promising progress, but no general superiority over autoregressive language models has yet been established.

For businesses, these methods are particularly relevant in scenarios where response times play a critical role. This applies to voice dialogue systems, interactive assistant applications, and AI agents, where the individual processing steps accumulate over time.

Text generation does not have to be exclusively sequential. Parallel and hierarchical methods open up new possibilities for acceleration, though their benefits must always be weighed against computational cost and output quality.

Another area of research focuses not primarily on text output, but on the internal processing of complex tasks. Language models work internally with mathematical representations. These consist of numerical values that map information and relationships in what is known as a latent space. In classical autoregressive processing, the next token is repeatedly determined based on these internal states.

For complex tasks, many so-called reasoning models use additional processing steps, some of which take the form of linguistic intermediate steps. While this can be helpful for problem-solving, it also incurs additional computational overhead. Therefore, researchers are investigating whether a model can directly process internal states without first converting every intermediate step into language.

An early approach is Coconut, which researchers at Meta presented in 2024. The method continues computations in latent space and only generates linguistic outputs when necessary. The studies show advantages for certain logical tasks but do not allow for a general conclusion regarding the superiority of this method.

The research paper *Hybrid Reasoning Agents*, published in October 2026, takes this a step further. Its approach combines internal latent processing with the ability to switch to a text-based mode when needed.

This is particularly interesting for AI agents. An agent could first develop a solution internally, then retrieve information via a programming interface, process it further, and finally formulate a response. Not every internal intermediate step would need to be generated as text. The following figure illustrates this transition between latent processing and linguistic or structured interaction with the environment.

Time: For the tasks studied, the researchers report a speedup of approximately two to three times compared to various text-based and latent comparison systems.

Cost: Under the conditions of the study, the number of tokens generated was significantly reduced, and the modeled usage costs were lowered accordingly. However, fewer tokens do not automatically mean a proportionally lower computational load. The key factor remains the resources required for the internal calculations.

Quality: The research reports average performance improvements compared to the reference systems examined, including high-performance agent-based language models. However, the results are derived from controlled experiments. Whether these advantages persist in practical business tasks remains to be seen.

In addition to these performance dimensions, another question arises: How can one evaluate processing whose internal calculations are not directly represented in linguistic terms? Here, a distinction should be made between transparency and controllability. Even an explicit chain of reasoning in today’s reasoning models is not a complete log of their actual internal calculations. Conversely, a hard-to-interpret internal state does not mean that an entire AI system must be uncontrollable.

For companies, what matters is which inputs, data sources, tool calls, and results can be traced. The extent to which an additional examination of internal model states is necessary depends on the use case, the risks, and the justifiable effort required for control.

Latent reasoning opens up the possibility of more clearly separating internal computation from linguistic interaction. Whether this results in a business advantage depends on the performance of the overall system and its actual resource requirements.

The performance of neural networks has improved in recent years through, among other things, larger models, more extensive training data, and increased computational effort. However, the number of model parameters alone is not a reliable measure of the quality of the results. Recursive approaches therefore investigate whether tasks can also be solved by repeatedly applying a smaller neural network.

The Tiny Recursive Model, introduced in 2025, uses a compact network that is applied multiple times to internal intermediate results. Thus, the model does not process a task just once, but improves its state representation in successive computational steps. In doing so, the same model parameters are reused. The following figure shows how a small network is run through repeatedly.

Time: Repeatedly applying a model generates additional computational steps. Recursion therefore does not necessarily lead to shorter response times. Its efficiency depends on how much progress each individual pass achieves.

Cost: A small number of parameters can reduce memory requirements and hardware demands. At the same time, numerous iterations can increase the actual computational effort. A small model is therefore not automatically cheaper than a larger one.

Quality: With approximately seven million parameters, the Tiny Recursive Model achieved remarkable results on tests for recognizing abstract patterns. On a more challenging test, however, the accuracy was significantly lower. The results show that small recursive networks can be powerful for certain tasks, but they do not replace universal language models.

For businesses, this development is particularly interesting in the long term. If the principle can be applied to other tasks, specialized systems with comparatively low memory requirements could emerge. Whether such models are economically superior, however, can only be assessed by considering their total computational costs.

Research shows that model size is not the only path to higher performance. For suitable tasks, repeated computations can compensate for some of the missing model capacity.

While recursive models modify the repetition of computations, concept models address the question of at which level information is represented. Classical language models primarily learn to predict the next token. In doing so, they already capture complex linguistic relationships. However, newer research approaches are investigating whether an additional learning objective at a more abstract level can improve these capabilities.

One example is ConceptLM, which was introduced in February 2026. In addition to traditional token prediction, the model also learns so-called discrete concepts. These are learned internal representations that can summarize information from multiple tokens.

These concepts are not predefined terms or immediately interpretable categories. They are mathematically learned representations that the model uses in addition to processing individual tokens. Concept prediction thus complements the existing training method rather than completely replacing it. The following figure illustrates the two complementary learning objectives.

Time: The publication does not yet provide a solid basis for concluding that ConceptLM is fundamentally faster than comparable classical language models in practical operation. The additional processing layer may itself incur computational overhead.

Cost: A general cost advantage has not yet been demonstrated either. In the long term, it would be interesting to see whether a more efficient internal representation—for a comparable model size—enables better results and can thus reduce the need for larger models.

Quality: This is where the most meaningful results have been reported so far. The authors report improvements over the respective comparison models for several model sizes examined. Positive effects were also observed when fine-tuning a larger language model. However, this does not yet imply general superiority in complex problem-solving or in understanding domain-specific contexts.

Concept models are therefore primarily an interesting area of research for the further development of existing language models. It is conceivable that such methods will be incorporated into larger models in the future without becoming visible to users as a separate product category.

More abstract representations could make language models more powerful without necessarily requiring an increase in the number of their parameters. Whether this will also result in economic benefits remains to be seen.

The five areas of development pursue different technical goals. Some are already commercially available, while others are still in the experimental phase. Furthermore, their performance metrics were determined under different conditions. A direct ranking would therefore be of little use. The following overview summarizes where the key potential and limitations currently lie.

The increasing diversity of models expands the possibilities for building targeted AI applications. For example, a company could use a decision model for document classification, a generative model for information extraction, and traditional software to check clearly defined conditions. Such architecture is already possible today. New specialized models can make it more economically attractive. However, each additional component also increases the integration effort and may create new sources of error or vendor lock-in.

Therefore, the key is not to deploy as many new model architectures as possible, but to choose the appropriate solution for each task. The relevant benchmark remains the performance of the entire process.

As specialization increases, data quality, interfaces, access permissions, and the monitoring of processing steps also gain in importance. A single model can deliver correct results, while the overall system may still be operating incorrectly. For companies, it is therefore important to distinguish between technical transparency, traceability, and controllability. Not every internal state needs to be immediately understandable. However, depending on the use case, it must be possible to verify what information was processed, what actions were executed, and what results were generated.

The European AI Regulation, the Artificial Intelligence Act, sets requirements for logging, transparency, and human oversight for certain high-risk AI systems. This does not result in a blanket obligation to disclose all internal model calculations. The specific design of the controls must be tailored to the respective system and its risks.

Technological development therefore does not necessarily change the fundamental requirements for responsible AI. Above all, it increases the number of technical options available for implementing these requirements.

ISO/IEC 42001 provides an organizational framework for managing such requirements. This international standard outlines requirements for an AI management system that enables organizations to systematically manage responsibilities, risks, and control measures throughout the entire lifecycle of their AI applications. It can support the implementation of regulatory requirements but does not replace independent verification of those requirements.

The research and product lines presented here show that progress in artificial intelligence is taking place at various levels. Some methods do not rely on free-form text generation, while others alter the process. Still others focus on internal calculations or the representation of information. This does not necessarily mean that large language models will be phased out. Rather, different approaches are likely to coexist, complement one another, or be integrated into existing model families.

For companies, it is therefore becoming more important not to plan AI applications based solely on comparisons of individual models. What matters most is which technical approach fulfills a specific task with adequate quality, reasonable processing time, and economically sound use of resources.

The next phase of AI development may be shaped less by how large we make models and more by how intelligently we utilize different approaches to information processing.

New AI architectures open up technical possibilities. The challenge for companies lies in deploying these in a controlled and economically sound manner. If you’re involved with AI governance, the EU AI Regulation, or ISO/IEC 42001, I’d be happy to discuss these topics with you.

Author: Sebastian Berger

The publications linked directly in the text include technical manufacturer information and scientific research papers. Performance figures from manufacturer comparisons and experimental benchmarks have not necessarily been independently reproduced and do not allow for a direct comparison of the architectural approaches presented.

Decision models: TypeSafe AI, Jev

Microsoft Decision-1

Mercury 3

DiffusionGemma Technical Report

Next-Concept Modeling

Coconut

Hybrid Reasoning Agents

Tiny Recursive Model

ConceptLM

EU AI Act