With the inclusion of PR 15795, the Theia 1.63.0 release that has been built today now supports Ollama release 0.9.0 which has introduced several improvements, which are now also available in Theia AI. This post highlights those improvements and gives a few general hints on how to improve your experience with Ollama in Theia AI.

Streaming Tool Calling

One of the most interesting features of the Ollama 0.9.0 release is support for streaming tool calling in the API. This makes working with the LLM much more convenient, as you can follow the output and tool calls during the generation. As a side effect, this also boosts output quality especially for tool calls for the models that support them. See this article for more information on this.

Starting with Theia 1.63.0, Theia AI uses the new streaming API including tool calling so that Ollama models now behave more like models from other providers, such as OpenAI, Google, and Anthropic.

Explicit Thinking Markup

Thinking or reasoning in LLMs have received much attention in the recent months, especially since deepseek-r1 was announced. While Ollama did support reasoning models for several months now, the reasoning steps have always been part of the actually generated response and could only be extracted by looking for <think>...</think> or similar markup generated by the LLM. 

Ollama 0.9.0 has added support for distinguishing reasoning output and actual output on the API level. The new Ollama provider in Theia propagates this feature so collapsible reasoning blocks are rendered on the UI level.

Token Usage Reporting for Ollama

Of course, as you are running Ollama most likely on your own hardware, the token usage report is not as crucial as with paid third-party services. But still, it might be at least interesting to know how many tokens you are using with your own Ollama server. 

The new Ollama provider released in Theia 1.63.0 now reports token usage statistics in the AI Configuration View in the tab Token Usage.

Support for Images in the Chat Context

Theia 1.63.0 has added preliminary support for images in the Theia AI chat. Using the + button in the AI Chat, it is now possible to add images which are passed to the LLM as part of the context. This is also supported for Ollama LLMs; but note that not many models support images. One example is the llava model.

If unsure, you can use the ollama show command in a terminal to check for the image capability of a model.

Configuration and Usage Hints

Using an Ollama server running on usually limited local hardware is most likely no competition for paid 3rd party services. But maybe the absence of token limits and privacy concerns are still sometimes more important than performance. To make Ollama models at least a bit more usable in practice, it is important to tweak the configuration to adapt it to your hardware and needs. This section gives some hints about possible optimizations. Note: these settings have been tested with an Apple Silicon M1 Max system; for other hardware, please refer to other blog articles that discuss Ollama optimizations.

Ollama Server Settings

To reduce the memory footprint of the request context, set these environment variables before executing ollama serve:

• OLLAMA_FLASH_ATTENTION=1

• OLLAMA_KV_CACHE_TYPE="q8_0"

See the Ollama FAQ for more details.

Theia AI Considerations

We might be tempted to download and use different models for different Theia AI agents, e.g. a universal model, such as qwen3 for universal tasks, and more specialized models, such as codellama for code-centric tasks like the Coder or Code Completion agent. However, we need to consider that each model that is loaded at the same time uses a lot of memory so that Ollama might decide to unload models that are not used at the moment. And since unloading and loading models takes some time, performance decreases if you use too many models.

Therefore, it is usually a better idea to use fewer models, for example one smaller model for tasks that should not take too long (such as Code Completion and Chat Session Naming), and one larger model for complex tasks (such as Architect or Coder agents). The best results can be achieved if both models fit in the available VRAM so that no model loading/unloading occurs.

Theia AI Request Settings

One important thing to note is that Ollama per default uses a context window of just 2048 tokens. Most tasks in Theia AI will not provide satisfactory results with this default. Therefore it is important to adjust the request settings and to increase the num_ctx parameter. As with the model size discussed before, the context size should be set to match the agent and task because a larger context leads to results with higher quality but longer runtimes. Therefore, it is important to play around with the settings and find the optimal settings matching the used models and the use case.

These hints provide some initial help:

• Check the Ollama server logs and keep an eye out for messages like this:
  "truncating input prompt" limit=2048 prompt=4313 keep=4 new=1024
  This message indicates that the input prompt exceeds the required context length, so the input prompt will be truncated (which will usually lead to bad answers from the model, as it will forget half of the question you were asking...)
• Also during the process of loading a model, if you see a message like 
  llama_context: n_ctx_per_seq (2048) < n_ctx_train (40960)
  this indicates that the current num_ctx (2048) setting is smaller than the context window the model was trained with (40960).
  Alternatively, you can also use ollama show to check for the context length property of the model. (Do not confuse the context length property, which is the theoretical maximum context length for this model with the num_ctx parameter, which is the actual context length used in requests).

To adjust the num_ctx parameter in Theia, go to Settings > AI Features > Model Settings and follow the link to open the settings.json file in the editor.

Then create a new setting like this:

"ai-features.modelSettings.requestSettings": [
{
  "scope": {
    "agentId": "Coder",
    "modelId": "ollama/qwen3:14b",
    "providerId": "ollama"
  },
  "requestSettings": { "num_ctx": 40960 }
}]

In the scope part, you declare the provider, model, and agent to which apply the settings. You can leave out either key to apply the setting for all providers, agents, and/or models, respectively. 

In the same way you can adjust and play around with the other Ollama parameters, such as temperature, top_k, etc. 

Details

Category: Eclipse

Published: 26 June 2025

This article is part of a series to help newcomers to get started with Eclipse Theia development by setting up a local docker-based development environment for Theia.

In the previous article, we have created our first Eclipse Theia extension, changed some code, and used the debugger to set and hit a breakpoint in the frontend.

As a next step, I wanted to demonstrate how to debug the backend. So the best way, I thought, was to create a backend service to provide the "Hello World" message instead of having it hard-coded in the frontend.

As described in the Theia Documentation, a Theia Extension can add logic to both the frontend and the backend. To facilitate communication between both components, any protocol could be used; the backend could register and open its own endpoint and the frontend could access it. But, of course, Theia already provides a JSON-RPC API which can be used quite easily.

All of the code below is available in the hello-world branch of my GitHub repository.

Let’s start by specifying a service which can provide our "Hello World“ message. Since it needs to be known by both the backend and the frontend, we put it in hello-world/src/common/hello-world.ts:

export namespace HelloWorldConstants {
    export const SERVICE_PATH = '/services/hello-world';
}

export const HelloWorld = Symbol("HelloWorld")

export interface HelloWorld {
    getHelloString(): Promise<string>;
}

This code defines an interface for our very simple service, a Symbol for it (which is required by the dependency injection framework), and a constant for the service path at which we want to publish/consume our service. Note that the service returns a Promise<string> instead of a plain string. Since we are dealing with a remote service, using promises makes the code behave better, because we can consume the result asynchronously as we receive it, as we will see below.

The service implementation in the backend goes into hello-world/src/node/hello-world-impl.ts and is as simple as:

@injectable()
export class HelloWorldImpl implements HelloWorld {
    getHelloString(): Promise<string> {
        return Promise.resolve("Hello from the backend!");
    }
}

We need to annotate the class with @injectable() because we want to use it as an injection binding in the backend module later.

Now, that we have the service and its implementation, let’s use it from the CommandContribution in the frontend (I am only showing the changed class HelloWorldCommandContribution):

@injectable()
export class HelloWorldCommandContribution implements CommandContribution {

    constructor(
        @inject(MessageService) private readonly messageService: MessageService,
        @inject(HelloWorld) private readonly helloWorldService: HelloWorld
    ) { }

    registerCommands(registry: CommandRegistry): void {
        registry.registerCommand(HelloWorldCommand, {
            execute: async () => {
                this.helloWorldService.getHelloString().then(helloString => this.messageService.info(helloString))
            }
        });
    }
}

Note that we have added an injection for the HelloWorld service, and in the execute logic, we chain the Promise with the callback logic via the then() function. So, we request the helloString asynchronously, and as soon as it is received (and the Promise is resolved), we call the messageService to show it.

The next step is to tell the dependency injection framework in the frontend how to provide the HelloWorld service we want to inject and use in the HelloWorldCommandContribution. To do this, we extend the existing hello-world-frontend-module.ts as follows:

export default new ContainerModule(bind => {
    // add your contribution bindings here
    bind(CommandContribution).to(HelloWorldCommandContribution);
    bind(MenuContribution).to(HelloWorldMenuContribution);
    bind(HelloWorld).toDynamicValue(ctx => {
        const connection = ctx.container.get(WebSocketConnectionProvider);
        return connection.createProxy<HelloWorld>(HelloWorldConstants.SERVICE_PATH);
    }).inSingletonScope();    
});

What we do here is to create a proxy implementation of the HelloWorld interface that is backed by a WebSocketConnectionProvider, which in turn is instructed to handle requests via the SERVICE_PATH path. Every method call on the proxy is encoded in a JSON-RPC request and sent to the backend via the given SERVICE_PATH.

At the backend-side in hello-world/src/node/hello-world-backend-module.ts, we create and register the peer instance:

export default new ContainerModule(bind => {
    bind(HelloWorld).to(HelloWorldImpl).inSingletonScope();
    bind(ConnectionHandler).toDynamicValue(ctx => 
        new JsonRpcConnectionHandler<HelloWorld>(HelloWorldConstants.SERVICE_PATH, (_client: any) => ctx.container.get<HelloWorld>(HelloWorld))
    ).inSingletonScope()
})

First, we bind the HelloWorld service to its actual implementation HelloWorldImpl. This is not strictly required for our use case, but would make sense as soon as HelloWorldImpl wanted to access other services which are provided via injection.

Next, we create the JsonRpcConnectionHandler which is the counterpart of the WebSocketConnectionProvider above. Like in the frontend, we bind the WebSocketConnectionProvider to the SERVICE_PATH. All incoming method invocation requests are then forwarded to a HelloWorldServiceImpl instance. (Note that there is also a _client argument. We don’t use it here, but we could use it to implement a bi-directional protocol in which the client can be called back by the server).

Now the only thing left to do is to register the backend ContainerModule configuration in the package.json file

  "theiaExtensions": [
    {
      "frontend": "lib/browser/hello-world-frontend-module",
      "backend": "lib/node/hello-world-backend-module"
    }
  ]

and we are all set. When we launch first the backend, then the frontend in the browser and click on the Say Hello menu item, the new message will appear: Hello from the backend!

Debugging

Backend debugging is easier than frontend debugging described in the previous article, because we don’t need to deal with a browser debugging engine. The VS Code debugger can natively attach to the backend process and the yeoman code generator already took care of creating a launch configuration for us in launch.json. So, we can just place a breakpoint in the getHelloString() method in the HelloWorldImpl class, launch the Launch Backend configuration, and when we click the menu item in the frontend, we see that the breakpoint is hit.

Besides plain breakpoints, VS Code also supports conditional breakpoints, breakpoints with hit counts, and logpoints. The latter are really useful when trying to find out which part of code is called when. Using a logpoint, you can inject an arbitrary log statement that is executed without the debugger suspending the execution.

To try it out, let’s add a logpoint in the node_modules/@theia/core/src/common/messaging/proxy-factory.ts at line 156 in the onRequest method by right-clicking on the line number and selecting Add Logpoint .... As the expression we enter Request: {method}. The expression inside the curly braces is evaluated when the breakpoint is hit.

Now, we can play around with the frontend and can see the requests that are issues to the backend:

Note: At the time of writing this, there is a bug in the VS Code JavaScript debugger that prevents logpoints from working correctly with Theia. This bug is already fixed when installing the vscode-js-debug nightly as described here, and will hopefully be resolved soon in the next VS Code release.

And with this, I close today’s article. Have fun playing around with logpoints.

Details

Category: Eclipse

Published: 07 February 2022