In this article we look at how to get started with Spring AI project and connect to Azure OpenAI Large Language Models (LLMs).

Feel free to browse other articles in this getting started with Spring AI series:

• How to get started with Spring AI

• Get started with Spring AI and Ollama

• Get started with Spring AI and OpenAI

• Getting started with Spring AI and Anthropic

Create a simple Spring Application (Beginners)

We can use Spring Initializr to generate a skeleton project for our simple application. You can do that either in IDE itself, or directly with a pre-configured project (e.g. Maven, Java, Spring Boot 3.4.4, spring-ai-demo, com.example package name. JAR packaging, Java 24 with Spring Web).

Clicking on “Generate” button will download a project skeleton zip archive, e.g. spring-ai-demo.zip which you can unzip and open in editor of your choice (e.g. IntelliJ, VS Code, etc.)

We can start adding specific AI model Spring Boot starters later on.

Let’s add a HomeController, e.g.

package com.example;

import org.springframework.web.bind.annotation.GetMapping;
import org.springframework.web.bind.annotation.RestController;

@RestController
public class HomeController {

    @GetMapping("/")
    public String home() {
        return "Hello World!";
    }
}

Let’s run our simple application, e.g.

./mvnw spring-boot:run

Feel free to browse from another terminal window, e.g. http localhost:8080

http localhost:8080

HTTP/1.1 200 
Connection: keep-alive
Content-Length: 12
Content-Type: text/plain;charset=UTF-8
Date: Tue, 11 Apr 2025 08:00:00 GMT
Keep-Alive: timeout=60

Hello World!

or a directly in the browser at http://localhost:8080

Feel free to explore Spring Boot Getting Started Guide for more details on how to create a simple Spring Boot web application.

Connecting your Spring Application to Azure OpenAI models

Configure OpenAI

In this section we will explore connecting your application to Azure OpenAI Large Language Models (LLMs). If you don’t already have an account on Azure, please sign up for an account, e.g.

Once you have created, authenticated and configured your Azure account, you can select Azure OpenAI from the Azure Marketplace to create an instance of Azure OpenAI, e.g.

Within a minute or so, your deployment will be ready. You should be able to find your key and your endpoint, e.g.

Please save the key (e.g. Ddlvau6w2KHzFC...) and endpoint (e.g. https://nevenc.openai.azure.com). Keep these secure, because anyone with your API key can access and consume your Azure OpenAI API platform credits.

Our Azure OpenAI instance does not have any models deployed, so let’s add a model we can use in your application. You need to go to Azure AI Foundry portal from your Azure OpenAI Summary page, and create a new deployment, e.g.

Fill in the details for the deployment name that we will use in our application later, e.g. azure-gpt-4o

Finally, let’s add Spring AI code to start using OpenAI API.

Add Spring AI for OpenAI

We will add Spring AI Spring Boot Starter for OpenAI in our Maven pom.xml, e.g.

        <dependency>
            <groupId>org.springframework.ai</groupId>
            <artifactId>spring-ai-starter-model-azure-openai</artifactId>
            <version>1.0.0-M7</version>
        </dependency>

Notice, we are using version 1.0.0-M7 that’s newest at the time of writing this article. Please replace with the latest Spring AI version.

We also need to add few Spring properties to our src/main/resources/application.properties file, e.g.

spring.application.name=spring-ai-demo
spring.ai.azure.openai.api-key=Ddlvau6w2KHzFC...
spring.ai.azure.openai.endpoint=https://nevenc.openai.azure.com
spring.ai.azure.openai.chat.options.deployment-name=azure-gpt-4o

Notice, we have used azure-gpt-4o deployment that uses gpt-4o large language model (LLM) for this simple use case. Feel free to explore all Azure OpenAI Service models. These models vary in size, speed, training data and eventually in price per 1M tokens. For example, gpt-4o-mini model costs per 1M tokens are: (1) for input US$0.15, (2) for cached input US$0.075 and (3) for output US$0.60, whereas gpt-4o model costs per 1M tokens are: (1) for input US$2.50, (2) for cached input US$1.25 and (3) for output US$10.00.

Finally, let’s add a call to a LLM from our HomeController, e.g.

package com.example;

import org.springframework.ai.chat.client.ChatClient;
import org.springframework.web.bind.annotation.GetMapping;
import org.springframework.web.bind.annotation.RestController;

@RestController
public class HomeController {

    private ChatClient chatClient;

    public HomeController(ChatClient.Builder builder) {
        this.chatClient = builder.build();
    }

    @GetMapping("/")
    public String home() {
        return chatClient
                .prompt()
                .user("who are you")
                .call()
                .content();
    } 
}

Run the application again, e.g.

./mvnw spring-boot:run

Test the application, in another terminal or browser, e.g.

http localhost:8080

HTTP/1.1 200 
Connection: keep-alive
Content-Length: 12
Content-Type: text/plain;charset=UTF-8
Date: Tue, 11 Apr 2025 08:05:00 GMT
Keep-Alive: timeout=60

Hello! I'm OpenAI's ChatGPT, an advanced AI language model designed to assist with
information, answer questions, and provide guidance on a wide range of topics.
Think of me as a helpful virtual assistant or conversational partner.
How can I assist you today? 😊

Congratulations! You have created your first application that talks to an Azure OpenAI service.

Possible Gotcha’s

If you run into ResourceNotFoundException (see below example) - this is probably because you have misconfigured Azure OpenAI instance, mistyped your deployment name, or have just created it. Maybe wait for few minutes and try running your application again. If the problem persists, please check your deployment name in the Azure AI Foundry (Azure Portal).

com.azure.core.exception.ResourceNotFoundException: Status code 404, 
"
    {
        "error": {
            "code":"DeploymentNotFound",
            "message": "The API deployment for this resource does not exist.
                        If you created the deployment within the last 5 minutes,
                        please wait a moment and try again."
            }
    }
"

Similarly, you might run into ClientAuthenticationException (see below example) - this is probably because you did not properly copied your Azure OpenAI API key, please check your key in Azure Portal.

com.azure.core.exception.ClientAuthenticationException: Status code 401,
"
    {
        "error": {
            "code":"401",
            "message": "Access denied due to invalid subscription key or wrong API endpoint.
                        Make sure to provide a valid key for an active subscription and use
                        a correct regional API endpoint for your resource."
        }
    }
"

Hope you enjoyed this beginners getting started with Spring AI article!

Connecting to other LLMs

If you want to run your Spring AI application with other LLM models, look at our articles:

• How to get started with Spring AI

• Get started with Spring AI and Ollama

• Get started with Spring AI and OpenAI

• Getting started with Spring AI and Anthropic

References

• Spring AI Azure OpenAI Chat

• Azure Portal

• Azure AI Foundry

• Azure OpenAI Service models

• Code Sample - Spring AI Quickstart Azure OpenAI

Please explore other Getting started with Spring AI articles in this series:

In this article we look at how to get started with Spring AI project and connect to OpenAI Large Language Models (LLMs).

Feel free to browse other articles in this getting started with Spring AI series:

• How to get started with Spring AI (with Docker Model Runner)

• Get started with Spring AI and Ollama

• Getting started with Spring AI and Anthropic

• Getting started with Spring AI and Azure OpenAI

Create a simple Spring Application (Beginners)

We can use Spring Initializr to generate a skeleton project for our simple application. You can do that either in IDE itself, or directly with a pre-configured project (e.g. Maven, Java, Spring Boot 3.4.4, spring-ai-demo, com.example package name. JAR packaging, Java 24 with Spring Web).

Clicking on “Generate” button will download a project skeleton zip archive, e.g. spring-ai-demo.zip which you can unzip and open in editor of your choice (e.g. IntelliJ, VS Code, etc.)

We can start adding specific AI model Spring Boot starters later on.

Let’s add a HomeController, e.g.

package com.example;

import org.springframework.web.bind.annotation.GetMapping;
import org.springframework.web.bind.annotation.RestController;

@RestController
public class HomeController {

    @GetMapping("/")
    public String home() {
        return "Hello World!";
    }
}

Let’s run our simple application, e.g.

./mvnw spring-boot:run

Feel free to browse from another terminal window, e.g. http localhost:8080

http localhost:8080

HTTP/1.1 200 
Connection: keep-alive
Content-Length: 12
Content-Type: text/plain;charset=UTF-8
Date: Tue, 11 Apr 2025 08:00:00 GMT
Keep-Alive: timeout=60

Hello World!

or a directly in the browser at http://localhost:8080

Feel free to explore Spring Boot Getting Started Guide for more details on how to create a simple Spring Boot web application.

Connecting your Spring Application to OpenAI models

Configure OpenAI

In this section we will explore connecting your application to OpenAI Large Language Models (LLMs). If you don’t already have an account on OpenAI platform, please sign up for an account, e.g.

Once you have created and authenticated your account, you can create a new API Key,

Please save the key (e.g. sk-proj-qqZJqCSCbI4xoXtB4Yn1aLWUq99...), as you won’t be able to view it again. Keep it secure, because anyone with your API key can access and consume your OpenAI API platform credits.

Finally, let’s add Spring AI code to start using OpenAI API.

Add Spring AI for OpenAI

We will add Spring AI Spring Boot Starter for OpenAI in our Maven pom.xml, e.g.

        <dependency>
            <groupId>org.springframework.ai</groupId>
            <artifactId>spring-ai-starter-model-openai</artifactId>
            <version>1.0.0-M7</version>
        </dependency>

Notice, we are using version 1.0.0-M7 that’s newest at the time of writing this article. Please replace with the latest Spring AI version.

We also need to add few Spring properties to our src/main/resources/application.properties file, e.g.

spring.application.name=spring-ai-demo
spring.ai.openai.api-key=sk-proj-qqZJqCSCbI4xoXtB4Yn1aLWUq99...
spring.ai.openai.chat.options.model=gpt-4o-mini

Notice, we have used gpt-4o-mini large language model (LLM) for this simple use case. Feel free to explore all OpenAI LLM models. These models vary in size, speed, training data and eventually in price per 1M tokens. For example, gpt-4o-mini model costs per 1M tokens are: (1) for input US$0.15, (2) for cached input US$0.075 and (3) for output US$0.60, whereas gpt-4o model costs per 1M tokens are: (1) for input US$2.50, (2) for cached input US$1.25 and (3) for output US$10.00.

Finally, let’s add a call to a LLM from our HomeController, e.g.

package com.example;

import org.springframework.ai.chat.client.ChatClient;
import org.springframework.web.bind.annotation.GetMapping;
import org.springframework.web.bind.annotation.RestController;

@RestController
public class HomeController {

    private ChatClient chatClient;

    public HomeController(ChatClient.Builder builder) {
        this.chatClient = builder.build();
    }

    @GetMapping("/")
    public String home() {
        return chatClient
                .prompt()
                .user("who are you")
                .call()
                .content();
    } 
}

Run the application again, e.g.

./mvnw spring-boot:run

Test the application, in another terminal or browser, e.g.

http localhost:8080

HTTP/1.1 200 
Connection: keep-alive
Content-Length: 227
Content-Type: text/plain;charset=UTF-8
Date: Tue, 11 Apr 2025 08:05:00 GMT
Keep-Alive: timeout=60

I am an AI language model created by OpenAI, designed to assist with a
variety of questions and tasks. I can provide information, answer
questions, and engage in conversation on a wide range of topics.
How can I help you today?

🏆 Congratulations! You have created your first application that talks to an OpenAI LLM service.

Possible Gotcha’s

If you run into Incorrect API key provided (see below example) - this is probably because you have mistyped the OpenAI API key.

org.springframework.ai.retry.NonTransientAiException: 401 - {
    "error": {
        "message": "Incorrect API key provided: \"sk-proj********\".
                    You can find your API key at
                    https://platform.openai.com/account/api-keys.",
        "type": "invalid_request_error",
        "param": null,
        "code": "invalid_api_key"
    }
}

Also, you might run into Insufficient quota (see below example) - this is probably because you have no funds available in your OpenAI account.

org.springframework.ai.retry.NonTransientAiException: 429 - {
    "error": {
        "message": "You exceeded your current quota, please check your plan and billing
                    details. For more information on this error, read the docs:
                    https://platform.openai.com/docs/guides/error-codes/api-errors.",
        "type": "insufficient_quota",
        "param": null,
        "code": "insufficient_quota"
    }
}

Check that you have some funds available in your account, e.g.

You might need to set the billing and add $5 to get started, e.g.

Hope you enjoyed this beginner’s getting started with Spring AI article!

Connecting to other LLMs

If you want to run your Spring AI application with other LLM models, look at our articles:

• Get started with Spring AI and OpenAI

• Get started with Spring AI and Ollama

• Getting started with Spring AI and Anthropic

• Getting started with Spring AI and Azure OpenAI

References

• Spring AI OpenAI Chat

• OpenAI API Docs

• Code Sample - Spring AI Quickstart OpenAI

In this article we look at how to get started with Spring AI project and connect to Ollama Large Language Models (LLMs) hosted locally on your desktop.

Feel free to browse other articles in this getting started with Spring AI series:

• How to get started with Spring AI

• Get started with Spring AI and OpenAI

• Getting started with Spring AI and Anthropic

Create a simple Spring Application (Beginners)

We can use Spring Initializr to generate a skeleton project for our simple application. You can do that either in IDE itself, or directly with a pre-configured project (e.g. Maven, Java, Spring Boot 3.4.4, spring-ai-demo, com.example package name. JAR packaging, Java 24 with Spring Web).

Clicking on “Generate” button will download a project skeleton zip archive, e.g. spring-ai-demo.zip which you can unzip and open in editor of your choice (e.g. IntelliJ, VS Code, etc.)

We can start adding specific AI model Spring Boot starters later on.

Let’s add a HomeController, e.g.

package com.example;

import org.springframework.web.bind.annotation.GetMapping;
import org.springframework.web.bind.annotation.RestController;

@RestController
public class HomeController {

    @GetMapping("/")
    public String home() {
        return "Hello World!";
    }
}

Let’s run our simple application, e.g.

./mvnw spring-boot:run

Feel free to browse from another terminal window, e.g. http localhost:8080

http localhost:8080

HTTP/1.1 200 
Connection: keep-alive
Content-Length: 12
Content-Type: text/plain;charset=UTF-8
Date: Tue, 11 Apr 2025 08:00:00 GMT
Keep-Alive: timeout=60

Hello World!

or a directly in the browser at http://localhost:8080

Feel free to explore Spring Boot Getting Started Guide for more details on how to create a simple Spring Boot web application.

Connecting your Spring Application to Ollama models

Configure Ollama

In this section we will explore connecting your application to Ollama Large Language Models (LLMs) hosted on your desktop. If you don’t already have Ollama installed, please download and install it on your machine, e.g.

You can start ollama process from command line too, e.g. ollama start

ollama start

2025/04/11 07:59:06 routes.go:1230: INFO server config env="map[HTTPS_PROXY: HTTP_PROXY: NO_PROXY: OLLAMA_CONTEXT_LENGTH:2048 OLLAMA_DEBUG:false OLLAMA_FLASH_ATTENTION:false OLLAMA_GPU_OVERHEAD:0 OLLAMA_HOST:http://127.0.0.1:11434 OLLAMA_KEEP_ALIVE:5m0s OLLAMA_KV_CACHE_TYPE: OLLAMA_LLM_LIBRARY: OLLAMA_LOAD_TIMEOUT:5m0s OLLAMA_MAX_LOADED_MODELS:0 OLLAMA_MAX_QUEUE:512 OLLAMA_MODELS:/Users/neven/.ollama/models OLLAMA_MULTIUSER_CACHE:false OLLAMA_NEW_ENGINE:false OLLAMA_NOHISTORY:false OLLAMA_NOPRUNE:false OLLAMA_NUM_PARALLEL:0 OLLAMA_ORIGINS:[http://localhost https://localhost http://localhost:* https://localhost:* http://127.0.0.1 https://127.0.0.1 http://127.0.0.1:* https://127.0.0.1:* http://0.0.0.0 https://0.0.0.0 http://0.0.0.0:* https://0.0.0.0:* app://* file://* tauri://* vscode-webview://* vscode-file://*] OLLAMA_SCHED_SPREAD:false http_proxy: https_proxy: no_proxy:]"
time=2025-04-11T07:59:06.948+02:00 level=INFO source=images.go:432 msg="total blobs: 79"
time=2025-04-11T07:59:06.956+02:00 level=INFO source=images.go:439 msg="total unused blobs removed: 0"
time=2025-04-11T07:59:06.962+02:00 level=INFO source=routes.go:1297 msg="Listening on 127.0.0.1:11434 (version 0.6.2)"
time=2025-04-11T07:59:07.020+02:00 level=INFO source=types.go:130 msg="inference compute" id=0 library=metal variant="" compute="" driver=0.0 name="" total="21.3 GiB" available="21.3 GiB"

Download Models for Ollama

You can use ollama command line to pull the models, e.g. ollama pull llama3.2

ollama pull llama3.2

pulling manifest 
pulling dde5aa3fc5ff... 100% ▕████████████████████████████████████████████████████████████████████▏ 2.0 GB                         
pulling 966de95ca8a6... 100% ▕████████████████████████████████████████████████████████████████████▏ 1.4 KB                         
pulling fcc5a6bec9da... 100% ▕████████████████████████████████████████████████████████████████████▏ 7.7 KB                         
pulling a70ff7e570d9... 100% ▕████████████████████████████████████████████████████████████████████▏ 6.0 KB                         
pulling 56bb8bd477a5... 100% ▕████████████████████████████████████████████████████████████████████▏   96 B                         
pulling 34bb5ab01051... 100% ▕████████████████████████████████████████████████████████████████████▏  561 B                         
verifying sha256 digest 
writing manifest 
success

You can list all the downloaded models, e.g. ollama list

ollama list

NAME                                                     ID              SIZE      MODIFIED      
llama3.2:latest                                          a80c4f17acd5    2.0 GB    2 minutes ago    
nomic-embed-text:v1.5                                    0a109f422b47    274 MB    8 weeks ago      
mistral:latest                                           f974a74358d6    4.1 GB    3 months ago     
mxbai-embed-large:latest                                 468836162de7    669 MB    3 months ago     
llama3.2:1b                                              baf6a787fdff    1.3 GB    3 months ago     
llama3.2:3b                                              a80c4f17acd5    2.0 GB    3 months ago     
nomic-embed-text:latest                                  0a109f422b47    274 MB    3 months ago

Finally, let’s run a model, e.g. ollama run llama3.2

ollama run llama3.2

>>> who are you
I'm an artificial intelligence model known as Llama. Llama stands for "Large Language Model Meta AI."

>>> Send a message (/? for help)

Finally, let’s add Spring AI code to start using Ollama AI API.

Add Spring AI for Ollama

We will add Spring AI Spring Boot Starter for OpenAI in our Maven pom.xml, e.g.

        <dependency>
            <groupId>org.springframework.ai</groupId>
            <artifactId>spring-ai-starter-model-ollama</artifactId>
            <version>1.0.0-M7</version>
        </dependency>

Notice, we are using version 1.0.0-M7 that’s newest at the time of writing this article. Please replace with the latest Spring AI version.

We also need to add few Spring properties to our src/main/resources/application.properties file, e.g.

spring.application.name=spring-ai-demo

spring.ai.ollama.chat.options.model=llama3.2

Notice, we have used llama3.2 large language model (LLM) for this simple use case. Feel free to explore all Ollama LLM models. These models vary in size, speed and training data.

Finally, let’s add a call to a LLM from our HomeController, e.g.

package com.example;

import org.springframework.ai.chat.client.ChatClient;
import org.springframework.web.bind.annotation.GetMapping;
import org.springframework.web.bind.annotation.RestController;

@RestController
public class HomeController {

    private ChatClient chatClient;

    public HomeController(ChatClient.Builder builder) {
        this.chatClient = builder.build();
    }

    @GetMapping("/")
    public String home() {
        return chatClient
                .prompt()
                .user("who are you")
                .call()
                .content();
    } 
}

Run the application again, e.g.

./mvnw spring-boot:run

Test the application, in another terminal or browser, e.g.

http localhost:8080

HTTP/1.1 200 
Connection: keep-alive
Content-Length: 12
Content-Type: text/plain;charset=UTF-8
Date: Tue, 11 Apr 2025 08:05:00 GMT
Keep-Alive: timeout=60

I am an AI language model created by OpenAI, designed to assist with a wide range of
questions and topics. I am here to provide information, answer questions, and engage
in conversation. How can I help you today?

🏆 Congratulations! You have created your first application that talks to an Ollama hosted LLM service.

Hope you enjoyed this beginners getting started with Spring AI article!

Connecting to other LLMs

If you want to run your Spring AI application with other LLM models, look at our articles:

• How to get started with Spring AI

• Get started with Spring AI and OpenAI

• Getting started with Spring AI and Anthropic

• Getting started with Spring AI and Azure OpenAI

References

• Spring AI Ollama Chat

• Ollama

• Ollama Models

• Code Sample - Spring AI Quickstart Ollama

In this article we look at how to get started with Spring AI project and connect to Anthropic Large Language Models (LLMs).

Feel free to browse other articles in this getting started with Spring AI series:

• How to get started with Spring AI

• Get started with Spring AI and Ollama

• Get started with Spring AI and OpenAI

• Getting started with Spring AI and Azure OpenAI

Create a simple Spring Application (Beginners)

We can use Spring Initializr to generate a skeleton project for our simple application. You can do that either in IDE itself, or directly with a pre-configured project (e.g. Maven, Java, Spring Boot 3.4.4, spring-ai-demo, com.example package name. JAR packaging, Java 24 with Spring Web).

Clicking on “Generate” button will download a project skeleton zip archive, e.g. spring-ai-demo.zip which you can unzip and open in editor of your choice (e.g. IntelliJ, VS Code, etc.)

We can start adding specific AI model Spring Boot starters later on.

Let’s add a HomeController, e.g.

package com.example;

import org.springframework.web.bind.annotation.GetMapping;
import org.springframework.web.bind.annotation.RestController;

@RestController
public class HomeController {

    @GetMapping("/")
    public String home() {
        return "Hello World!";
    }
}

Let’s run our simple application, e.g.

./mvnw spring-boot:run

Feel free to browse from another terminal window, e.g. http localhost:8080

http localhost:8080

HTTP/1.1 200 
Connection: keep-alive
Content-Length: 12
Content-Type: text/plain;charset=UTF-8
Date: Tue, 11 Apr 2025 08:00:00 GMT
Keep-Alive: timeout=60

Hello World!

or a directly in the browser at http://localhost:8080

Feel free to explore Spring Boot Getting Started Guide for more details on how to create a simple Spring Boot web application.

Connecting your Spring Application to Anthropic models

Configure Anthropic

In this section we will explore connecting your application to Anthropic Large Language Models (LLMs). If you don’t already have an account on Anthropic API platform, please sign up for an account, e.g.

Once you have created and authenticated your account, you can create a new API Key,

Please save the key (e.g. sk-ant-api03-IywAWAvco...), as you won’t be able to view it again. Keep it secure, because anyone with your API key can access and consume your Anthropic API platform credits.

NOTE: Of course, I regenerated my API key after taking the screenshots. 🤣

Finally, let’s add Spring AI code to start using Anthropic API.

Add Spring AI for Anthropic

We will add Spring AI Spring Boot Starter for Anthropic in our Maven pom.xml, e.g.

        <dependency>
            <groupId>org.springframework.ai</groupId>
            <artifactId>spring-ai-starter-model-anthropic</artifactId>
            <version>1.0.0-M7</version>
        </dependency>

Notice, we are using version 1.0.0-M7 that’s newest at the time of writing this article. Please replace with the latest Spring AI version.

We also need to add few Spring properties to our src/main/resources/application.properties file, e.g.

spring.application.name=spring-ai-demo
spring.ai.anthropic.api-key=sk-ant-api03-IywAWAvco...
spring.ai.anthropic.chat.options.model=claude-3-5-haiku-latest

/quoteNotice, we have used claude-3-5-haiku-latest large language model (LLM) for this simple use case. Feel free to explore all Antropic LLM models. These models vary in size, speed, training data and eventually in price per 1M tokens. For example, claude-3-5-haiku-latest is the fastest model and costs a third of the price of more powerful claude-3-7-sonnet-latest model. The claude-3-5-haiku-latest costs per 1M tokens (MTOK) are: (1) for input US$0.80 and (2) for output US$4.00, whereas claude-3-7-sonnet-latest costs are (1) for input $3.00 and (2) for output $15.00 per 1M tokens.

Finally, let’s add a call to a LLM from our HomeController, e.g.

package com.example;

import org.springframework.ai.chat.client.ChatClient;
import org.springframework.web.bind.annotation.GetMapping;
import org.springframework.web.bind.annotation.RestController;

@RestController
public class HomeController {

    private ChatClient chatClient;

    public HomeController(ChatClient.Builder builder) {
        this.chatClient = builder.build();
    }

    @GetMapping("/")
    public String home() {
        return chatClient
                .prompt()
                .user("who are you")
                .call()
                .content();
    } 
}

Run the application again, e.g.

./mvnw spring-boot:run

Test the application, in another terminal or browser, e.g.

http localhost:8080

HTTP/1.1 200 
Connection: keep-alive
Content-Length: 266
Content-Type: text/plain;charset=UTF-8
Date: Tue, 11 Apr 2025 08:05:00 GMT
Keep-Alive: timeout=60

I'm Claude, an AI created by Anthropic. I aim to be helpful, honest, and
harmless. I won't pretend to be human, and I'm always direct about being
an AI. I'm happy to help you with tasks or have a conversation, while being
clear about my capabilities and limitations.

🏆 Congratulations! You have created your first application that talks to an Anthropic LLM service.

Possible Gotcha’s

If you run into Incorrect API key provided (see below example) - this is probably because you have no funds available in your Anthropic account, or you mistyped the Anthropic API key.

org.springframework.ai.retry.NonTransientAiException: 401 - {
    "type" : "error",
    "error": {
        "type":"authentication_error",
        "message":"invalid x-api-key"
    }
}

Check that you have some funds available in your account, e.g.

You might need to set the billing and add $5 to get started, e.g.

Hope you enjoyed this beginners getting started with Spring AI article!

Connecting to other LLMs

If you want to run your Spring AI application with other LLM models, look at our articles:

• How to get started with Spring AI

• Get started with Spring AI and OpenAI

• Get started with Spring AI and Ollama

• Getting started with Spring AI and Azure OpenAI

References

• Spring AI Anthropic Chat

• Anthropic Console

• Anthropic Models

• Code Sample - Spring AI Quickstart Anthropic

In this article we look at how to get started with Spring AI project and connect to various Large Language Model (LLM) services, both locally, or hosted in your datacenters, or publicly provided AI services, such as OpenAI, Azure OpenAI, Google Gemini, AWS Bedrock, Anthropic Claude, Groq, Mistral or any other AI service provider.

Feel free to look at other articles in this Getting Started with Spring AI series:

• Get started with Spring AI and OpenAI

• Get started with Spring AI and Ollama

• Getting started with Spring AI and Anthropic

• Getting started with Spring AI and Azure OpenAI

Create a simple Spring Application (Beginners)

We can use Spring Initializr to generate a skeleton project for our simple application. You can do that either in IDE itself, or directly with a pre-configured project (e.g. Maven, Java, Spring Boot 3.5.0, spring-ai-demo, com.example package name. JAR packaging, Java 24 with Spring Web).

Clicking on “Generate” button will download a project skeleton zip archive, e.g. spring-ai-demo.zip which you can unzip and open in editor of your choice (e.g. IntelliJ, VS Code, etc.)

We can start adding specific AI model Spring Boot starters later on.

Let’s add a HomeController, e.g.

package com.example;

import org.springframework.web.bind.annotation.GetMapping;
import org.springframework.web.bind.annotation.RestController;

@RestController
public class HomeController {

    @GetMapping("/")
    public String home() {
        return "Hello World!";
    }
}

Let’s run our simple application, e.g.

./mvnw spring-boot:run

Feel free to browse from another terminal window, e.g. http localhost:8080

http localhost:8080

HTTP/1.1 200 
Connection: keep-alive
Content-Length: 12
Content-Type: text/plain;charset=UTF-8
Date: Tue, 11 Apr 2025 08:00:00 GMT
Keep-Alive: timeout=60

Hello World!

or a directly in the browser at http://localhost:8080

Feel free to explore Spring Boot Getting Started Guide for more details on how to create a simple Spring Boot web application.

Connecting your Spring Application to Docker Model Runner

In this section we will explore connecting your application to Large Language Models (LLMs) hosted on your Docker Desktop. If you don’t already have Docker Desktop installed, please download and install it on your machine, e.g.

Configure Docker Model Runner

Docker Model Runner is a new experimental feature of Docker Desktop to host and run various Large Language Models (LLMs), directly hosted on your Docker Desktop. Please enable this experimental feature and , e.g.

Note: Docker Model Runner requires Docker Desktop v4.40+

Download Models for Docker Model Runner

First, we need to download a Large Language Model (LLM) that we will use in our Docker Model Runner, e.g. docker model pull ai/llama3.2

docker model pull ai/llama3.2

Downloaded: 0.00 MB
Model ai/llama3.2 pulled successfully

You can see a list of downloaded models, e.g. docker model list

docker model list

MODEL                 PARAMETERS  QUANTIZATION    ARCHITECTURE  MODEL ID      CREATED      SIZE   
ai/llama3.2           3.21 B      IQ2_XXS/Q4_K_M  llama         436bb282b419  2 weeks ago  1.87 GiB
ai/gemma3             3.88 B      IQ2_XXS/Q4_K_M  gemma3        0b329b335467  2 weeks ago  2.31 GiB 
ai/mxbai-embed-large  334.09 M    F16             bert          e5e025b145ac  2 weeks ago  638.85 MiB 
ai/mistral            7.25 B      IQ2_XXS/Q4_K_M  llama         395e9e2070c7  2 weeks ago  4.07 GiB 
ai/phi4               14.66 B     IQ2_XXS/Q4_K_M  phi3          03c0bc8e0f5a  2 weeks ago  8.43 GiB

You can also test the model locally, e.g. docker model run ai/llama3.2 “who are you”

docker model run ai/llama3.2 "who are you"

I'm an artificial intelligence model known as Llama. Llama stands for "Large Language Model Meta AI."

Check the Docker repository for a complete list of available models (fourteen models available at the moment), e.g.

Add Spring AI for OpenAI

Docker Model Runner uses OpenAI wire protocol, so we will add Spring AI Spring Boot Starter for OpenAI in our Maven pom.xml, e.g.

        <dependency>
            <groupId>org.springframework.ai</groupId>
            <artifactId>spring-ai-starter-model-openai</artifactId>
            <version>1.0.0</version>
        </dependency>

Notice, we are using version 1.0.0-M6 that’s newest at the time of writing this article. Please replace with the latest Spring AI version.

We also need to add few Spring properties to our src/main/resources/application.properties file, e.g.

spring.application.name=spring-ai-demo

spring.ai.openai.api-key=_NOT_NEEDED_
spring.ai.openai.chat.base-url=http://localhost:12434/engines
spring.ai.openai.chat.options.model=ai/llama3.2

Notice, we have used ai/llama3.2 large language model (LLM) for this simple use case. Feel free to explore other Docker Model Runner models. These models vary in size, speed and training data.

Finally, let’s add a call to a LLM from our HomeController, e.g.

package com.example;

import org.springframework.ai.chat.client.ChatClient;
import org.springframework.web.bind.annotation.GetMapping;
import org.springframework.web.bind.annotation.RestController;

@RestController
public class HomeController {

    private ChatClient chatClient;

    public HomeController(ChatClient.Builder builder) {
        this.chatClient = builder.build();
    }

    @GetMapping("/")
    public String home() {
        return chatClient
                .prompt()
                .user("who are you")
                .call()
                .content();
    } 
}

Run the application again, e.g.

./mvnw spring-boot:run

Test the application, in another terminal or browser, e.g.

http localhost:8080

HTTP/1.1 200 
Connection: keep-alive
Content-Length: 101
Content-Type: text/plain;charset=UTF-8
Date: Tue, 11 Apr 2025 08:05:00 GMT
Keep-Alive: timeout=60

I'm an artificial intelligence model known as Llama. Llama stands for "Large Language Model Meta AI."

🏆 Congratulations! You have created your first application that talks to a Llama3.2 LLM service hosted on Docker Model Runner in your Docker Desktop.

Connecting to other LLMs

If you want to run your Spring AI application with other LLM models, look at our articles:

• Get started with Spring AI and OpenAI

• Get started with Spring AI and Ollama

• Getting started with Spring AI and Anthropic

• Getting started with Spring AI and Azure OpenAI

References

• Spring AI Docker Model Runner Chat

• Spring AI with Docker Model Runner

• Run AI Models Locally: Zero API Keys, Zero Fees with Docker Desktop Model Runner (youtube video 12min 34s)

• Code Sample - Spring AI Quickstart Docker Model Runner