Implement generative AI and agentic solutions 2.1 Build generative applications by using Foundry

Field technicians use a Windows app on laptops in places with no network connection.

Study guide skill: Deploy and consume LLMs, small models, code models, and multimodal models

Question B3-010

Field technicians use a Windows app on laptops in places with no network connection. The app must summarize inspection notes with a language model, the notes must never leave the device, and the developers want to reuse their existing OpenAI SDK code. What should you use?

Select one answer.

  • A A Data Zone provisioned deployment, so the data stays in the region
  • B Foundry Local with a model from its on-device catalog, called through its OpenAI-compatible API ✓ Correct answer
  • C Foundry Local installed on one central server that all technicians share
  • D A Global Standard deployment of a GPT model, called from the app over HTTPS

Answer: B

Foundry Local runs models entirely on the user's device, so the app works offline and data never leaves the device, and its OpenAI-compatible API lets apps that already use the OpenAI SDK switch with minimal code changes. Cloud deployments, including Data Zone ones, need network access and send data off the device. Foundry Local is optimized for single-user, on-device inference and isn't designed as a shared server inference stack.

Deep dive

How it works

Foundry Local is Microsoft's runtime for shipping apps that run AI entirely on the user's device. The page lists its reasons for being: run AI locally, work offline, minimize latency, or keep data private. This scenario hits all of them:

  • Offline: the app works offline, with no network latency and no backend infrastructure.
  • Privacy: your data never leaves the device.
  • Existing code: an OpenAI-compatible API supports OpenAI request and response formats. If the app already uses the OpenAI SDK, you point it at a Foundry Local endpoint with minimal code changes.
  • Models: a curated catalog of optimized models, with the best-performing variant selected for the device's hardware.

For embedded apps like this, the guidance is to use the SDK directly so inference runs in-process. The optional local web server is for serving several processes or tools such as LangChain on the same machine.

The other options, and when you'd use them

  • Data Zone provisioned deployment: Data zone deployments keep processing within a data zone such as the EU or US, but they're still cloud deployments. Notes would leave the laptop, and offline use is impossible.
  • Foundry Local on one shared server: It's optimized for devices where a single user accesses the model at a time, and it isn't designed as a server inference stack. A shared server also needs a network.
  • Global Standard over HTTPS: That's a cloud call, so it fails offline and sends the notes off the device.

Exam tip

Offline + data stays on device + reuse OpenAI SDK code = Foundry Local on each device. Shared or multi-user serving = a server runtime, not Foundry Local.

References

Last reviewed:

Ready to test yourself? The quiz asks this and every other AI-103 question, with instant feedback. Practice this in a quiz →

All Generative AI & agents questions · AI-103 study guide

Thanks for your feedback!