Plan and manage an Azure AI solution 1.3 Manage, monitor, and secure AI systems

Three teams use shared model deployments through separate Foundry projects.

Study guide skill: Manage quotas, scaling, rate limits, and cost footprints for model and agent workloads

Question B4-009

Three teams use shared model deployments through separate Foundry projects. Finance wants each project capped at a tokens-per-minute rate and at a monthly token total, enforced centrally by sending requests through AI Gateway. Which statement describes the setup and the responses callers get?

Select one answer.

  • A Set token limits per project and deployment in AI Gateway (Foundry Control Plane); callers get 429 Too Many Requests above the TPM limit and 403 Forbidden once the token quota is used up ✓ Correct answer
  • B Set token limits in AI Gateway; callers get 429 Too Many Requests both above the TPM limit and once the monthly quota is used up
  • C Lower the TPM of the shared model deployment; Foundry then splits that TPM evenly across the projects that use it
  • D Set token limits in AI Gateway; enforcement is exact, so concurrent bursts can never go over the configured values

Answer: A

With AI Gateway, Foundry Control Plane enforces TPM rate limits and total token quotas at the project scope, so each project can have its own settings. Exceeding the TPM limit returns 429, and exhausting the quota for its period returns 403. Many concurrent requests can temporarily exceed the limits until responses are processed.

Deep dive

How it works

Foundry Control Plane integrates with AI Gateway (backed by Azure API Management) to enforce two token limits on model deployments, at the project scope:

  • TPM rate limit: caps token consumption per minute. Above it, the caller gets 429 Too Many Requests.
  • Total token quota: caps consumption per quota period (hourly, daily, weekly, monthly or yearly). Once it's used up, the caller gets 403 Forbidden.

When AI Gateway is enabled for a project, all requests flow through the associated API Management instance, and limits apply at the project level, so each project can have its own TPM and quota settings. That's exactly the finance requirement: three teams, shared deployments, separate caps. Microsoft lists "multiple-team token containment (prevent one project from monopolizing capacity)" and cost control among the reasons to use it.

Two operational details: if many requests arrive concurrently, consumption can temporarily exceed the limits until responses are processed, and new limits apply immediately to subsequent requests after you save them. Enabling AI Gateway for a project needs the API Management Service Contributor (or Owner) role on the API Management resource.

When you test, validate both codes: 429 for the rate limit and 403 for the exhausted quota.

The other options, and when you'd use them

  • 429 for both: The quota has its own code. Exhausting the quota returns 403 Forbidden, not 429.
  • Lower the shared deployment's TPM: Nothing splits a deployment's TPM evenly across projects. Per-project limits are what AI Gateway adds.
  • Exact enforcement: Concurrent requests can temporarily push consumption over the configured limits.

Exam tip

AI Gateway token limits: per project. TPM exceeded = 429. Quota exhausted = 403. Bursts can briefly overshoot.

References

Last reviewed:

Ready to test yourself? The quiz asks this and every other AI-103 question, with instant feedback. Practice this in a quiz →

All Plan and manage questions · AI-103 study guide

Thanks for your feedback!