Sarvam Launches Sarvam Token Factory: Bringing India-Hosted Inference to 8 Open Models
Sarvam has opened Sarvam Token Factory to everyone. Sarvam Token Factory is an inference platform from Sarvam that serves its own models and open-weight models from GPU clusters in India. Developers reach every model through one OpenAI-compatible API. For India-based teams, this means low-latency tokens without sending traffic to overseas clouds. You can check the deployment options and specs here through our GPULeaderboard.com chrome extension.
TL;DR
- Size: 8 models listed at launch, including Sarvam 105B (105B total, 9B active, 128K context), GLM 5.3, DeepSeek V4.1 Flash 1017, Kimi K3.1 Pro and Gemma 4 31B.
- Runs on: NVIDIA B200 clusters in India. Sarvam 105B now fits on 1 B200 per replica. On-premise deployment is offered to enterprises.
- Performance: Fastest first token from Mumbai on most tested cases, per Sarvam’s own measurements.
- Best: Kimi K3.1 Pro at 6,177 tokens/s per GPU vs 3,213 for InferenceX, at 70 tokens/s per user (1.9x).
What is Sarvam Token Factory?
It is a managed inference stack, not a single model. Sarvam first previewed it as Sarvam Inference at its Epoch event. The launch post lists 8 models: Sarvam 105B, GLM 5.3, GLM 5.3 Flash, DeepSeek V4.1 Flash 1017, DeepSeek V4 Pro, Kimi K3, Kimi K3.1 Pro and Gemma 4 31B.
The docs split them into 2 groups. Pay-as-you-go models run on shared endpoints. DeepSeek V4 Pro, Kimi K3.1 Pro, Qwen 3.8 27B, Clef and EmbeddingGemma 2 need Provisioned Throughput, enabled per organisation.
How does it work?
The stack has 3 layers around a shared cache:
- Control plane: a cache-aware router, admission control and a planner.
- Inference engine: separate GPU pools for prefill and decode.
- Cache: GPU memory plus a shared G3 store built on Mooncake.
It builds on vLLM, SGLang, Dynamo, Mooncake, NIXL, DeepEP, DeepGEMM and FlashInfer. An orchestrator agent proposes experiments, and an engineer approves each deployment.
Caching drives most of the gains. DeepSeek V4.1 Flash serves 450B+ tokens per day, with 95.7% of prompt tokens served from cache. On GLM 5.3, the planner cut prefill GPU-hours from 32.3 to 17.1 under the same replayed load.
For Sarvam 105B, MXFP8 weights plus speculative decoding raised output throughput about 2.2x. That cut each replica from 2 B200s to 1. Sarvam says none of these changes reduced accuracy.
How fast is it? Benchmarks
All numbers below come from Sarvam’s launch post. They are self-reported.
- Sarvam 105B: 0.2 s first token and 330 tokens/s for a single user.
- Gemma 4 31B (10K prompt, Mumbai): 0.40 s first token and 308 tokens/s, vs 2.00 s and 193 tokens/s at the only other provider.
- Kimi K3.1 Pro (16 B200s): 6,177 vs 3,213 tokens/s per GPU against InferenceX at 70 tokens/s per user.
- Heavy agent load (72 sessions): median first token 0.8 s vs 11.6 s for InferenceX.
- GLM 5.3 Flash (64 sessions): p99 first token 3.2 s vs 14.1 s for stock vLLM and Dynamo.
The results are not a clean sweep. On DeepSeek V4.1 Flash with a 10K prompt, Together posts 0.71 s first token vs Sarvam’s 0.90 s. Together also streams faster, at 278 tokens/s vs 130 on Sarvam Standard.
<html lang="en"><head><meta charset="utf-8"><meta name="viewport" content="width=device-width,initial-scale=1">
<style>
*{box-sizing:border-box;margin:0;padding:0}
html,body{background:transparent!important}
#stf-x{–ink:#1F1F1F;–mut:#666666;–bg:#FFFFFF;–soft:#F5F5F5;–line:#E6E6E6;–b:#556ADC;–b2:#A5BBFC;–or:#E6651B;–or2:#EE7944;–nv:#1E2033;
font-family:Inter,system-ui,-apple-system,Segoe UI,sans-serif!important;max-width:880px;margin:0 auto;background:var(–bg)!important;color:var(–ink)!important;border:1px solid var(–line)!important;border-radius:16px;overflow:hidden}
#stf-x .hd{padding:22px 24px 16px;text-align:center;background:radial-gradient(60% 120% at 50% -20%,#F7801E 0%,#FBB27A 35%,rgba(255,255,255,0) 75%),radial-gradient(45% 90% at 0% 110%,#D2DFF9 0%,rgba(255,255,255,0) 70%),radial-gradient(45% 90% at 100% 110%,#D2DFF9 0%,rgba(255,255,255,0) 70%),#FFFFFF!important;border-bottom:1px solid var(–line)!important}
#stf-x .tag{display:inline-block;font-size:11px;font-weight:600;letter-spacing:.06em;color:#3333CC!important;background:rgba(255,255,255,.75)!important;border:1px solid #C7D2FE!important;padding:4px 10px;border-radius:99px}
#stf-x h3{font-family:Lora,Georgia,'Times New Roman',serif!important;font-size:26px;font-weight:500;margin-top:10px;color:var(–ink)!important;letter-spacing:-.01em}
#stf-x .sub{font-size:13px;color:var(–mut)!important;margin-top:4px}
#stf-x .tabs{display:flex;gap:4px;padding:10px 16px 0;background:var(–soft)!important;border-bottom:1px solid var(–line)!important;overflow-x:auto}
#stf-x .tab{border:0!important;background:transparent!important;font:600 13px Inter,sans-serif;color:var(–mut)!important;padding:10px 14px;border-radius:10px 10px 0 0;cursor:pointer;white-space:nowrap;transition:.2s}
#stf-x .tab.on{background:#fff!important;color:var(–ink)!important;box-shadow:0 -2px 0 var(–or) inset}
#stf-x .tab:hover{color:var(–ink)!important}
#stf-x .pn{display:none;padding:20px 24px;animation:stfIn .35s ease}
#stf-x .pn.on{display:block}
@keyframes stfIn{from{opacity:0;transform:translateY(6px)}to{opacity:1;transform:none}}
#stf-x .flow{display:grid;grid-template-columns:repeat(4,1fr);gap:10px;position:relative}
#stf-x .node{border:1.5px solid var(–line)!important;border-radius:12px;padding:12px;background:#fff!important;transition:.3s;min-height:96px}
#stf-x .node b{display:block;font-size:13px;color:var(–ink)!important}
#stf-x .node span{font-size:12px;color:var(–mut)!important;line-height:1.4;display:block;margin-top:4px}
#stf-x .node .n{font-size:11px;font-weight:800;color:var(–or)!important;margin-bottom:4px}
#stf-x .node.act{border-color:var(–or)!important;background:#FFF4EC!important;transform:translateY(-3px);box-shadow:0 8px 20px rgba(230,101,27,.18)}
#stf-x .pipe{height:6px;background:var(–line)!important;border-radius:6px;margin:14px 0 6px;overflow:hidden;position:relative}
#stf-x .pipe i{position:absolute;inset:0;width:30%;background:linear-gradient(90deg,transparent,var(–or2),var(–b2),transparent)!important;animation:stfFlow 1.6s linear infinite}
@keyframes stfFlow{from{left:-30%}to{left:100%}}
#stf-x .desc{background:var(–soft)!important;border-radius:10px;padding:12px 14px;font-size:13.5px;line-height:1.5;margin-top:10px;min-height:66px;color:var(–ink)!important}
#stf-x .ctl{display:flex;gap:8px;margin-top:12px;flex-wrap:wrap}
#stf-x .btn{border:1px solid var(–line)!important;background:#fff!important;color:var(–ink)!important;font:600 12.5px Inter,sans-serif;padding:7px 16px;border-radius:99px;cursor:pointer;transition:.2s}
#stf-x .btn:hover,#stf-x .btn.on{background:linear-gradient(#3A3F5C,#1E2033)!important;color:#fff!important;border-color:#1E2033!important}
#stf-x .row{display:grid;grid-template-columns:150px 1fr 70px;align-items:center;gap:10px;margin:9px 0;font-size:13px}
#stf-x .row .lb{color:var(–ink)!important;font-weight:600}
#stf-x .bar{height:20px;background:var(–soft)!important;border-radius:6px;overflow:hidden}
#stf-x .bar i{display:block;height:100%;width:0;border-radius:6px;background:#C7D2FE!important;transition:width 1s cubic-bezier(.2,.8,.2,1)}
#stf-x .bar i.s{background:linear-gradient(90deg,#E6651B,#F7801E)!important}
#stf-x .row .v{font-weight:700;text-align:right;color:var(–ink)!important}
#stf-x .note{font-size:11.5px;color:var(–mut)!important;margin-top:10px;line-height:1.5}
#stf-x .hl{font-size:13px;margin:4px 0 10px;color:var(–ink)!important}
#stf-x .cards{display:grid;grid-template-columns:repeat(3,1fr);gap:10px;margin-bottom:12px}
#stf-x .cd{background:var(–soft)!important;border-radius:12px;padding:12px;cursor:pointer;transition:.2s;border:1.5px solid transparent!important}
#stf-x .cd:hover{border-color:var(–or2)!important}
#stf-x .cd .k{font-family:Lora,Georgia,serif!important;font-size:26px;font-weight:600;color:var(–or)!important}
#stf-x .cd .t{font-size:12px;color:var(–mut)!important;margin-top:2px}
#stf-x .cd .r{font-size:12px;color:var(–ink)!important;margin-top:8px;max-height:0;overflow:hidden;transition:max-height .3s}
#stf-x .cd.op .r{max-height:120px}
#stf-x table{width:100%;border-collapse:collapse;font-size:13px}
#stf-x th,#stf-x td{padding:9px 8px;border-bottom:1px solid var(–line)!important;text-align:left;color:var(–ink)!important;background:transparent!important}
#stf-x th{font-size:11.5px;text-transform:uppercase;letter-spacing:.05em;color:var(–mut)!important}
#stf-x td.c{font-weight:700;color:var(–or)!important}
#stf-x .calc{display:grid;grid-template-columns:1fr 1fr;gap:12px;margin-bottom:12px}
#stf-x label{font-size:12px;font-weight:600;color:var(–mut)!important;display:block;margin-bottom:4px}
#stf-x input[type=range]{width:100%;accent-color:#E6651B}
#stf-x .lim li{list-style:none;padding:10px 12px;border-left:3px solid var(–or)!important;background:#FFF4EC!important;border-radius:0 8px 8px 0;margin:8px 0;font-size:13.5px;color:var(–ink)!important}
#stf-x .ft{display:flex;justify-content:space-between;padding:10px 24px;border-top:1px solid var(–line)!important;font-size:11.5px;color:var(–mut)!important;background:var(–soft)!important}
#stf-x .ft b{color:#76B900!important}
#stf-x hr,#stf-x p:empty,#stf-x del,#stf-x s{display:none!important}
@media(max-width:640px){#stf-x .flow{grid-template-columns:1fr 1fr}#stf-x .cards{grid-template-columns:1fr}#stf-x .row{grid-template-columns:96px 1fr 56px;font-size:12px}#stf-x .calc{grid-template-columns:1fr}#stf-x h3{font-size:18px}#stf-x .pn{padding:16px}#stf-x table{font-size:12px}#stf-x .ft{flex-direction:column;gap:4px;padding:10px 16px}}
</style></head><body>
<div id="stf-x">
<div class="hd"><span class="tag">SARVAM TOKEN FACTORY</span><h3>How Sarvam serves open-weight models from India</h3><div class="sub">One OpenAI-compatible API, B200 clusters in India, a 3-tier cache. All speed figures are Sarvam-reported.</div></div>
<div class="tabs">
<button class="tab on" data-t="0">How it works</button><button class="tab" data-t="1">First token</button><button class="tab" data-t="2">Throughput</button><button class="tab" data-t="3">Pricing</button><button class="tab" data-t="4">Limits</button>
</div>
<div class="pn on" data-p="0">
<div class="flow">
<div class="node" data-s="0"><div class="n">01</div><b>Request</b><span>OpenAI-compatible call to api.sarvam.ai/v2</span></div>
<div class="node" data-s="1"><div class="n">02</div><b>Control plane</b><span>Cache-aware router, admission control, planner</span></div>
<div class="node" data-s="2"><div class="n">03</div><b>Prefill + decode</b><span>Separate GPU pools on B200s</span></div>
<div class="node" data-s="3"><div class="n">04</div><b>Shared cache</b><span>GPU memory plus G3 store (Mooncake)</span></div>
</div>
<div class="pipe"><i></i></div>
<div class="desc" id="stf-d"></div>
<div class="ctl"><button class="btn" id="stf-prev">← Prev</button><button class="btn" id="stf-next">Next →</button><button class="btn" id="stf-play"> Auto-play</button></div>
</div>
<div class="pn" data-p="1">
<div class="hl" id="stf-th">Time to first token from Mumbai (lower is better)</div>
<div class="ctl" style="margin:0 0 8px"><button class="btn on" data-k="d1">DeepSeek, 1K</button><button class="btn" data-k="d10">DeepSeek, 10K</button><button class="btn" data-k="g1">GLM 5.3, 1K</button><button class="btn" data-k="g10">GLM 5.3, 10K</button></div>
<div id="stf-ttft"></div>
<div class="note">Seconds. Source: Sarvam technical blog, measured by Sarvam from Mumbai. Not independently verified.</div>
</div>
<div class="pn" data-p="2">
<div class="cards">
<div class="cd"><div class="k">1.9x</div><div class="t">Kimi K3.1 Pro tokens/s per GPU</div><div class="r">6,177 vs 3,213 for InferenceX at 70 tok/s per user, 16 B200s.</div></div>
<div class="cd"><div class="k">0.8 s</div><div class="t">Median first token, 72 agent sessions</div><div class="r">vs 11.6 s for InferenceX. p95 3.1 s vs 76.6 s.</div></div>
<div class="cd"><div class="k">95.7%</div><div class="t">DeepSeek prompt tokens from cache</div><div class="r">On 450B+ tokens served per day.</div></div>
</div>
<div class="hl">Output speed, DeepSeek V4.1 Flash (tokens/s, higher is better)</div>
<div id="stf-tp"></div>
<div class="note">Click a card to reveal detail. Source: Sarvam technical blog (self-reported).</div>
</div>
<div class="pn" data-p="3">
<div class="calc">
<div><label>Input tokens per month: <span id="stf-iv"></span>M</label><input type="range" id="stf-in" min="1" max="1000" value="100"></div>
<div><label>Output tokens per month: <span id="stf-ov"></span>M</label><input type="range" id="stf-out" min="1" max="500" value="20"></div>
</div>
<table><thead><tr><th>Model</th><th>₹ per 1M in / out</th><th>Your monthly cost</th></tr></thead><tbody id="stf-pt"></tbody></table>
<div class="note">Prices from Sarvam docs. Sarvam 105B and DeepSeek rates are introductory through the end of 2026. Excludes cache discounts.</div>
</div>
<div class="pn" data-p="4">
<ul class="lim">
<li>Every speed and throughput number is Sarvam-measured.</li>
<li>Together beats Sarvam on DeepSeek V4.1 Flash: 0.71 s vs 0.90 s first token at 10K, and 278 vs 130 tok/s streaming.</li>
<li>Uptime, SLA, rate limits and context windows are not on the launch post or docs page.</li>
<li>Kimi K3.1 Pro and DeepSeek V4 Pro need Provisioned Throughput with unpublished rates.</li>
</ul>
</div>
<div class="ft"><span>Sources: sarvam.ai/blogs/sarvam-token-factory · docs.sarvam.ai</span><b>© Marktechpost</b></div>
</div>
<script>
(function(){
var R=document.getElementById(‘stf-x’);
function post(){try{parent.postMessage({stfHeight:R.offsetHeight+40},’*’)}catch(e){}}
R.querySelectorAll(‘.tab’).forEach(function(t){t.onclick=function(){R.querySelectorAll(‘.tab’).forEach(function(x){x.classList.remove(‘on’)});t.classList.add(‘on’);R.querySelectorAll(‘.pn’).forEach(function(p){p.classList.toggle(‘on’,p.dataset.p==t.dataset.t)});if(t.dataset.t==’1′)ttft(cur);if(t.dataset.t==’2′)tp();setTimeout(post,60)}});
var steps=[‘A developer calls Chat Completions with the OpenAI SDK. Only the base_url and API key change.’,’The cache-aware router sends the request to the worker most likely to hold its prefix. Misrouted requests fell from 17.6% to 3.5%.’,’Prefill and decode run on separate GPU pools. On GLM 5.3 the planner cut prefill GPU-hours from 32.3 to 17.1.’,’Prompt tokens are served from GPU memory or the G3 store when possible. DeepSeek V4.1 Flash hits cache on 95.7% of prompt tokens.’];
var si=0,timer=null,nodes=R.querySelectorAll(‘.node’);
function show(i){si=(i+4)%4;nodes.forEach(function(n){n.classList.toggle(‘act’,n.dataset.s==si)});document.getElementById(‘stf-d’).textContent=steps[si]}
nodes.forEach(function(n){n.onclick=function(){show(+n.dataset.s)}});
document.getElementById(‘stf-prev’).onclick=function(){show(si-1)};
document.getElementById(‘stf-next’).onclick=function(){show(si+1)};
document.getElementById(‘stf-play’).onclick=function(){var b=this;if(timer){clearInterval(timer);timer=null;b.textContent=”
Auto-play”}else{timer=setInterval(function(){show(si+1)},2200);b.textContent=”❚❚ Pause”}};
show(0);
var D={d1:[[‘Sarvam’,.51],[‘Together’,.61],[‘Fireworks’,1.01],[‘DeepInfra’,1.08]],d10:[[‘Sarvam’,.90],[‘Together’,.71],[‘Fireworks’,1.55],[‘DeepInfra’,1.75]],g1:[[‘Sarvam’,.68],[‘Together’,.82],[‘Fireworks’,.84],[‘DeepInfra’,2.17]],g10:[[‘Sarvam’,1.19],[‘Together’,1.19],[‘Fireworks’,1.57],[‘DeepInfra’,2.18]]};
var cur=”d1″;
function bars(el,rows,max,unit){el.innerHTML=rows.map(function(r){return ‘<div class="row"><div class="lb">’+r[0]+'</div><div class="bar"><i class="’+(r[0].indexOf(‘Sarvam’)==0?’s’:”)+’" data-w="’+(r[1]/max*100)+’"></i></div><div class="v">’+r[1]+unit+'</div></div>’}).join(”);requestAnimationFrame(function(){setTimeout(function(){el.querySelectorAll(‘i’).forEach(function(i){i.style.width=i.dataset.w+’%’})},30)})}
var TN={d1:’DeepSeek V4.1 Flash, 1K prompt’,d10:’DeepSeek V4.1 Flash, 10K prompt’,g1:’GLM 5.3, 1K prompt’,g10:’GLM 5.3, 10K prompt’};function ttft(k){cur=k;document.getElementById(‘stf-th’).textContent=”Time to first token from Mumbai, “+TN[k]+’ (lower is better)’;bars(document.getElementById(‘stf-ttft’),D[k],2.3,’ s’)}
R.querySelectorAll(‘[data-k]’).forEach(function(b){b.onclick=function(){R.querySelectorAll(‘[data-k]’).forEach(function(x){x.classList.remove(‘on’)});b.classList.add(‘on’);ttft(b.dataset.k)}});
function tp(){bars(document.getElementById(‘stf-tp’),[[‘Sarvam Interactive’,200],[‘Sarvam Standard’,130],[‘Together’,278],[‘Fireworks’,193],[‘Baseten’,165],[‘DeepSeek API’,60]],290,”)}
R.querySelectorAll(‘.cd’).forEach(function(c){c.onclick=function(){c.classList.toggle(‘op’);setTimeout(post,350)}});
var P=[[‘Sarvam 105B’,10,25],[‘DeepSeek V4.1 Flash 1017’,14,55],[‘GLM 5.3 Flash’,13.5,45],[‘Gemma 4 31B’,35,90],[‘GLM 5.3’,126,390]];
function price(){var a=+document.getElementById(‘stf-in’).value,b=+document.getElementById(‘stf-out’).value;document.getElementById(‘stf-iv’).textContent=a;document.getElementById(‘stf-ov’).textContent=b;
document.getElementById(‘stf-pt’).innerHTML=P.map(function(p){return ‘<tr><td>’+p[0]+'</td><td>₹’+p[1]+’ / ₹’+p[2]+'</td><td class="c">₹’+Math.round(a*p[1]+b*p[2]).toLocaleString(‘en-IN’)+'</td></tr>’}).join(”)}
document.getElementById(‘stf-in’).oninput=price;document.getElementById(‘stf-out’).oninput=price;price();
ttft(‘d1’);
window.addEventListener(‘load’,post);window.addEventListener(‘resize’,post);setTimeout(post,300);
})();
</script>
</body></html>
“>
window.addEventListener(‘message’,function(e){if(e.data&&e.data.stfHeight){var f=document.getElementById(‘stf-frame’);if(f&&e.source===f.contentWindow){f.style.height=e.data.stfHeight+’px’;}}});
How much does it cost?
Prices are in Indian rupees per 1M tokens, per the docs:
Introductory pricing runs through the end of 2026. A faster DeepSeek Interactive endpoint costs ₹25 input and ₹105 output. Provisioned model rates are not published on the docs page.
How does it compare with other inference providers?
Key Takeaways
- Sarvam Token Factory serves Sarvam and open-weight models from India via one OpenAI-compatible API.
- Sarvam 105B starts at ₹10 input and ₹25 output per 1M tokens.
- Sarvam reports 1.9x per-GPU throughput over InferenceX on Kimi K3.1 Pro.
- Together still beats it on DeepSeek V4.1 Flash streaming speed.
- All benchmark numbers are self-reported.
Check out the technical blog, the API docs and the launch post on X. You can check the deployment options and specs here through our GPULeaderboard.com chrome extension.
Also, feel free to follow us on Twitter and don’t forget to join our 150k+ML SubReddit and Subscribe to our Newsletter. Wait! are you on telegram? now you can join us on telegram as well.
Need to partner with us for promoting your GitHub Repo OR Hugging Face Page OR Product Release OR Webinar etc.? Connect with us
The post Sarvam Launches Sarvam Token Factory: Bringing India-Hosted Inference to 8 Open Models appeared first on MarkTechPost.



