TokenPost topic
AI inference
49 TokenPost stories are tagged AI inference. The newest appear first.
Latest AI inference stories
- Baseten Brings AI Inference Platform to Microsoft Foundry
Azure enterprise customers can use the service to run open AI models.
· Business
- NVIDIA Finds Key Scoring and Selection Took 47% of Layer Prefill Time
The DeepSeek V3.2 example used a 128K-token input. Separate tests found that more indexer key heads and smaller selection blocks increased component runtimes.
· Technology
- AMD Adds Instinct MI350P Support to Windows Server for AI Inference
The support covers Windows Server 2025 and the upcoming Windows Server 2028. GPU partitioning can divide an accelerator’s capacity among workloads.
· Technology
- AMD Completes Taalas Acquisition to Expand Its AI Inference Roadmap
AMD plans to integrate Taalas technology into its accelerator roadmap and develop system-level solutions with AMD Instinct GPUs.
· Business
- MegaETH TVL Falls More Than 96% to $8.71 Million in Six Months
MEGA’s first product under its $30 million treasury commitment is an AI inference marketplace planned for December.
· Investing
- AntSeed’s Open Inference Market Faces a Test Under New Lab Rules
The test centers on whether the market can hold up without discounted access to frontier models.
· Technology
- MEGA Completes Restructuring That Reduced Its Team by 20%
Departing employees had worked on public blockchain development, wallets and ecosystem support. Kong said the founders, early employees and core engineering team will remain.
· Business
- Luminal Raises Series A to Expand AI Inference Across More Chips
It plans to use the proceeds to expand engineering and go-to-market operations while broadening model and chip support.
· Business
- CoreWeave Moves to CNCF’s Platinum Tier as AI Inference Work Expands
Its engineers contribute to Kubernetes, Kueue and CloudNativePG, while CoreWeave was a founding contributor to llm-d.
· Technology
- Fractile Raises $720 Million in Series C at $7 Billion Valuation
Thrive Capital, Founders Fund, Lightspeed and Redpoint led the round; the UK government also invested through its Sovereign AI fund and the British Business Bank.
· Business
- AWS CEO Says AI Inference Is “Not Even 1%” of Its Eventual Scale
AWS revenue rose 39.3% in the third quarter, while Bedrock spending and token volume also increased.
· Business
- MEGA Board to Guide Token Decisions as MegaETH Targets 2027 Decentralization
Co-founder Shuyao Kong said the board will weigh community tokenomics, programmatic buybacks and product investment. The team is pursuing four upgrades, with decentralization targeted by the end of the first quarter o…
· Technology
- NVIDIA Snapshot Cuts Kubernetes GPU Inference Starts to Seconds
Snapshot preserves initialized CPU and GPU memory so new containers can skip setup steps. Tests on NVIDIA HGX B300 systems showed restore times of 3.6 to 42.7 seconds.
· Technology
- Blackfuel Launches First Inference-Only Deployment in Barcelona
The site uses AMD Instinct MI355X accelerators and open serving software. Customers can reserve capacity based on throughput, latency and data-location requirements.
· Technology
- Mira Launches Sterling AI Inference Service With x402, USDC Payments
The service is in testing, supports Claude, GPT and Gemini, and offers launch prices up to 50% below list prices.
· Technology
- Together AI Compiler Cuts AI Engine Setup Time to 9 Seconds
Together AI says the compiler delivered up to three times the throughput on selected open-source models and reached 252 times the experimentation throughput of baseline agentic systems.
· Technology
- Boundless Plans AI Credits for ZKC Stakers as ZK Protocol Pauses
The program would allocate model credits by eligible stake, while new ZKC emissions are scheduled to end in Q4 2026.
· Technology
- Cysic Shifts GPU Network Toward AI Inference as ZK Proof Demand Eases
Its network has processed more than 13 million transactions and produced more than 10 million proofs since its Sept. 24, 2025, mainnet launch. CEO Leo Fan says Cysic still aims to prove Ethereum blocks in under eight …
· Technology
- Arthur Hayes Says FLOP Devnet Opens for Verified AI Inference Tests
Developers can obtain test FLOP and inspect blocks, but the test units have no value or reward eligibility, and the network may reset.
· Technology
- Baseten Seeks $2 Billion in Funding at $26 Billion and $30 Billion Valuations
The higher proposed valuation is more than double the $13 billion valuation from Baseten’s June financing.
· Business
- NVIDIA Tests Show DSpark Speeds Nemotron on 64GB DGX Spark
Generation rates rose across three workloads, while median time to first token increased by 0.6 to 2.1 milliseconds.
· Technology
- AI Inference Demand Keeps High-End MLCC Supply Tight Through First Half of 2027
NVIDIA's Vera Rubin shipments are expected to rise month by month in the fourth quarter, while Advanced Micro Devices has begun sourcing specialized high-capacitance MLCCs for MI450.
· Business
- AI Inference Price Measure Falls 1.9% Over Seven Days Through Oct. 27
The token expenditure price index stood at $0.94 per million tokens, 54.48% below its May 28 period high.
· Technology
- SK hynix Details Memory-Tiering Technologies for AI Inference
SALT-KV evaluates cache reuse value and storage cost, while CMM-Hybrid stages predicted data from SSD into DRAM. SK hynix also plans to develop High Bandwidth Flash between HBM and SSDs.
· Technology
- NVIDIA Reports 42.5% Throughput Gain for Ling-3.0-flash-fp4 on DGX Spark
NVIDIA's test across all 1,319 GSM8K questions also cut mean latency 24.5%; a synthetic benchmark showed a much larger 185.1% increase.
· Technology
- Cerebras and Oracle Plan a 2027 AI Inference Service on OCI
The offering is planned for calendar 2027.
· Business
- Vera Rubin Inference Results Leave Nvidia’s 75% Training Claim Untested
At about 190 output tokens per second per user, Vera Rubin reached 5.7 times GB300’s total throughput per megawatt of GPU and memory power, using a four-GPU GB300 comparison.
· Technology
- DeepSeek Throughput Implies $38 in Theoretical Billings per GPU-Hour
DeepSeek disclosed daily throughput of 155 trillion tokens. The annualized estimate exceeds $330,000 per GPU under continuous operation, before costs and utilization.
· Technology
- Arcium’s Blackthorn Alpha Combines MPC With NVIDIA GPU Confidential Computing
Organizations can apply through a portal using an organization email for API access, installation on compatible GPUs they operate, or dedicated Blackthorn units by request.
· Technology
- NEAR AI’s Staking Service Tops 1 Million NEAR for AI Compute
The service supports confidential inference and IronClaw hosting. Customers retain ownership and control of their staked tokens.
· Technology
- Apple’s M5 Ultra Mac Studio Starts at $5,499 as AI API Rates Fall
The system supports up to 512GB of unified memory, while OpenAI and Anthropic announced lower per-token prices on Sept. 22.
· Technology
- Ant Group and vLLM Community Open-Source Humming Kernel Library
The library supports quantization formats for dense and mixture-of-experts models, with performance gains across many representative workloads.
· Technology
- Surplus Intelligence Logs 312.6 Billion Tokens in One Day
The Oct. 18 count was about 2.3 times its Sept. 23 level, but the dashboard lists user savings without showing customer spending.
· Technology
- General Compute Plans Positron Atlas Service for First Quarter of 2027
General Compute will buy Atlas systems and has committed to buying Positron’s planned Titan systems. The companies plan to use different chips to process prompts and generate responses.
· Technology
- IBM-Backed llm-d 1.0 Adds Controls for AI Inference Workloads
The release also supports coordinated upgrades and context-aware routing. A pre-release benchmark reached 6.60 million output tokens per minute with 3,000 coding agents.
· Technology
- DeepSeek Opens DeepInfer Code for Offline Model Inference
The release omits components needed for a complete inference service. DeepSeek plans to add code and optimizations for future models.
· Technology
- DeepSeek Fully Opens DeepInfer Framework Used for V4.1 Flash
The company says DeepInfer already supports large-scale V4.1 Flash inference and will host code and optimizations for future models.
· Technology
- Surplus Intelligence Sets $18.21 Million Market Cap Record on Base
The AI inference marketplace was valued at $18.17 million after a 24-hour gain of more than 61%.
· Investing
- Bitcoin Mining Estimate Puts Electricity at 45.9% of Total Costs
Vitalik Buterin’s comparison assigns 76% of estimated AI inference costs to computing capital, using a per-GPU-year measure that differs from mining’s cost per Bitcoin.
· Technology
- Architect Launches Liquid Inference Auction for AI Model Requests
Buyers can set spending, model, speed and data rules. The test phase covered hundreds of tasks across more than 700 models.
· Technology