World's smallest AI supercomputer achieves world record with 120B-parameter LLM support on-device what I don't understand, though, is how it does OTA hardware upgrades

Date:
Mon, 15 Dec 2025 22:15:00 +0000

Description:
Tiiny AI Pocket Lab is a pocket-sized Mini PC running large language models offline with optimized CPU and NPU performance.

FULL STORY ======================================================================Tiiny
AI Pocket Lab runs large models locally, avoiding cloud dependence The mini
PC executes advanced inference tasks without discrete GPU support Models from 10B to 120B parameters operate offline within 65W of power

Tiiny, an American startup, has released the AI Pocket Lab, a pocket-sized AI supercomputer capable of running large language models locally.

The device is a mini PC designed to execute advanced inference workloads without cloud access, external servers, or discrete accelerators.

The company states that all processing remains offline, which removes network latency and limits external data exposure. Built to run large models without the cloud

"Cloud AI has brought remarkable progress, but it also created dependency, vulnerability, and sustainability challenges," said Samar Bhoj, GTM Director of Tiiny AI.

"With Tiiny AI Pocket Lab, we believe intelligence shouldn't belong to data centers, but to people. This is the first step toward making advanced AI
truly accessible, private, and personal, by bringing the power of large
models from the cloud to every individual device."

The Pocket Lab targets large personal models designed for complex reasoning and long-context tasks while operating within a constrained 65W power envelope.

Tiiny claims consistent performance for models in the 10B100B parameter
range, with support extending to 120B.

This upper limit approaches the capability of leading cloud systems, enabling advanced reasoning and extended context to run locally.

Guinness World Records has reportedly certified the hardware for local 100B-class model execution.

The system uses a 12-core ARMv9.2 CPU paired with a custom heterogeneous AI module that delivers roughly 190 TOPS of compute.

The system includes 80GB of LPDDR5X memory alongside a 1TB SSD, with total power draw reportedly staying within a 65W system envelope.

Its physical size more closely resembles a large external drive than a workstation , reinforcing its pocket-oriented branding.

While the specifications resemble a Houmo Manjie M50-style chip, independent real-world performance data is not yet available.

Tiiny also emphasizes an open-source ecosystem that supports one-click installation of major models and agent frameworks.

The company states that it will provide continuous updates, including what it describes as OTA hardware upgrades.

This phrasing is problematic, since over-the-air mechanisms traditionally apply to software.

The statement suggests either imprecise wording or a marketing error rather than literal hardware modification.

The technical approach relies on two software-driven optimizations rather
than scaling raw silicon performance.

TurboSparse focuses on selective neuron activation to reduce inference cost without altering model structure.

PowerInfer distributes workloads across heterogeneous components,
coordinating the CPU with a dedicated NPU to approach server-grade throughput at lower power.

The system includes no discrete GPU, with the company arguing that careful scheduling removes the need for expensive accelerators.

These claims indicate that efficiency gains, rather than brute force
hardware, serve as the primary differentiator.

Tiiny AI positions the Pocket Lab as a response to sustainability, privacy, and cost pressures affecting centralized AI services.

Running large language models locally could reduce recurring cloud expenses and limit exposure of sensitive data.

However, claims regarding capability, server-grade performance, and seamless scaling on such constrained hardware remain difficult to independently
verify.

Via TechPowerUp

Follow TechRadar on Google News and add us as a preferred source to get our expert news, reviews, and opinion in your feeds. Make sure to click the
Follow button!

And of course you can also follow TechRadar on TikTok for news, reviews, unboxings in video form, and get regular updates from us on WhatsApp too.



======================================================================
Link to news story: https://www.techradar.com/pro/worlds-smallest-ai-supercomputer-achieves-world- record-with-120b-parameter-llm-support-on-device-what-i-dont-understand-though -is-how-it-does-ota-hardware-upgrades


--- Mystic BBS v1.12 A49 (Linux/64)
* Origin: tqwNet Technology News (1337:1/100)