Science & Tech · AI · 4 hrs ago

Tiny Corp's $30,000 workstation aims to run Kimi K3 at home

Tiny Corp's $30,000 workstation aims to run Kimi K3 at home

Tiny Corp has introduced a $30,000 workstation designed to run Moonshot AI's Kimi K3 model on a desk.

The system combines four AMD GPUs with 128 GB of memory and sixteen 1 TB SSDs.

Kimi K3 is a very large model, with weights measuring about 1.56 TB, far more than the GPUs can hold at once.

Tiny Corp says its software lets the GPUs access the SSD array directly, which is rated to read data at 238.4 GB per second.

The company claims the machine can generate 40 tokens per second, but no one outside the company has tested that speed.

Reaching it appears to depend on keeping nearly all the weights needed for each token in GPU memory, so the SSDs cannot supply most of the data on their own.

The workstation is available to order, though it weighs about 56 pounds and its GPU fans can be audible.

1 source

Related

What people are saying

Tiny CorpCompany
frontier models at home at usable speeds for a good price
via TechRadar