Science & Tech · AI · 5 hrs ago

Intel Arc B580 runs a 125B AI model nearly three times faster with XeStrata

Intel Arc B580 runs a 125B AI model nearly three times faster with XeStrata

A test in Japan found XeStrata generated text nearly three times faster than Strata on the same computer.

The comparison used Intel's 12GB Arc B580 graphics card to run a 125-billion-parameter Qwen model.

XeStrata produced 31.1 tokens per second, while Strata produced 10.9.

The test also depended on 64GB of system memory, because the model setup required about 50GB of RAM; the full model was not shown to fit in the card's 12GB of graphics memory.

XeStrata is derived from Strata and includes additional optimisations for supported graphics hardware, but the test did not identify which changes caused the speed difference.

The result shows that inference software can affect text-generation speed without changing the graphics card.

The test measured text generation only, and the reviewer said the card may not suit heavier AI workloads; no further test or next step was given.

1 source

Related