Science & Tech · AI · 16 hrs ago

Huawei Ascend tests LoRA fine-tuning of Kimi K3.1 Pro on 128 cards

Huawei Ascend tests LoRA fine-tuning of Kimi K3.1 Pro on 128 cards

Ascend says it validated LoRA fine-tuning of the Kimi K3.1 Pro model.

The test used a 128-card Atlas 950 SuperPoD, a computing system built around 128 cards.

Kimi K3.1 Pro has 2.8 trillion parameters, the adjustable values that help a model produce its outputs.

LoRA is a way to fine-tune a model without updating all of its parameters.

Ascend says Swap Core caching improved performance by 15% on average in the test.

This suggests the caching method can help when fine-tuning a very large model, though no further details or independent comparison are given.

The account does not say what happens next.

1 source

Related