Google Releases TPU Sync: Open-Source KV Cache Transfer for TPUs
📝 内容摘要
Google released TPU Sync, an open-source C++ library that moves KV caches and weights across TPU HBM, host DRAM, storage
Google released TPU Sync, an open-source C++ library that moves KV caches and weights across TPU HBM, host DRAM, storage and network. Google reports over 90% lower KV transfer latency on 235B to 480B MoE models.
The post Google Releases TPU Sync: Open-Source KV Cache Transfer for TPUs appeared first on MarkTechPost.
打开微信「扫一扫」，分享给好友或朋友圈
关闭