[Cherry-Pick][BugFix] support deepgemm without bias input(#7559)#7565
Conversation
|
Thanks for your contribution! |
PaddlePaddle-bot
left a comment
There was a problem hiding this comment.
🤖 AI Code Review |
2026-04-22 19:51:03
📋 Review 摘要
PR 概述:适配 DeepGEMM FP8 GEMM 算子在不支持 bias 关键字参数时的兼容处理,通过模块加载时运行时探测自动选择调用路径。
变更范围:model_executor/layers/quantization/block_wise_fp8.py
影响面 Tag:OP Quantization
📝 PR 规范检查
Modifications 节未填写,建议补充说明本次变更的具体修改内容,供可直接复制的示例如下:
Modifications(可直接复制):
- 在模块加载时通过
inspect.signature探测fp8_gemm_nt是否支持bias关键字参数,结果缓存至全局变量_fp8_gemm_nt_has_bias_kwarg。deep_gemm_fp8_gemm_nt函数中增加分支:支持bias时原样调用,不支持时先调用无 bias 版本再通过paddle.add追加 bias。
问题
| 级别 | 文件 | 概述 |
|---|---|---|
| 🟡 建议 | block_wise_fp8.py:62 |
异常捕获注释与实现不符,误导读者 |
| 🟡 建议 | block_wise_fp8.py:157 |
paddle.add 创建额外 Tensor,可考虑原地操作 |
总体评价
整体思路清晰,运行时 capability detection 是处理 pybind11 函数签名差异的合理方案,兼容性设计正确。建议修正误导性注释并评估 paddle.add 的显存影响后合入。
| (layer_weight, layer_weight_scale_inv), | ||
| linear_out, | ||
| ) | ||
| if bias is not None: |
There was a problem hiding this comment.
🟡 建议 额外 Tensor 分配
linear_out = paddle.add(linear_out, bias) 会产生一个新 Tensor,而非原地操作,在大 batch / 长序列场景下会带来额外显存峰值。建议评估是否可使用 paddle.add_(linear_out, bias) 原地加法,或与上游确认 linear_out 的生命周期是否允许原地操作。
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## release/2.6 #7565 +/- ##
==============================================
Coverage ? 73.75%
==============================================
Files ? 376
Lines ? 53097
Branches ? 8303
==============================================
Hits ? 39161
Misses ? 11188
Partials ? 2748
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
Cherry-pick of #7559 (authored by @zoooo0820) to
release/2.6.devPR:#7559
Motivation
适配deepgemm缺少bias参数时的处理
Modifications
Usage or Command
Accuracy Tests
Checklist
[FDConfig],[APIServer],[Engine],[Scheduler],[PD Disaggregation],[Executor],[Graph Optimization],[Speculative Decoding],[RL],[Models],[Quantization],[Loader],[OP],[KVCache],[DataProcessor],[BugFix],[Docs],[CI],[Optimization],[Feature],[Benchmark],[Others],[XPU],[HPU],[GCU],[DCU],[Iluvatar],[Metax]]pre-commitbefore commit.releasebranch, make sure the PR has been submitted to thedevelopbranch, then cherry-pick it to thereleasebranch with the[Cherry-Pick]PR tag.