Feat/multi format audit report export by LordofAvernus · Pull Request #3238 · actiontech/sqle

LordofAvernus · 2026-04-10T03:26:34Z

User description

关联的 issue

描述你的变更

https://github.com/actiontech/sqle-ee/issues/2689

确认项（pr提交后操作）

Tip

请在指定复审人之前，确认并完成以下事项，完成后✅

我已完成自测
我已记录完整日志方便进行诊断
我已在关联的issue里补充了实现方案
我已在关联的issue里补充了测试影响面
我已确认了变更的兼容性，如果不兼容则在issue里标记 not_compatible
我已确认了是否要更新文档，如果要更新则在issue里标记 need_update_doc

Description

新增审核报告多格式导出功能，支持CSV和HTML格式（企业版支持PDF/WORD）
实现通用报告生成器接口及导出格式规范化处理
重构下载审核报告接口，移除原有CSV构建逻辑，直接调用统一导出服务
补充单元测试、API文档、Swagger配置和本地化消息更新以支持新功能

Diagram Walkthrough

flowchart LR
  A["API请求: export_format参数"] --> B["DownloadTaskSQLReportFile处理"]
  B --> C["构建AuditReportData数据"]
  C --> D["调用ExportAuditReport接口"]
  D --> E["CSVReportGenerator (for csv)"]
  D --> F["HTMLReportGenerator (for html)"]
  D -- 企业版扩展 --> G["PDF/WORD导出错误提示"]

File Walkthrough

Relevant files

Enhancement

8 files

report_data_builder.go `构建审核报告数据模型并统计SQL审核信息`	+265/-0
task.go `重构下载报告接口，增加export_format查询参数`	+16/-45
export_format.go `定义审核报告导出格式常量及规范化处理方法`	+36/-0
report_csv.go `实现CSV格式报告生成器，复用CSVBuilder`	+75/-0
report_generator.go `定义审核报告数据模型与通用ReportGenerator接口`	+112/-0
report_generator_ce.go `实现CE版统一导出入口，限制PDF/WORD格式`	+30/-0
report_html.go `实现HTML格式报告生成器，基于html/template渲染`	+51/-0
report_html_template.go `嵌入审核报告HTML模板文件，供HTML渲染使用`	+17/-0

Tests

3 files

report_data_builder_test.go `添加审核报告数据构建功能单元测试`	+451/-0
report_generator_ce_test.go `添加CE版导出功能限制的单元测试`	+109/-0
report_generator_test.go `为CSV与HTML报告生成器增加覆盖率测试`	+751/-0

Documentation

7 files

docs.go `更新API文档，增加导出格式参数说明`	+14/-1
message_zh.go `添加审核报告导出相关本地化消息支持`	+20/-0
swagger.json `更新Swagger文档，增加export_format参数描述`	+14/-1
swagger.yaml `调整Swagger YAML，补充导出格式参数入参及响应描述`	+11/-1
active.en.toml `增加英文本地化审核报告导出标签`	+16/-0
active.zh.toml `增加中文本地化审核报告导出标签`	+16/-0
audit_report.html `添加审核报告HTML模板，定义各模块样式及布局`	+280/-0

Miscellaneous

1 files

file.go `更新文件保存函数注释，说明导出格式用途`	+1/-1

Configuration changes

1 files

ci.yml `升级CI工作流中actions版本以提高稳定性`	+5/-12

…or interface Add backend infrastructure for multi-format audit report export: - Extend ExportFormat enum with HTML/PDF/WORD constants - Add NormalizeExportFormatStr function for string-based format normalization - Define AuditReportData, AuditSummary, AuditStatistics, LevelCount, RuleHit, AuditSQLItem, and ReportLabels data model structs - Define ReportGenerator interface with Generate and Format methods - Add CSVHeaders() and ToCSVRow() helper methods for CSV export - Add unit tests for NormalizeExportFormatStr with map case pattern

Implement CSVReportGenerator that reuses the existing CSVBuilder to generate CSV audit reports via the ReportGenerator interface. Add comprehensive unit tests covering normal data, empty SQL list, and special character escaping (comma, newline, double quote).

…ests - Add self-contained HTML report template (templates/audit_report.html) with 4 content modules: audit summary, result statistics, problem SQL list, and rule hit statistics. All text uses Go template i18n variables, CSS is inlined, SQL displayed in <pre> tags, logo supports base64 embedding. - Add report_html_template.go using embed.FS to embed the HTML template file. - Add report_html.go implementing HTMLReportGenerator with html/template for automatic XSS prevention via HTML escaping. Content-Type: text/html, filename format: SQL_audit_report_{instance}_{taskId}.html. - Add 4 unit tests: Normal (ContentType, HTML tags, SQL content, labels), XSSPrevention (script/img tags escaped), EmptyData (empty SQL list renders), LargeData (10000 SQL items performance check).

… export Refactor the DownloadTaskSQLReportFile API handler to support exporting audit reports in multiple formats (CSV, HTML, PDF, WORD) via the new export_format query parameter. Default to CSV for backward compatibility. Key changes: - Add BuildAuditReportData function in report_data_builder.go that converts Task + SQL data into AuditReportData (placed in controller layer to avoid circular dependency between utils and model packages) - Refactor DownloadTaskSQLReportFile to use BuildAuditReportData and ExportAuditReport for format-agnostic report generation - Add 16 new i18n locale messages for report labels (zh/en TOML + Go vars) - Update Swagger annotations with export_format parameter - Add comprehensive unit tests for helper functions: toLevelCounts, toRuleHits, extractRuleInfo, buildReportLabels (13 test functions, map case pattern, covering normal/empty/nil/edge cases) Requirement refs: REQ-6.1, REQ-6.2, REQ-6.4, REQ-NF-2.1, REQ-NF-3.1

Update swagger.json, swagger.yaml, and docs.go to include the new export_format query parameter (type: string, default: csv) for the /v1/tasks/audits/{task_id}/sql_report endpoint. Also update response description from "sql report csv file" to "sql report file" to reflect multi-format support.

… fallback (CE part)

github-actions · 2026-04-10T03:27:56Z

PR Reviewer Guide 🔍

(Review updated until commit `2c3d81f`)

🎫 Ticket compliance analysis ✅ 2689 - Fully compliant Compliant requirements: 审核报告多格式导出（CSV、HTML、PDF/WORD 的处理逻辑） API 文档和 Swagger 参数更新完整的单元测试覆盖
⏱️ Estimated effort to review: 5 🔵🔵🔵🔵🔵
🧪 PR contains tests
🔒 No security concerns identified
⚡ Recommended focus areas for review 错误处理在解析导出格式参数以及报告生成流程中，错误返回时的提示信息和处理方式稍显分散，不同场景下错误信息可能不够统一，建议进一步提升错误信息的规范性，以便客户端更好地识别和处理异常情况。 ctx := c.Request().Context() // 解析导出格式参数，缺失时默认返回 CSV（向后兼容），无效格式返回 400 错误 exportFormat, err := auditreport.NormalizeExportFormatStr(c.QueryParam("export_format")) if err != nil { return controller.JSONBaseErrorReq(c, err) } // 构建报告数据 reportData, err := BuildAuditReportData(task, s, req.NoDuplicate, ctx) if err != nil { return controller.JSONBaseErrorReq(c, err) } // 根据格式生成报告 result, err := auditreport.ExportAuditReport(exportFormat, reportData) if err != nil {

github-actions · 2026-04-10T03:28:38Z

Failed to generate code suggestions for PR

…fixes

github-actions · 2026-04-10T06:33:15Z

Persistent review updated to latest commit 2d973a0

github-actions · 2026-04-10T06:33:31Z

PR Code Suggestions ✨

No code suggestions found for the PR.

…tData function Updated the BuildAuditReportData function to initialize sqlList and problemSQLs with a predefined capacity based on taskSQLsDetail length, improving performance and memory usage.

github-actions · 2026-04-10T07:30:59Z

Persistent review updated to latest commit 0fc0407

github-actions · 2026-04-10T07:31:32Z

Failed to generate code suggestions for PR

winfredLIN · 2026-04-10T10:20:34Z

+// NormalizeExportFormatStr 规范化导出格式参数（字符串版本）。
+// 空字符串默认返回 CSV（向后兼容）；无效格式返回错误。
+// 支持 html/pdf/word/docx/excel/xlsx/csv 等输入的规范化。
+func NormalizeExportFormatStr(format string) (ExportFormat, error) {
+	switch strings.ToLower(strings.TrimSpace(format)) {
+	case "html":
+		return ExportFormatHTML, nil
+	case "pdf":
+		return ExportFormatPDF, nil
+	case "word", "docx":
+		return ExportFormatWORD, nil
+	case "excel", "xlsx":
+		return ExcelExportFormat, nil
+	case "csv", "":
+		return CsvExportFormat, nil
+	default:
+		return "", fmt.Errorf("unsupported export format: %s", format)
+	}
+}
+


ExportFormat也是导出格式的业务对象，应当移动到业务层

winfredLIN · 2026-04-10T10:34:58Z

+	// ReportType 返回生成器对应的导出格式
+	ReportType() utils.ExportFormat


目前这个接口没有被除了单元测试之外的地方用到

可以保留

winfredLIN · 2026-04-10T10:36:29Z

 // @Param task_id path string true "task id"
 // @Param no_duplicate query boolean false "select unique (fingerprint and audit result) for task sql"
-// @Success 200 file 1 "sql report csv file"
+// @Param export_format query string false "export format: csv, html, pdf, word" default(csv)


这里是不是应该加一个enum
参考：
// @param sort_field query string false "sort field" Enums(first_appear_timestamp,last_receive_timestamp,fp_count)
// @param sort_order query string false "sort order" Enums(asc,desc)

winfredLIN · 2026-04-10T10:37:45Z

+                    {
+                        "type": "string",
+                        "description": "export format: csv, html, pdf, word",
+                        "name": "export_format",
+                        "in": "query",
+                        "default": "csv"
                    }


确实缺少枚举

winfredLIN · 2026-04-10T10:44:20Z

+// toLevelCounts 将等级分布 map 转换为有序的 LevelCount 切片。
+// 按 error > warn > notice > normal 顺序排列。
+func toLevelCounts(dist map[string]int) []auditreport.LevelCount {
+	if len(dist) == 0 {
+		return []auditreport.LevelCount{}
+	}
+
+	levelOrder := map[string]int{
+		"error":  0,
+		"warn":   1,
+		"notice": 2,
+		"normal": 3,
+	}
+
+	result := make([]auditreport.LevelCount, 0, len(dist))
+	for level, count := range dist {
+		result = append(result, auditreport.LevelCount{
+			Level: level,
+			Count: count,
+		})
+	}
+
+	sort.Slice(result, func(i, j int) bool {
+		oi, ok := levelOrder[result[i].Level]
+		if !ok {
+			oi = 99
+		}
+		oj, ok := levelOrder[result[j].Level]
+		if !ok {
+			oj = 99
+		}
+		return oi < oj
+	})
+
+	return result
+}


可以移除，没必要这么复杂，一开始定义一个数组就可以了

levelCounts := []LevelCount{
{Level: "normal", Count: 0},
{Level: "notice", Count: 0},
{Level: "warn", Count: 0},
{Level: "error", Count: 0},
}

根据level类型，在对应下标的元素里面++，不需要在Map和Array/Slice之间杂耍

// LevelCount 等级统计
type LevelCount struct {
Level string json:"level" // normal/notice/warn/error
Count int json:"count"
}

Define ExportFormat and NormalizeExportFormatStr in auditreport; keep utils.ExportFormat for generic csv/excel data export only.

Use one time.Now for GeneratedAt and fallback audit time; build level stats with fixed-order slices instead of map plus sort.

Use auditreport.NormalizeExportFormatStr; document Enums in swagger.

github-actions · 2026-04-13T07:22:45Z

Persistent review updated to latest commit 2c3d81f

github-actions · 2026-04-13T07:23:05Z

PR Code Suggestions ✨

No code suggestions found for the PR.

winfredLIN · 2026-04-14T02:43:45Z

+	// 1. 获取 SQL 列表
+	data := map[string]interface{}{
+		"task_id":      fmt.Sprintf("%d", task.ID),
+		"no_duplicate": noDuplicate,
+	}
+


应该定义一个FilterOption 结构体，明确能用哪些参数进行筛选

winfredLIN · 2026-04-14T02:46:46Z

+	return strings.Join(ruleNames, ", "), strings.Join(suggestions, "; ")
+}
+
+// formatLevelDistribution 将各等级计数转为 LevelCount 切片：先按固定顺序输出非零的标准等级，


和上次比，没有进步啊，非得排序一下，直接做一个[]auditreport.LevelCount的数组，长度4
每次统计的时候在对应的元素里面 LevelArray[index].count ++就行了

winfredLIN · 2026-04-14T02:47:11Z

+}
+
+// toLevelCounts 将等级分布 map 转为 LevelCount 切片（供测试与 map 输入场景）。
+func toLevelCounts(dist map[string]int) []auditreport.LevelCount {


不再被使用也不删掉

winfredLIN · 2026-04-14T02:48:19Z

+		})
+	}
+
+	sort.Slice(result, func(i, j int) bool {


这里排序是合理的，在界面上规则顺序保持一致，提高影响用户体验

winfredLIN · 2026-04-14T02:49:43Z

-		"task_id":      taskId,
-		"no_duplicate": req.NoDuplicate,
-	}
+	ctx := c.Request().Context()


没有被第二次使用，就直接作为入参就行

winfredLIN · 2026-04-14T02:51:40Z

+
+// LevelCount 等级统计
+type LevelCount struct {
+	Level string `json:"level"` // normal/notice/warn/error


没有Enumerate

LordofAvernus added 8 commits April 10, 2026 02:46

feat(utils): implement CE format dispatch for ExportAuditReport

19ac482

fix(report): fix BUG-001,003,005,007 in multi-format export (CE part)

a01d8a0

fix(utils): return error for unsupported export format instead of CSV…

8f6d839

… fallback (CE part)

actiontech-bot requested a review from winfredLIN April 10, 2026 03:26