Skip to content

Commit c972471

Browse files
authored
test(kubevirt): Add eval configs directly to kubevirt tasks (#691)
Provide some basic Evals for use with the KubeVirt tasks. Eventually gevals should allow agent and taskset configuration to be provided separately. Signed-off-by: Lee Yarwood <lyarwood@redhat.com>
1 parent a32b99b commit c972471

2 files changed

Lines changed: 31 additions & 0 deletions

File tree

Lines changed: 15 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,15 @@
1+
kind: Eval
2+
metadata:
3+
name: "kubevirt-basic-operations"
4+
config:
5+
agent:
6+
type: "builtin.claude-code"
7+
mcpConfigFile: ../../../mcp-config.yaml
8+
taskSets:
9+
- glob: ../*/task.yaml
10+
assertions:
11+
toolsUsed:
12+
- server: kubernetes
13+
toolPattern: ".*"
14+
minToolCalls: 1
15+
maxToolCalls: 20
Lines changed: 16 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,16 @@
1+
kind: Eval
2+
metadata:
3+
name: "kubevirt-basic-operations"
4+
config:
5+
agent:
6+
type: "builtin.openai-agent"
7+
model: "gemini-2.0-flash"
8+
mcpConfigFile: ../../../mcp-config.yaml
9+
taskSets:
10+
- glob: ../*/task.yaml
11+
assertions:
12+
toolsUsed:
13+
- server: kubernetes
14+
toolPattern: ".*"
15+
minToolCalls: 1
16+
maxToolCalls: 20

0 commit comments

Comments
 (0)