Add AKS deployment tests using aspire deploy pipeline (C# + TypeScript)#16077
Add AKS deployment tests using aspire deploy pipeline (C# + TypeScript)#16077mitchdenny wants to merge 5 commits intomainfrom
Conversation
|
/deployment-test |
|
🚀 Dogfood this PR with:
curl -fsSL https://raw.githubusercontent.com/microsoft/aspire/main/eng/scripts/get-aspire-cli-pr.sh | bash -s -- 16077Or
iex "& { $(irm https://raw.githubusercontent.com/microsoft/aspire/main/eng/scripts/get-aspire-cli-pr.ps1) } 16077" |
|
🚀 Deployment tests starting on PR #16077... This will deploy to real Azure infrastructure. Results will be posted here when complete. |
| { | ||
| var (promptText, value) = parameterResponses[i]; | ||
|
|
||
| await auto.WaitUntilTextAsync(promptText, timeout: TimeSpan.FromMinutes(5)); |
There was a problem hiding this comment.
Pull request overview
Adds a new AKS end-to-end deployment test that validates the full aspire deploy Kubernetes/Helm pipeline (build → push → chart gen → deploy) against real Azure infrastructure, and introduces a small terminal-automation helper to drive interactive aspire deploy parameter prompts.
Changes:
- Added
AksDeployStarterDeploymentTestsE2E test that provisions AKS+ACR, deploys the Aspire starter viaaspire deploy, verifies endpoints, and cleans up resources. - Added
AspireDeployInteractiveAsyncterminal automator helper for answering interactive deploy parameter prompts and waiting for pipeline completion.
Reviewed changes
Copilot reviewed 2 out of 2 changed files in this pull request and generated 3 comments.
| File | Description |
|---|---|
| tests/Aspire.Deployment.EndToEnd.Tests/Helpers/DeploymentE2EAutomatorHelpers.cs | Adds reusable terminal automation helper to run aspire deploy interactively in deployment E2E tests. |
| tests/Aspire.Deployment.EndToEnd.Tests/AksDeployStarterDeploymentTests.cs | New AKS E2E test that exercises the integrated aspire deploy pipeline end-to-end and validates deployed services. |
| } | ||
|
|
||
| // Wait for pipeline completion | ||
| await auto.WaitUntilTextAsync("PIPELINE SUCCEEDED", timeout: timeout); |
There was a problem hiding this comment.
In this reusable helper, waiting only for "PIPELINE SUCCEEDED" means a failed deploy will typically burn the full timeout before surfacing, making failures slower and harder to diagnose. Consider waiting for either "PIPELINE SUCCEEDED" or "PIPELINE FAILED" and throwing immediately when the failed marker is observed (similar to other deployment E2E tests).
| await auto.WaitUntilTextAsync("PIPELINE SUCCEEDED", timeout: timeout); | |
| var pipelineSucceededPattern = new CellPatternSearcher().Find("PIPELINE SUCCEEDED"); | |
| var pipelineFailedPattern = new CellPatternSearcher().Find("PIPELINE FAILED"); | |
| var pipelineFailed = false; | |
| await auto.WaitUntilAsync( | |
| s => | |
| { | |
| if (pipelineFailedPattern.Search(s).Count > 0) | |
| { | |
| pipelineFailed = true; | |
| return true; | |
| } | |
| return pipelineSucceededPattern.Search(s).Count > 0; | |
| }, | |
| timeout: timeout, | |
| description: "pipeline completion"); | |
| if (pipelineFailed) | |
| { | |
| throw new InvalidOperationException("Deployment pipeline failed."); | |
| } |
| /// <summary> | ||
| /// Runs <c>aspire deploy</c> interactively, answering parameter prompts via terminal automation. | ||
| /// The deploy pipeline handles image building, pushing, Helm chart generation, and deployment. | ||
| /// </summary> | ||
| /// <param name="auto">The terminal automator.</param> | ||
| /// <param name="counter">Sequence counter for prompt tracking.</param> | ||
| /// <param name="parameterResponses"> | ||
| /// Ordered list of (promptSubstring, valueToType) tuples. | ||
| /// Each entry matches by the parameter name appearing in the prompt text. | ||
| /// Entries are consumed in order — first match wins. | ||
| /// </param> | ||
| /// <param name="pipelineTimeout">Timeout for the entire deploy pipeline. Defaults to 15 minutes.</param> | ||
| internal static async Task AspireDeployInteractiveAsync( | ||
| this Hex1bTerminalAutomator auto, | ||
| SequenceCounter counter, | ||
| IReadOnlyList<(string PromptText, string Value)> parameterResponses, | ||
| TimeSpan? pipelineTimeout = null) | ||
| { |
There was a problem hiding this comment.
This method largely duplicates AspireDeployInteractiveAsync already present in tests/Aspire.Cli.EndToEnd.Tests/Helpers/KubernetesDeployTestHelpers.cs. If both projects need the same behavior, consider moving the shared implementation to tests/Shared (or another common helper) to avoid future drift (e.g., prompt handling, failure detection, timeouts).
| // Configure container registry for image push | ||
| var registryEndpoint = builder.AddParameter("registryendpoint"); | ||
| var registry = builder.AddContainerRegistry("registry", registryEndpoint); | ||
|
|
||
| // Add Kubernetes environment with Helm deployment | ||
| builder.AddKubernetesEnvironment("k8s") | ||
| .WithHelm(helm => | ||
| { | ||
| helm.WithNamespace(builder.AddParameter("namespace")); | ||
| helm.WithChartVersion(builder.AddParameter("chartversion")); | ||
| }); |
There was a problem hiding this comment.
The injected AppHost.cs snippet declares var registry = builder.AddContainerRegistry(...) but never uses registry. If the generated template/project treats warnings as errors, this unused local could break the deploy build. Consider either dropping the assignment (just call AddContainerRegistry without storing the result) or using the registry (e.g., via .WithContainerRegistry(registry) where appropriate).
|
/deployment-test |
|
🚀 Deployment tests starting on PR #16077... This will deploy to real Azure infrastructure. Results will be posted here when complete. |
|
/deployment-test |
|
🚀 Deployment tests starting on PR #16077... This will deploy to real Azure infrastructure. Results will be posted here when complete. |
|
Re-running the failed jobs in the CI workflow for this pull request because 1 job was identified as retry-safe transient failures in the CI run attempt.
|
|
/deployment-test |
|
🚀 Deployment tests starting on PR #16077... This will deploy to real Azure infrastructure. Results will be posted here when complete. |
Add two new E2E deployment tests that deploy Aspire applications to Azure Kubernetes Service using the full 'aspire deploy' pipeline instead of the manual 'aspire publish' + helm install workflow. AksDeployStarterDeploymentTests (C# AppHost): - Deploys the standard Aspire starter template to AKS - Uses AddContainerRegistry + AddKubernetesEnvironment with Helm - Verifies apiservice and webfrontend endpoints via port-forward AksDeployTypeScriptDeploymentTests (TypeScript AppHost): - Deploys the Express/React starter template to AKS - Modifies apphost.ts with addContainerRegistry + addKubernetesEnvironment - Runs aspire restore to regenerate TypeScript SDK with K8s types - Verifies the bundled app service endpoint via port-forward Both tests exercise the end-to-end aspire deploy flow for Kubernetes: provision AKS + ACR, create project, run 'aspire deploy' (which builds images, pushes to ACR, generates Helm charts, and deploys), then verify services respond. Also adds a reusable AspireDeployInteractiveAsync helper to DeploymentE2EAutomatorHelpers for future deployment tests. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Fix all deployment E2E test failures caused by CLI/bundle behavior drift: Bundle install (3 tests): PythonFastApi, AppServicePython, AcrPurgeTask were only sourcing the CLI binary but Python templates need the full bundle with AppHost server. Switched to InstallAspireBundleFromPullRequestAsync + SourceAspireBundleEnvironmentAsync. NuGet prompt (3 tests): TypeScriptExpress, TypeScriptVnetSqlServer, AcrPurgeTask waited for '(based on NuGet.config)' after aspire add, but bundle-based installs don't show this prompt. Added PR number guard or removed the wait entirely. Init flow (2 tests): AcaCompactNamingUpgrade and TypeScriptVnetSqlServer used hardcoded init prompt expectations that no longer match the CLI. Replaced with a flexible state machine that handles language selection, template version, agent init, and success prompts in any order. Output path prompt (1 test): AcaManagedRedis matched 'Enter the output path:' with trailing colon but the CLI text changed. Relaxed to match without the colon. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
|
/deployment-test |
|
🚀 Deployment tests starting on PR #16077... This will deploy to real Azure infrastructure. Results will be posted here when complete. |
Revert the GetPrNumber() guard on the NuGet version prompt wait. The '(based on NuGet.config)' prompt DOES appear when running aspire add from a PR bundle install, so we must always handle it in CI. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
|
/deployment-test |
|
🚀 Deployment tests starting on PR #16077... This will deploy to real Azure infrastructure. Results will be posted here when complete. |
TypeScript AppHost with bundle install: aspire add completes without a NuGet version selection prompt. The bundle provides package versions directly. This differs from Python projects which still show the prompt. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
|
/deployment-test |
|
🚀 Deployment tests starting on PR #16077... This will deploy to real Azure infrastructure. Results will be posted here when complete. |
The CLI now prompts 'Would you like to configure AI agent environments?' after aspire new completes. This test uses the SequenceBuilder pattern and was missing the agent init prompt handling, causing it to hang. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
|
/deployment-test |
|
🚀 Deployment tests starting on PR #16077... This will deploy to real Azure infrastructure. Results will be posted here when complete. |
|
🎬 CLI E2E Test Recordings — 68 recordings uploaded (commit View recordings
📹 Recordings uploaded automatically from CI run #24304301289 |
|
❌ Deployment E2E Tests failed — 23 passed, 9 failed, 0 cancelled View test results and recordings
|
JamesNK
left a comment
JamesNK
left a comment
There was a problem hiding this comment.
2 issues found (1 potential bug, 1 resource leak).
| .Type("n").Enter() | ||
| .WaitForSuccessPrompt(counter, TimeSpan.FromMinutes(1)); | ||
|
|
||
| // Step 4: Navigate to project directory |
There was a problem hiding this comment.
The WaitUntil matches on either the agent init prompt or the success prompt. But .Wait(500).Type("n").Enter() runs unconditionally. If aspire new completes without showing the agent init prompt (success prompt matches instead), typing "n" + Enter at the shell runs n as a command (exit code 127 / ERR), causing WaitForSuccessPrompt to time out.
This needs to check which condition was actually matched and only type "n" when the agent init prompt was seen.
|
|
||
| private async Task CleanupResourceGroupAsync(string resourceGroupName) | ||
| { | ||
| try |
There was a problem hiding this comment.
Process implements IDisposable and holds OS handles but is never disposed here. Should be:
using var process = new System.Diagnostics.Process { ... };This is a pre-existing pattern copied across ~16 deployment test classes — consider fixing it in the existing locations as well.
Description
Adds two new E2E deployment tests that deploy Aspire applications to Azure Kubernetes Service using the full
aspire deploypipeline.Motivation: We recently merged end-to-end
aspire deploysupport for Kubernetes (#15723). The existingAksStarterDeploymentTestsuses the manual workflow (aspire publish→dotnet publish /t:PublishContainer→helm install). These new tests validate that the integratedaspire deploypipeline works end-to-end against a real AKS cluster for both C# and TypeScript AppHosts.AksDeployStarterDeploymentTests (C# AppHost)
AddContainerRegistry+AddKubernetesEnvironmentwith Helmaspire deploywhich automatically builds images, pushes to ACR, generates Helm charts, and deployskubectl port-forwardAksDeployTypeScriptDeploymentTests (TypeScript AppHost)
Aspire.Hosting.Kubernetes, runsaspire restorefor TypeScript SDKapphost.tswithaddContainerRegistry+addKubernetesEnvironmentwith Helmaspire deployand verifies the bundled app service endpoint viakubectl port-forwardAlso adds a reusable
AspireDeployInteractiveAsynchelper toDeploymentE2EAutomatorHelpers.Validated by: Both tests pass in CI via
/deployment-teston this PR.Checklist