Add watchlist ConditionalAccessBenignStatusCodes and Update the Conditional Access bypass analytic rule for Microsoft Entra Id by v-kasghosh · Pull Request #14016 · Azure/Azure-Sentinel

v-kasghosh · 2026-04-08T10:02:33Z

Change(s):

See guidance below- Added guidance for 'ConditionalAccessBenignStatusCodes' watchlist
Updated query to include optional watchlist filtering (commented out by default)
Added documentation note about legitimate failures and tuning thresholds

Reason for Change(s):

"Attempt to bypass conditional access rule" (BypassCondAccessRule.yaml) triggers false positives on benign Entra AADSTS error codes (e.g., 50076, 70043) #13815

Version Updated:

1.0.8
-3.0.10

- Added guidance for 'ConditionalAccessBenignStatusCodes' watchlist - Updated query to include optional watchlist filtering (commented out by default) - Added documentation note about legitimate failures and tuning thresholds

contentautomationbot · 2026-04-08T10:02:58Z

Hello how are you I am GitHub bot
😀😀
I see that you changed templates under the detections/analytic rules folder. Did you remember to update the version of the templates you changed?
If not, and if you want customers to be aware that a new version of this template is available, please update the version property of the template you changed.

Copilot

Pull request overview

Note

Copilot was unable to run its full agentic suite in this review.

Adds a new Microsoft Sentinel watchlist for benign Conditional Access status codes and updates the “Bypass conditional access rule(s)” analytic rule to optionally filter those known-benign failures and include tuning guidance.

Changes:

Added ConditionalAccessBenignStatusCodes watchlist (ARM template + CSV) for reducing false positives.
Updated BypassCondAccessRule to include optional (commented) watchlist-based filtering and expanded rule guidance.
Updated solution manifest to include the new watchlist and updated solution UI/package assets.

Reviewed changes

Copilot reviewed 6 out of 8 changed files in this pull request and generated 4 comments.

Show a summary per file

File	Description
Solutions/Microsoft Entra ID/Watchlists/ConditionalAccessBenignStatusCodes.json	Adds an ARM template to deploy the new watchlist into Sentinel.
Solutions/Microsoft Entra ID/Watchlists/ConditionalAccessBenignStatusCodes.csv	Provides the initial list of benign status codes for the watchlist.
Solutions/Microsoft Entra ID/Package/testParameters.json	Updates packaging test parameters for the watchlist (ignored from review per repo guidance).
Solutions/Microsoft Entra ID/Package/createUiDefinition.json	Updates installer UI text and adds a Watchlists blade (ignored from review per repo guidance).
Solutions/Microsoft Entra ID/Data/Solution_AAD.json	Registers the new watchlist in the solution content manifest.
Solutions/Microsoft Entra ID/Analytic Rules/BypassCondAccessRule.yaml	Adds optional watchlist filtering and expands rule description/tuning guidance.

Copilot · 2026-04-09T08:15:11Z

Solutions/Microsoft Entra ID/Data/Solution_AAD.json

 		"Solutions/Microsoft Entra ID/Playbooks/Revoke-AADSignInSessions/entity-trigger/azuredeploy.json"
 	],
+	"Watchlists": [
+      "Watchlists/ConditionalAccessBenignStatusCodes.json"


The watchlist path is not consistent with the rest of this solution manifest (which uses paths like Solutions/Microsoft Entra ID/...) while BasePath is set to the repo root. As written, this likely resolves to C:\\GitHub\\Azure-Sentinel\\Watchlists\\... (nonexistent) and will break packaging/validation. Use the repo-root-relative path Solutions/Microsoft Entra ID/Watchlists/ConditionalAccessBenignStatusCodes.json, or change BasePath to the solution folder and update all other component paths accordingly.

Suggested change

"Watchlists/ConditionalAccessBenignStatusCodes.json"

"Solutions/Microsoft Entra ID/Watchlists/ConditionalAccessBenignStatusCodes.json"

Copilot · 2026-04-09T08:15:11Z

Solutions/Microsoft Entra ID/Data/Solution_AAD.json

 	"BasePath": "C:\\GitHub\\Azure-Sentinel",
 	"Version": "3.3.6",


PR description states the solution version was updated to 3.0.10, but Solution_AAD.json still shows 3.3.6. Please align the PR description with the actual version bump, or update the solution manifest version to match the stated release version.

Copilot · 2026-04-09T08:15:12Z

Solutions/Microsoft Entra ID/Watchlists/ConditionalAccessBenignStatusCodes.json

+        {
+        "name": "[concat(parameters('workspaceName'), '/Microsoft.SecurityInsights/ConditionalAccessBenignStatusCodes')]",
+        "type": "Microsoft.OperationalInsights/workspaces/providers/Watchlists",
+        "kind": "",


The watchlist ARM resource sets kind to an empty string. This is likely invalid for deployment/validation and can cause the template to fail. Remove the kind property entirely (if not required by the API version) or set it to the correct expected value for a watchlist resource.

Suggested change

"kind": "",

This is fair comment.

Copilot · 2026-04-09T08:15:12Z

Solutions/Microsoft Entra ID/Analytic Rules/BypassCondAccessRule.yaml

+  ConditionalAccessStatus == 3 // unknown
+  Note:
+  This analytic rule is designed to identify potential attempts to bypass Conditional Access policies. Depending on your organization's configuration, some Conditional Access failures may be legitimate and expected.
+  It is recommended to review alerts generated by this rule in the context of your organization's Conditional Access policies and adjust the detection threshold as needed to reduce false positives.
+  This rule includes an optional filter that leverages a watchlist (ConditionalAccessBenignStatusCodes) to exclude known benign status codes. You can customize this watchlist with status codes that are expected in your environment and do not indicate malicious activity.
+  By default, the watchlist-based filtering is commented out to provide full visibility into all relevant events. You may choose to enable this filter by uncommenting the corresponding line in the query to reduce noise from known benign scenarios.'


The description field is now very long and ends with a stray apostrophe. Analytic rule templates typically enforce description constraints (e.g., max length / limited sentences), so this may fail repo validation. Please (1) remove the trailing ', and (2) shorten the description to fit the repo’s detection-template requirements (keep it to a few sentences); consider moving the extended tuning guidance into a more concise summary or other appropriate documentation location.

rahul0216

Please address inline comment.

rahul0216 · 2026-04-09T14:49:40Z

Solutions/Microsoft Entra ID/Watchlists/ConditionalAccessBenignStatusCodes.json

+        {
+        "name": "[concat(parameters('workspaceName'), '/Microsoft.SecurityInsights/ConditionalAccessBenignStatusCodes')]",
+        "type": "Microsoft.OperationalInsights/workspaces/providers/Watchlists",
+        "kind": "",


This is fair comment.

Copilot

Pull request overview

Copilot reviewed 7 out of 9 changed files in this pull request and generated 2 comments.

Solutions/Microsoft Entra ID/ReleaseNotes.md

Copilot · 2026-04-13T13:42:22Z

Solutions/Microsoft Entra ID/Data/Solution_AAD.json

 	],
+	"Watchlists": [
+      "Watchlists/ConditionalAccessBenignStatusCodes.json"
+    ],
 	"BasePath": "C:\\GitHub\\Azure-Sentinel",
 	"Version": "3.3.6",


This PR updates solution content and adds a new Watchlist, but Solution_AAD.json still has \"Version\": \"3.3.6\" while ReleaseNotes.md adds an entry for 3.3.11 and the PR description mentions different versioning. Please align the solution data Version with the version being released/documented in release notes for this change.

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

add watchlist, update the analytic rule

8e4bb74

- Added guidance for 'ConditionalAccessBenignStatusCodes' watchlist - Updated query to include optional watchlist filtering (commented out by default) - Added documentation note about legitimate failures and tuning thresholds

contentautomationbot bot added the Content-Package label Apr 8, 2026

v-maheshbh assigned v-shukore Apr 8, 2026

v-maheshbh added the Solution Solution specialty review needed label Apr 8, 2026

v-sabiraj requested a review from Copilot April 9, 2026 08:13

Copilot AI reviewed Apr 9, 2026

View reviewed changes

v-kasghosh assigned rahul0216 Apr 9, 2026

rahul0216 reviewed Apr 9, 2026

View reviewed changes

v-kasghosh added 2 commits April 13, 2026 16:16

Merge branch 'master' into v-kasghosh/issues_number/13815

daae9a9

Updated to Latest Zip 3.3.11

56c7089

v-kasghosh marked this pull request as ready for review April 13, 2026 13:33

v-kasghosh requested review from a team as code owners April 13, 2026 13:33

v-kasghosh requested review from Copilot and rahul0216 April 13, 2026 13:40

Copilot AI reviewed Apr 13, 2026

View reviewed changes

Update Solutions/Microsoft Entra ID/ReleaseNotes.md

5e4eef4

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add watchlist ConditionalAccessBenignStatusCodes and Update the Conditional Access bypass analytic rule for Microsoft Entra Id#14016

Add watchlist ConditionalAccessBenignStatusCodes and Update the Conditional Access bypass analytic rule for Microsoft Entra Id#14016
v-kasghosh wants to merge 4 commits intomasterfrom
v-kasghosh/issues_number/13815

v-kasghosh commented Apr 8, 2026

Uh oh!

contentautomationbot bot commented Apr 8, 2026

Uh oh!

Copilot AI left a comment

Uh oh!

Copilot AI Apr 9, 2026

Uh oh!

Copilot AI Apr 9, 2026

Uh oh!

Copilot AI Apr 9, 2026

Uh oh!

rahul0216 Apr 9, 2026

Uh oh!

Copilot AI Apr 9, 2026

Uh oh!

rahul0216 left a comment

Uh oh!

rahul0216 Apr 9, 2026

Uh oh!

Copilot AI left a comment

Uh oh!

Uh oh!

Copilot AI Apr 13, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

	"Watchlists/ConditionalAccessBenignStatusCodes.json"
	"Solutions/Microsoft Entra ID/Watchlists/ConditionalAccessBenignStatusCodes.json"

		"BasePath": "C:\\GitHub\\Azure-Sentinel",
		"Version": "3.3.6",

Conversation

v-kasghosh commented Apr 8, 2026

Uh oh!

contentautomationbot bot commented Apr 8, 2026

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Copilot AI Apr 9, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Apr 9, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Apr 9, 2026

Choose a reason for hiding this comment

Uh oh!

rahul0216 Apr 9, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Apr 9, 2026

Choose a reason for hiding this comment

Uh oh!

rahul0216 left a comment

Choose a reason for hiding this comment

Uh oh!

rahul0216 Apr 9, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Copilot AI Apr 13, 2026

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants