Skip to content

[HDX-2300] introduce Shared Filters for team-wide filter visibility and discoverability#2047

Open
brandon-pereira wants to merge 24 commits intomainfrom
brandon/global-pinned-filters
Open

[HDX-2300] introduce Shared Filters for team-wide filter visibility and discoverability#2047
brandon-pereira wants to merge 24 commits intomainfrom
brandon/global-pinned-filters

Conversation

@brandon-pereira
Copy link
Copy Markdown
Member

@brandon-pereira brandon-pereira commented Apr 2, 2026

Summary

Introduces a "Shared Filters" feature (in addition to locally pinned items in HyperDX.

Especially helpful for teams with lots of filters and team members, allows users to highlight the top filters easily for all members.

This has been one of the most requested features we have received from enterprise customers. \

Note: - currently any user on a team can modify shared filters - we may want/need to introduce role limits to this, but that is oos

Screenshots or video

export-1775685397652.mp4

How to test locally or on Vercel

  1. Start the dev server (yarn dev)
  2. Navigate to the Search page
  3. Pin a filter field using the 📌 icon on any filter group header —you should be asked to pin (existing) or add to shared filters.
  4. Share a specific value by hovering over a filter checkbox row and clicking the pin icon — it should also appear in Shared Filters
  5. Reload the page — pins should persist (MongoDB-backed)
  6. Open a second browser/incognito window with the same team — pins should be visible there too
  7. Click the ⚙ gear icon next to "Filters" — toggle "Show Shared Filters" off/on
  8. Click "Reset Shared Filters" in the gear popover to clear all team pins

References

- Add PinnedFilter Mongoose model with team+source+user compound index
- Add GET/PUT /pinned-filters API routes with Zod validation
- Replace localStorage-based pinned filters with MongoDB-backed TanStack Query hooks
- Add SharedFilters UI section showing team-pinned fields at top of filter sidebar
- Add FilterSettingsPanel popover with show/hide toggle and reset button
- Add optimistic local state to prevent stale reads during rapid toggles
- Add one-time localStorage → MongoDB migration for existing users
- Add local-mode fallback for development without API
- Remove pinned fields from main filters list when shown in SharedFilters
- Persist 'Show Shared Filters' preference in localStorage
- Add integration tests for pinned filters API (14 tests)
- Add unit tests for merge logic and migration (12 tests)
- Add E2E tests for shared filters UI flow (8 tests)
@brandon-pereira brandon-pereira added the ai-generated AI-generated content; review carefully before merging. label Apr 2, 2026
@vercel
Copy link
Copy Markdown

vercel bot commented Apr 2, 2026

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated (UTC)
hyperdx-oss Ready Ready Preview, Comment Apr 10, 2026 5:54pm

Request Review

@changeset-bot
Copy link
Copy Markdown

changeset-bot bot commented Apr 2, 2026

🦋 Changeset detected

Latest commit: 8ab5199

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 4 packages
Name Type
@hyperdx/common-utils Minor
@hyperdx/api Minor
@hyperdx/app Minor
@hyperdx/otel-collector Minor

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

@github-actions
Copy link
Copy Markdown
Contributor

github-actions bot commented Apr 2, 2026

E2E Test Results

All tests passed • 141 passed • 3 skipped • 1060s

Status Count
✅ Passed 141
❌ Failed 0
⚠️ Flaky 0
⏭️ Skipped 3

Tests ran across 4 shards in parallel.

View full report →

@brandon-pereira brandon-pereira requested review from a team, fleon and wrn14897 and removed request for a team and wrn14897 April 2, 2026 20:34
@brandon-pereira brandon-pereira marked this pull request as ready for review April 2, 2026 20:34
@github-actions
Copy link
Copy Markdown
Contributor

github-actions bot commented Apr 2, 2026

PR Review

Good overall implementation — source ownership verification, Zod validation, and optimistic UI with proper sourceId keying all look correct. A few things worth addressing:

  • ⚠️ Stale debounce timeout on sourceId change (searchFilters.tsx) → The cleanup useEffect has [] deps, so it only fires on unmount. If the user switches sources mid-debounce, the old 300ms timer fires setOptimisticTeam(null), which clears any optimistic state the new source may have set. Add sourceId to the useEffect deps array so the timeout is cancelled on source change:

    useEffect(() => {
      return () => { /* cleanup */ };
    }, [sourceId]); // ← add sourceId
  • ⚠️ unshareField E2E helper silently breaks for nested facets (FilterComponent.ts) → getByTestId(\shared-filter-group-${filterName}`)only matches non-grouped facets; nested ones get testidshared-nested-filter-group-${filterName}. The tests pass today because SeverityTextis non-nested, but the helper is a hidden trap. Consider handling both testid patterns or scoping withfilter-group-+ fallback tonested-filter-group-`.

  • ⚠️ Cross-team isolation has no automated coverage → The PR comment acknowledges this (register returns 409). For a feature that writes team-wide shared state to MongoDB, consider tracking this as a follow-up test/story rather than just a code comment. At minimum, a note in the PR or a linked issue would help ensure it doesn't get forgotten.

  • ℹ️ DBSearchPageFilters.tsx is now ~1800+ lines → Per AGENTS.md, files should stay under 300 lines. The renderFacetList callback alone is ~100 lines inside the component. Consider splitting this into a dedicated file (e.g. FacetList.tsx) as a follow-up.

- Fix integration test team scoping by inserting PinnedFilter directly
  in MongoDB instead of trying to register a second user (409 Conflict)
- Fix E2E shared-filters test isolation by resetting pinned filters via
  API in beforeEach so state doesn't leak between tests
- Add title attribute to field pin ActionIcons so E2E pinField selector
  can find them
- Use dispatchEvent in scrollAndClick to avoid flaky CSS hover state
- Replace waitForTimeout with expect.poll and Playwright assertions
  for proper wait mechanisms
return PinnedFilterModel.findOne({
team: new mongoose.Types.ObjectId(teamId),
source: new mongoose.Types.ObjectId(sourceId),
user: null,
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I see that the code includes the capability to store user-level pinned filters, but I don't see that implemented anywhere.

IMO, we shouldn't remove user-specific pinned filters, since different users are likely to care about different filter keys and values. Having no special UI for "pin for team" vs "pin for me" is likely to result in various teammates modifying the filters for the entire team without realizing they're doing so.

It's also odd that the migration from local --> team/db filters is automatic, and applies to the entire team. The first user to login after deploying this will have their filters set for the entire team, possibly unexpectedly.

I would suggest we implement both user-level and team-level filters, and have the migration create user-level pinned filters.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@pulpdrew you're right, I was trying to keep the scope small and minimize UX changes, but it didn't make sense.

I have changed the UX of this so that clicking the pin icons opens a menu where you can select between pin and share. I have updated the PR description with a video.

Previously localStorage was still used for local mode, but now localStorage is used for pinning and mongo is used for sharing with team.

I have also removed the migration since it's not needed.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Makes sense, though with user-specific pinned filters always using local storage, should we remove the user field from this model now?

@github-actions github-actions bot added the review/tier-4 Critical — deep review + domain expert sign-off label Apr 6, 2026
@github-actions
Copy link
Copy Markdown
Contributor

github-actions bot commented Apr 6, 2026

🔴 Tier 4 — Critical

Touches auth, data models, config, tasks, OTel pipeline, ClickHouse, or CI/CD.

Why this tier:

  • Large diff: 1772 lines changed (threshold: 1000)

Additional context: touches API routes or data models

Review process: Deep review from a domain expert. Synchronous walkthrough may be required.
SLA: Schedule synchronous review within 2 business days.

Stats
  • Files changed: 20
  • Lines changed: 1772 (+ 665 in test files, excluded from tier calculation)
  • Branch: brandon/global-pinned-filters
  • Author: brandon-pereira

To override this classification, remove the review/tier-4 label and apply a different review/tier-* label. Manual overrides are preserved on subsequent pushes.

@brandon-pereira brandon-pereira requested a review from pulpdrew April 8, 2026 22:28
@brandon-pereira brandon-pereira changed the title [HDX-2300] Store pinned filters in MongoDB for team-wide sharing [HDX-2300] introduce Shared Filters for team-wide filter visibility and discoverability Apr 8, 2026
@pulpdrew
Copy link
Copy Markdown
Contributor

The behavior is odd for shared filters that don't load in the initial round of filters. Clicking Load More doesn't work, until clicking More filters in the non-shared section. See ScopeName here

Screen.Recording.2026-04-10.at.9.13.50.AM.mov

Copy link
Copy Markdown
Contributor

@pulpdrew pulpdrew left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I like this approach much better - just a couple more comments

@brandon-pereira brandon-pereira removed ai-generated AI-generated content; review carefully before merging. waiting-on-author labels Apr 10, 2026
@brandon-pereira
Copy link
Copy Markdown
Member Author

The behavior is odd for shared filters that don't load in the initial round of filters. Clicking Load More doesn't work, until clicking More filters in the non-shared section. See ScopeName here

Screen.Recording.2026-04-10.at.9.13.50.AM.mov

Screenshot 2026-04-10 at 11 42 15 AM

Good catch - now shared filters will always be fetched on load

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

review/tier-4 Critical — deep review + domain expert sign-off

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants