feat(openapi): Dynamic config for /poll, based on application.yaml

[skrydal]

Topics should be (for now) controlled via application.yaml. A new field is added to the list of topics pollEnabled, which controls whether topic can be polled through the OpenAPI. This contrasts old state where those topics were hardcoded in the service code. The PR also introduces a new topic for work events, as well as, a generic privilege to allow polling any topic.

[codecov]

Codecov Report

❌ Patch coverage is 38.88889% with 11 lines in your changes missing coverage. Please review.
✅ All tests successful. No failed tests found.

Files with missing lines	Patch %	Lines
...ms/factory/event/ExternalEventsServiceFactory.java	0.00%	10 Missing ⚠️
...din/metadata/config/kafka/TopicsConfiguration.java	50.00%	1 Missing ⚠️

❌ Your patch status has failed because the patch coverage (38.88%) is below the target coverage (75.00%). You can increase the patch coverage or adjust the target coverage.

📢 Thoughts on this report? Let us know!

[github-actions]

Linear: CAT-1558

[chakru-r]

A doc update may be helpful since the display name can now be modified and the events poll api needs to use the configured name (which earlier was hardcoded).Though the defaults ensure things remain exactly same as before.

[chakru-r]

Need a displayName here so that the API user can use a static name without having to use the actual topic name that has a prefix. This replaces the hardcoded constants we had. Without this, all clients polling need to use a deployment specific name (instead of a static MetadataChangeLog_Timeseries_v1 name that can work across deployments

[chakru-r]

i am wondering if externalName is a better name for this property -- the name that clients or API polling users reference this topic

[skrydal]

I have made a change to use displayName, now with settings:

WORK_EVENT_TOPIC_NAME=2391478-acme_WorkEvent_v1

When I try to poll topics (via the API) I get results:

• PlatformEvent_v1 -> HTTP 200
• WorkEvent_v1 -> HTTP 200
• 2391478-acme_WorkEvent_v1 -> HTTP 500 (Unsupported events topic provided)
• MetadataChangeLog_Versioned_v1 -> HTTP 200
• MetadataChangeLog_Timeseries_v1 -> HTTP 200

With topics:

❯ kcl topic list | awk '{print $1}'
NAME
2391478-acme_WorkEvent_v1
DataHubUpgradeHistory_v1
DataHubUsageEvent_v1
FailedMetadataChangeProposal_v1
MetadataChangeEvent_v4
MetadataChangeLog_Timeseries_v1
MetadataChangeLog_Versioned_v1
MetadataChangeProposal_v1
PlatformEvent_v1

[codecov]

Bundle Report

Changes will increase total bundle size by 23.78MB (100.0%) ⬆️⚠️, exceeding the configured threshold of 5%.

Bundle name	Size	Change
datahub-react-web-esm	23.78MB	23.78MB (100%) ⬆️⚠️