diff --git a/docs/README.md b/docs/README.md index f421eb3b1c..1428db561a 100644 --- a/docs/README.md +++ b/docs/README.md @@ -32,6 +32,7 @@ This section points to sources that explain why ServiceControl is designed the w - [Handling unavailable runtime dependencies](handling-unavailable-runtime-dependencies.md) — how instances react when a dependency is unavailable - [Telemetry](telemetry.md) — telemetry configuration and emitted metrics - [Throughput collection](throughput-collection.md) — why and how usage data is collected +- [Usage report contents](usage-report.md): every key the usage report carries, every setting it leaves out, and why ## Decisions and rationale diff --git a/docs/decisions/2026-10-01-usage-report-feature-coverage.md b/docs/decisions/2026-10-01-usage-report-feature-coverage.md new file mode 100644 index 0000000000..dd0ad91e82 --- /dev/null +++ b/docs/decisions/2026-10-01-usage-report-feature-coverage.md @@ -0,0 +1,90 @@ +# Usage report covers every customer choice + +- Date: 2026-10-01 +- Revised: 2026-10-06. Security configuration is no longer reported, and Particular deletes the security values it has already received. `SqlVersion` reports only the major version. Email notification details and licensed endpoint details are now out of scope. +- Status: Accepted +- Implementation: [#5945](https://github.com/Particular/ServiceControl/pull/5945). Link further pull requests here as they open. + +## Context + +The usage report tells Particular how customers run ServiceControl. The primary instance builds the report when a user downloads it from ServicePulse. The customer then sends the file to Particular. The report's `EnvironmentData` section is a flat dictionary of strings. Since 6.20.0 it has carried about 30 keys describing the host, the storage, security, a handful of features and retention. + +Those keys were added one at a time, each when someone needed an answer. A key only appears in reports from the release that adds it. So the first question about a feature always arrives with no data to answer it. The proposal to ingest audit messages in the primary was withdrawn in September 2026, largely for that reason. + +An audit on 1 October 2026 compared the report with every setting and runtime choice in the primary, audit and monitoring instances. The report carries about a third of what could be reported. The gaps include: + +- Error ingestion turned off, OTLP metrics export, and logging providers and level. +- The RabbitMQ queue type and routing topology. Whenever the transport has a broker throughput query, the report's `MessageTransport` carries only the broker family name. +- The transport connection options that change behaviour, such as the Azure Service Bus topology and the SQL Server queue schema. +- The runtime choices users make in ServicePulse, apart from email notifications being on or off. Heartbeat instance tracking, retry redirects and report masks are all missing. +- Every tuning number, including concurrency, batch sizes, timeouts and thresholds. +- Nearly everything about the audit and monitoring instances. + +The audit also found security settings missing from the report, such as CORS, forwarded headers, and the database and transport authentication modes. Whether to report them is part of this decision. + +The decision has to respect seven constraints: + +- The report names the licensee in `CustomerName`. Its queue list carries queue names and, on the SQL Server and PostgreSQL transports, a `Scope` of `[Database].[Schema]`. These are masked only when the customer configures report masks. Otherwise they are in clear text. `EnvironmentData` must add nothing that identifies the customer's infrastructure, people or data. Since August 2026 every value has been a fixed enum member, a boolean, a count, a number or a version. +- The report is signed, so a customer cannot remove one key and send the rest. Security reviewers in regulated organisations often classify security configuration as confidential. A reviewer who does would block the whole report, and with it the licensing data. +- Licensing has no need to know how a named customer secures its instances. +- Only the primary builds the report. The primary already fetches each audit instance's `GET /api/configuration` once a day. Its only link to the monitoring instance is the throughput message monitoring sends every five minutes, and the primary reads only that message's body. +- `EnvironmentData` accepts new keys without a change to the signed report schema in `Particular.LicensingComponent.Report`. Any other change to the report needs a new version of that package. +- A value that fails to read must not stop the report. The `ReadFailed` value already covers this. +- Analysis compares reports across versions. Renaming a key forces analysis to read both spellings, as the `Persistence.*` to `Storage.*` rename did. + +A breach of the store where Particular keeps reports is a real risk, but a secondary one for security settings. The values carry no host, URL or secret. Exploiting any of them needs network access to the instance or to the database, broker or storage it uses, and an attacker with that access can observe the same facts directly. + +## Decision + +Every setting or runtime choice that changes how ServiceControl behaves gets a key in `EnvironmentData`. The only exception is a setting that [the catalog](../usage-report.md) lists as excluded. An exclusion gives one of four reasons: + +- Identifies: the setting is a name, address, path or secret. +- Not a choice: the setting is a test hook, a dead setting, an action, or a mode in which the instance cannot build a report. +- Security configuration: the setting describes how the instance authenticates callers or connections, encrypts traffic, validates certificates or tokens, or restricts access. +- Out of scope: this decision excludes it. + +The report carries no security configuration. It says nothing about how the customer authenticates callers or connections, encrypts traffic, validates certificates or tokens, or restricts access. A setting that identifies and is also security configuration is excluded as Identifies. + +Values follow fixed shapes: + +- A switch reports `Enabled` or `Disabled`. A mode reports a fixed enum member. +- A runtime feature reports a count that shows whether it is in use, for example the number of retry redirects. +- A tuning number reports `Default` when the setting is absent from configuration. Otherwise it reports the configured number, in the unit the key name ends with. +- A value derived from an identifying setting goes through a fixed classifier or becomes a count. `DatabaseHostClassifier` is the model for a classifier. +- `SqlVersion` reports only the major version of the database engine, the same shape as `Storage.ServerVersion` on SQL Server and PostgreSQL. The full version text carries the patch level, edition and operating system build. That tells a reader which known vulnerabilities apply, and no analysis needs it. Azure SQL Database always reports version 12, which cannot be compared with SQL Server's version numbers. On Azure SQL Database, `SqlVersion` is `AzureSql` instead. + +Each component reports its own keys. The primary and each persister do it through an `IEnvironmentDataProvider`. A transport does it through `ITransportCustomization.GetEnvironmentData`, which returns nothing by default, and the primary's `TransportEnvironmentDataProvider` adds what it returns to the report. Runtime choices are read from storage when the report is built. + +Audit instance keys come only from the `GET /api/configuration` response the primary already fetches. That response gives retention, audit forwarding, maximum body size, log level and storage type. Several audit instances combine into one key per fact. A number reports the largest value, and an enum reports `Mixed` when the instances differ. + +The monitoring instance is out of scope. `MonitoringEnabled` stays as it is, and the catalog documents what the key means. + +Email notification details beyond on or off are out of scope. The question is whether notifications are in use, and `Features.EmailNotifications` already answers it. Licensed endpoint details are out of scope as well. They apply only to Endpoint Size licenses with endpoint metadata, and no analysis needs them. + +[`docs/usage-report.md`](../usage-report.md) is the single list of what the report contains. It names every key, its values and its source. It also names every excluded setting and the reason. A pull request that adds or changes a setting updates the catalog in the same change. A missing entry is a review finding. The gaps from the audit that this decision covers are listed in the catalog as `Planned` rows. + +## Consequences + +- The report grows. A typical instance goes from about 30 `EnvironmentData` keys to about 65. Keys specific to a transport or a storage engine appear only on that transport or engine. The extra size is a few kilobytes in a file that is already zipped. +- Every new setting costs a little more. It needs a key or an exclusion, a catalog row, and a review against the privacy and security rules. Accepted. The four exclusion reasons keep that review short. +- Reporting `Default` needs the instance to know whether a setting was configured. Several settings parse straight to an effective value with the default folded in, `HeartbeatGracePeriod` for example. Each of those needs a small change to keep that information. Installers write some settings explicitly, such as `ShutdownTimeout`. Those report the installer's value, so analysis has to know the installer defaults. +- Transports report through a second mechanism. `ServiceControl.Transports` is strong-named and `Particular.LicensingComponent.Contracts` is not, so the transports do not reference `IEnvironmentDataProvider`. They return their keys from `ITransportCustomization.GetEnvironmentData` instead, and the primary wraps them. Accepted. The keys follow the same rules and land in the same dictionary. +- Particular has no report data on security configuration. That includes how many customers keep the insecure defaults. Accepted. Evidence for changing a secure default has to come from another source. +- Reports from 6.20.0 and 6.21.0 carry `Security.Authentication`, `Security.RoleBasedAuthorization`, `Security.Https` and `Persistence.BodyStorage.Auth`. Later versions do not. Particular deletes these values from the reports it has already received, and from reports that still arrive from instances on those versions. Instances that stay on 6.20.0 or 6.21.0 keep sending the keys until they upgrade, so a security reviewer can still object to their reports. +- `SqlVersion` changes shape but keeps its name. Reports up to 6.21.0 carry the full `@@VERSION` or `version()` text. Later reports carry the major version. Analysis reads the major version, or `AzureSql`, from either shape. +- Audit coverage stays thin. On the audit side, ingestion on or off, full-text search, embedded or external RavenDB, logging providers and OTLP stay invisible. Combining several audit instances also hides their differences behind `Mixed` or the largest value. Both are accepted for this decision. A dedicated audit environment endpoint has already been prototyped for the audit telemetry work. That endpoint is the channel if these facts are needed later. +- Monitoring stays invisible, and `MonitoringEnabled` stays misleading. The key is `False` when monitoring runs but no endpoint sends metrics. It stays `True` for up to 14 months after monitoring is removed. The catalog states what the key means, so analysis does not read it as "monitoring is installed". +- Some counts are imprecise. `Heartbeats.MonitoredInstances` cannot separate instances a user stopped monitoring from instances that never sent a heartbeat. The catalog says so. +- Each value is read when the user downloads the report, so the report holds no history over its window. Accepted. +- Older versions do not emit the new keys. Analysis treats a missing key as unknown, never as `Disabled` or zero. +- Error ingestion workers stay invisible. A worker started with `--error-ingestion-only` never builds a report. This gap was already accepted for the ingestion telemetry work. + +## Alternative approaches + +- Add keys on demand. This is how keys were added until now, and it costs nothing until a question comes up. It is rejected because data starts only at the release that adds the key. This is the main point of leverage. A key is cheap to add while the setting is being written. Waiting a year for the evidence is expensive. +- Report the whole settings object, minus a deny-list of identifying values. This gives full coverage for very little ongoing effort. It is rejected because a deny-list fails open: a new identifying setting would leak until someone noticed. The catalog fails closed, because a setting nobody reviewed is missing from the report. +- Enforce coverage with a convention test. A test could list every `SettingsReader` key and fail when one has neither a key nor an exclusion. It is rejected because many settings never pass through `SettingsReader`. Examples are environment variables read directly (`OTEL_EXPORTER_OTLP_ENDPOINT` and the integrated ServicePulse variables), transport connection string options, and runtime state in storage. A scanner would miss all of them and still look complete. The catalog and review cover them all, at the cost of depending on reviewers. +- Report security configuration per area, never per flag. This was the rule from 1 October 2026 until 6 October 2026. For example, `Security.TokenValidation` would be `Relaxed` when any token validation flag is off. It answers how often customers relax a protection without listing which flag a named customer turned off. It is rejected because an area-level key is still security configuration tied to a named customer. The signed report cannot be sent in part, so a reviewer who treats that as confidential blocks all of it. Licensing does not need the data either. +- Replace `CustomerName` with the license id, so that security keys could be reported. It is rejected because this is only pseudonymisation: Particular must map the id back to the customer for licensing. The report also identifies the customer in other ways. The zip and JSON file names are built from `CustomerName`. The report arrives from the customer's email domain. Queue names and the SQL Server and PostgreSQL `Scope` are in clear text unless the customer configured report masks. `NameHash` and `ScopeHash` are unsalted SHA-256 hashes of the unmasked value, so a guessed name can be confirmed. The `Report` model in `Particular.LicensingComponent.Report` 1.3.0 also has no license id field. +- A new audit endpoint for audit-side facts. It would cover full-text search and ingestion on or off. It is deferred because it needs an audit release, and audit instances already in the field would never report through it. The existing `/api/configuration` gives five facts from every audit version deployed today. +- A header on the monitoring throughput message. It is cheap, and older primaries ignore unknown headers. It is deferred because it needs a monitoring release. A monitoring instance with no monitored endpoints also sends no message. The header would therefore miss the same case where `MonitoringEnabled` reports `False` today. diff --git a/docs/usage-report.md b/docs/usage-report.md new file mode 100644 index 0000000000..445e5628ff --- /dev/null +++ b/docs/usage-report.md @@ -0,0 +1,258 @@ +# Usage report contents + +The primary instance builds the usage report when a user downloads it from ServicePulse (`GET api/licensing/report/file`). The customer sends the signed file to Particular. This page lists every key in the report's `EnvironmentData`, with its values and its source. The page also lists every setting that is deliberately left out, with the reason. The reasoning behind these rules is in [the coverage decision](decisions/2026-10-01-usage-report-feature-coverage.md). How usage data itself is collected is covered in [throughput-collection.md](throughput-collection.md). + +## Rules for a value + +The report names the licensee in `CustomerName`. Its queue list carries queue names and, on the SQL Server and PostgreSQL transports, a `Scope` of `[Database].[Schema]`. These are masked only when the customer configures report masks. Otherwise they are in clear text. `EnvironmentData` must add nothing that identifies the customer's infrastructure, people or data. That rules out host names, URLs, connection strings, paths, queue and endpoint names, addresses, user names, client ids and secrets. It also rules out a hash or prefix of any of them. + +`EnvironmentData` also says nothing about security configuration: how the customer authenticates callers or connections, encrypts traffic, validates certificates or tokens, or restricts access. + +A value is always one of these: + +- `Enabled` or `Disabled` for a switch. +- A fixed enum member for a mode. +- A count or a number. +- A version. + +Tuning numbers report `Default` when the setting is absent from configuration. Otherwise they report the configured number in invariant culture, in the unit the key name ends with. Installers write some settings explicitly, for example `ShutdownTimeout`. Those report the installer's value, not `Default`. + +A value derived from an identifying setting goes through a fixed classifier or becomes a count. `DatabaseHostClassifier` is the model for a classifier. + +Three values have a fixed meaning on every key: + +- `NotApplicable` means the feature the key describes is off or absent. +- `Unknown` means the instance cannot tell. +- `ReadFailed` means reading the value threw. The report is still generated. + +Audit keys combine every live audit instance. Numbers report the largest value. An enum reports `Mixed` when the instances differ. + +A key name is permanent. Analysis compares reports across versions, and a rename forces it to read both spellings. + +## Adding or changing a setting + +A pull request that adds a setting, a runtime choice, or a new mode of an existing setting does one of two things: + +1. It adds a key. That means an `IEnvironmentDataProvider` in the component that owns the setting, a row in the tables below, and an entry in `ExpectedKeys` in `When_reporting_the_environment` when the key is emitted on every storage. A transport adds its key to `GetEnvironmentData` on its transport customization instead of an `IEnvironmentDataProvider`. +2. It adds a row to [Not reported](#not-reported) with one of the four reasons. + +A pull request that does neither gets a review finding. + +## Reported keys + +`Status` is the first release that emits the key, `Unreleased` for a key on master that has not shipped, or `Planned` for a key this page commits to. Planned names and value sets are final at implementation review. + +### Versions and instance counts + +| Key | Values | Source | Status | +| --- | --- | --- | --- | +| `ServiceControlVersion` | version | the running primary | 5.4.0 | +| `ServicePulseVersion` | version | the `spVersion` query parameter ServicePulse sends | 5.4.0 | +| `AuditEnabled` | `True`, `False` | any audit throughput in the report window | 5.4.0 | +| `MonitoringEnabled` | `True`, `False` | any monitoring throughput in the report window | 5.4.0 | +| `RabbitMQVersion` | version | the broker throughput query on RabbitMQ | 5.4.0 | +| `SqlVersion` | major version, or `AzureSql` on Azure SQL Database | the broker throughput query on SQL Server and PostgreSQL | 5.4.0 | +| `Audit.ConfiguredInstances` | count | entries in `ServiceControl/RemoteInstances` | Unreleased | +| `Audit.LiveInstances` | count | remotes that answer as an audit instance | Unreleased | + +`MonitoringEnabled` does not mean a monitoring instance is installed. It means a monitoring instance delivered non-zero throughput to this primary on at least one day of the report window, which covers the last 14 months and excludes today. It is `False` when monitoring is installed but no endpoint sends metrics, or when the throughput queue names do not match. It stays `True` for up to 14 months after monitoring is removed. + +Releases up to 6.21.0 report `SqlVersion` as the full text of `@@VERSION` on SQL Server or `version()` on PostgreSQL. That text includes the patch level, edition and operating system build. Later releases report only the major version. Azure SQL Database always reports version 12, which cannot be compared with SQL Server's version numbers. On Azure SQL Database, `SqlVersion` is `AzureSql` instead. + +### Host + +| Key | Values | Source | Status | +| --- | --- | --- | --- | +| `Host.Model` | `Container`, `WindowsService`, `Console` | the process | 6.20.0 | +| `Host.Orchestrator` | `Kubernetes`, `None` | `KUBERNETES_SERVICE_HOST` | 6.20.0 | +| `Host.OSPlatform` | `Windows`, `Linux`, `macOS`, `Unknown` | the runtime | 6.20.0 | +| `Host.OSVersion` | major.minor | the runtime | 6.20.0 | +| `Host.Architecture` | the process architecture | the runtime | 6.20.0 | +| `Host.RuntimeVersion` | version | the runtime | 6.20.0 | +| `Host.ProcessorCount` | count | the runtime | 6.20.0 | +| `Host.AvailableMemoryGB` | number | the GC memory limit, which honours a container limit | 6.20.0 | +| `Host.VirtualDirectory` | `None`, `Configured` | `ServiceControl/VirtualDirectory` | Planned | +| `Host.ShutdownTimeoutSeconds` | `Default` or number | `ServiceControl/ShutdownTimeout` | Planned | + +### Storage + +6.20.0 and 6.21.0 emit these keys as `Persistence.*`. The rename to `Storage.*` is unreleased. + +| Key | Values | Source | Status | +| --- | --- | --- | --- | +| `Storage.Type` | `RavenDB`, `SQLServer`, `PostgreSQL` | the persister | 6.20.0 | +| `Storage.RavenServer` | `Embedded`, `External` | RavenDB only | 6.20.0 | +| `Storage.Hosting` | a fixed hosting class from `DatabaseHostClassifier` | the database host, and the engine's own answer where it gives one | 6.20.0 | +| `Storage.HostingSource` | how `Storage.Hosting` was decided | the persister | 6.20.0 | +| `Storage.ServerVersion` | major version on SQL Server and PostgreSQL, major.minor on RavenDB. Always 12 on Azure SQL Database, so read it with `Storage.Hosting` | the engine | 6.20.0 | +| `Storage.FullTextSearch` | `Enabled`, `Disabled` | `EnableFullTextSearchOnBodies` | 6.20.0 | +| `Storage.BodyStorage.Type` | `RavenAttachments`, `FileSystem`, `AzureBlob`, `S3` | the persister | 6.20.0 | +| `Limits.MaxBodySizeToStore` | bytes | `MaxBodySizeToStore`, SQL Server and PostgreSQL only | 6.20.0 | +| `Storage.Schema` | `Default`, `Custom` | `Database/Schema`, SQL Server and PostgreSQL only | Planned | +| `Storage.LogLevel` | `None`, `Information`, `Operations` | `RavenDBLogLevel`, RavenDB only | Planned | +| `Storage.CommandTimeoutSeconds` | `Default` or number | `Database/CommandTimeout`, SQL Server and PostgreSQL only | Planned | +| `Storage.QueryTimeoutSeconds` | `Default` or number | `QueryTimeoutInSeconds` | Planned | +| `Storage.SubscriptionCacheSeconds` | `Default` or number | `SubscriptionCacheDuration`, SQL Server and PostgreSQL only | Planned | +| `Storage.BodyStorage.MinCompressionBytes` | `Default` or number | `MessageBody/MinCompressionSize`, SQL Server and PostgreSQL only | Planned | +| `Storage.FreeSpaceThresholdPercent` | `Default` or number. `NotApplicable` on SQL Server and PostgreSQL when body storage is not the file system | `DataSpaceRemainingThreshold` on RavenDB, `MessageBody/FileSystem/DataSpaceRemainingThreshold` on file system body storage | Planned | +| `Storage.MinimumFreeSpaceForIngestionPercent` | `Default` or number | `MinimumStorageLeftRequiredForIngestion`, RavenDB only | Planned | +| `Storage.ExpirationIntervalSeconds` | `Default` or number | `ExpirationProcessTimerInSeconds`, RavenDB only | Planned | + +### Transport + +The report's top-level `MessageTransport` carries the broker family name (`RabbitMQ`) whenever the transport has a broker throughput query. The RabbitMQ queue type and routing topology are lost there, so `Transport.Type` carries the full manifest name. `MessageTransport` stays as it is for analysis that already reads it. + +The primary's `TransportEnvironmentDataProvider` emits `Transport.Type` and adds the keys the transport returns from `ITransportCustomization.GetEnvironmentData`. Keys under `Transport..*` are emitted only by that transport. + +| Key | Values | Source | Status | +| --- | --- | --- | --- | +| `Transport.Type` | the manifest name, for example `RabbitMQ.QuorumConventionalRouting` | `ServiceControl/TransportType`, resolved through the manifest | Planned | +| `Transport.AzureServiceBus.Topology` | `TopicPerEvent`, `Migration`, `Custom` | `TopicName` in the connection string, then `ServiceControl.Transport.ASBS/Topology` | Planned | +| `Transport.AzureServiceBus.Partitioning` | `Enabled`, `Disabled` | `EnablePartitioning` | Planned | +| `Transport.AzureServiceBus.WebSockets` | `Enabled`, `Disabled` | `TransportType=AmqpWebSockets` | Planned | +| `Transport.AzureServiceBus.HierarchyNamespace` | `None`, `Configured` | `HierarchyNamespace` | Planned | +| `Transport.AmazonSQS.NamePrefixes` | `None`, `Queue`, `Topic`, `QueueAndTopic` | `QueueNamePrefix`, `TopicNamePrefix` | Planned | +| `Transport.AmazonSQS.LargeMessageBucket` | `None`, `Configured` | `S3BucketForLargeMessages` | Planned | +| `Transport.AmazonSQS.MessageWrapping` | `Enabled`, `Disabled` | `DoNotWrapOutgoingMessages` | Planned | +| `Transport.AmazonSQS.ReservedBytesInMessageSize` | `Default` or number | `ReservedBytesInMessageSize` | Planned | +| `Transport.RabbitMQ.DeliveryLimitValidation` | `Enabled`, `Disabled` | `ValidateDeliveryLimits` | Planned | +| `Transport.RabbitMQ.ManagementApi` | `Default`, `Configured` | `ManagementApiUrl` | Planned | +| `Transport.SQLServer.QueueSchema` | `Default`, `Custom` | `Queue Schema` in the connection string | Planned | +| `Transport.SQLServer.SubscriptionsTable` | `Default`, `Custom` | `Subscriptions Table` in the connection string | Planned | +| `Transport.PostgreSQL.QueueSchema` | `Default`, `Custom` | `Queue Schema` in the connection string | Planned | +| `Transport.PostgreSQL.SubscriptionsTable` | `Default`, `Custom` | `Subscriptions Table` in the connection string | Planned | + +### Features + +| Key | Values | Source | Status | +| --- | --- | --- | --- | +| `Features.IntegratedServicePulse` | `Enabled`, `Disabled` | `ServiceControl/EnableIntegratedServicePulse` | 6.13.0 | +| `Features.MessageEditing` | `Enabled`, `Disabled` | `ServiceControl/AllowMessageEditing` | 6.20.0 | +| `Features.ExternalIntegrationsPublishing` | `Enabled`, `Disabled` | `ServiceControl/DisableExternalIntegrationsPublishing`, inverted | 6.20.0 | +| `Features.ForwardErrorMessages` | `Enabled`, `Disabled` | `ServiceControl/ForwardErrorMessages` | 6.20.0 | +| `Features.EmailNotifications` | `Enabled`, `Disabled`, `NotConfigured` | the stored email settings | 6.20.0 | +| `Features.ErrorIngestion` | `Enabled`, `Disabled` | `ServiceControl/IngestErrorMessages` | Planned | +| `Features.ConfigurationValidation` | `Enabled`, `Disabled` | `ServiceControl/ValidateConfig` | Planned | +| `Limits.ExternalIntegrationsBatchSize` | `Default` or number | `ExternalIntegrationsDispatchingBatchSize` | Planned | + +### Integrated ServicePulse + +These keys are `NotApplicable` when `Features.IntegratedServicePulse` is `Disabled`. The values come from the environment variables the integrated ServicePulse reads. + +| Key | Values | Source | Status | +| --- | --- | --- | --- | +| `ServicePulse.MonitoringUrl` | `Default`, `Custom`, `Disabled`, `NotApplicable` | `MONITORING_URL`, or the legacy `MONITORING_URLS`. `!` means disabled | Planned | +| `ServicePulse.DefaultRoute` | `Default`, `Custom`, `NotApplicable` | `DEFAULT_ROUTE` | Planned | +| `ServicePulse.ShowPendingRetry` | `Enabled`, `Disabled`, `NotApplicable` | `SHOW_PENDING_RETRY` | Planned | + +### Error ingestion + +| Key | Values | Source | Status | +| --- | --- | --- | --- | +| `Ingestion.Error.MaxConcurrency` | `Default` or number | `ServiceControl/MaximumConcurrencyLevel` | Planned | +| `Ingestion.Error.BatchSize` | `Default` or number | `ServiceControl/ErrorIngestionBatchSize` | Planned | +| `Ingestion.Error.MaxParallelWriters` | `Default` or number | `ServiceControl/ErrorIngestionMaxParallelWriters` | Planned | +| `Ingestion.Error.BatchTimeoutMs` | `Default` or number | `ServiceControl/ErrorIngestionBatchTimeout` | Planned | +| `Ingestion.Error.RestartAfterFailureSeconds` | `Default` or number | `ServiceControl/TimeToRestartErrorIngestionAfterFailure` | Planned | + +### Retention + +| Key | Values | Source | Status | +| --- | --- | --- | --- | +| `Retention.ErrorHours` | whole hours | `ErrorRetentionPeriod` | 6.20.0 | +| `Retention.EventsHours` | whole hours | `EventRetentionPeriod` | 6.20.0 | + +`Retention.EventsHours` reports `ServiceControl/EventRetentionPeriod`, the spelling the instance validates and the public documentation uses. Both persisters enforce `ServiceControl/EventsRetentionPeriod` instead, with a 14 day default. Until the two are reconciled, the reported value can differ from the retention the storage applies. + +### Heartbeats, recoverability and licensing + +Most of these are choices users make in ServicePulse, or the heartbeat state those choices apply to. `Heartbeats.GracePeriodSeconds` and `Recoverability.RetryHistoryDepth` are configuration settings. Every value is read when the report is built. + +| Key | Values | Source | Status | +| --- | --- | --- | --- | +| `Heartbeats.TrackInstancesDefault` | `Enabled`, `Disabled` | the stored default endpoint settings row. Falls back to `ServiceControl/TrackInstancesInitialValue` when no row is stored | Planned | +| `Heartbeats.TrackInstancesOverrides` | count | stored endpoint settings rows whose value differs from the default | Planned | +| `Heartbeats.KnownInstances` | count | `IEndpointInstanceMonitoring.GetEndpoints` | Planned | +| `Heartbeats.MonitoredInstances` | count | the same, where `Monitored` is true | Planned | +| `Heartbeats.GracePeriodSeconds` | `Default` or number | `ServiceControl/HeartbeatGracePeriod` | Planned | +| `Recoverability.Redirects` | count | `IMessageRedirectsDataStore.GetRedirects` | Planned | +| `Recoverability.RetryHistoryDepth` | `Default` or number | `ServiceControl/RetryHistoryDepth` | Planned | +| `Licensing.ReportMasks` | count | `ILicensingDataStore.GetReportMasks` | Planned | + +An instance becomes monitored on its first heartbeat. An instance first seen in an ingested message starts unmonitored. So `KnownInstances` minus `MonitoredInstances` counts instances that a user stopped monitoring together with instances that have never sent a heartbeat. Storage cannot separate the two. + +### Logging and telemetry + +| Key | Values | Source | Status | +| --- | --- | --- | --- | +| `Logging.Providers` | the active providers from `NLog`, `Seq`, `Otlp`, comma separated in that order | `ServiceControl/LoggingProviders`. `NLog` when unset | Planned | +| `Logging.Level` | `Trace`, `Debug`, `Information`, `Warning`, `Error`, `Critical`, `None` | `ServiceControl/LogLevel` | Planned | +| `Telemetry.OtlpMetrics` | `Enabled`, `Disabled` | whether `OTEL_EXPORTER_OTLP_ENDPOINT` is set | Planned | + +### Audit instances + +These come from the `GET /api/configuration` response of each live audit instance. The primary already fetches that response once a day. No change to the audit instance is needed. + +| Key | Values | Source | Status | +| --- | --- | --- | --- | +| `Audit.RetentionHours` | whole hours, largest across instances | `data_retention.audit_retention_period` | Planned | +| `Audit.Features.ForwardAuditMessages` | `Enabled`, `Disabled`, `Mixed` | `transport.forward_audit_messages` | Planned | +| `Audit.Limits.MaxBodySizeToStore` | bytes, largest across instances | `performance_tunning.max_body_size_to_store` | Planned | +| `Audit.Logging.Level` | the `Logging.Level` values, or `Mixed` | `host.logging.logging_level` | Planned | +| `Audit.Storage.Type` | `RavenDB`, `SQLServer`, `PostgreSQL`, `Mixed` | `persistence.persistence_type`, normalised because the raw value can be a legacy type name | Planned | + +## Not reported + +Every setting below is left out for one of four reasons: + +- Identifies: the setting is a name, address, path or secret, or a hash or prefix of one. +- Not a choice: the setting is a test hook, a dead setting, an action, or a mode in which the instance cannot build a report. +- Security configuration: the setting describes how the instance authenticates callers or connections, encrypts traffic, validates certificates or tokens, or restricts access. +- Out of scope: the coverage decision leaves it out. + +A setting that identifies and is also security configuration is listed as Identifies. + +| Setting | Reason | Notes | +| --- | --- | --- | +| `ServiceControl/InstanceName`, `InternalQueueName` | Identifies | | +| `ServiceBus/ErrorQueue`, `ServiceBus/ErrorLogQueue` | Identifies | | +| `ServiceControl/Hostname`, `ServiceControl/Port` | Identifies | Together they form the instance's address. | +| `ServiceControl/VirtualDirectory` value | Identifies | Reported as `Host.VirtualDirectory`. | +| Transport connection string | Identifies | Some of its options are reported under Transport as fixed values. Its authentication, encryption and certificate validation options are security configuration. | +| `ServiceControl/RemoteInstances` | Identifies | Counted by `Audit.ConfiguredInstances` and `Audit.LiveInstances`. | +| `Database/ConnectionString`, `RavenDB/ConnectionString`, `RavenDB/DatabaseName`, `DbPath` | Identifies | The hosting class and server version are reported under Storage. | +| `RavenDB/ClientCertificatePath`, `ClientCertificateBase64`, `ClientCertificatePassword` | Identifies | | +| `Database/Schema` value | Identifies | Reported as `Storage.Schema`. | +| `MessageBody/*` path, container, bucket, key prefix, region, service URL, credentials, managed identity client id, authority host | Identifies | The type is reported as `Storage.BodyStorage.Type`. | +| `LogPath`, `SeqAddress`, the `OTEL_EXPORTER_OTLP_ENDPOINT` value | Identifies | The providers and OTLP use are reported. | +| `Https.CertificatePath`, `Https.CertificatePassword` | Identifies | | +| `Authentication.Authority`, `Audience`, `ServicePulse.ClientId`, `ServicePulse.ApiScopes`, `ServicePulse.Authority`, claim names | Identifies | | +| `Cors.AllowedOrigins`, `ForwardedHeaders.KnownProxies`, `ForwardedHeaders.KnownNetworks` | Identifies | | +| Email server, sender, recipients, account and password | Identifies | Only whether email notifications are on is reported, as `Features.EmailNotifications`. | +| `ServiceControl/NotificationsFilter` check ids | Identifies | | +| Report mask strings, redirect addresses, endpoint names, licensed endpoint details | Identifies | Masks and redirects are reported as counts. | +| `MONITORING_URL`, `DEFAULT_ROUTE` values, `SERVICECONTROL_URL` | Identifies | The integrated ServicePulse keys report `Default` or `Custom`. | +| `Authentication.Enabled`, `RoleBasedAuthorizationEnabled`, `ValidateIssuer`, `ValidateAudience`, `ValidateLifetime`, `ValidateIssuerSigningKey`, `RequireHttpsMetadata`, `ServicePulse.OfflineAccessScopeEnabled`, and whether claim names differ from the defaults | Security configuration | 6.20.0 and 6.21.0 report the first two as `Security.Authentication` and `Security.RoleBasedAuthorization`. | +| `Https.Enabled`, `RedirectHttpToHttps`, `EnableHsts`, `HstsMaxAgeSeconds`, `HstsIncludeSubDomains`, `Https.Port` | Security configuration | 6.20.0 and 6.21.0 report `Https.Enabled` as `Security.Https`. | +| `Cors.AllowAnyOrigin` and whether origins are restricted | Security configuration | | +| `ForwardedHeaders.Enabled`, `TrustAllProxies` and whether proxies are restricted | Security configuration | | +| Transport authentication, encryption and certificate validation options in the connection string, for example RabbitMQ `UseExternalAuthMechanism` and `DisableRemoteCertificateValidation` | Security configuration | | +| Database authentication, encryption and certificate validation options, for example SQL Server `Encrypt` and `TrustServerCertificate` or PostgreSQL `SSL Mode`, and whether a RavenDB client certificate is used | Security configuration | | +| Body storage authentication mode | Security configuration | 6.20.0 and 6.21.0 report it as `Persistence.BodyStorage.Auth`. | +| Email notification TLS and SMTP authentication | Security configuration | | +| Audit instance security settings | Security configuration | `GET /api/configuration` does not return them. | +| `ServiceControl/PrintMetrics` | Not a choice | Nothing reads it. | +| `ServiceControl/AuditRetentionPeriod` on the primary | Not a choice | It is displayed and returned by `api/configuration`, but nothing acts on it. | +| `EmailDropFolder`, `MessageFilter` | Not a choice | Acceptance tests only. | +| `RunCleanupBundle`, `DisableHealthChecks` | Not a choice | Set by commands, never by configuration. | +| `--error-ingestion-only` | Not a choice | A worker mode. Workers do not build reports. | +| RavenDB `MaintenanceMode` | Not a choice | An instance in maintenance mode serves no API, so it cannot build a report. | +| `ASPNETCORE_ENVIRONMENT=Development` | Not a choice | A development mode. | +| `DOTNET_RUNNING_IN_CONTAINER` | Not a choice | Already covered by `Host.Model`. | +| Retry, archive, unarchive, resolve, edit and group comment operations | Not a choice | These are actions. Their on/off switch, where one exists, is reported. | +| Custom checks | Not a choice | Endpoints report them and ServiceControl stores them. Deleting one does not keep a muted state. | +| Email notification details: whether a filter is set, port, number of recipients and mail provider | Out of scope | Only `Features.EmailNotifications` is reported. | +| Whether licensed endpoint details are uploaded, and whether they match the license | Out of scope | They only apply to Endpoint Size licenses with endpoint metadata. | +| Transport `QueueLengthQueryDelayInterval`, `QueueLengthQueryMaxDelayInterval` | Out of scope | Only the monitoring instance reads them. | +| Every monitoring instance setting | Out of scope | | +| Audit instance settings that `GET /api/configuration` does not return | Out of scope | Includes `IngestAuditMessages`, full-text search, embedded or external RavenDB, logging providers, OTLP, ingestion and RavenDB tuning, `ServiceControlQueueAddress`, `VirtualDirectory` and maintenance mode. Audit security settings are listed under Security configuration. |