v12 integrations: groundwork for guide-only integrations (guide list, card names, View logs, syslog guide fix, templates)

[kryonsx]

Part of: #2731. The four technology lists depend on this, so it goes first.

Why

Every guide-only technology rides on one of four channels we already have: syslog, NetFlow, the Windows agent or the Linux agent. Doing the shared parts once keeps each technology down to a list entry, an icon, some text and a link. The research behind this found a few things in today's Integrations page that block the idea or mislead customers; they are part of this task.

What to build

1. Where the entries live.
The catalog cannot hold them. integrations has a unique index on tenant and data type (idx_integration_tenant_data_type in backend/modules/integrations/domain/integration.go); WINDOWS_AGENT and LINUX_AGENT already own wineventlog and linux, and only one row could ever use syslog. Two ways out:

• Recommended: a front-end list of guide-only entries next to frontend/src/features/integrations/constants/systemModules.ts (name, icon, category, parent channel, guide, Log Explorer filter), merged into the grid by IntegrationsPage.tsx. No backend change. The entries will not appear in the API or the MCP tools.
• Database rows with a new ingest type (for example guide) left out of that unique index and out of /integrations/data-types, which feeds the rule and pipeline pickers. This also needs the chk_integration_ingest_type check changed. Only worth it if the API must list these entries.

2. Open the right guide.
The drawer picks the setup screen by ingest type: an agent row always renders AgentSetup, which chooses install commands from the display name (buildAgentInstall checks startsWith('windows' | 'linux' | 'macos')). Guide entries must render their own guide, with a link to the Windows or Linux agent install.
Guide lookup is also loose: getCollector() falls back to substring matches, so any name containing cisco, fortinet, sophos, vmware, oracle, palo, sonic, sentinel, eset, aix, kaspersky, meraki, mikrotik, pfsense, suricata or deceptive opens that vendor's existing guide (a "Cisco ISE" card would open the Cisco ASA guide). Register every guide under its exact name. A name with no guide falls back to GenericCollectorGuide, which shows a made-up installer address with a fake token and port 514/udp; replace it with the fixed syslog guide (item 6).

3. Card names show internal identifiers today.
IntegrationsPage.tsx reads integrations.modules.<NAME>.name, but the locale files store a plain description string at integrations.modules.<NAME>, so every card falls back to its internal name (for example WINDOWS_AGENT). Split the key into name and description in all seven locale files (frontend/src/shared/i18n/locales/{en,es,pt,fr,de,it,ru}.json). New entries need both.

4. A per-technology log filter.
Give each entry a filter and use it wherever the page links to logs:

Channel	Data type	Field that names the product	Example
Windows agent	wineventlog	log.providerName (the event source), plus log.channel	log.providerName=MSSQLSERVER
Linux agent	linux	log.syslogIdentifier (the program name in the journal)	log.syslogIdentifier=nginx
Syslog	syslog	none today: the generic parser keeps the whole line in log.message	only dataSource (the sender)
NetFlow	netflow	none: flow records carry no product name	log.exporter (the exporter's address; dataSource also carries its source port)

Use "is one of" in these links (see the dashboards groundwork issue #2697 for why). Keep all filters in this one list: the Windows and Linux names exist only because the event engine used to strip underscores from top-level field names (note at the top of definitions/filters/linux/linux.yaml). The engine stopped doing that on 24 Sep 2026 (go-sdk v1.1.35), and the v12 parsers still have to catch up, so these names may change. Check them against the v12 parser when you build this.

5. "View logs" for every integration.
The drawer shows "View logs" only for integrations marked configured, and only cloud pullers can ever be configured (configured in backend/modules/integrations/usecase/integration.go), so agent, syslog and guide-only cards never get the button. Show it everywhere and pass the entry's own filter (today it passes only ?dataType=). The dashboards groundwork issue #2697 asks for the same change; do it once, in whichever lands first.

6. Fix the syslog guide and add a SYSLOG card.
Generic syslog is received by the UTMStack Collector on port 7014, UDP or TCP, and TCP can use TLS (collectors/forwarder/config/const.go). The current SYSLOG guide (collectors/syslog.tsx), CollectorEndpointInfo and GenericCollectorGuide all show 514/udp, the default port of the four Cisco data types, so today's guide sends customers to the wrong port. There is also no SYSLOG catalog row in v12, so the guide never shows, and syslog is missing from the data type pickers on the rules and pipelines pages. Add the row (data type syslog, ingest type forwarder); no other row uses that data type. The guide should:

• recommend TCP for chatty devices: the Collector reads each UDP message into a 2,048-byte buffer (collectors/forwarder/collector/syslog/framing.go) and cuts anything longer without an error;
• tell devices that can only send to 514 to use utmstack_forwarder change-port syslog udp 514 on a Collector that does not run the Cisco integrations (they own UDP 514 by default and the command refuses a busy port). enable-integration syslog 514 udp does not do it: for a built-in data type the port argument is ignored.

7. Four guide templates.
Build them once, so each technology only supplies text, a screenshot and its settings:

• SyslogDeviceGuide: Collector address, port 7014 and protocol (reuse ForwarderGuide), a slot for the vendor's steps, then how to check the logs arrived.
• WindowsEventLogGuide: agent install link, which channel and source the product writes to, any setting to switch on, then how to check.
• LinuxJournalGuide: agent install link, the setting that sends the product's logs to the journal or to syslog, then how to check.
• NetflowExporterGuide: Collector address and UDP 2055, which export version to pick, the vendor's steps, then how to check.

8. Clean up the orphan guides.
Several guide components are left over from v11 and describe things v12 no longer has (Filebeat, an agent-side syslog listener, a JSON endpoint that is not published). No card opens them today, but they would mislead anyone who wires them up. Delete iis.tsx, filebeat.tsx, json.tsx, ldap.tsx, salesforce.tsx, webroot.tsx, awsTrafficMirror.tsx, vulnerabilities.tsx, assetScanner.tsx and socAi.tsx; merge windowsFaa.tsx and fileIntegrity.tsx into the "Windows file access auditing" entry of the Windows list; leave the AWS log-group guides unregistered until the AWS form collects a log group name. Details per file in the research notes below.

9. Fix the remote-enable "master" option.
ForwarderGuide.tsx offers https://<host>:50052/v1/ingest, but the installer does not publish port 50052 and nginx does not route /v1/ingest, so anyone who follows it fails. Fix the address or remove the option.

10. Optional, high value: tell syslog products apart.
The generic syslog parser (definitions/filters/syslog/syslog-generic.yaml) keeps only the whole line, so a Ubiquiti gateway and a Synology box look the same except for the sender. Parsing the standard syslog header into the same fields the Linux parser uses (origin.host, log.syslogIdentifier, log.priority) would give every syslog guide a filter that survives address changes. It is a parser change: hand it to the parser owners if it is outside your area. The field list and the rules for the change are in 4(a) of the syslog list issue.

Needs other owners (link, don't wait)

• Windows agent keeps only named event data. Unnamed <Data> values and <UserData> are dropped (convertEventToJSON in agent/collector/platform/windows_amd64.go), so SQL Server failed logins arrive without the user, reason or client address, and AD FS, Remote Desktop sessions, AppLocker and Hyper-V lose their details. Two small agent changes fix most of it (research note 4(c)(7) below).
• Windows parser drop list removes events several guides need: Sysmon event 1 (process creation), 4778 and 4779, 5137 to 5141, 4898 and 4899, 6274 and 6275, 5145, 7040, and IIS 5057 and 5059 (note 4(c)(8) below; tracked with Integration Update: Windows #1728).
• Windows channels the agent does not read (IIS request logs, DNS Server audit, LAPS, OpenSSH, DHCP and others): one line each in the agent's channel list; see 4(b) in the Windows list issue.
• NetFlow Collector and parser lose ports, protocol and byte counts for most export versions; see 4(d) in the NetFlow list issue and Integration Update: NetFlow #1736.
• Linux agent reads the journal without --all, so any field over 4 KB arrives empty.

Done when

• Guide-only entries show on the Integrations page with real names, icons and categories, and each opens its own guide.
• Every card shows its display name, not its internal identifier.
• Every integration's drawer shows "View logs"; for guide-only entries it opens the Log Explorer filtered to that technology.
• A SYSLOG card exists and its guide shows port 7014.
• The orphan guides are deleted or merged as listed.
• The four templates exist and one technology from each list uses them.

Research notes

The full research behind this project, apart from the four technology lists (which are in their own issues).

How to write a Log Explorer filter (0.6)

Log Explorer offers field filters (is, is not, contains, exists), free text (a case-insensitive substring search of the raw line) and SQL mode. It also accepts a deep link where every query parameter becomes an "is" filter and a comma list becomes "is one of" (frontend/src/features/log-explorer/pages/LogExplorerPage.tsx):

/log-explorer?dataType=linux&log.syslogIdentifier=sshd,sshd-session
/log-explorer?dataType=wineventlog&log.providerName=MSSQLSERVER
/log-explorer?dataType=syslog&dataSource=10.0.0.1

The same filters in SQL mode: SELECT * FROM logs WHERE dataType = 'wineventlog' AND toString(log.providerName) = 'MSSQLSERVER'.

Note: event data names that contain spaces (Microsoft Defender uses "Threat Name", "Detection User") cannot be typed as a plain field filter; filter on provider and event ID and use free text for the rest.

Three more rules the tables in section 3 follow (checked in go-sdk store/clickhouse/filter.go and backend/pkg/eventstore/eventstore.go):

• "is one of" on a log.* field compares as text (toString(field) IN (...)), while "is" compares the stored value as it is. log.eventCode is stored as a number, so the tables always use "is one of" for event IDs, even for a single ID.
• "contains" and free text are case-insensitive substring searches. Free text searches the raw column, which for Windows and Linux is the JSON the agent sent and for syslog is the line as received.
• For NetFlow, dataSource is <exporter IP>:<exporter source port>. Filter on log.exporter, which holds the IP alone (table 3d).

Version 11 integrations and what happened to them (1)

Source: the last definition of register_integrations on branch v11 (changeset 20260105001_adding_crowdstrike_integration.xml), the separate "adding_*" changesets (pfSense, FortiWeb, AIX, AS/400, Oracle, Suricata, UTMStack, CrowdStrike), the removal changesets, UtmModulesEnum (frontend/src/app/app-module/shared/enum/utm-module.enum.ts) and ModuleName.java.

All 33 v12 catalog rows existed in v11. The v11 names below are the ones missing from the v12 catalog.

v11 name (label)	State in v11	Works in v12 today? How
SYSLOG (Syslog)	Registered until the end of v11	Yes. Collector, data type syslog, port 7014. There is no v12 catalog row, and the orphan guide syslog.tsx gives the wrong port. Top candidate to restore.
JSON (Json Input)	Registered until the end of v11	Partly. The Collector can open an HTTP listener for data type json-input (utmstack_forwarder enable-integration json-input <port> http), and the json-input filter still exists. The listener binds to 127.0.0.1 only, so only programs on the Collector host can post.
FILE_INTEGRITY (File Classification)	Registered until the end of v11	Yes, through the Windows agent, after the customer enables "Audit File System" and puts auditing entries on the folders. Events 4656, 4660, 4663, 4670 arrive in Security. The agent does not enable Object Access itself (auditpolicy_windows.go), and the filter drops 5145.
SOC_AI	Registered until the end of v11	Not an integration in v12. Configured on its own settings page (features/settings/pages/SocAiSettingsPage.tsx).
APACHE (Apache)	Removed from v11 on 2026-02-18 (20260218004)	Yes on Linux after a setting: error log to syslog, access log through a piped logger. Then the Linux agent collects it. No path on Windows. See table 3c.
NGINX	Removed 2026-02-18 (20260218002)	Yes on Linux after a setting (error_log and access_log to syslog). See 3c.
HAPROXY (High Availability Proxy)	Removed 2026-02-18 (20260218014)	Yes on Linux; see 3c for the distribution default.
TRAEFIK	Removed 2026-02-18 (20260218012)	See 3c.
IIS (Internet Information Services)	Removed 2026-02-18 (20260218015)	Partly. Service and worker events in System and Application arrive today. Request logs are files by default; the optional event logging goes to channel Microsoft-IIS-Logging/Logs, which the agent does not read. See 3b.
MYSQL	Removed 2026-02-18 (20260218005)	Yes on Linux after a setting; see 3c.
POSTGRESQL	Removed 2026-02-18 (20260218003)	Yes on Linux after a setting; see 3c.
MONGODB	Removed 2026-02-18 (20260218006)	Yes on Linux after a setting; see 3c.
REDIS	Removed 2026-02-18 (20260218001)	Yes on Linux after a setting; see 3c.
OSQUERY (OsQuery)	Removed 2026-02-18 (20260218016)	Yes on Linux after a setting; see 3c.
AUDITD (Linux Auditing Demon)	Removed 2026-02-18 (20260218013)	Yes, built in: the v12 Linux agent tails the audit log itself. Not a separate card.
ELASTICSEARCH, KIBANA, LOGSTASH, KAFKA	Removed 2026-02-18 (20260218007 to 20260218010)	Not confirmed. They write their own files by default; sending them to syslog needs a logging-framework change that was not verified. Low value; leave out.
NATS	Removed 2026-02-18 (20260218011)	Not confirmed. Leave out.
UFW	Removed 2024-04-05 (20240405001)	Yes. Kernel log lines ([UFW BLOCK]) reach the journal as soon as UFW is enabled, because the shipped logging level is low. See 3c.
LINUX_LOGS (Linux Logs)	Removed 2024-04-05 (20240405001)	Replaced by LINUX_AGENT. A Linux host without the agent can forward with rsyslog to the Collector on 7014.
AD_AUDIT (AD Audit)	Function existed, never called by the last register_integrations	Nothing to configure: the v12 analysis plugin plugins/ad-audit works from Windows agent events on domain controllers.
VULNERABILITIES, ASSET_SCANNER	Function existed for VULNERABILITIES, never called; ASSET_SCANNER only in the front-end enum	No v12 component.
SALESFORCE	Call commented out in v11	No v12 collector.
APACHE2	Call commented out in v11	Same answer as APACHE.
WEBROOT, WINDOWS_FAA, AWS_CLOUDTRAIL, AWS_TRAFFIC_MIRROR, AWS_SQL_SERVER, AWS_POSTGRESQL, AWS_BEANSTALK, AWS_FARGATE, AWS_LAMBDA, MACOS_AGENT, SYSTEM	Front-end enum only; never registered by the last v11 database scripts	WEBROOT: no. WINDOWS_FAA: same as FILE_INTEGRITY. AWS_*: see section 2.

Orphan v12 guide components (2)

Folder: frontend/src/features/integrations/components/setup/collector/collectors/. Text: frontend/src/shared/i18n/locales/en.json under integrations.setup.collector.*. None of these names is in the v12 catalog, so no card ever opens them. syslog.tsx was not on the list I was given, but it is also an orphan (no SYSLOG row).

Component (registered name)	Does the text match v12?	Recommendation
syslog.tsx (SYSLOG)	No. Says 514 over UDP or TCP "to the UTMStack agent" and "enable the syslog integration in the configuration file where your UTMStack agent is installed". v12 uses the Collector on 7014.	Keep and fix: rebuild on ForwarderGuide with sourceType="syslog" and port 7014, and add a SYSLOG catalog row (data type syslog, ingest type forwarder). This becomes the parent guide for every device in table 3a.
iis.tsx (IIS)	No. Tells the user to run filebeat modules enable iis under ...\UTMStack Agent\beats\filebeat. v12 has no Filebeat.	Delete now. Replace with the guide-only IIS entry in table 3b: it can cover the service and worker events that arrive today; request logs need Microsoft-IIS-Logging/Logs added to the agent (3b and 4(b)).
filebeat.tsx (LINUX_LOGS)	No. Tells the user to forward rsyslog to "the UTMStack agent" on 514 and enable "Linux Logs" in the agent configuration.	Delete. Move one useful paragraph into the fixed SYSLOG guide: "Linux host without the agent: *.* @@<collector host>:7014 in /etc/rsyslog.d/90-utmstack.conf, then restart rsyslog".
json.tsx (JSON)	No. Shows POST https://<host>:8080/v1/logs with a "federation token". Nothing serves that: port 8080 of log-input is its health endpoint, the real ingest path is /v1/ingest on 50052 with header Utm-Api-Key, and 50052 is neither published by the installer nor routed by nginx.	Delete. The Custom integration guide (CustomSetup.tsx) already covers JSON over HTTP through the Collector.
ldap.tsx (AD_AUDIT)	No. Walks through installing Active Directory Certificate Services to get LDAPS for the old LDAP-based AD Audit module.	Delete. Replace with a guide-only "Active Directory (domain controllers)" entry under the Windows agent (row 3b).
salesforce.tsx (SALESFORCE)	No. Asks for OAuth credentials, but v12 has no Salesforce collector (not in plugins/, not in the config schema).	Delete.
webroot.tsx (WEBROOT)	No. Asks for API credentials; no v12 collector.	Delete.
windowsFaa.tsx (WINDOWS_FAA)	Mostly yes: "Audit File System" plus folder auditing is still how file access events are produced. The "Activate the UTMStack module" step is wrong (nothing to activate).	Merge with fileIntegrity.tsx into one guide-only entry "Windows file access auditing" (row 3b).
fileIntegrity.tsx (FILE_INTEGRITY)	Mostly yes, same content plus "verify 4663/4670 in Event Viewer". Same wrong "Activate" step.	Keep as the merged guide above; add that 5145 (detailed file share access) is dropped by the filter.
awsBeanstalk.tsx, awsFargate.tsx, awsLambda.tsx, awsSqlServer.tsx, awsPostgresql.tsx	Partly. They explain how to get each service's logs into CloudWatch Logs. But the v12 AWS plugin reads one log group per configuration group through the key aws_log_group_name (plugins/aws/main.go), and the v12 AWS form and backend schema never collect that key (builders/aws.ts, schema_provider.go). As far as the code shows, no log group name ever reaches the plugin (not tested).	Re-home later as sections of the AWS guide, not as cards, after the log group field exists. Until then, delete or leave unregistered.
awsTrafficMirror.tsx	No. Mirrors packets (VXLAN, port 4789) to "the UTMStack agent VM"; nothing in v12 receives mirrored packets.	Delete.
vulnerabilities.tsx	No. v12 has no vulnerability scanner.	Delete.
assetScanner.tsx (ASSET_MANAGEMENT)	No. v12 has no asset scanner (the Data Sources page is fed by log statistics).	Delete.
socAi.tsx	No. SOC AI has its own settings page in v12.	Delete.

Other things that block a guide-only approach (4(c))

1. One catalog row per data type. backend/modules/integrations/domain/integration.go puts a unique index on (tenant_id, data_type) (idx_integration_tenant_data_type). WINDOWS_AGENT already owns wineventlog and LINUX_AGENT owns linux, so no second system row can use them, and only one row can ever use syslog. Options: (1) keep guide-only entries in the front end only (a static list next to constants/systemModules.ts with name, icon, category, parent channel, guide and a Log Explorer link; IntegrationsPage.tsx merges them into the grid); no backend change, but they are invisible to the API and the MCP tools. (2) Database rows with a new ingest type such as guide and a unique index that excludes them; needs a model change (the chk_integration_ingest_type check allows only agent, collector, forwarder, plugin) and a filter so these rows do not show up in /integrations/data-types, which feeds the rule and pipeline pickers. Option 1 is the cheaper path.
2. "Configured" only exists for pullers. configured() in backend/modules/integrations/usecase/integration.go returns false for every ingest type except plugin. IntegrationDrawer.tsx shows "View logs" only when status === 'configured', so agent, collector, forwarder and any guide-only card never get the button. Fix: always show "View logs" for guide-only entries and pass the entry's own filter (Log Explorer already turns every query parameter into a filter, section 0.6); today the button passes only ?dataType=. A real "receiving data" badge could come from the datasources table (backend/modules/datasources, rebuilt every 10 minutes from log statistics per data type and sender) for syslog senders, or from a count query with the entry's filter.
3. The drawer routes by ingest type, not by guide. Any row with ingest type agent renders AgentSetup and never a technology guide. AgentSetup picks the install commands from the display name (buildAgentInstall checks startsWith('windows' | 'linux' | 'macos')); any other name gets an empty command. Windows and Linux application guides therefore cannot be agent rows without a drawer change.
4. Loose guide matching. getCollector() falls back to matches() substring tests when no guide is registered under the exact name: any name containing cisco (without switch, meraki, firepower) opens the Cisco ASA guide, fortinet opens FortiGate, sophos without xg opens Sophos Central, vmware or esxi opens ESXi, and likewise oracle, palo, sonic, sentinel, eset, aix, kaspersky, meraki, mikrotik, pfsense, suricata, deceptive. So CISCO_ISE, FORTINET_FORTIMAIL, SONICWALL_SMA or VMWARE_VCENTER would show the wrong vendor's guide unless each gets its own registered guide. Names with no match fall back to GenericCollectorGuide, which shows a made-up installer (https://install.utmstack.com/collector.sh with a fake token) and 514/udp; it should be replaced by the fixed SYSLOG guide.
5. Card names. mapModuleToIntegration asks for integrations.modules.<NAME>.name, but the translation files store only a description string at integrations.modules.<NAME>, so every card shows the raw identifier (for example WINDOWS_AGENT). Guide-only entries need a real display name.
6. No syslog row means no syslog in the data-type pickers. /integrations/data-types lists catalog rows only, so rules and pipeline pages cannot offer syslog until a row exists.
7. Windows agent data loss (section 0.4): only named event data is kept; unnamed data, UserData and message text are dropped. This limits every Windows application guide more than the channel list does, and it also affects channels already collected (remote desktop session events 21-25 and 1149, print event 307, WMI 5857-5861, log-cleared 104 and 1102; see table 3b). Two agent changes in convertEventToJSON (agent/collector/platform/windows_amd64.go) would recover most of it:
   • Keep unnamed <Data> values under numbered keys (for example log.data.param1, log.data.param2). This alone gives SQL Server failed logins their user, reason and client address (one text value), and gives Veeam, ScreenConnect, Citrix and the Application-log antivirus events their details.
   • Flatten the first child element of <UserData> into log.data (for example <EventXML><User> becomes log.data.User). This gives Remote Desktop session events, Remote Desktop Gateway, AppLocker, Hyper-V, print event 307 and WMI event 5861 their user, address, file or virtual machine name.
     Both are agent code, not guides, but they decide whether most Windows rows in table 3b can say "Yes".
8. Windows drop list (section 0.4): event IDs 0 and 1 are dropped for every provider, which removes Sysmon process creation (event 1) even though the Sysmon channel is collected. The drop step should be limited to log.channel = Security (or to the provider Microsoft-Windows-Security-Auditing). Checking every event ID named in table 3b against the list found more losses, and most of them are Security events, so limiting the step to Security does not save them. They should come off the list: 4778 and 4779 (remote desktop session reconnected and disconnected), 5137, 5138, 5139 and 5141 (directory object created, undeleted, moved and deleted), 4898 and 4899 (certificate template loaded and changed), 6274 and 6275 (NPS discarded a request), 5145 (detailed file share access) and 7040 (a service's start type changed, for example Defender set to disabled). Outside Security the list also removes Exchange cmdlet event 1 (MSExchange Management, if that channel is ever added) and Windows Update events 43 and 44.
9. Collector limits (section 0.2): UDP messages over 2 KB are cut; TCP frames must start with < or a length; length-framed messages over 8 KB close the connection. Devices that can only send to port 514 (for example Ubiquiti EdgeRouter) need utmstack_forwarder change-port syslog udp 514 on a Collector that does not also run the Cisco integrations (they default to UDP 514, and the command refuses a port an enabled integration already uses). enable-integration syslog 514 udp does not work for this: for a built-in data type it ignores the port argument (collectors/forwarder/cmd/enable_integration.go, collector/datatype.go).
10. Linux agent limits (section 0.5): fields over 4 KB arrive as null because --all is not passed to journalctl; journald rate limits apply to chatty services.
11. The remote-enable "master" option points at an unreachable address. ForwarderGuide.tsx shows https://<host>:50052/v1/ingest, but log-input does not publish 50052 (compose.go) and the nginx templates do not route /v1/ingest (installer/templates/front-end.go, proxy.go). Not a guide-only blocker, but any guide that points people there will fail.
12. Collector file readers with no parser. The Collector can tail /var/log/nginx/ and /var/log/postgresql/ as data types nginx and postgresql (FilePaths in collectors/forwarder/config/const.go, command utmstack_forwarder change-paths), but definitions/filters has no filter for either data type and the catalog has no row, so such logs are stored unparsed. Either add parsers or remove the readers; until then the Linux agent route in table 3c is the one to document.
13. Windows forwarded events. Forwarded events keep the source computer name (log.computer), and dataSource becomes the collector host. Whether log.channel keeps the original channel name was not confirmed, so the WEF workaround in 4(b) filters on provider and computer.