[cisco_meraki] Fix Blocked RA Packet grok pattern in events pipeline. - #20394
[cisco_meraki] Fix Blocked RA Packet grok pattern in events pipeline.#20394ie-ops wants to merge 7 commits into
Conversation
ie-ops
added
enhancement
✅ Elastic Docs Style Checker (Vale)No issues found on modified lines! The Vale linter checks documentation changes against the Elastic Docs style guide. To use Vale locally or report issues, refer to Elastic style guide for Vale. |
🚀 Benchmarks reportTo see the full report comment with |
|
Pinging @elastic/integration-experience (Team:Integration-Experience) |
![vera-review-bot[bot]](https://avatars.githubusercontent.com/in/3297022?s=60&v=4){kind=small}
| if: ctx._temp?.blocked_arp != null | ||
| - set: | ||
| field: cisco_meraki.event_subtype | ||
| value: ra_blocked |
There was a problem hiding this comment.
Severity: 🟡 Medium confidence: medium path: packages/cisco_meraki/data_stream/log/elasticsearch/ingest_pipeline/events.yml:150
The new ra_blocked and dhcp_blocked subtypes are missing from the eventmap in default.yml, so blocked packets are categorized as event.type: ["info"] with event.action falling back to the raw subtype string. Add eventmap entries so these denial events get event.type: ["denied"] and a hyphenated action.
Details
These two set processors introduce the new subtype values ra_blocked and dhcp_blocked, but neither value exists in the eventmap parameter of the set-event-type-for-meraki-events script processor in data_stream/log/elasticsearch/ingest_pipeline/default.yml. The script's fallback branch runs (if (eventData == null) { ctx.event.action = ctx.cisco_meraki.event_subtype; return; }), so the documents end up with event.action: "ra_blocked" / "dhcp_blocked" and only the pipeline-wide defaults event.category: ["network"], event.type: ["info"]. This is visible in the new expected documents in test-events.log-expected.json. These are packet-denial events, so ECS event.type should include denied; leaving them as info means detection rules and dashboards filtering on event.type: denied will not see them. It also leaves event.action in underscore form while every mapped subtype in the eventmap uses a hyphenated action string.
Recommendation:
Add entries for the new subtypes to the eventmap params in data_stream/log/elasticsearch/ingest_pipeline/default.yml (the pre-existing arp_blocked subtype has the same gap and can be fixed in the same map):
"arp_blocked":
type:
- denied
action: arp-packet-blocked
"ra_blocked":
type:
- denied
action: ra-packet-blocked
"dhcp_blocked":
type:
- denied
action: dhcp-packet-blockedThen regenerate the pipeline test expectations so event.type and event.action reflect the new mapping.
🤖 AI-Generated Review | Vera Review Bot | 📚 Knowledge base: integration-skills
⚠️ Automated review — verify suggestions before applying.
| <134>1 1736917275.649842796 game_Office_DMC_47_Floor events type=disassociation radio='1' vap='0' client_mac='E4-F4-18-79-1F-E1' band='5' channel='161' reason='1' da_vendor='none' duration='31.22725353' auth_neg_dur='0.012057865' last_auth_ago='31.152091238' is_8021x='1' full_conn='12.115982391' ip_resp='12.115982391' ip_src='89.160.20.112' arp_resp='0.241727656' arp_src='20.22.20.39' identity='xxxx_xxx' aid='1607913618' | ||
| <134>1 1736917275.649842796 game_Office_DMC_47_Floor events type=8021x_client_deauth radio='1' vap='0' client_mac='E4:F4:18:79:1F:E1' client_ip='172.25.25.74' identity='[email protected]' aid='1442734124' | ||
| <134>1 1736917275.649842796 game_Office_DMC_47_Floor events type=8021x_eap_failure radio='1' vap='0' client_mac='AC:50:DE:7B:B0:3D' identity='DOMAIN\jdoe' | ||
| <134>1 1782557203.245381197 blvl_01 events Blocked RA Packet from 00-00-5E-00-53-23 (2001:db8::1) on VLAN 201 by default |
There was a problem hiding this comment.
Severity: 🔵 Low confidence: medium path: packages/cisco_meraki/data_stream/log/_dev/test/pipeline/test-events.log:57
Both new grok patterns end with an optional (?: by default)? group, but every new fixture line includes the " by default" suffix, so the branch without it is never exercised. Add fixture lines without the suffix.
Details
The two patterns added to events.yml make the trailing " by default" optional and anchor on $, so the message shape without that suffix is a distinct branch. Both fixture lines added here carry the suffix, so a regression that broke the suffix-less form (for example, making the group mandatory) would not be caught by elastic-package test pipeline. Both new lines also use hyphen-separated MAC addresses, while the existing Blocked ARP fixture on line 25 uses the colon-separated form Meraki emits, so the gsub/uppercase normalization of source.mac is not exercised for the new patterns.
Recommendation:
Add fixture lines covering the suffix-less form and a colon-separated MAC, then regenerate test-events.log-expected.json:
<134>1 1782557203.245381197 blvl_01 events Blocked RA Packet from 00:00:5E:00:53:23 (2001:db8::1) on VLAN 201
<134>1 1782557203.245381197 blvl_01 events Blocked DHCP Packet from 00:00:5E:00:53:23 (2001:db8::1) on VLAN 201
🤖 AI-Generated Review | Vera Review Bot | 📚 Knowledge base: integration-skills
⚠️ Automated review — verify suggestions before applying.
| # newer versions go on top | ||
| - version: "1.31.2" | ||
| changes: | ||
| - description: Fix grok pattern to support `Blocked RA Packet` and `Blocked DHCP Packet` events in addition to `Blocked ARP Packet` events. |
There was a problem hiding this comment.
Severity: 🟡 Medium confidence: high path: packages/cisco_meraki/changelog.yml:4
The changelog only mentions the grok fix, but this PR also changes event.action for already-supported Blocked ARP Packet events from arp_blocked to arp-packet-blocked and adds denied to event.type. Mention that user-visible change so existing queries and rules can be updated.
Details
The new arp_blocked entry in the default.yml eventmap (line 218) replaces the script's fallback branch, which previously set ctx.event.action = ctx.cisco_meraki.event_subtype (i.e. the literal arp_blocked). The regenerated fixture confirms the change: test-events.log-expected.json line 1264 now reads "action": "arp-packet-blocked" and event.type gains denied. That is a semantic change to an event subtype the package already emitted before this PR, so saved searches, dashboards, and detection rules filtering on event.action: arp_blocked stop matching after upgrade. The changelog entry describes only the grok pattern addition for RA and DHCP packets, so the ARP behaviour change is undocumented.
Recommendation:
Extend the changelog description (or add a second entry) to cover the ECS categorization change for existing ARP events:
- version: "1.31.2"
changes:
- description: Fix grok pattern to support `Blocked RA Packet` and `Blocked DHCP Packet` events in addition to `Blocked ARP Packet` events.
type: bugfix
link: https://github.com/elastic/integrations/pull/20394
- description: Set `event.action` to `arp-packet-blocked` (previously `arp_blocked`) and add `denied` to `event.type` for `Blocked ARP Packet` events.
type: enhancement
link: https://github.com/elastic/integrations/pull/20394🤖 AI-Generated Review | Vera Review Bot | 📚 Knowledge base: integration-skills
⚠️ Automated review — verify suggestions before applying.
| if: ctx._temp?.blocked_ra != null | ||
| - set: | ||
| field: cisco_meraki.event_subtype | ||
| value: dhcp_blocked |
There was a problem hiding this comment.
Severity: 🔵 Low confidence: low path: packages/cisco_meraki/data_stream/log/elasticsearch/ingest_pipeline/events.yml:154
The new dhcp_blocked subtype does not set network.protocol, unlike the other DHCP subtypes in this pipeline (dhcp_offer, dhcp_no_offer, multiple_dhcp_servers_detected), which all set network.protocol: dhcp. Add the same set processor for consistency.
Details
events.yml sets network.protocol: dhcp for dhcp messages (line 77-80) and for multiple_dhcp_servers_detected (line 247-250). The newly added dhcp_blocked subtype describes a blocked DHCP packet but leaves network.protocol unset, so DHCP-scoped queries such as network.protocol: dhcp will not surface these blocked-packet events.
Recommendation:
Add a matching set processor after the dhcp_blocked subtype assignment:
- set:
field: cisco_meraki.event_subtype
value: dhcp_blocked
if: ctx._temp?.blocked_dhcp != null
- set:
field: network.protocol
value: dhcp
if: ctx._temp?.blocked_dhcp != nullRegenerate test-events.log-expected.json afterwards so the two dhcp_blocked fixture events include the new field.
🤖 AI-Generated Review | Vera Review Bot | 📚 Knowledge base: integration-skills
⚠️ Automated review — verify suggestions before applying.
| field: cisco_meraki.event_subtype | ||
| value: arp_blocked | ||
| if: ctx._temp?.blocked_arp != null | ||
| - set: |
There was a problem hiding this comment.
Severity: 🔵 Low confidence: medium path: packages/cisco_meraki/data_stream/log/elasticsearch/ingest_pipeline/events.yml:148
The three new set processors for ra_blocked, dhcp_blocked and network.protocol have no tag, so a failure in one of them lands in error.message with no way to tell which processor produced it. Add a tag to each.
Details
docs/extend/_publish_an_integration.md (step 7) asks for tag fields on ingest pipeline processors. This pipeline has no pipeline-local on_failure on these processors, so any failure falls through to the pipeline-level on_failure at the bottom of events.yml, which appends _ingest.on_failure_message without a processor tag. Most of the surrounding processors in this file predate the convention, so this is a suggestion scoped to the processors this PR adds rather than a request to retag the file.
Recommendation:
Add a tag to each of the newly added processors:
- set:
field: cisco_meraki.event_subtype
tag: set_event_subtype_ra_blocked
value: ra_blocked
if: ctx._temp?.blocked_ra != null
- set:
field: cisco_meraki.event_subtype
tag: set_event_subtype_dhcp_blocked
value: dhcp_blocked
if: ctx._temp?.blocked_dhcp != null
- set:
field: network.protocol
tag: set_network_protocol_dhcp_blocked
value: dhcp
if: ctx._temp?.blocked_dhcp != null🤖 AI-Generated Review | Vera Review Bot | 📚 Knowledge base: integration-skills
⚠️ Automated review — verify suggestions before applying.
| - grok: | ||
| field: event.original | ||
| patterns: | ||
| patterns: |
There was a problem hiding this comment.
Severity: 🔵 Low confidence: medium path: packages/cisco_meraki/data_stream/log/elasticsearch/ingest_pipeline/events.yml:123
The grok processor this PR extends with the RA/DHCP patterns still has no tag, unlike the set processors just below it that were given one. Add a tag (and a description) so a pattern mismatch is attributable in error.message.
Details
This grok processor now carries three alternative patterns (ARP, RA, DHCP) and has no on_failure, so when none of them match the failure propagates to the pipeline-level on_failure in this file, which appends {{{_ingest.on_failure_message}}} to error.message. Without a tag that message identifies neither the processor nor which of the three blocked-packet formats failed to parse. The three set processors immediately below (set_event_subtype_ra_blocked, set_event_subtype_dhcp_blocked, set_network_protocol_dhcp_blocked) were given tags in this PR, so the grok that feeds them is now the only untagged processor in the block.
Recommendation:
Add a tag and a description to the grok processor:
- grok:
description: Parse Blocked ARP, RA and DHCP packet events
tag: grok_blocked_packet_events
field: event.original
patterns:
- '^%{SYSLOGHDR}%{SPACE}%{NUMBER}%{SPACE}%{WORDORHOST}%{SPACE}events%{SPACE}(?<message>%{BLOCKEDARP:_temp.blocked_arp} from %{MAC:source.mac} with IP %{IP:source.ip} on %{NOTSPACE} %{GREEDYDATA:observer.ingress.vlan.id})$'
- '^%{SYSLOGHDR}%{SPACE}%{NUMBER}%{SPACE}%{WORDORHOST}%{SPACE}events%{SPACE}(?<message>%{BLOCKEDRA:_temp.blocked_ra} from %{MAC:source.mac} \(%{IP:source.ip}\) on VLAN %{WORD:observer.ingress.vlan.id}(?: by default)?)$'
- '^%{SYSLOGHDR}%{SPACE}%{NUMBER}%{SPACE}%{WORDORHOST}%{SPACE}events%{SPACE}(?<message>%{BLOCKEDDHCP:_temp.blocked_dhcp} from %{MAC:source.mac} \(%{IP:source.ip}\) on VLAN %{WORD:observer.ingress.vlan.id}(?: by default)?)$'🤖 AI-Generated Review | Vera Review Bot | 📚 Knowledge base: integration-skills
⚠️ Automated review — verify suggestions before applying.
|
✅ All changelog entries have the correct PR link. |
| action: dynamic-frequency-selection-detected | ||
| "aps_association_reject": | ||
| action: association-rejected-for-load-balancing | ||
| "ra_blocked": |
There was a problem hiding this comment.
Severity: 🔵 Low confidence: medium path: packages/cisco_meraki/data_stream/log/elasticsearch/ingest_pipeline/default.yml:218
The new eventmap entries give ra_blocked and dhcp_blocked event.type 'denied', but the sibling arp_blocked subtype still has no eventmap entry, so add one so all three blocked-packet events categorize consistently.
Details
This PR adds ra_blocked and dhcp_blocked to the set-event-type-for-meraki-events eventmap with type: [denied] and hyphenated actions (ra-packet-blocked, dhcp-packet-blocked). arp_blocked — set by the same grok processor in events.yml, for the same class of blocked-packet event — has no eventmap entry, so the painless script falls through to ctx.event.action = ctx.cisco_meraki.event_subtype. The committed expected output confirms the divergence: the ARP document (test-events.log-expected.json:1264) has event.action: "arp_blocked" and event.type: ["info"], while the new RA/DHCP documents have event.action: "ra-packet-blocked" / "dhcp-packet-blocked" and event.type: ["info", "denied"]. A detection rule or dashboard filtering blocked packets on event.type: denied will match RA and DHCP but silently miss ARP.
Recommendation:
Add an arp_blocked entry alongside the two new ones. Keeping action: arp_blocked preserves the value users already query on; if the hyphenated form is preferred for consistency, that is a behaviour change and deserves its own changelog entry.
"arp_blocked":
type:
- denied
action: arp_blocked
"ra_blocked":
type:
- denied
action: ra-packet-blocked
"dhcp_blocked":
type:
- denied
action: dhcp-packet-blockedRegenerate the pipeline test fixtures afterwards with elastic-package test pipeline -g so test-events.log-expected.json picks up the new event.type on the ARP document.
🤖 AI-Generated Review | Vera Review Bot | 📚 Knowledge base: integration-skills
⚠️ Automated review — verify suggestions before applying.
Review summaryIssues found across the latest commits 4e0b0cc — 1 low
Issues found across earlier commits 2dbf4f1 — 1 low
Issues found across earlier commits c234325 — 1 high, 1 low
Issues found across earlier commits cd33362 — 1 medium, 1 low
Issues found across earlier commits d59dbdd — 1 medium, 2 low
🤖 AI-Generated Review | Vera Review Bot | 📚 Knowledge base: integration-skills
|
💚 Build Succeeded
History
|
2 participants
Executive summary
The cisco_meraki events pipeline only handled 'Blocked ARP Packet' events via Grok, causing parse failures for 'Blocked RA Packet' and 'Blocked DHCP Packet' events which share a similar but distinct log format (IP in parentheses, explicit 'VLAN' keyword). The fix adds two new Grok patterns plus corresponding BLOCKEDRA/BLOCKEDDHCP pattern definitions, and two new set processors to assign the correct event_subtype values (ra_blocked, dhcp_blocked). Test fixtures covering both new event types are included and all pipeline tests pass.
Proposed commit message
Root cause
The BLOCKEDARP grok pattern definition in events.yml (line 130) hardcodes the literal
'Blocked ARP Packet'and the rest of the pattern useswith IP %{IP:source.ip}syntax, which does not match Cisco Meraki's 'Blocked RA Packet' events where the IPv6 source address is enclosed in parentheses and the message ends with 'by default'. Both the literal string and the IP capture syntax must be extended to handle the RA variant.Approach
Add a second grok pattern for the RA (Router Advertisement) packet variant to the existing 'Handle Blocked ARP' grok processor in events.yml. The RA format differs structurally from ARP: it wraps the IPv6 address in parentheses and appends 'by default', so a dedicated alternative pattern with
%{BLOCKEDRA:_temp.blocked_ra}is needed. A follow-onsetprocessor maps the new temp capture to a distinctra_blockedevent subtype, preserving existingarp_blockedbehaviour unchanged. A test fixture is added for the sanitized RA event.Implementation
packages/cisco_meraki/data_stream/log/elasticsearch/ingest_pipeline/events.yml(lines 121-143), add a second pattern entry to the existing blocked-ARP grok processor that matches the RA format:'^%{SYSLOGHDR}%{SPACE}%{NUMBER}%{SPACE}%{WORDORHOST}%{SPACE}events%{SPACE}(?<message>%{BLOCKEDRA:_temp.blocked_ra} from %{MAC:source.mac} \(%{IP:source.ip}\) on VLAN %{WORD:observer.ingress.vlan.id}(?: by default)?)$'. Add a newBLOCKEDRA: 'Blocked RA Packet'entry to thepattern_definitionsblock.set event_subtype=arp_blockedprocessor (line 140-143), add a newsetprocessor:field: cisco_meraki.event_subtype, value: ra_blocked, if: ctx._temp?.blocked_ra != null.packages/cisco_meraki/data_stream/log/_dev/test/pipeline/test-events.log, append the sanitized RA test event:<134>1 1782557203.245381197 blvl_01 events Blocked RA Packet from 00-00-5E-00-53-23 (2001:db8::1) on VLAN 201 by default.packages/cisco_meraki/data_stream/log/_dev/test/pipeline/test-events.log-expected.json, append the expected document for the RA event withcisco_meraki.event_subtype: ra_blocked,source.mac: 00-00-5E-00-53-23,source.ip: 2001:db8::1,observer.ingress.vlan.id: 201,message: 'Blocked RA Packet from 00-00-5E-00-53-23 (2001:db8::1) on VLAN 201 by default',log.syslog.priority: 134, andrelated.ip: ['2001:db8::1'].packages/cisco_meraki/changelog.yml, prepend a newversion: 1.31.2entry withtype: bugfixand description:Fix grok pattern to support Blocked RA Packet events in addition to Blocked ARP Packet events.packages/cisco_meraki/manifest.yml, updateversionfrom1.31.1to1.31.2.elastic-package test pipelinefor the cisco_meraki log data stream to confirm both ARP and RA test cases pass.Pipeline changes
BLOCKEDRA(literal 'Blocked RA Packet') to the existing blocked-ARP grok processor's patterns list; new pattern captures MAC tosource.mac, IPv6 (in parens) tosource.ip, and VLAN number toobserver.ingress.vlan.id— with an optional(?: by default)?terminal groupBLOCKEDRA: 'Blocked RA Packet'to the grok processor'spattern_definitionsblocksetprocessor after existingarp_blockedset:field: cisco_meraki.event_subtype, value: ra_blocked, if: ctx._temp?.blocked_ra != nullField / mapping changes
—
Sanitized error message
Provided Grok expressions do not match field value: [on_failure_message]Sanitized log (
event_sanitizedexcerpt)<134>1 1782557203.245381197 blvl_01 events Blocked RA Packet from 00-00-5E-00-53-23 (2001:db8::1) on VLAN 201 by defaultReviewer concerns
on VLAN %{WORD:observer.ingress.vlan.id}while the ARP pattern useson %{NOTSPACE} %{GREEDYDATA:observer.ingress.vlan.id}— reviewers should confirm this difference is intentional and reflects a real format difference between ARP and RA/DHCP log lines.ifguard requirescisco_meraki.event_subtype == "blocked"to be set by an upstream processor; the test passing confirms this path works, but reviewers should verify that RA/DHCP blocked events are reliably classified asblockedsubtype in the earlier pipeline stage.Self-review findings
—
Risk and classification
Links
17f5589529993a8f