HOME Resources Blog Leveraging Metrics in Google SecOps SIEM for Enhanced Threat Detection

|

Daniel Koifman

Leveraging Metrics in Google SecOps SIEM for Enhanced Threat Detection

Google Security Operations (SecOps) SIEM offers powerful tools for threat detection, and one of its most valuable features is the use of metrics within YARA-L rules. Metrics allow you to aggregate historical data over extended time periods, enabling the creation of sophisticated detections for risk analytics. This article will guide you through the use of metrics, demonstrating how to apply them in practical scenarios to enhance your security posture.

Metrics in Google SecOps SIEM provide a method to aggregate historical data that can then be applied in a YARA-L rule. These metrics span various datasets, including network, authentication, and endpoint data. The metrics function is always located within the outcome section of a YARA-L rule. To find the list of possible metrics, and their associated allowed combinations of variables, visit Google’s documentation page.

Named Parameters for Metric Functions

Metric functions use named parameters, which can be specified in any order. These parameters include:

  • Period: The length of time over which individual log events are combined into a single observation. The allowed values are 1h and 1d.
    • Note: Hourly metrics (1h) can only be used in rules that also use daily metrics (1d).
  • Window: The length of time over which individual observations are aggregated into a single value. Allowed values depend on the period:
    • period:1h : window:today
    • period:1d : window:30d
  • Metric: The specific type of data to aggregate. Five metric types are supported:
    • event_count_sum: Number of unique log events within each period.
    • first_seen: First seen timestamp of a matching log event within each period.
    • last_seen: Last seen timestamp of a matching log event within each period.
    • value_sum: Sum of the number of bytes in all log events combined within the period. This is only used with metrics that have “bytes” in the name.
    • num_unique_filter_values: Metric that is computed during rule execution, not pre-computed.
  • Agg: The type of aggregation applied to the metric over the entire window.
    • avg: Average value per period.
    • max: Greatest value per period.
    • min: Smallest value per period.
    • num_metric_periods: Number of periods within the time window that had a non-zero metric value.
    • stddev: Standard deviation of the value per period.
    • sum: Sum of each value per period, over the entire window.
  • Filter: Allows filtering metrics before aggregation by a value in the pre-computed metric. Filters can be any valid events expression without event fields or placeholders.
  • UDM Fields: Metrics are filtered by 1, 2, or 3 UDM fields, depending on the function.
    • Dimensions (Required): Different combinations are listed in the documentation.
    • Namespaces (Optional): Can be used for entities specified in dimensions.

The metric function groups data using a combination of time-based bucketing (defined by the period and window) and UDM field-based grouping. When a rule includes a placeholder variable in the events section, and that variable is also used in the metric function, the metric data will be grouped by that UDM field.

For instance, if an event includes $net.principal.ip = $ip and the metric function uses principal.asset.ip: $ip, the calculation will be done for each unique IP address represented by the placeholder variable $ip. Some metrics can group data across multiple UDM fields (up to three), which allows for more specific and precise analysis. For example, you could group file execution data by metadata.event_type, principal.asset.hostname, and principal.process.file.sha256 to analyze the execution of specific files on specific hosts.

Let’s explore how to use these metrics in real-world scenarios.

Example 1: Monitoring Network Bytes

The rule below calculates the maximum number of outbound bytes for a specific IP address over a 30-day window: 

rule metric_examples_network {
  meta:
    author = "Google Cloud Security"
  events:
    $net.metadata.event_type = "NETWORK_CONNECTION"
    $net.network.sent_bytes > 0
    $net.principal.ip = $ip
    $net.principal.ip = "10.128.0.21"
  match:
    $ip over 1d
  outcome:
    $max_byte_count_window = 
max(metrics.network_bytes_outbound(
      period:1d,
      window:30d,
      metric:value_sum,
      agg:max,
      principal.asset.ip: $ip
    ))
  condition:
    $net and $max_byte_count_window > 0
}

More precisely, the metric function first divides the data into time-based buckets:

  • The period:1d parameter specifies that the data is initially grouped into buckets, each representing one day’s worth of log events.
  • The window:30d parameter defines a 30-day window over which these daily buckets will be aggregated

Within each of these daily buckets, the metric function aggregates the data based on the specified criteria:

  • The example uses metrics.network_bytes_outbound, which focuses on events where network.sent_bytes is greater than 0.
  • The metric:value_sum parameter instructs the function to sum the values of the network.sent_bytes field for all relevant events within each daily bucket.

The metric function groups the data by the UDM field principal.asset.ip. The parameter principal.asset.ip:$ip specifies that the calculations should be done separately for each unique IP address represented by the placeholder variable $ip.

After calculating the sum of outbound bytes for each day in the 30-day window, the metric function performs a final aggregation across those daily sums:

  • The agg:max parameter tells the function to return the highest of the daily sums calculated in the previous step, effectively identifying the day with the greatest number of outbound bytes for that IP address within the 30-day window.

In summary, the rule uses the metric function to look at a 30-day window, creating daily buckets of data. Within each of those daily buckets, the function calculates the sum of outbound bytes for the specified IP address. Finally, it returns the maximum of these daily sums, giving the greatest number of outbound bytes in the 30 day window.

Example 2: First-Seen Logins

This next rule calculates whether a successful user login is the first successful login for that user within the past 30 days.

rule metric_examples_success_authentication {
  meta:
    author = "Google Cloud Security"
  
  events:
    $login.metadata.event_type = "USER_LOGIN"
    $login.security_result.action = "ALLOW"
    $login.target.user.userid != /\$$/
    strings.to_lower($login.target.user.userid) = $userid
  
  match: 
    $userid over 1d
  
  outcome:
    $first_seen_login_window = max(metrics.auth_attempts_success(
      period:1d,
      window:30d,
      metric:first_seen,
      agg:min,
      target.user.userid: $userid
    
    ))
    
    $systems_accessed = array_distinct($login.principal.hostname)

  condition:
    
    $login and $first_seen_login_window = 0
}

To be more precise, the metric function first divides the historical data into time-based observations:

  • The period:1d parameter specifies that the historical data is initially considered in observations, each representing one day.
  • The window:30d parameter defines a 30-day lookback window during which these daily observations are analyzed.

Within each of these daily observations, the metric function considers specific login events:

  • The example uses metrics.auth_attempts_success, which focuses on user login events where at least one SecurityResult.Action was ALLOW.
  • The metric:first_seen parameter instructs the function to identify the timestamp of the very first successful login event for a given user within each daily period. If there are no successful logins for a user in a day within the window, there is no “first seen” timestamp for that day for that user.

The metric function groups the analysis by the UDM field target.user.userid.

The parameter target.user.userid: $userid specifies that the calculation of the “first seen” timestamp should be done separately for each unique user ID represented by the placeholder variable $userid.

After identifying the “first seen” timestamp (if any) for each user within each day of the 30-day window, the metric function performs a final aggregation across these daily “first seen” timestamps:

  • The agg:min parameter tells the function to find the earliest “first seen” timestamp recorded for a given user across the entire 30-day window. If a user has no successful logins within the 30-day window, the first_seen metric will return a default value of 0. The outer max() function around the metric function in the outcome section is required for multi-event rules to aggregate outcome variables, and in this case, since we are looking at the first seen event for a user over 30 days, the earliest timestamp (or 0) will be the effective result.

In summary, this part of the rule uses the metric function to examine a 30-day history of successful user logins. For each user, it determines the timestamp of their earliest successful login within this period. The rule then checks if this earliest timestamp is 0, indicating that this successful login within the current one-day match window is the first successful login for that user in the last 30 days, thus flagging a potentially new or unusual login event.

About the Author

Daniel Koifman is a Security Researcher currently working at CardinalOps. Previously, he worked as a Detection Engineer for a Fortune 500 financial institution and a top Israeli MSSP. He is also an active contributor of various open-source repositories and loves to participate in various capture the flag (CTF) competitions.