mv-apply operator - Azure Data Explorer & Real-Time Intelligence (2024)

Applies a subquery to each record, and returns the union of the results ofall subqueries.

For example, assume a table T has a column Metric of type dynamicwhose values are arrays of real numbers. The following query locates thetwo biggest values in each Metric value, and return the records correspondingto these values.

T | mv-apply Metric to typeof(real) on ( top 2 by Metric desc)

The mv-apply operator has the followingprocessing steps:

1. Uses the mv-expand operator to expand each record in the input into subtables (order is preserved).
2. Applies the subquery for each of the subtables.
3. Adds zero or more columns to the resulting subtable. These columns contain the values of the source columns that aren't expanded, and are repeated where needed.
4. Returns the union of the results.

The mv-apply operator gets the following inputs:

1. One or more expressions that evaluate into dynamic arrays to expand.The number of records in each expanded subtable is the maximum length ofeach of those dynamic arrays. Null values are added where multiple expressions are specified and the corresponding arrays have different lengths.

2. Optionally, the names to assign the values of the expressions after expansion.These names become the columns names in the subtables.If not specified, the original name of the column is used when the expression is a column reference. A random name is used otherwise.

   Note

   It is recommended to use the default column names.

   See Also

   Approximate, partial and combined lookups in Azure SentinelKQL’s mv-apply command – Yet Another Security BlogCalculating downtime using Window functions and Time Series functions in Azure Data Explorermv-apply-Operator - Azure Data Explorer & Real-Time Intelligence

3. The data types of the elements of those dynamic arrays, after expansion.These become the column types of the columns in the subtables.If not specified, dynamic is used.

4. Optionally, the name of a column to add to the subtables that specifies the0-based index of the element in the array that resulted in the subtable record.

5. Optionally, the maximum number of array elements to expand.

The mv-apply operator can be thought of as a generalization of themv-expand operator (in fact, the latter can be implementedby the former, if the subquery includes only projections.)

Syntax

T | mv-apply [ItemIndex] ColumnsToExpand [RowLimit] on ( SubQuery )

Where ItemIndex has the syntax:

with_itemindex = IndexColumnName

ColumnsToExpand is a comma-separated list of one or more elements of the form:

[Name =] ArrayExpression [to typeof (Typename)]

RowLimit is simply:

limit RowLimit

and SubQuery has the same syntax of any query statement.

Learn more about syntax conventions.

Parameters

Examples

Getting the largest element from the array

let _data = range x from 1 to 8 step 1 | summarize l=make_list(x) by xMod2 = x % 2;_data| mv-apply element=l to typeof(long) on ( top 1 by element )

Output

Calculating the sum of the largest two elements in an array

let _data = range x from 1 to 8 step 1 | summarize l=make_list(x) by xMod2 = x % 2;_data| mv-apply l to typeof(long) on ( top 2 by l | summarize SumOfTop2=sum(l) )

Output

Select elements in arrays

datatable (Val:int, Arr1:dynamic, Arr2:dynamic)[ 1, dynamic(['A1', 'A2', 'A3']), dynamic([10, 30, 7]), 7, dynamic(['B1', 'B2', 'B5']), dynamic([15, 11, 50]), 3, dynamic(['C1', 'C2', 'C3', 'C4']), dynamic([6, 40, 20, 8])] | mv-apply NewArr1=Arr1, NewArr2=Arr2 to typeof(long) on ( top 2 by NewArr2 | summarize NewArr1=make_list(NewArr1), NewArr2=make_list(NewArr2))

Output

Using with_itemindex for working with a subset of the array

let _data = range x from 1 to 10 step 1 | summarize l=make_list(x) by xMod2 = x % 2;_data| mv-apply with_itemindex=index element=l to typeof(long) on ( // here you have 'index' column where index >= 3 )| project index, element

Output

Using mutiple columns to join element of 2 arrays

datatable (Val: int, Arr1: dynamic, Arr2: dynamic)[ 1, dynamic(['A1', 'A2', 'A3']), dynamic(['B1', 'B2', 'B3']), 5, dynamic(['C1', 'C2']), dynamic(['D1', 'D2'])] | mv-apply Arr1, Arr2 on ( extend Out = strcat(Arr1, "_", Arr2) | summarize Arr1 = make_list(Arr1), Arr2 = make_list(Arr2), Out= make_list(Out) )

Output

Applying mv-apply to a property bag

In the following example, mv-apply is used in combination with aninner mv-expand to remove values that don't start with "555" from a property bag:

datatable(SourceNumber: string, TargetNumber: string, CharsCount: long)[ '555-555-1234', '555-555-1212', 46, '555-555-1212', '', int(null)]| extend values = pack_all()| mv-apply removeProperties = values on ( mv-expand kind = array values | where values[1] !startswith "555" | summarize propsToRemove = make_set(values[0]) )| extend values = bag_remove_keys(values, propsToRemove)| project-away propsToRemove

Output

Related content

• mv-expand operator