pyspark.sql.functions.get

pyspark.sql.functions.get(col, index)

Array function: Returns the element of an array at the given (0-based) index. If the index points outside of the array boundaries, then this function returns NULL.

New in version 3.4.0.

Parameters

colColumn or str

Name of the column containing the array.

indexColumn or str or int

Index to check for in the array.

Returns

Column

Value at the given position.

See also

element_at()

Notes

The position is not 1-based, but 0-based index. Supports Spark Connect.

Examples

Example 1: Getting an element at a fixed position

>>> from pyspark.sql import functions as sf
>>> df = spark.createDataFrame([(["a", "b", "c"],)], ['data'])
>>> df.select(sf.get(df.data, 1)).show()
+------------+
|get(data, 1)|
+------------+
|           b|
+------------+

Example 2: Getting an element at a position outside the array boundaries

>>> from pyspark.sql import functions as sf
>>> df = spark.createDataFrame([(["a", "b", "c"],)], ['data'])
>>> df.select(sf.get(df.data, 3)).show()
+------------+
|get(data, 3)|
+------------+
|        NULL|
+------------+

Example 3: Getting an element at a position specified by another column

>>> from pyspark.sql import functions as sf
>>> df = spark.createDataFrame([(["a", "b", "c"], 2)], ['data', 'index'])
>>> df.select(sf.get(df.data, df.index)).show()
+----------------+
|get(data, index)|
+----------------+
|               c|
+----------------+

Example 4: Getting an element at a position calculated from another column

>>> from pyspark.sql import functions as sf
>>> df = spark.createDataFrame([(["a", "b", "c"], 2)], ['data', 'index'])
>>> df.select(sf.get(df.data, df.index - 1)).show()
+----------------------+
|get(data, (index - 1))|
+----------------------+
|                     b|
+----------------------+

Example 5: Getting an element at a negative position

>>> from pyspark.sql import functions as sf
>>> df = spark.createDataFrame([(["a", "b", "c"], )], ['data'])
>>> df.select(sf.get(df.data, -1)).show()
+-------------+
|get(data, -1)|
+-------------+
|         NULL|
+-------------+