pyspark.sql.functions.to_timestamp#

pyspark.sql.functions.to_timestamp(col, format=None)[source]#

Converts a Column into pyspark.sql.types.TimestampType using the optionally specified format. Specify formats according to datetime pattern. By default, it follows casting rules to pyspark.sql.types.TimestampType if the format is omitted. Equivalent to col.cast("timestamp").

New in version 2.2.0.

Changed in version 3.4.0: Supports Spark Connect.

Parameters

colColumn or str: column values to convert.
format: str, optional: format to use to convert timestamp values.

Returns

Column: timestamp value as pyspark.sql.types.TimestampType type.

Examples

Example 1: Convert string to a timestamp

>>> import pyspark.sql.functions as sf
>>> df = spark.createDataFrame([('1997-02-28 10:30:00',)], ['t'])
>>> df.select(sf.try_to_timestamp(df.t).alias('dt')).show()
+-------------------+
|                 dt|
+-------------------+
|1997-02-28 10:30:00|
+-------------------+

Example 2: Convert string to a timestamp with a format

>>> import pyspark.sql.functions as sf
>>> df = spark.createDataFrame([('1997-02-28 10:30:00',)], ['t'])
>>> df.select(sf.try_to_timestamp(df.t, sf.lit('yyyy-MM-dd HH:mm:ss')).alias('dt')).show()
+-------------------+
|                 dt|
+-------------------+
|1997-02-28 10:30:00|
+-------------------+