35 Pull requests opened by 27 people

• [SPARK-48549][SQL] Restrict the number of parameters for function `sentences` to `1` or `3`

  #46880 opened Jun 5, 2024

• [SPARK-48541][CORE] Add a new exit code for executors killed by TaskReaper

  #46883 opened Jun 5, 2024

• [SPARK-48543][SS] Track state row validation failures using explicit error class

  #46885 opened Jun 5, 2024

• [SPARK-48547][DEPLOY] Add opt-in flag to have SparkSubmit automatically call System.exit after user code main method exits

  #46889 opened Jun 5, 2024

• [Test Only]

  #46893 opened Jun 6, 2024

• [SPARK-48577][SQL] Invalid UTF-8 byte sequence replacement

  #46899 opened Jun 6, 2024

• [SPARK-48555][SQL][PYTHON][CONNECT] Support using Columns as parameters for several functions in pyspark/scala

  #46901 opened Jun 6, 2024

• [SPARK-48557][SQL] Support scalar subquery with group-by on column equal to constant

  #46902 opened Jun 6, 2024

• [SPARK-48558][CORE] Improve accumulator V2 error message

  #46904 opened Jun 6, 2024

• [SPARK-47777] fix python streaming data source connect test

  #46906 opened Jun 6, 2024

• [WIP] [PROOF OF CONCEPT] [SPARK] [SQL] Collation Mode

  #46917 opened Jun 7, 2024

• [SPARK-48566][PYTHON] Fix bug where partition indices are incorrect when UDTF analyze() uses both select and partitionColumns

  #46918 opened Jun 7, 2024

• [SPARK-48567][SS] StreamingQuery.lastProgress should return the actual StreamingQueryProgress

  #46921 opened Jun 8, 2024

• [PYTHON] Fix pyspark.sql.functions.reduce docstring typo

  #46923 opened Jun 9, 2024

• [WIP][SPARK-48311][SQL] Fix nested pythonUDF in groupBy and aggregate Bind…

  #46925 opened Jun 10, 2024

• [WIP][SQL] Collation support for Char/Varchar (v2)

  #46926 opened Jun 10, 2024

• Corrected row index usage when exploding packed arrays in vectorized reader

  #46928 opened Jun 10, 2024

• [DO-NOT-REVIEW][SPARK-48480][SS][Connect] StreamingQueryListener should not be affected by spark.interrupt()

  #46929 opened Jun 10, 2024

• [SPARK-48580] Add consistency check and fallback for mapIds in push-merged block meta

  #46934 opened Jun 11, 2024

• [SPARK-48583][SQL][TESTS] Replace deprecated classes and methods of `commons-io` called in Spark

  #46935 opened Jun 11, 2024

• [SPARK-48585][SQL] Make `JdbcDialect.classifyException` throw out the `original` exception

  #46937 opened Jun 11, 2024

• [SPARK-48587][VARIANT] Avoid storage amplification when accessing a sub-Variant

  #46941 opened Jun 11, 2024

• [WIP] [SPARK-48586] Remove lock contention between maintenance and task threads

  #46942 opened Jun 11, 2024

• [WIP][SPARK-48572] Fix DateSub, DateAdd, WindowTime, TimeWindow and SessionWindow expressions

  #46943 opened Jun 11, 2024

• [SPARK-48589][SQL][SS] Add option snapshotStartBatchId and snapshotPartitionId to state data source

  #46944 opened Jun 11, 2024

• [SPARK-48590][BUILD] Upgrade netty to `4.1.111.Final`

  #46945 opened Jun 11, 2024

• [SPARK-48591][PYTHON] Simplify the if-else branches with `F.lit`

  #46946 opened Jun 12, 2024

• [SPARK-48592][INFRA] Add scala style check for logging message inline variables

  #46947 opened Jun 12, 2024

• [SPARK-48593][PYTHON][CONNECT] Fix the string representation of lambda function

  #46948 opened Jun 12, 2024

• [SPARK-48594][PYTHON][CONNECT] Rename `parent` field to `child` in `ColumnAlias`

  #46949 opened Jun 12, 2024

• [SPARK-47577][SPARK-47579] Correct misleading usage of log key TASK_ID

  #46951 opened Jun 12, 2024

• [SPARK-48596][SQL] Perf improvement for calculating hex string for long

  #46952 opened Jun 12, 2024

• [SPARK-48597][SQL] Introduce a marker for isStreaming property in text representation of logical plan

  #46953 opened Jun 12, 2024

• [SPARK-48598][PYTHON][CONNECT] Propagate cached schema in dataframe operations

  #46954 opened Jun 12, 2024

• [SPARK-48601] Give a more user friendly error message when setting a null value for JDBC Option

  #46955 opened Jun 12, 2024

35 Unresolved conversations

Sometimes conversations happen on old items that aren’t yet closed. Here is a list of all the Issues and Pull Requests with unresolved conversations.

• [SPARK-48578][SQL] UTF8 string validation

  #46845 commented on Jun 11, 2024 • 36 new comments

• [SPARK-47172][CORE] Add support for AES-GCM for RPC encryption

  #46515 commented on Jun 12, 2024 • 29 new comments

• [SPARK-48342][SQL] Introduction of SQL Scripting Parser

  #46665 commented on Jun 12, 2024 • 22 new comments

• [SPARK-48280][SQL] Improve collation testing surface area using expression walking

  #46801 commented on Jun 12, 2024 • 15 new comments

• [SPARK-48511][SS] Remove TimeMode None from TransformWithState.

  #46825 commented on Jun 10, 2024 • 11 new comments

• [SPARK-36680][SQL] Supports Dynamic Table Options for Spark SQL

  #46707 commented on Jun 10, 2024 • 8 new comments

• [SPARK-48532][BUILD] Upgrade maven plugin to latest version

  #46870 commented on Jun 11, 2024 • 7 new comments

• [SPARK-48421][SQL] SPJ: Add documentation

  #46745 commented on Jun 11, 2024 • 5 new comments

• [SPARK-47910][CORE] close stream when DiskBlockObjectWriter closeResources to avoid memory leak

  #46131 commented on Jun 12, 2024 • 4 new comments

• [SPARK-48495][SQL][DOCS] Describe shredding scheme for Variant

  #46831 commented on Jun 8, 2024 • 4 new comments

• [SPARK-47353][SQL] Enable collation support for the Mode expression

  #46597 commented on Jun 11, 2024 • 4 new comments

• [SPARK-48466][SQL] Create dedicated node for EmptyRelation in AQE

  #46830 commented on Jun 12, 2024 • 4 new comments

• [SPARK-48416][SQL] Support related nested WITH expression

  #46741 commented on Jun 10, 2024 • 3 new comments

• [SPARK-48510][2/2] Support UDAF `toColumn` API in Spark Connect

  #46849 commented on Jun 7, 2024 • 2 new comments

• [SPARK-24497][SQL] Support recursive SQL

  #40744 commented on Jun 11, 2024 • 1 new comment

• [SPARK-48302][PYTHON] Preserve nulls in map columns in PyArrow Tables

  #46837 commented on Jun 7, 2024 • 1 new comment

• [SPARK-46798][Structured Streaming] Kafka custom partition location assignment (rack awareness)

  #46863 commented on Jun 12, 2024 • 1 new comment

• [SPARK-43301][CORE][SHUFFLE] BlockStoreClient getHostLocalDirs RPC supports IOException retry

  #46805 commented on Jun 10, 2024 • 1 new comment

• [SPARK-46714][SQL] Overwrite a partition with custom location

  #44725 commented on Jun 6, 2024 • 1 new comment

• [SPARK-46741][SQL] Cache Table with CTE won't work

  #44767 commented on Jun 11, 2024 • 1 new comment

• [SPARK-47050][SQL] Collect and publish partition level metrics

  #45123 commented on Jun 6, 2024 • 1 new comment

• [SPARK-47547][SQL] Fix inaccurate false positive rates when N is large for Bloom Filter

  #46370 commented on Jun 8, 2024 • 1 new comment

• [SPARK-47336][SQL][CONNECT] Provide to PySpark a functionality to get estimated size of DataFrame in bytes

  #46368 commented on Jun 7, 2024 • 1 new comment

• [SPARK-47050][SQL] Collect and publish partition level metrics for V1

  #46188 commented on Jun 6, 2024 • 1 new comment

• [SPARK-48524][SQL] Make Not IsNull semantically equal to IsNotNull and vice versa

  #46865 commented on Jun 6, 2024 • 0 new comments

• [WIP][SQL] Collation support for Char/Varchar

  #46868 commented on Jun 10, 2024 • 0 new comments

• [SPARK-48537][SQL] Support Hive UDAF inherited from `GenericUDAFResolver` or `GenericUDAFResolver2`

  #46878 commented on Jun 5, 2024 • 0 new comments

• [SPARK-48505][CORE] Simplify the implementation of `Utils#isG1GC`

  #46873 commented on Jun 12, 2024 • 0 new comments

• [SPARK-48497][PYTHON][DOCS] Add an example for Python data source writer in user guide

  #46833 commented on Jun 6, 2024 • 0 new comments

• [SPARK-48493][PYTHON] Enhance Python Datasource Reader with direct Arrow Batch support for improved performance

  #46826 commented on Jun 6, 2024 • 0 new comments

• [SPARK-48441][SQL] Fix StringTrim behaviour for non-UTF8_BINARY collations

  #46762 commented on Jun 10, 2024 • 0 new comments

• [SPARK-48440][SQL] Fix StringTranslate behaviour for non-UTF8_BINARY collations

  #46761 commented on Jun 10, 2024 • 0 new comments

• [SPARK-44473][SQL] [BUG] Overwriting the same partition of a partitioned table multiple times with empty data yields non-idempotent results

  #46699 commented on Jun 12, 2024 • 0 new comments

• [SPARK-47482] Add HiveDialect to sql module

  #45644 commented on Jun 11, 2024 • 0 new comments

• [SPARK-19335][SPARK-38200][SQL] Add upserts for writing to JDBC using MERGE INTO with temp table

  #41611 commented on Jun 7, 2024 • 0 new comments