DataFrame count method #5071
DataFrame count method #5071
Conversation
datafusion/core/src/dataframe.rs
Outdated
| /// # Ok(()) | ||
| /// # } | ||
| /// ``` | ||
| pub async fn len(self) -> Result<usize> { |
There was a problem hiding this comment.
Hmm, could it be count? For DataFrame-like API (pandas, Spark, etc.), there is commonly count for this.
There was a problem hiding this comment.
yeah that makes sense, changed
|
clippy failures can be fixed by merging latest from master |
|
Benchmark runs are scheduled for baseline = 7d2318b and contender = a780a0b. a780a0b is a master commit associated with this PR. Results will be available as each benchmark for each run completes. |
| let rows = self | ||
| .aggregate( | ||
| vec![], | ||
| vec![datafusion_expr::count(Expr::Literal(ScalarValue::Null))], |
There was a problem hiding this comment.
This was actually working by accident as a result of a quirk of NullArray whereby it doesn't have a null buffer despite all values being null. The count reported by this query should be 0 as only non-null values are counted.
Which issue does this PR close?
Closes #1926.
Rationale for this change
What changes are included in this PR?
New
count()method on DataFrameAre these changes tested?
New unit test
Are there any user-facing changes?