Make as_data_frame a generic so can work on matrices?

[jennybc]

I love love love data_frame(). I wish I could get that behaviour when converting an existing matrix to a data.frame.

> foo <- matrix(LETTERS[1:6], nrow = 3)
> data_frame(foo)      # nope
Error: data_frames can not contain data.frames, matrices or arrays
> data.frame(foo)      # this works, BTW
  X1 X2
1  A  D
2  B  E
3  C  F
> as.data.frame(foo)   # but this is probably better
  V1 V2
1  A  D
2  B  E
3  C  F
> str(data.frame(foo)) # but the factor conversion is annoying
'data.frame':   3 obs. of  2 variables:
 $ X1: Factor w/ 3 levels "A","B","C": 1 2 3
 $ X2: Factor w/ 3 levels "D","E","F": 1 2 3


> ## what I WISH I could do -- warning: I made this up!
> (foo_dream <- as_data_frame(foo))
Source: local data frame [3 x 2]
            V1           V2
1            A            D
2            B            E
3            C            F
> str(foo_dream)
Classes ‘tbl_df’, ‘tbl’ and 'data.frame':   3 obs. of  2 variables:
 $ V1: chr  "A" "B" "C"
 $ V2: chr  "D" "E" "F"

[romainfrancois]

We already have as_data_frame but it works on lists, not matrices:

> as_data_frame( list( a = 1:10, b = letters[1:10] ) )
Source: local data frame [10 x 2]

    a b
1   1 a
2   2 b
3   3 c
4   4 d
5   5 e
6   6 f
7   7 g
8   8 h
9   9 i
10 10 j
> as_data_frame( list( a = 1:10, b = letters[1:10] ) ) %>% str
Classes ‘tbl_df’, ‘tbl’ and 'data.frame':   10 obs. of  2 variables:
 $ a: int  1 2 3 4 5 6 7 8 9 10
 $ b: chr  "a" "b" "c" "d" ...

[hadley]

Maybe we need to make as_data_frame() generic? But that would make it slower. We could just add a second path for matrices

[jennybc]

I have amused myself with this workaround now that I know about existing as_data_frame():

> plyr::alply(foo, 2) %>% as_data_frame
Source: local data frame [3 x 2]

  1 2
1 A D
2 B E
3 C F

[klmr]

To chime in, I do the following all over the place:

result = some_function_call() %>%
    as.data.frame() %>%
    as_data_frame()

Furthermore, I’m not sure I understand the need for the function tbl_df, which does just one additional check. So I suggest merging the current functionality of as_data_frame into tbl_df, and providing a more helpful as_data_frame.

[davharris]

This issue was referenced on @jennybc's Twitter account today, so I thought now might be a good time to chime in.

I also use @klmr's pattern very frequently (except my code usually looks more like as.data.frame(stringsAsFactors = FALSE) %>% tbl_df).

I'm not sure I understand @hadley's concern about speed. I thought that generics only took a few extra thousandths of a second. Is as_data_frame called often enough that this extra time is likely to matter?

If speed is still a concern, then I'd also be nearly as happy with a function like this one:

tbl_df_generic = function(x){
    x%>% as.data.frame(stringsAsFactors = FALSE) %>% tbl_df`)
}

It would add a small amount of cognitive overhead compared to just using a generic tbl_df, but it would still be way less overhead than needing to remember stringsAsFactors = FALSE.

[hadley]

@davharris I'm concerned about speed because sometimes I apply as_data_frame() to a list that contains >100,000 things. Some of that might go away if rbind_rows() could work with lists directly, but I need to think it through and do some benchmarking.

[davharris]

@hadley Okay, I hadn't realized it would be called that many times.

Here's some quick benchmarking that might be useful.

# a new generic for as_data_frame
as_data_frame_generic = function(x){
  UseMethod("as_data_frame_generic")
}

# a list method for the new generic
as_data_frame_generic.list = function(x){
  as_data_frame(x)
}

# A small list to use as an example.
# If the list is small, then dispatch will take proportionately longer
my_list = list(
  a = letters,
  A = LETTERS,
  one = 1:26,
  two = 26:1
)

library(microbenchmark)

microbenchmark(
  no_dispatch = as_data_frame(my_list),
  dispatch = as_data_frame_generic(my_list),
  times = 5E5
)

Here are the results on my laptop:

Unit: microseconds
        expr    min     lq     mean median     uq      max neval
 no_dispatch 40.307 44.849 51.16531 46.135 51.646 88714.88 5e+05
    dispatch 42.263 46.865 53.54021 48.212 54.082 84428.73 5e+05

It looks like this dispatch implementation increases the computation time on a half-million lists from 25.6 seconds to 26.8 seconds.

It probably makes sense to do more extensive benchmarking (for example, would things slow down if we added more methods?). So far, things look pretty good, though.

[hadley]

@davharris thanks - that seems reasonable and suggests that I shouldn't worry about it. I'll make the change when I'm next working on dplyr. An efficient implementation of as_data_frame.matrix() in C++ should be reasonable simple.

[davharris]

@hadley Perfect. Thanks.

On May 13, 2015, at 6:13 AM, Hadley Wickham [email protected] wrote:

@davharris thanks - that seems reasonable and suggests that I shouldn't worry about it. I'll make the change when I'm next working on dplyr. An efficient implementation of as_data_frame.matrix() in C++ should be reasonable simple.

—
Reply to this email directly or view it on GitHub.

[jennybc]

Déjà vu: trying to turn a character matrix into a tbl and rediscovered my own issue.

Bump.

library(dplyr)
ip <- installed.packages() %>% as.tbl()
#> Error in UseMethod("as.tbl"): no applicable method for 'as.tbl' applied to an object of class "c('matrix', 'character')"
ip <- installed.packages() %>% as_data_frame()
#> Error: is.list(x) is not TRUE

[hadley]

I actually just wrote an efficient implementation for this for tidyr - I'll hoist it up into dplyr.