Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Correct the Spread command in xbrl_get_data( ) #13

Open
Karnner2 opened this issue Apr 22, 2020 · 0 comments
Open

Correct the Spread command in xbrl_get_data( ) #13

Karnner2 opened this issue Apr 22, 2020 · 0 comments

Comments

@Karnner2
Copy link

Karnner2 commented Apr 22, 2020

Hi! There may have been a change to the SEC reporting of the XBRL. I spent some time correcting the code so it works.

In the XBRL_get_data() (called by XBRL_get_statements function) you have a set of contextId and elementId filters. Brilliant work in getting this to adapt the correct filters from the underlying taxonomy -- this is truly a work of art in how you managed it but! Spread is prone to error because it fails if there are duplicate elements with different row ids (which there are if there is a new row in the financial statements). This is a catastrophic error -- I say catastrophic because the program virtually fails at random over various years.

Please remove the dyplr::spread__(elementId, fact ) command and replace it with the following:

reshape2::dcast(formula = contextId + 
                  startDate + endDate + decimals ~ elementId, value.var = "fact", 
                fun.aggregate = mean)

While I have chosen fun.aggregate = mean, this does not actually affect anything and I have validated that it does the correct pull. Sum also seems to work -- but I figured if there was some sort of aberrant issue it would be better to average than sum.

The reshape2 will l do a specific long-to-wide conversion where Spread fails. We can see in various R templates that reshape2 is the obsidian blade to Spread's steel surgeon's knife with respect to directness. I've tested it and it seems to be a general correction to the problems of the package. You can simply paste it where the spread_ command is and it should work fine.

I am also forking this project because I am getting tired of doing trace('xbrl_get_data', edit=TRUE) every time I load, R. Hope you don't mind! But I hope you will consider the adjustment so I can download directly from your repository in the future!

All the best, JTFJ

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant