Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Buffer the files in preprocessor to prevent EACCESS errors. #566

Closed
iammerrick opened this issue Jun 10, 2013 · 1 comment
Closed

Buffer the files in preprocessor to prevent EACCESS errors. #566

iammerrick opened this issue Jun 10, 2013 · 1 comment

Comments

@iammerrick
Copy link
Contributor

Currently the preprocessor functionality is not resilient to a large number of files... At the small number of 256 you will reach the Mountain Lion limit... The solution to this is to implement a buffer that queues requests beyond a particular limit and flushes them accordingly. @isaacs has an implementation of this as a drop in replacement for fs called node-graceful-fs. The tests do not pass when we drop this in however, I think because karma is mocking fs.

From my perspective we have two options...

  1. We can either update the tests fs mocking to not collide with node-graceful-fs, not really sure how to do this currently, the problem is still unclear to me.
  2. We can implement our own queue system inside of preprocessor for read's only.

Please let me know which approach you want to take and I will look into implementing it. Currently our project can't use preprocessing because of this limitation.

(I don't consider calling ulimit as a viable solution.)

@vojtajina
Copy link
Contributor

@iammerrick thanks a bunch for looking into this!

I know about this issue and yep, it has to be fixed asap. I need to look into it more, there are multiple things how to improve it. Using graceful-fs is probably the first thing we should do...

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants