Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

sum distributed array on cluster very very slow #8

Closed
armgong opened this issue Mar 26, 2015 · 3 comments · Fixed by #9
Closed

sum distributed array on cluster very very slow #8

armgong opened this issue Mar 26, 2015 · 3 comments · Fixed by #9

Comments

@armgong
Copy link

armgong commented Mar 26, 2015

following code very very slow on sum(DA), it takes ten minutes and still no result,so I interrupted it use ctrl+c.
this shouldn't happens if DistributedArrays operations are performed over the localpart chunks and then aggregated. maybe there are some issues?

 x=Array(ASCIIString,6)
 x[1]="arch01"
 x[2]="arch02"
 x[3]="arch03"
 x[4]="arch04"
 x[5]="arch05"
 x[6]="arch06"
addprocs(x)
@everywhere using DistributedArrays
A = fill(1.1, (1000,1000));
sum(A)
DA = distribute(A);
sum(DA)
sum(A) == sum(DA)
@amitmurthy
Copy link
Contributor

Most of the standard functions are yet to be written to support DArrays. The generic sum is currently fetching elements of the DArray onto the calling process one-by-one and summing locally.

At least for the functions that operate on Arrays - sum, minimum, maximum, etc. the corresponding DArray versions could be generated with the implementation using a distributed map ?

@ViralBShah
Copy link
Member

👍

@armgong
Copy link
Author

armgong commented Mar 29, 2015

thank you very much

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants