Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Provide Job Name in submitterDID submitting a Job to sciserver compute?? #1830

Closed
umesh-timalsina opened this issue Aug 4, 2020 · 2 comments

Comments

@umesh-timalsina
Copy link
Contributor

umesh-timalsina commented Aug 4, 2020

It might be nice to submit a Job in sciserver-compute with its name, it will be easier to debug in dashboard when executing a pipeline with multiple jobs. However, there's no clear advantage other than that.

image

@umesh-timalsina umesh-timalsina changed the title Provide Job Name in submitterDID submitting a Job to sciserver compute Provide Job Name in submitterDID submitting a Job to sciserver compute?? Aug 4, 2020
@brollb
Copy link
Contributor

brollb commented Aug 5, 2020

This is a good idea but will require a little thought about what sort of additional metadata will be useful for job creation. (It would be good to update the compute client interface for createJob to not only accept the hash but also a metadata argument.) Compute adapters may be used for other tasks, such as interactive compute sessions, so it might be good to think a bit about what sort of additional, generic metadata might be useful to the variety of compute adapters.

Maybe "name" is the only thing we care about but it's possible that other info could be of interest in the future. If there is additional info in the future, it might be good to simply create a metadata argument that can have a lot of optional values (maybe start time?). If we are confident there are not many other types of info that would be added, we could simply add a name argument...

Anyway, the short answer is that this is a good idea but requires modification of the compute adapter interface. Although it is only a minor change, it would be nice to think about it carefully before adding it to prevent an annoying or confusing interface in the future.

@brollb
Copy link
Contributor

brollb commented Aug 10, 2020

It might be nice to revamp the JobOriginsAPI in favor of a richer compute API as the JobOriginsAPI is tied to the GME executor framework.

One tricky thing is that interactive compute will not have an associated job, pipeline, etc. Maybe we should have them as optional? Something like the following for interactive compute sessions:

{
  name: 'Train Neural Network',
}

or

{
  job: 'Train',
  pipeline: 'cifar10',
  branch: 'master',
}

The WebGME Executor Framework compute client would need to record the latter metadata info in the database for use with the compute dashboard (or it could simply be changed to show a name like SciServer compute's dashboard).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

2 participants