Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fixed integer Data type issue for UDF #694

Merged

Conversation

sutyag
Copy link
Contributor

@sutyag sutyag commented Sep 11, 2018

int is not valid type in spark, so fixed it to integer

@skaarthik
Copy link
Contributor

@sutyag thanks for the PR. Can you include a pointer to Spark documentation in your PR that confirms "int is not valid type in spark"? It will be helpful to those who are interested in this PR and make the review faster.

@sutyag
Copy link
Contributor Author

sutyag commented Sep 17, 2018

@skaarthik I tried with few samples for UDFs. When i try to register below function;

int UDF1(string a){
return 10;
}

it is failing with below error:
Name of the function to register UDF1, method info
[2018-09-17 16:54:58,218] [1] [ERROR] [Microsoft.Spark.CSharp.Interop.Ipc.JvmBridge] - JVM method execution failed: Static method fromJson failed for class org.apache.spark.sql.types.DataType when called with 1 parameters ([Index=1, Type=String, Value="int"], )
[2018-09-17 16:54:58,218] [1] [ERROR] [Microsoft.Spark.CSharp.Interop.Ipc.JvmBridge] - java.util.NoSuchElementException: key not found: int

while method 
string UDF2(int a){
return "10";
} is working without any issue.

@skaarthik skaarthik merged commit 221dfbd into microsoft:master Sep 21, 2018
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants