diff --git a/CHANGELOG.md b/CHANGELOG.md index 268adc282e..b3f330e02f 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -1,39 +1,38 @@ # Changelog ## 0.5.0 -Performance improvements: New features and bug fixes: -- Evaluators check for empty data [#178](https://github.com/salesforce/TransmogrifAI/pull/178) -- Refactored splitter tests [#176](https://github.com/salesforce/TransmogrifAI/pull/176) + +- XGBoost classification & regression models - EXPERIMENTAL [#44](https://github.com/salesforce/TransmogrifAI/pull/44) - Add default param grid for xgboost [#175](https://github.com/salesforce/TransmogrifAI/pull/175) -- Added aggregate & conditonal readers for parquet [#172](https://github.com/salesforce/TransmogrifAI/pull/172) - Fix ModelInsights for xgboost [#170](https://github.com/salesforce/TransmogrifAI/pull/170) +- Added Parquet reader [#169](https://github.com/salesforce/TransmogrifAI/pull/169) +- Added aggregate & conditional readers for Parquet [#172](https://github.com/salesforce/TransmogrifAI/pull/172) +- Evaluators check for empty data [#178](https://github.com/salesforce/TransmogrifAI/pull/178) +- Refactored splitter tests [#176](https://github.com/salesforce/TransmogrifAI/pull/176) - Return scoring feature distributions from RawFeatureFilter [#171](https://github.com/salesforce/TransmogrifAI/pull/171) -- Added parquet reader [#169](https://github.com/salesforce/TransmogrifAI/pull/169) - Using MapReduce Api for Avro Read Write [#150](https://github.com/salesforce/TransmogrifAI/pull/150) - Improve test coverage for VectorsCombiner and make vector aggregator efficient [#168](https://github.com/salesforce/TransmogrifAI/pull/168) - Time based aggregators [#167](https://github.com/salesforce/TransmogrifAI/pull/167) - Ignore null values in meta + support floats [#166](https://github.com/salesforce/TransmogrifAI/pull/166) - CLI command name fix + bump shadow plugin version + cleanup [#164](https://github.com/salesforce/TransmogrifAI/pull/164) -- fix build.sbt example in readme [#165](https://github.com/salesforce/TransmogrifAI/pull/165) +- Fix build.sbt example in readme [#165](https://github.com/salesforce/TransmogrifAI/pull/165) - Removed an old test I added to check if Spark ran out of memory when calculating a correlation matrix (this is unnecessary and unhelpful) [#160](https://github.com/salesforce/TransmogrifAI/pull/160) - Replace assert with require [#159](https://github.com/salesforce/TransmogrifAI/pull/159) - Streaming histogram implementation [#152](https://github.com/salesforce/TransmogrifAI/pull/152) - Added test and removed dead code for Sanity Checker dealing with map with same key [#153](https://github.com/salesforce/TransmogrifAI/pull/153) -- XGBoost classification & regression models [#44](https://github.com/salesforce/TransmogrifAI/pull/44) -- Added Maven Central badge [#149](https://github.com/salesforce/TransmogrifAI/pull/149) -- fixed model insights exception when features are excluded from sanity checker correlation calculations [#147](https://github.com/salesforce/TransmogrifAI/pull/147) -- added logging of response distribution to RFF [#146](https://github.com/salesforce/TransmogrifAI/pull/146) +- Fixed model insights exception when features are excluded from sanity checker correlation calculations [#147](https://github.com/salesforce/TransmogrifAI/pull/147) +- Added logging of response distribution to RFF [#146](https://github.com/salesforce/TransmogrifAI/pull/146) - Use proper test ranges in feature converter test [#143](https://github.com/salesforce/TransmogrifAI/pull/143) - Added support for DateType and TimestampType primitive spark types [#135](https://github.com/salesforce/TransmogrifAI/pull/135) - Standardizing timezone to UTC [#138](https://github.com/salesforce/TransmogrifAI/pull/138) -Dependency upgrades: -- Upgrade XGBoost to 0.81 [#180](https://github.com/salesforce/TransmogrifAI/pull/180) +Dependency upgrades & misc: +- XGBoost 0.81 [#180](https://github.com/salesforce/TransmogrifAI/pull/180) - Spark 2.3.2 [#44](https://github.com/salesforce/TransmogrifAI/pull/44) +- Gradle 4.10.2 [#142](https://github.com/salesforce/TransmogrifAI/pull/142) - Use OpenJDK8 for CircleCI builds + refactor build config [#140](https://github.com/salesforce/TransmogrifAI/pull/140) -- Update Gradle to 4.10.2 [#142](https://github.com/salesforce/TransmogrifAI/pull/142) ## 0.4.0 @@ -86,7 +85,6 @@ BinaryClassificationModelSelector ``` For more example on how to use new model selectors please refer to our documentation and helloworld examples. - Dependency upgrades & misc: - CI/CD runtime improvements for CircleCI and TravisCI - Updated Gradle to 4.10 diff --git a/README.md b/README.md index 37da41eba7..bcada5ca7d 100644 --- a/README.md +++ b/README.md @@ -1,6 +1,6 @@ # TransmogrifAI -[![Maven Central](https://img.shields.io/maven-central/v/com.salesforce.transmogrifai/transmogrifai-core_2.11.svg?colorB=blue)](https://search.maven.org/search?q=g:com.salesforce.transmogrifai) [![Download](https://api.bintray.com/packages/salesforce/maven/TransmogrifAI/images/download.svg)](https://bintray.com/salesforce/maven/TransmogrifAI/_latestVersion) [![Javadocs](https://www.javadoc.io/badge/com.salesforce.transmogrifai/transmogrifai-core_2.11/0.5.0.svg?color=blue)](https://www.javadoc.io/doc/com.salesforce.transmogrifai/transmogrifai-core_2.11/0.5.0) [![Spark version](https://img.shields.io/badge/spark-2.2-brightgreen.svg)](https://spark.apache.org/downloads.html) [![Scala version](https://img.shields.io/badge/scala-2.11-brightgreen.svg)](https://www.scala-lang.org/download/2.11.12.html) [![License](http://img.shields.io/:license-BSD--3-blue.svg)](./LICENSE) [![Chat](https://badges.gitter.im/salesforce/TransmogrifAI.svg)](https://gitter.im/salesforce/TransmogrifAI?utm_source=badge&utm_medium=badge&utm_campaign=pr-badge&utm_content=badge) +[![Maven Central](https://img.shields.io/maven-central/v/com.salesforce.transmogrifai/transmogrifai-core_2.11.svg?colorB=blue)](https://search.maven.org/search?q=g:com.salesforce.transmogrifai) [![Download](https://api.bintray.com/packages/salesforce/maven/TransmogrifAI/images/download.svg)](https://bintray.com/salesforce/maven/TransmogrifAI/_latestVersion) [![Javadocs](https://www.javadoc.io/badge/com.salesforce.transmogrifai/transmogrifai-core_2.11/0.5.0.svg?color=blue)](https://www.javadoc.io/doc/com.salesforce.transmogrifai/transmogrifai-core_2.11/0.5.0) [![Spark version](https://img.shields.io/badge/spark-2.3-brightgreen.svg)](https://spark.apache.org/downloads.html) [![Scala version](https://img.shields.io/badge/scala-2.11-brightgreen.svg)](https://www.scala-lang.org/download/2.11.12.html) [![License](http://img.shields.io/:license-BSD--3-blue.svg)](./LICENSE) [![Chat](https://badges.gitter.im/salesforce/TransmogrifAI.svg)](https://gitter.im/salesforce/TransmogrifAI?utm_source=badge&utm_medium=badge&utm_campaign=pr-badge&utm_content=badge) [![TravisCI Build Status](https://travis-ci.com/salesforce/TransmogrifAI.svg?token=Ex9czVEUD7AzPTmVh6iX&branch=master)](https://travis-ci.com/salesforce/TransmogrifAI) [![CircleCI Build Status](https://circleci.com/gh/salesforce/TransmogrifAI.svg?&style=shield&circle-token=e84c1037ae36652d38b49207728181ee85337e0b)](https://circleci.com/gh/salesforce/TransmogrifAI) [![Codecov](https://codecov.io/gh/salesforce/TransmogrifAI/branch/master/graph/badge.svg)](https://codecov.io/gh/salesforce/TransmogrifAI) [![CodeFactor](https://www.codefactor.io/repository/github/salesforce/transmogrifai/badge)](https://www.codefactor.io/repository/github/salesforce/transmogrifai) diff --git a/docs/installation/index.md b/docs/installation/index.md index 1b0bc644f5..694c3e9208 100644 --- a/docs/installation/index.md +++ b/docs/installation/index.md @@ -1,7 +1,7 @@ # Installation * Download and install Java 1.8, then set an environment variable: `export JAVA_HOME=$(/usr/libexec/java_home -v 1.8)` -* Get Spark 2.2.x: [Download](https://spark.apache.org/downloads.html), unzip it and then set an environment variable: `export SPARK_HOME=` +* Get Spark 2.3.x: [Download](https://spark.apache.org/downloads.html), unzip it and then set an environment variable: `export SPARK_HOME=` * Clone the TransmogrifAI repo: `git clone https://github.com/salesforce/TransmogrifAI.git` * Build the project: `cd TransmogrifAI && ./gradlew compileTestScala installDist` * Start hacking diff --git a/helloworld/build.gradle b/helloworld/build.gradle index 5f813d19d2..d6c4bde474 100644 --- a/helloworld/build.gradle +++ b/helloworld/build.gradle @@ -36,7 +36,7 @@ ext { scalaVersion = '2.11' scalaVersionRevision = '12' junitVersion = '4.11' - sparkVersion = '2.2.1' + sparkVersion = '2.3.2' scalatestVersion = '3.0.0' transmogrifaiVersion ='0.5.0' collectionsVersion = '3.2.2' diff --git a/local/README.md b/local/README.md index e55c15d71d..4f881f95c6 100644 --- a/local/README.md +++ b/local/README.md @@ -12,13 +12,13 @@ Add the `transmogrifai-local` dependency into your project together with `hadria For Gradle in `build.gradle` add: ```gradle dependencies { - compile 'com.salesforce.transmogrifai:transmogrifai-local_2.11:0.4.0' + compile 'com.salesforce.transmogrifai:transmogrifai-local_2.11:0.5.0' runtime 'com.opendatagroup:hadrian:0.8.5' } ``` For SBT in `build.sbt` add: ```sbt -libraryDependencies += "com.salesforce.transmogrifai" %% "transmogrifai-local" % "0.4.0" +libraryDependencies += "com.salesforce.transmogrifai" %% "transmogrifai-local" % "0.5.0" libraryDependencies += libraryDependencies += "com.opendatagroup" % "hadrian" % "0.8.5" % Runtime ```