2015-9-7

Bias correction added

.Rproj.user/132DF987/pcs/files-pane.pper

@@ -1,5 +1,5 @@
 {
- "path" : "~/hyfo/R",
+ "path" : "~/hyfo",
  "sortOrder" : [
  {
  "ascending" : false,

.Rproj.user/132DF987/pcs/source-pane.pper

@@ -1,3 +1,3 @@
 {
- "activeTab" : 2
+ "activeTab" : 0
 }

.Rproj.user/132DF987/pcs/windowlayoutstate.pper

@@ -1,13 +1,13 @@
 {
  "left" : {
  "panelheight" : 941,
- "splitterpos" : 375,
+ "splitterpos" : 374,
  "topwindowstate" : "NORMAL",
  "windowheight" : 979
  },
  "right" : {
  "panelheight" : 941,
- "splitterpos" : 577,
+ "splitterpos" : 576,
  "topwindowstate" : "NORMAL",
  "windowheight" : 979
  }

.Rproj.user/132DF987/pcs/workbench-pane.pper

@@ -1,4 +1,4 @@
 {
  "TabSet1" : 0,
- "TabSet2" : 1
+ "TabSet2" : 3
 }

.Rproj.user/132DF987/sdb/per/t/579C533F

@@ -0,0 +1,18 @@
+{
+ "contents" : "\n\n#' Biascorrect the input timeseries or hyfo dataset\n#' \n#' Biascorrect the input time series or dataset, the input time series or dataset should consist of observation, hindcast, and forecast.\n#' observation and hindcast should belong to the same period, in order to calibrate. Then the modified forecast\n#' will be returned. If the input is a time series, first column should be date column and rest columns should be \n#' the value column. If the input is a hyfo dataset, the dataset should be the result of \\code{loadNcdf}, or a list\n#' file with the same format.\n#' \n#' @param frc a hyfo grid data output or a dataframe (time series) consists of Date column and one or more value columns, \n#' representing the forecast to be calibrated.\n#' @param hindcast a hyfo grid data output or a dataframe(time series) consists of Date column and one or more value columns, \n#' representing the hindcast data. This data will be used in the calibration of the forecast, so it's better to have the same date period as\n#' observation data. Check details for more information.\n#' @param obs a hyfo grid data output or a dataframe (time series) consists of Date column and one or more value columns, \n#' representing the observation data.\n#' @param method bias correct method, including 'delta', 'scaling'...\n#' @param scaleType only when the method \"scaling\" is chosen, scaleType will be available. Two different types\n#' of scaling method, 'add' and 'mult', which means additive and multiplicative scaling method. More info check \n#' details.\n#' @param input If input is a time series, \\code{input = 'TS'} needs to be assigned, or hyfo will take it as \n#' an hyfo output grid file. Default is time series input, where in most of the cases we prefer. If your input\n#' is a hyfo output file, \\code{input = 'hyfo'}.\n#' @param preci If the precipitation is biascorrected, then you have to assign \\code{preci = TRUE}. Since for\n#' precipitation, some biascorrect methods may not apply to, or some methods are specially for precipitation. \n#' Default is FALSE.\n#' @details \n#' \n#' Since climate forecast is based on global condition, when downscaling to different regions, it may include\n#' some bias, biascorrection is used then to fix the bias.\n#' \n#' \\strong{Hindcast}\n#' \n#' In order to bias correct, we need to pick up some\n#' data from the forecast to train with the observation, which is called hindcast in this function. Hindcast\n#' should have \\strong{EVERY} attributes that forecast has.\n#' \n#' Hindcast is also called re-forecast, is the forecast of the past. E.g. you have a forecast from year 2000-2010, assuming now you are in 2005. So from 2000-2005, this period\n#' is the hindcast period, and 2005-2010, this period is the forecast period.\n#'\n#'\n#' \\strong{How it works}\n#' \n#' Forecast product has to be calibrated, usually the system is doing forecast in real time. So, e.g., if the \n#' forecast starts from year 2000, assuming you are in year 2003, then you will have 3 years' hindcast \n#' data (year 2000 - 2003), which can be used to calibrate. And your forecast period is (2003-2004)\n#' \n#' E.g. you have observation from 2001 - 2002, this is your input obs. Then you can take the same \n#' period (2001-2002) from the forecast, which is the hindcast period. For forecast, you can take any period.\n#' The program will evaluate the obs and hindcast, to get the modification of the forecast, and then add the \n#' modification to the forecast data.\n#' \n#' \\strong{method}\n#' \n#' Different methods used in the bias correction.\n#' \n#' \\strong{delta}\n#' \n#' This method consists on adding to the observations the mean change signal (delta method). \n#' This method is applicable to any kind of variable but it is preferable to avoid it for bounded variables\n#' (e.g. precipitation, wind speed, etc.) because values out of the variable range could be obtained \n#' (e.g. negative wind speeds...)\n#' \n#' \\strong{scaling}\n#' \n#' This method consists on scaling the simulation with the difference (additive) or quotient (multiplicative) \n#' between the observed and simulated means in the train period. The \\code{additive} or \\code{multiplicative}\n#' correction is defined by parameter \\code{scaling.type} (default is \\code{additive}).\n#' The additive version is preferably applicable to unbounded variables (e.g. temperature) \n#' and the multiplicative to variables with a lower bound (e.g. precipitation, because it also preserves the frequency). \n#' \n#' \n#' @examples \n#' \n#' # Use testdl as an example, we take frc, hindcast and obs fro testdl.\n#' data(testdl)\n#' \n#' # common period has to be extracted in order to better train the forecast.\n#' \n#' datalist <- extractPeriod(testdl, startDate = '1994-1-1', endDate = '1995-10-1')\n#' \n#' frc <- datalist[[1]]\n#' hindcast <- datalist[[2]]\n#' obs <- datalist[[3]]\n#' \n#' # default method is delta\n#' frc_new <- biasCorrect(frc, hindcast, obs)\n#' \n#' # If the variable is precipitation, it cannot be negative value, so use multi scale method\n#' frc_new <- biasCorrect(frc, hindcast, obs, method = 'scaling', scaleType = 'multi')\n#' \n#' @references \n#' Bias correction methods come from \\code{biasCorrection} from \\code{dowscaleR}\n#' \n#' Santander Meteorology Group (2015). downscaleR: Climate data manipulation and statistical downscaling. R\n#' package version 0.6-0. https://github.com/SantanderMetGroup/downscaleR/wiki\n#' @export\n\nbiasCorrect <- function(frc, hindcast, obs, method = 'delta', scaleType = 'multi', input = 'TS', preci = FALSE){\n \n if (input == 'TS') {\n # First check if the first column is Date\n if (!grepl('-|/', obs[1, 1]) | !grepl('-|/', hindcast[1, 1]) | !grepl('-|/', frc[1, 1])) {\n stop('First column is not date or Wrong Date formate, check the format in ?as.Date{base} \n and use as.Date to convert.If your input is a hyfo dataset, put input = \"hyfo\" as an\n argument, check help for more info.')\n } \n \n \n # change to date type is easier, but in case in future the flood part is added, Date type doesn't have\n # hour, min and sec, so, it's better to convert it into POSIxlt.\n \n # if condition only accepts one condition, for list comparison, there are a lot of conditions, better\n # further process it, like using any.\n if (any(as.POSIXlt(hindcast[, 1]) != as.POSIXlt(obs[, 1]))) {\n warning('time of obs and time of hindcast are not the same, which may cause inaccuracy in \n the calibration.')\n }\n \n if (ncol(frc) == 2) {\n frc_data <- biasCorrect_core(frc[, 2], hindcast[, 2], obs[, 2], method = method, \n scaleType = scaleType, preci = preci)\n } else if (ncol(frc) > 2) {\n # In this case more than one value columns exist in the dataset, both frc and hindcast.\n \n n <- ncol(frc)\n \n # For every column, it's biascorrected respectively.\n frc_data <- lapply(2:n, function(x) biasCorrect_core(frc[, x], hindcast[, x], obs[, 2], method = method,\n scaleType = scaleType, preci = preci))\n frc_data <- do.call('cbind', frc_data)\n \n } else stop('Wrong TS input, check your TS dimension.')\n \n \n } else if (input == 'hyfo') {\n print('Under development...')\n }\n\n names <- colnames(frc)\n frc <- data.frame(frc[, 1], frc_data)\n colnames(frc) <- names\n \n return(frc)\n}\n\n\n# this is only used to calculate the value column, \nbiasCorrect_core <- function(frc, hindcast, obs, method = 'delta', scaleType = 'multi', preci = FALSE){\n \n\n # default is the simplest method in biascorrection, just do simple addition and subtraction.\n if (method == 'delta') {\n # comes from downscaleR biascorrection method\n frcMean <- mean(obs, na.rm = TRUE)\n hindcastMean <- mean(hindcast, na.rm = TRUE)\n frc <- obs - hindcastMean + frcMean\n \n } else if (method == 'scaling') {\n obsMean <- mean(obs, na.rm = TRUE)\n hindcastMean <- mean(hindcast, na.rm = TRUE)\n \n if (scaleType == 'multi') {\n frc <- frc / hindcastMean * obsMean\n \n } else if (scaleType == 'add') {\n frc <- frc - hindcastMean + obsMean\n }\n \n \n } else if (method == 'eqm') {\n \n # To be added, right now too complicated and not so much use.\n \n }\n \n \n return(frc)\n}",
+ "created" : 1440499519868.000,
+ "dirty" : false,
+ "encoding" : "ASCII",
+ "folds" : "",
+ "hash" : "2108272061",
+ "id" : "579C533F",
+ "lastKnownWriteTime" : 1441616505,
+ "path" : "~/hyfo/R/biasCorrect.R",
+ "project_path" : "R/biasCorrect.R",
+ "properties" : {
+ "tempName" : "Untitled1"
+ },
+ "relative_order" : 1,
+ "source_on_save" : false,
+ "type" : "r_source"
+}

.Rproj.user/132DF987/sdb/per/t/606BD0E

@@ -0,0 +1,17 @@
+{
+ "contents" : "% Generated by roxygen2 (4.1.0.9001): do not edit by hand\n% Please edit documentation in R/biasCorrect.R\n\\name{biasCorrect}\n\\alias{biasCorrect}\n\\title{Biascorrect the input timeseries or hyfo dataset}\n\\usage{\nbiasCorrect(frc, hindcast, obs, method = \"delta\", scaleType = \"multi\",\n input = \"TS\", preci = FALSE)\n}\n\\arguments{\n\\item{frc}{a hyfo grid data output or a dataframe (time series) consists of Date column and one or more value columns,\nrepresenting the forecast to be calibrated.}\n\n\\item{hindcast}{a hyfo grid data output or a dataframe(time series) consists of Date column and one or more value columns,\nrepresenting the hindcast data. This data will be used in the calibration of the forecast, so it's better to have the same date period as\nobservation data. Check details for more information.}\n\n\\item{obs}{a hyfo grid data output or a dataframe (time series) consists of Date column and one or more value columns,\nrepresenting the observation data.}\n\n\\item{method}{bias correct method, including 'delta', 'scaling'...}\n\n\\item{scaleType}{only when the method \"scaling\" is chosen, scaleType will be available. Two different types\nof scaling method, 'add' and 'mult', which means additive and multiplicative scaling method. More info check\ndetails.}\n\n\\item{input}{If input is a time series, \\code{input = 'TS'} needs to be assigned, or hyfo will take it as\nan hyfo output grid file. Default is time series input, where in most of the cases we prefer. If your input\nis a hyfo output file, \\code{input = 'hyfo'}.}\n\n\\item{preci}{If the precipitation is biascorrected, then you have to assign \\code{preci = TRUE}. Since for\nprecipitation, some biascorrect methods may not apply to, or some methods are specially for precipitation.\nDefault is FALSE.}\n}\n\\description{\nBiascorrect the input time series or dataset, the input time series or dataset should consist of observation, hindcast, and forecast.\nobservation and hindcast should belong to the same period, in order to calibrate. Then the modified forecast\nwill be returned. If the input is a time series, first column should be date column and rest columns should be\nthe value column. If the input is a hyfo dataset, the dataset should be the result of \\code{loadNcdf}, or a list\nfile with the same format.\n}\n\\details{\nSince climate forecast is based on global condition, when downscaling to different regions, it may include\nsome bias, biascorrection is used then to fix the bias.\n\n\\strong{Hindcast}\n\nIn order to bias correct, we need to pick up some\ndata from the forecast to train with the observation, which is called hindcast in this function. Hindcast\nshould have \\strong{EVERY} attributes that forecast has.\n\nHindcast is also called re-forecast, is the forecast of the past. E.g. you have a forecast from year 2000-2010, assuming now you are in 2005. So from 2000-2005, this period\nis the hindcast period, and 2005-2010, this period is the forecast period.\n\n\n\\strong{How it works}\n\nForecast product has to be calibrated, usually the system is doing forecast in real time. So, e.g., if the\nforecast starts from year 2000, assuming you are in year 2003, then you will have 3 years' hindcast\ndata (year 2000 - 2003), which can be used to calibrate. And your forecast period is (2003-2004)\n\nE.g. you have observation from 2001 - 2002, this is your input obs. Then you can take the same\nperiod (2001-2002) from the forecast, which is the hindcast period. For forecast, you can take any period.\nThe program will evaluate the obs and hindcast, to get the modification of the forecast, and then add the\nmodification to the forecast data.\n\n\\strong{method}\n\nDifferent methods used in the bias correction.\n\n\\strong{delta}\n\nThis method consists on adding to the observations the mean change signal (delta method).\nThis method is applicable to any kind of variable but it is preferable to avoid it for bounded variables\n (e.g. precipitation, wind speed, etc.) because values out of the variable range could be obtained\n (e.g. negative wind speeds...)\n\n \\strong{scaling}\n\nThis method consists on scaling the simulation with the difference (additive) or quotient (multiplicative)\nbetween the observed and simulated means in the train period. The \\code{additive} or \\code{multiplicative}\ncorrection is defined by parameter \\code{scaling.type} (default is \\code{additive}).\nThe additive version is preferably applicable to unbounded variables (e.g. temperature)\nand the multiplicative to variables with a lower bound (e.g. precipitation, because it also preserves the frequency).\n}\n\\examples{\n# Use testdl as an example, we take frc, hindcast and obs fro testdl.\ndata(testdl)\n\n# common period has to be extracted in order to better train the forecast.\n\ndatalist <- extractPeriod(testdl, startDate = '1994-1-1', endDate = '1995-10-1')\n\nfrc <- datalist[[1]]\nhindcast <- datalist[[2]]\nobs <- datalist[[3]]\n\n# default method is delta\nfrc_new <- biasCorrect(frc, hindcast, obs)\n\n# If the variable is precipitation, it cannot be negative value, so use multi scale method\nfrc_new <- biasCorrect(frc, hindcast, obs, method = 'scaling', scaleType = 'multi')\n}\n\\references{\nBias correction methods come from \\code{biasCorrection} from \\code{dowscaleR}\n\nSantander Meteorology Group (2015). downscaleR: Climate data manipulation and statistical downscaling. R\npackage version 0.6-0. https://github.com/SantanderMetGroup/downscaleR/wiki\n}\n\n",
+ "created" : 1441615447496.000,
+ "dirty" : false,
+ "encoding" : "ASCII",
+ "folds" : "",
+ "hash" : "1599471990",
+ "id" : "606BD0E",
+ "lastKnownWriteTime" : 1441616512,
+ "path" : "~/hyfo/man/biasCorrect.Rd",
+ "project_path" : "man/biasCorrect.Rd",
+ "properties" : {
+ },
+ "relative_order" : 6,
+ "source_on_save" : false,
+ "type" : "r_doc"
+}

.Rproj.user/132DF987/sdb/per/t/A9B56DE9

@@ -6,7 +6,7 @@
  "folds" : "",
  "hash" : "297121889",
  "id" : "A9B56DE9",
- "lastKnownWriteTime" : 1441195685,
+ "lastKnownWriteTime" : 1441354441,
  "path" : "~/hyfo/vignettes/hyfo.Rmd",
  "project_path" : "vignettes/hyfo.Rmd",
  "properties" : {

.Rproj.user/132DF987/sdb/per/t/E7F036D3

@@ -0,0 +1,17 @@
+{
+ "contents" : "Package: hyfo\nType: Package\nTitle: Hydrology and Climate Forecasting R Package for Data Analysis and Visualization\nVersion: 1.1.7\nDate: 2015-07-02\nAuthors@R: person(\"Yuanchao\", \"Xu\", email = \"[email protected]\",\n role = c(\"aut\", \"cre\"))\nDescription: This package can be used as a tool for hydrology and climate forecasting. \n There are several tools including data processing, data visualization and data analysis. \n For hydrological and hydraulic modellers, hyfo can be a good pre-processing and post-processing \n tool for you.\n hyfo has been tested stable on windows platform.\nLicense: GPL-2\nDepends: R (>= 3.1.0), stats (>= 3.1.3), utils(>= 3.1.3)\nImports: ggplot2 (>= 1.0.1),\n reshape2 (>= 1.4.1),\n zoo (>= 1.7-12),\n rgdal (>= 0.9-3),\n plyr (>= 1.8.3),\n moments (>= 0.14),\n lmom (>= 2.5),\n maps(>= 2.3-9),\n maptools (>= 0.8-36),\n rgeos (>= 0.3-8),\n ncdf (>= 1.6.8)\nSuggests: gridExtra,\n knitr\nVignetteBuilder: knitr\nLazyData: true\nURL: http:https://yuanchao-xu.github.io/hyfo/\nBugReports: https://github.com/Yuanchao-Xu/hyfo/issues\nrepository: github",
+ "created" : 1441616319813.000,
+ "dirty" : false,
+ "encoding" : "ASCII",
+ "folds" : "",
+ "hash" : "712805123",
+ "id" : "E7F036D3",
+ "lastKnownWriteTime" : 1441616324,
+ "path" : "~/hyfo/DESCRIPTION",
+ "project_path" : "DESCRIPTION",
+ "properties" : {
+ },
+ "relative_order" : 7,
+ "source_on_save" : false,
+ "type" : "dcf"
+}