Merge pull request #5 from YoungFaithful/singlepkg

Single Pkg, Examples, small var cost for storage

README.md

@@ -2,7 +2,7 @@
 ===
 [![](https://img.shields.io/badge/docs-stable-blue.svg)](https://YoungFaithful.github.io/CEP.jl/stable)
 [![](https://img.shields.io/badge/docs-dev-blue.svg)](https://YoungFaithful.github.io/CEP.jl/dev)
-[![Build Status](https://travis-ci.com/holgerteichgraeber/ClustForOpt.jl.svg?token=HRFemjSxM1NBCsbHGNDG&branch=master)](https://travis-ci.com/YoungFaithful/CEP.jl)
+[![Build Status](https://travis-ci.com/YoungFaithful/CEP.jl.svg?branch=master)](https://travis-ci.com/YoungFaithful/CEP.jl)
 
 [CEP](https://github.com/YoungFaithful/CEP.jl) is a [julia](https://www.juliaopt.com) implementation of a input-data-scaling capacity expansion modeling framework.
 
@@ -32,20 +32,27 @@ This package is developed by Elias Kuepper [@YoungFaithful](https://github.com/y
 This package runs under julia v1.0 and higher.
 First install the dependencies:
 - `JuMP.jl` - for the modeling environment
-- `StatsKit.jl` - for handling of `.csv`-Files
+- `CSV.jl` - for handling of `.csv`-Files
+- `DataFrames.jl` - for handling of tables
+- `StatsBase.jl` - for handling of basic 
 - `JLD2` - for saving your result data
 - `FileIO` - for file accessing
 - `ClustForOpt.jl` - for time-series data
 
 ```julia
 using Pkg
-Pkg.add(["JuMP","StatsKit","JLD2","FileIO"])
+Pkg.add(["JuMP","CSV","DataFrames","StatsBase","JLD2","FileIO"])
 Pkg.add(PackageSpec(url="https://github.com/holgerteichgraeber/ClustForOpt.jl", rev="dev"))
 ```
 Second install `CEP.jl`:
+- If you like to install CEP.jl on the default master branch:
 ```julia
 Pkg.add(PackageSpec(url="https://github.com/YoungFaithful/CEP.jl.git"))
 ```
+- If you like to install CEP.jl on the development dev branch:
+```julia
+Pkg.add(PackageSpec(url="https://github.com/YoungFaithful/CEP.jl.git", rev="dev"))
+```
 ## Example Workflow
 ```julia
 using CEP

REQUIRE

@@ -1,7 +1,9 @@
 julia 0.7-
 Reexport
 ClustForOpt
-StatsKit
+CSV
+DataFrames
+StatsBase
 JLD2
 FileIO
 JuMP

data/CA_1/README_CA_1.md

@@ -19,8 +19,9 @@ California (modeling CAISO) one node
 - `wind, pv, gas, bat, h2, oil, coal`: NREL
 - `trans`: assumption no fix costs
 ### `var`-costs
-- `pv, wind, bat, coal, gas, oil`: NREL
-- `h2, trans`: assumption no var costs
+- `pv, wind, coal, gas, oil`: NREL
+- `trans`: assumption no var costs
+- `bat, h2`: assumption of minimal var costs to avoid charge and discharge in same hour in case of energy excess
 
 ## LCIA Recipe H Midpoint, GWP 100a
 - `pv, wind, trans, coal, gas, oil`: Ecoinvent v3.3

data/CA_1/costs.csv

@@ -4,7 +4,7 @@ bat_e,all,2016,fix,|,3020,0
 bat_e,all,2016,var,|,0,0
 bat_in,all,2016,cap,|,280000,0
 bat_in,all,2016,fix,|,2800,0
-bat_in,all,2016,var,|,0,0
+bat_in,all,2016,var,|,0.01,0
 bat_out,all,2016,cap,|,0,0
 bat_out,all,2016,fix,|,0,0
 bat_out,all,2016,var,|,0,0

data/CA_14/README_CA_14.md

@@ -19,8 +19,9 @@ California (modeling CAISO) 14-node model, no existing capacity (currently not p
 - `wind, pv, gas, bat, h2, oil, coal`: NREL
 - `trans`: assumption no fix costs
 ### `var`-costs
-- `pv, wind, bat, coal, gas, oil`: NREL
-- `h2, trans`: assumption no var costs
+- `pv, wind, coal, gas, oil`: NREL
+- `trans`: assumption no var costs
+- `bat, h2`: assumption of minimal var costs to avoid charge and discharge in same hour in case of energy excess
 
 ## LCIA Recipe H Midpoint, GWP 100a
 - `pv, wind, trans, coal, gas, oil`: Ecoinvent v3.3

data/CA_14/costs.csv

@@ -4,7 +4,7 @@ bat_e,all,2016,fix,|,3020,0
 bat_e,all,2016,var,|,0,0
 bat_in,all,2016,cap,|,280000,0
 bat_in,all,2016,fix,|,2800,0
-bat_in,all,2016,var,|,0,0
+bat_in,all,2016,var,|,0.01,0
 bat_out,all,2016,cap,|,0,0
 bat_out,all,2016,fix,|,0,0
 bat_out,all,2016,var,|,0,0

data/GER_1/README_GER_1.md

@@ -25,8 +25,9 @@ Germany one node, with existing infrastructure of year 2015, no nuclear
 - `trans`: assumption no fix costs
 ### `var`-costs
 - `coal, gas, oil`: Calculation: varcosts_th(Masterthesis Christiane Reinert)/eff(median(eff in ELMOD-DE))
-- `pv, wind, bat, trans`: assumption no var costs
+- `pv, wind, trans`: assumption no var costs
 - `h2`: Glenk, "Shared Capacity and Levelized Cost with Application to Power-to-Gas Technology", Glenk, 2019
+- `bat`: assumption of minimal var costs to avoid charge and discharge in same hour in case of energy excess
 
 ## LCIA Recipe H Midpoint, GWP 100a
 - `pv, wind, trans, coal, gas, oil`: Ecoinvent v3.3

data/GER_1/costs.csv

@@ -29,7 +29,7 @@ coal,GER,2015,var,|,10.708,1217.49
 oil,GER,2015,var,|,153.586,874.6
 gas,GER,2015,var,|,86.564,728.71
 bat_e,GER,2015,var,|,0,0
-bat_in,GER,2015,var,|,0,0
+bat_in,GER,2015,var,|,0.01,0
 bat_out,GER,2015,var,|,0,0
 h2_e,GER,2015,var,|,0,0
 h2_in,GER,2015,var,|,1.9,0

data/GER_18/README_GER_18.md

@@ -25,7 +25,9 @@ Germany 18 (dena) nodes, with existing infrastructure of year 2015, no nuclear
 - `h2`: "Shared Capacity and Levelized Cost with Application to Power-to-Gas Technology", Glenk, 2019
 ### `var`-costs
 - `coal, gas, oil`: Calculation: varcosts_th(Masterthesis Christiane Reinert)/eff(median(eff in ELMOD-DE))
-- `pv, wind, bat, h2, trans`: assumption no var costs
+- `pv, wind, trans`: assumption no var costs
+- `h2`: Glenk, "Shared Capacity and Levelized Cost with Application to Power-to-Gas Technology", Glenk, 2019
+- `bat`: assumption of minimal var costs to avoid charge and discharge in same hour in case of energy excess
 
 ## LCIA Recipe H Midpoint, GWP 100a
 - `pv, wind, trans, coal, gas, oil`: Ecoinvent v3.3

data/GER_18/costs.csv

@@ -29,7 +29,7 @@ coal,GER,2015,var,|,10.708,1217.49
 oil,GER,2015,var,|,153.586,874.6
 gas,GER,2015,var,|,86.564,728.71
 bat_e,GER,2015,var,|,0,0
-bat_in,GER,2015,var,|,0,0
+bat_in,GER,2015,var,|,0.01,0
 bat_out,GER,2015,var,|,0,0
 h2_e,GER,2015,var,|,0,0
 h2_in,GER,2015,var,|,1.9,0

docs/src/README_CA_1.md

@@ -19,8 +19,9 @@ California (modeling CAISO) one node
 - `wind, pv, gas, bat, h2, oil, coal`: NREL
 - `trans`: assumption no fix costs
 ### `var`-costs
-- `pv, wind, bat, coal, gas, oil`: NREL
-- `h2, trans`: assumption no var costs
+- `pv, wind, coal, gas, oil`: NREL
+- `trans`: assumption no var costs
+- `bat, h2`: assumption of minimal var costs to avoid charge and discharge in same hour in case of energy excess
 
 ## LCIA Recipe H Midpoint, GWP 100a
 - `pv, wind, trans, coal, gas, oil`: Ecoinvent v3.3

docs/src/README_CA_14.md

@@ -19,8 +19,9 @@ California (modeling CAISO) 14-node model, no existing capacity (currently not p
 - `wind, pv, gas, bat, h2, oil, coal`: NREL
 - `trans`: assumption no fix costs
 ### `var`-costs
-- `pv, wind, bat, coal, gas, oil`: NREL
-- `h2, trans`: assumption no var costs
+- `pv, wind, coal, gas, oil`: NREL
+- `trans`: assumption no var costs
+- `bat, h2`: assumption of minimal var costs to avoid charge and discharge in same hour in case of energy excess
 
 ## LCIA Recipe H Midpoint, GWP 100a
 - `pv, wind, trans, coal, gas, oil`: Ecoinvent v3.3

docs/src/README_GER_1.md

@@ -25,8 +25,9 @@ Germany one node, with existing infrastructure of year 2015, no nuclear
 - `trans`: assumption no fix costs
 ### `var`-costs
 - `coal, gas, oil`: Calculation: varcosts_th(Masterthesis Christiane Reinert)/eff(median(eff in ELMOD-DE))
-- `pv, wind, bat, trans`: assumption no var costs
+- `pv, wind, trans`: assumption no var costs
 - `h2`: Glenk, "Shared Capacity and Levelized Cost with Application to Power-to-Gas Technology", Glenk, 2019
+- `bat`: assumption of minimal var costs to avoid charge and discharge in same hour in case of energy excess
 
 ## LCIA Recipe H Midpoint, GWP 100a
 - `pv, wind, trans, coal, gas, oil`: Ecoinvent v3.3

docs/src/README_GER_18.md

@@ -25,7 +25,9 @@ Germany 18 (dena) nodes, with existing infrastructure of year 2015, no nuclear
 - `h2`: "Shared Capacity and Levelized Cost with Application to Power-to-Gas Technology", Glenk, 2019
 ### `var`-costs
 - `coal, gas, oil`: Calculation: varcosts_th(Masterthesis Christiane Reinert)/eff(median(eff in ELMOD-DE))
-- `pv, wind, bat, h2, trans`: assumption no var costs
+- `pv, wind, trans`: assumption no var costs
+- `h2`: Glenk, "Shared Capacity and Levelized Cost with Application to Power-to-Gas Technology", Glenk, 2019
+- `bat`: assumption of minimal var costs to avoid charge and discharge in same hour in case of energy excess
 
 ## LCIA Recipe H Midpoint, GWP 100a
 - `pv, wind, trans, coal, gas, oil`: Ecoinvent v3.3

docs/src/index.md

@@ -32,17 +32,24 @@ This package is developed by Elias Kuepper [@YoungFaithful](https://github.com/y
 This package runs under julia v1.0 and higher.
 First install the dependencies:
 - `JuMP.jl` - for the modeling environment
-- `StatsKit.jl` - for handling of `.csv`-Files
+- `CSV.jl` - for handling of `.csv`-Files
+- `DataFrames.jl` - for handling of tables
+- `StatsBase.jl` - for handling of basic 
 - `JLD2` - for saving your result data
 - `FileIO` - for file accessing
 - `ClustForOpt.jl` - for time-series data
 
 ```julia
 using Pkg
-Pkg.add(["JuMP","StatsKit","JLD2","FileIO"])
+Pkg.add(["JuMP","CSV","DataFrames","StatsBase","JLD2","FileIO"])
 Pkg.add(PackageSpec(url="https://github.com/holgerteichgraeber/ClustForOpt.jl", rev="dev"))
 ```
 Second install `CEP.jl`:
+- If you like to install CEP.jl on the default master branch:
 ```julia
 Pkg.add(PackageSpec(url="https://github.com/YoungFaithful/CEP.jl.git"))
 ```
+- If you like to install CEP.jl on the development dev branch:
+```julia
+Pkg.add(PackageSpec(url="https://github.com/YoungFaithful/CEP.jl.git", rev="dev"))
+```

docs/src/opt_cep.md

@@ -57,13 +57,13 @@ An overview is provided in the following table:
 
 | description | unit | configuration | values | type | default value |
 |--------------------------------------------------------------------------------------|------------------|-------------------------|---------------------------------------------|----------------|---------------|
-| enforce a CO2-limit | kg-CO2-eq./MW | co2_limit | >0 | ::Number | Inf |
-| including existing infrastructure (no extra costs) | - | existing_infrastructure | true or false | ::Bool | false |
-| type of storage implementation | - | storage | "none", "simple" or "seasonal" | ::String | "none" |
-| allowing transmission | - | transmission | true or false | ::Bool | false |
-| fix. var and CEO to dispatch problem | - | fixed_design_variables | design variables from design run or nothing | ::OptVariables | nothing |
-| allowing lost load (necessary for dispatch) | price/MWh | lost_el_load_cost | >1e6 | ::Number | Inf |
-| allowing lost emission (necessary for dispatch) | price/kg_CO2-eq. | lost_CO2_emission_cost | >700 | ::Number | Inf |
+| enforce a CO2-limit | kg-CO2e/MWh | `co2_limit` | >0 | ::Number | Inf |
+| including existing infrastructure (no extra costs) | - | `existing_infrastructure` | true or false | ::Bool | false |
+| type of storage implementation | - | `storage` | "none", "simple" or "seasonal" | ::String | "none" |
+| allowing transmission | - | `transmission` | true or false | ::Bool | false |
+| fix. var and CEO to dispatch problem | - | `fixed_design_variables` | design variables from design run or nothing | ::OptVariables | nothing |
+| allowing lost load (necessary for dispatch) | price/MWh | `lost_el_load_cost` | >1e6 | ::Number | Inf |
+| allowing lost emission (necessary for dispatch) | price/kg-CO2e | `lost_CO2_emission_cost` | >700 | ::Number | Inf |
 
 They can be applied in the following way:
 ```@docs

examples/workflow_example_cep.jl

@@ -20,30 +20,30 @@ ts_seg_data = run_clust(ts_input_data;method="kmeans",representation="centroid",
 optimizer=Clp.Optimizer
 
 # tweak the CO2 level
-co2_result = run_opt(ts_clust_data.best_results,cep_data,optimizer;descriptor="co2",co2_limit=50) #generally values between 1250 and 10 are interesting
+co2_result = run_opt(ts_clust_data.best_results,cep_data,optimizer;co2_limit=50) #generally values between 1250 and 10 are interesting
 
 # Include a Slack-Variable
-slack_result = run_opt(ts_clust_data.best_results,cep_data,optimizer;descriptor="slack",lost_el_load_cost=1e6, lost_CO2_emission_cost=700)
+slack_result = run_opt(ts_clust_data.best_results,cep_data,optimizer;lost_el_load_cost=1e6, lost_CO2_emission_cost=700)
 
 
 # Include existing infrastructure at no COST
-ex_result = run_opt(ts_clust_data.best_results,cep_data,optimizer;descriptor="ex",existing_infrastructure=true)
+ex_result = run_opt(ts_clust_data.best_results,cep_data,optimizer;existing_infrastructure=true)
 
 # Intraday storage (just within each period, same storage level at beginning and end)
-simplestor_result = run_opt(ts_clust_data.best_results,cep_data,optimizer;descriptor="simple storage",storage="simple")
+simplestor_result = run_opt(ts_clust_data.best_results,cep_data,optimizer;storage="simple")
 
 # Interday storage (within each period & between the periods)
-seasonalstor_result = run_opt(ts_clust_data.best_results,cep_data,optimizer;descriptor="seasonal storage",storage="seasonal")
+seasonalstor_result = run_opt(ts_clust_data.best_results,cep_data,optimizer;storage="seasonal")
 
 # Transmission
-transmission_result = run_opt(ts_clust_data.best_results,cep_data,optimizer;descriptor="transmission",transmission=true)
+transmission_result = run_opt(ts_clust_data.best_results,cep_data,optimizer;transmission=true)
 
 # Segmentation
-seg_result = run_opt(ts_seg_data.best_results,cep_data,optimizer;descriptor="segmentation")
+seg_result = run_opt(ts_seg_data.best_results,cep_data,optimizer)
 
 # Desing with clusered data and operation with ts_full_data
 # First solve the clustered case
-design_result = run_opt(ts_clust_data.best_results,cep_data,optimizer;descriptor="design&operation", co2_limit=50)
+design_result = run_opt(ts_clust_data.best_results,cep_data,optimizer;co2_limit=50)
 
 #capacity_factors
 design_variables=get_cep_design_variables(design_result)

examples/workflow_example_cep_plot.jl

@@ -4,25 +4,29 @@ using Clp
 using Plots
 
 ## LOAD DATA ##
-state="GER_1" # or "GER_18" or "CA_1" or "TX_1"
+state="GER_18" # or "GER_18" or "CA_1" or "TX_1"
 # laod ts-data
-ts_input_data = load_timeseries_data("CEP", state; T=24) #CEP
+ts_input_data = load_timeseries_data_provided(state; T=24) #CEP
 # load cep-data
 cep_data = load_cep_data_provided(state)
 
 ## CLUSTERING ##
 # run aggregation with kmeans
-ts_clust_data = run_clust(ts_input_data;method="kmeans",representation="centroid",n_init=5,n_clust=5) # default k-means make sure that n_init is high enough otherwise the results could be crap and drive you crazy
+ts_clust_data = run_clust(ts_input_data;method="kmeans",representation="centroid",n_init=100,n_clust=5) # Increase n_init to 10000 for a "real" run
 ## OPTIMIZATION EXAMPLES##
 # select optimizer
 optimizer=Clp.Optimizer
 
-# Create a Scenario of the clustered data and the corresponding OptResult
-cep = Scenario("co2",ts_clust_data, run_opt(ts_clust_data.best_results,cep_data;optimizer=optimizer,descriptor="co2",co2_limit=1000)) #generally values between 1250 and 10 are interesting
+# Optimize the capacity expansion problem with a co2_limit of 1000
+cep = run_opt(ts_clust_data.best_results,cep_data,optimizer;co2_limit=1000)
 
-# use the get variable set in order to get the labels: indicate the varible as "CAP" and the set-number as 1 to recieve this sets values
-labels=get_cep_variable_set(cep,"CAP",1)
-# use the get variable value function to recieve the values of CAP[:,:,1]
-data=get_cep_variable_value(cep,"CAP",[:,:,1])
-# use the data provided for a simple bar-plot without a legend
-bar(data,title="Cap", xticks=(1:length(labels),labels),legend=false)
+# Extract the CAP-Variable
+cap=cep.variables["CAP"]
+
+# Prepare the data that shall be plotted
+dat=sum(cap[:, "new", :], dims=2)
+# Prepare the xticks
+xticks=(1:length(axes(cap,"tech")),axes(cap,"tech"))
+
+# Plot as a bar-plot
+bar(dat,title="Cap", xticks=xticks ,legend=false)

examples/workflow_introduction.jl

@@ -9,7 +9,7 @@ The data is for a Capacity Expansion Problem "CEP"
 and for the single node representation of Germany "GER_1"
 The original timeseries has 8760 entries (one for each hour of the year)
 It should be cut into K=365 periods (365 days) with T=24 timesteps per period (24h per day) =#
-ts_input_data, = load_timeseries_cep("CEP", "GER_1"; K=365, T=24)
+ts_input_data = load_timeseries_data_provided("GER_1"; T=24)
 
 #= ClustData
 How the struct is setup:
@@ -46,17 +46,17 @@ Each column in `[file name].csv` file will be added to the ClustData.data called
 - region is an additional String to specify the loaded time series data
 - K describes the number of periods in the input data
 - T describes the length of each period =#
-load_your_own_data=true
-#if load_your_own_data
-# Single file at the path e.g. homedir/tutorial/solar.csv
-# It will automatically call the data 'solar' within the datastruct
-my_path=joinpath(homedir(),"tutorial","solar.csv")
-your_data_1=load_timeseries_data(my_path; region="elias", K=365, T=24)
-# Multiple files in the folder e.g. homedir/tutorial/
-# Within the data struct, it will automatically call the data the names of the csv filenames
-my_path=joinpath(homedir(),"tutorial")
-your_data_2=load_timeseries_data(my_path; region="GER_18", K=365, T=24)
-#end
+load_your_own_data=false
+if load_your_own_data
+ # Single file at the path e.g. homedir/tutorial/solar.csv
+ # It will automatically call the data 'solar' within the datastruct
+ my_path=joinpath(homedir(),"tutorial","solar.csv")
+ your_data_1=load_timeseries_data(my_path; T=24)
+ # Multiple files in the folder e.g. homedir/tutorial/
+ # Within the data struct, it will automatically call the data the names of the csv filenames
+ my_path=joinpath(homedir(),"tutorial")
+ your_data_2=load_timeseries_data(my_path; T=24)
+end
 
 
 #############
@@ -107,23 +107,17 @@ ts_clust_result_2 = run_clust(ts_input_data; method="kmedoids", representation="
 ######
 # Using a Solver called Clp (if not installed run `using Pkg; Pkg.add("Clp")`):
 using Clp
-solver=ClpSolver()
+optimizer=Clp.Optimizer
 # Some extra data for nodes, costs and so on:
-cep_data = load_cep_data_providedts_clust_data.region)
+cep_data = load_cep_data_provided(ts_clust_data.region)
 # Running a simple CEP with a co2-limit of 1000 kg/MWh
 co2_result = run_opt(ts_clust_data,cep_data,optimizer;descriptor="co2",co2_limit=200)
 # co2_result.
 gen_var=co2_result.variables["GEN"]
-# show axes names and axis
-gen_var.axes_names
-# get specific parts of axis names
-gen_var.axes
 # get all operating decision variables (GEN)
-get_cep_variable_value(gen_var,[:,:,:,:,:])
+gen_var[:,:,:,:,:]
 
-# get specific operating decision variables (GEN) indicated with numbers
-get_cep_variable_value(gen_var,[1,:,:,:,1])
 # get specific operating decision variables (the one for the electricity sector, the coal technology at the node germany) indicated with names
-coal_gen_var=get_cep_variable_value(gen_var,["el","wind",:,:,"germany"])
+coal_gen_var=gen_var["el","wind",:,:,"germany"]
 # plot the specific operation decision variables
-plot(coal_gen_var, xlabel="Time [h]", ylabel="Wind Generation [MW]", labels="K ".*string.(get_cep_variable_set(gen_var, 4)))
+plot(axes(gen_var,"time_T"),coal_gen_var, xlabel="Time [h]", ylabel="Wind Generation [MW]", label=axes(gen_var,"time_K"), legendtitle="Period K", linewidth=2)