Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

EV charging support/integration #121

Open
mergwyn opened this issue Feb 13, 2024 · 203 comments
Open

EV charging support/integration #121

mergwyn opened this issue Feb 13, 2024 · 203 comments
Labels
enhancement New feature or request long term Good idea but not a priority

Comments

@mergwyn
Copy link

mergwyn commented Feb 13, 2024

Is your feature request related to a problem? Please describe.
I'm all electric at home with ASHP, Solar, storage and and an EV charger. PV Opt does a good job of managing my solar and storage but I also need to be able schedule EV charging

Describe the solution you'd like
I'd really like pvopt to ensure that charging the EV is synchronised such that ir does not discharge the storage battery. At its simplest level the EV is just another battery which needs be charged to a target SOC in the cheapest slots so hopefully there is an overlap in the algorithm. This could either be done fully within PV Opt or via hooks that allow users to build their own automation using the results (eg publishing x cheapest slots)

Describe alternatives you've considered
At the moment, I am using zappi 'agile knowledge' to set a schedule that only charges when the tariff is below a certain level, but obviously this may or may not cause the storage battery to discharge. (I also have to change schedule manually depending on the latest tariff)

I've tried using Predbat, but this is soo complicated and I've not been able to understand the charging plans it puts together and am struggling with the amount of 'hold' and 'reserve' charging that does on.

Additional context
I understand that this might not be where you want or need to take PV Opt and appreciate all that you have done so far. I wish I could program in python to help out but that it beyond me! If I can help in any other way please let me know.

@SzosszeNET
Copy link
Contributor

SzosszeNET commented Feb 13, 2024

Like this idea. While you get a response as an alternative not sure if you looked into it, but zappi seems to have its own integration.
https://github.com/CJNE/ha-myenergi

Could imagine to have both, and create an automation (under settings in HA) if not already one in the myenergi repo, that would watch the myenergi integration and if it's scheduled to charge, at the beginning of the charge window:
Stop AppDaemon (or maybe less drastic to set pv_opt to read only)
Set a static charge window to the inverter (to hold charge
Just to be sure 00 the discharge window and current

And potentially a second one that would start AppDaemon when the myenergi charge is finished/is outside the charge window?

@fboundy
Copy link
Owner

fboundy commented Feb 13, 2024

Keen to support this as an idea but I don't want to get as complicated as PredBat!

One starting point would be the entity that already exists called switch.pvopt_charge_active - this is On when pv_opt is charging so is a good starting point. If I knew how much charge your needed for Zappi in a given window and at what rate I could probably call the zappi integration too.

It sounds like installing the Zappi integration would be a start. I would then need to know what sensors and services it exposes.

@mergwyn
Copy link
Author

mergwyn commented Feb 14, 2024 via email

@fboundy fboundy added the long term Good idea but not a priority label Mar 20, 2024
@stevebuk1
Copy link
Collaborator

stevebuk1 commented Mar 23, 2024

Seconded for an EV integration. I'm on IOG so all I'd be looking for is a "hold SOC" whilst the EV is charging. I'd prefer a charge current = 0 to do the hold function rather than setting Backup/Reserve, as in Backup/Reserve my inverter doesnt respect Bit 6 in the Energy Storage register to prevent grid charging, causing oscillations between full rate charging and full rate discharging during EV charging. This rules me out of Pv-opt for now.

I'm currently running Predbat and am contributing to Solis code changes for IOG, but I can see that the fundamental way it works is never going to work properly for Agile due to GivEnergy functions Solis inverters just don't have (like Freeze Charging) - and I figure one day I'll be moving to Agile.

I have an ID4 and Zappi so similar to [mergwyn] and will happily assist with Alpha/Beta testing.

@fboundy
Copy link
Owner

fboundy commented Mar 25, 2024

I think that setting charge current to zero for the Solis is a better method of holding SOC anyway so I'm about to start testing that.

How would you flag up the EV charging slots with IOG?

@mergwyn
Copy link
Author

mergwyn commented Mar 25, 2024

BottlecapDave's integration has a sensor: binary_sensor.octopus_energy_{{ACCOUNT_ID}}_intelligent_dispatching which is used to determine if you're currently in a planned dispatch period (i.e. "smart-charge" determined by Octopus Energy) or are within the standard off peak period. This sensor will not come on during a bump charge which I think is an OK limitation.

There are some limitations: https://github.com/BottlecapDave/HomeAssistant-OctopusEnergy/blob/b06499c86548847b5c75c17a487a87178f59b64a/_docs/entities/intelligent.md?plain=1#L9

This doesn't cover how to make EV charging work with Agile rather than IOG which is a harder problem I suspect. I'm probably going to switch to IOG shortly until the Autumn, so again this is a limitation I can live with for a while at least!

Thanks again for your great work on pv_opt.

@fboundy
Copy link
Owner

fboundy commented Mar 25, 2024

OK - so looking at the attributes for this would it be reasonable to assume that pv_opt should simply "hold SOC" for all entries in planned_dispatches?

@stevebuk1 - I'm a bit confused by your comment regarding Bit 6 for Energy Storage as this is the Feed In Priority bit. Do you mean this or Bit 5 which is Grid Charging? How does this interact with setting Backup Mode (Bit 4) and Backup SOC? One of the current limitations with the Solax integration is that the control switch isn't set bit-wise or by number but using a select. That said I've successfully added modes to the integration before.

@mergwyn
Copy link
Author

mergwyn commented Mar 25, 2024

For me, the ideal situation would be for pv_opt to do its thing regarding whether the solar batteries need to be charged. If this is at the same time as the EV needs to be charged (indicated by intelligent_dispatching for IOG) is it possible just to set the discharge current to zero (or some equivalent) such that both the EV and solar batteries can be charged at the same time?

@stevebuk1
Copy link
Collaborator

I'm a bit confused by your comment regarding Bit 6 for Energy Storage as this is the Feed In Priority bit. Do you mean this or Bit 5 which is Grid Charging?

Apologies, I was responding to an Issue in the Solis Inverters Facebook group where people were labelling the 8 bits as Bit1 to Bit 8, and I forgot to reset my language to the traditional LSB = 0, MSB = 7 nomenclature. It is indeed Bit 5.

Regarding the Solax integration, it currently has two options for Backup/Reserve:

"Backup/Reserve"
"Backup/Reserve - No Grid Charging".

and I imagine the 2nd one clears Bit 5.

What I was trying to say in my previous post is that there is no change in behavior between these modes (both charge from grid when battery SOC is below backup SOC), so I'd prefer using charge/discharge slots and current (amps) setting to do it instead.

@stevebuk1
Copy link
Collaborator

OK - so looking at the attributes for this would it be reasonable to assume that pv_opt should simply "hold SOC" for all entries in planned_dispatches?

Yes, this is the way Predbat works, or rather it does now in my local copy since the addition of setting charge slots and current = zero for Solis inverters. Note: I'd toyed with using discharge start/end times and discharge current = 0, but Predbat doesnt clear out old charge start/end times anyway, so I just used the charge start/end times. I guess either would work as long as there are no conflicts between charge and discharge times.

@fboundy
Copy link
Owner

fboundy commented Mar 25, 2024 via email

@fboundy
Copy link
Owner

fboundy commented Mar 25, 2024 via email

@stevebuk1
Copy link
Collaborator

I wrote some of the initial Solis code for PredBat but I gave up on it because I found it too heavily predicated on GivEnergy behaviour. I also couldn’t get my head around the optimiser which often didn’t seem very optimal for Agile.

I remember you saying in the Solis Facebook group you authored some of the code in Predbat. I've carried on making some changes and additions for getting it working in IOG (e.g. I've disabled it using Backup/Reserve mode to fix low rate charging) but the target SOC changes wildly through the 6 hours cheap rate period for what looks like no input changes (solcast, power forecast etc), resulting in a mess of high rate and low rate charging with constant writes to the inverter all night. For this reason I've just started trying to figure out the optimizer too but not encouraged by your experience. With EV hold functionality PV_opt will do the job, although getting in an electrician to add a Henley block for the Zappi is the other way ;-)

@stevebuk1
Copy link
Collaborator

Regarding consumption data, I'm not sure if the current algorithm just uses what the Solis supplied to the house as energy, or takes account of the house load coming from grid during charging slots and hold slots. If one load shifts to co-inside with charging slots to avoid battery losses (as I do) then its likely direct use of grid energy will be a significant portion of consumption, so I assume grid energy forms part of the consumption data calculations?

If the assumption is correct, then car charging is going to add to (and corrupt) any existing method of grid use being factored into consumption data.
This means the energy consumed from the grid use for car charging will need discounting from any consumption data, so it doesn't influence the charging plans for the house battery. Ultimately, we are compensating for the fact that the Solis CT clamp sees both the EV and the house as consumption.

I see 3 different ways of doing this:

  1. Assume any grid draw above 6kW will mean the EV is charging, so subtract 7kw (the actual EV charge rate) from the consumption figures
  2. Utilise a Zappi entity that tracks consumption "sensor.myenergi_zappi_{Serial number}_charge_added_session", and subtract this from the consumption figures. Note this resets to zero when the car is plugged in.
  3. Utilise the "completed dispatches" attribute within the "_intelligent_dispatching" entity mentioned above and subtract it from the consumption figures.

Apologies if this was already obvious.

@fboundy
Copy link
Owner

fboundy commented Mar 26, 2024 via email

@stevebuk1
Copy link
Collaborator

stevebuk1 commented Mar 26, 2024

Make sense?

Not really. Looking at the log file, consumption is loaded from "sensor.solis_house_load_today". The plot of that sensor is below, and it includes the 26kWh ish of car charging that I did last night. It can't be blind to it, otherwise the we wouldn't have the problem of the house battery discharging into the EV in the first place?

image

@fboundy
Copy link
Owner

fboundy commented Mar 27, 2024

OK - got it.

I think all 3 of your suggestions would work. I'd probably prioritise them in terms of what is available on a per case basis:

  1. Zappi (or other EV charger entity)
  2. IO dispatches (would only work for IO I presume)
  3. Charge threshold power which is the crudest and might pick up
    cooking Sunday Roast in error

@fboundy
Copy link
Owner

fboundy commented Mar 27, 2024

I'm hoping to have something out to test later today as a pre-release I don't think it will work fully but hopefully it will log enough stuff to gather the necessary info.

@fboundy
Copy link
Owner

fboundy commented Mar 27, 2024

v3.13.0-ev-beta-1 should now be available as a pre-release via HACS. It adds:

If you can load it up and check it doesn;t bust anything. If it doesn't then could you run it for at least 24 hours and then send me the pv_opt.log and error.log (if any errors occur)?

If this works OK then next step is to isolate the IO charging intervals from the optimisation.

Thanks

@fboundy fboundy added the enhancement New feature or request label Mar 27, 2024
@stevebuk1
Copy link
Collaborator

Great - I've now installed it and its up and running. Its scheduled a house battery charge for this evening (the weather in Scotland is currently awful solar-wise) and I'll plug in the car.

I assume there aren't any changes needed to apps.yaml?

@stevebuk1
Copy link
Collaborator

stevebuk1 commented Mar 27, 2024

I think all 3 of your suggestions would work. I'd probably prioritise them in terms of what is available on a per case basis:

  1. Zappi (or other EV charger entity)
  2. IO dispatches (would only work for IO I presume)
  3. Charge threshold power which is the crudest and might pick up
    cooking Sunday Roast in error

Agree that Option1 is the best, and should easily be configurable for other chargers.
2) agree thats only useful for IO, so no use for Agile or the other tariffs
3) is crude and it would probably work for me as during the day as I try and keep all consumption below 3kW (max battery discharge rate in my 3.6kW inverter) so I don't draw from grid. Overnight is similar (I offset all appliances), but I doubt it would work for those having G99 inverters and bigger battery banks.

@fboundy
Copy link
Owner

fboundy commented Mar 27, 2024

Great - I've now installed it and its up and running. Its scheduled a house battery charge for this evening (the weather in Scotland is currently awful solar-wise) and I'll plug in the car.

I assume there aren't any changes needed to apps.yaml?

Shouldn't be any changes needed. It should just check for the io and zappi entities and then spit out their contents to the log file.

@stevebuk1
Copy link
Collaborator

Apologies, I spoke too soon. I flicked the toggle on Consumption to go fixed, and have since realised that it did a restart of PV_opt ok but the first run had hung. I've now restarted AppDeamon and Pv_Opt is stuck in "Loading Tarrifs".

Error log code is:

15:58:03 WARNING pv_opt: ------------------------------------------------------------
15:58:03 WARNING pv_opt: Unexpected error running initialize() for pv_opt
15:58:03 WARNING pv_opt: ------------------------------------------------------------
15:58:03 WARNING pv_opt: Traceback (most recent call last):
File "/usr/lib/python3.11/site-packages/appdaemon/app_management.py", line 162, in initialize_app
await utils.run_in_executor(self, init)
File "/usr/lib/python3.11/site-packages/appdaemon/utils.py", line 304, in run_in_executor
response = future.result()
^^^^^^^^^^^^^^^
File "/usr/lib/python3.11/concurrent/futures/thread.py", line 58, in run
result = self.fn(*self.args, **self.kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/homeassistant/appdaemon/apps/pv_opt/pv_opt.py", line 407, in initialize
self._check_for_zappi()
File "/homeassistant/appdaemon/apps/pv_opt/pv_opt.py", line 472, in _check_for_zappi
for entity_id in self.self.io:
^^^^^^^^^
AttributeError: 'PVOpt' object has no attribute 'self'

15:58:03 WARNING pv_opt: ------------------------------------------------------------

I thought it might be a duplicate "self" at line 472, but removing it didnt make any difference:

16:03:22 WARNING pv_opt: ------------------------------------------------------------
16:05:16 WARNING pv_opt: ------------------------------------------------------------
16:05:16 WARNING pv_opt: Unexpected error in worker for App pv_opt:
16:05:16 WARNING pv_opt: Worker Ags: {}
16:05:16 WARNING pv_opt: ------------------------------------------------------------
16:05:16 WARNING pv_opt: Traceback (most recent call last):
File "/usr/lib/python3.11/site-packages/appdaemon/app_management.py", line 162, in initialize_app
await utils.run_in_executor(self, init)
File "/usr/lib/python3.11/site-packages/appdaemon/utils.py", line 304, in run_in_executor
response = future.result()
^^^^^^^^^^^^^^^
File "/usr/lib/python3.11/concurrent/futures/thread.py", line 58, in run
result = self.fn(*self.args, **self.kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/homeassistant/appdaemon/apps/pv_opt/pv_opt.py", line 407, in initialize
self._check_for_zappi()
File "/homeassistant/appdaemon/apps/pv_opt/pv_opt.py", line 472, in _check_for_zappi
for entity_id in self.io:
TypeError: 'bool' object is not iterable

@fboundy
Copy link
Owner

fboundy commented Mar 27, 2024

If you are happy to edit it yourself it's just a simple typo. Line 472 should be:

        for entity_id in self.self.io_entities:

Didn't flag in my system because it was an empty list for me and the prior if caught it

@stevebuk1
Copy link
Collaborator

stevebuk1 commented Mar 27, 2024 via email

@fboundy
Copy link
Owner

fboundy commented Mar 27, 2024

Any changes to the .py files or config.yaml should cause AD to restart it

@mergwyn
Copy link
Author

mergwyn commented Mar 27, 2024

I've also installed this version and made the edit to py_opt.py, but I get a different error:

16:45:14 WARNING pv_opt: ------------------------------------------------------------
16:46:15 WARNING pv_opt: ------------------------------------------------------------
16:46:15 WARNING pv_opt: Unexpected error running initialize() for pv_opt
16:46:15 WARNING pv_opt: ------------------------------------------------------------
16:46:15 WARNING pv_opt: Traceback (most recent call last):
  File "/usr/local/lib/python3.10/site-packages/appdaemon/app_management.py", line 162, in initialize_app
    await utils.run_in_executor(self, init)
  File "/usr/local/lib/python3.10/site-packages/appdaemon/utils.py", line 304, in run_in_executor
    response = future.result()
  File "/usr/local/lib/python3.10/concurrent/futures/thread.py", line 58, in run
    result = self.fn(*self.args, **self.kwargs)
  File "/home-assistant/appdaemon/apps/pv_opt/pv_opt.py", line 407, in initialize
    self._check_for_zappi()
  File "/home-assistant/appdaemon/apps/pv_opt/pv_opt.py", line 472, in _check_for_zappi
    for entity_id in self.self.io_entities:
AttributeError: 'PVOpt' object has no attribute 'self'

I'm also seeing something I've not noticed before:

16:45:14 WARNING pv_opt: ------------------------------------------------------------
16:45:14 WARNING pv_opt: Unexpected error running initialize() for pv_opt
16:45:14 WARNING pv_opt: ------------------------------------------------------------
16:45:14 WARNING pv_opt: Traceback (most recent call last):
  File "/usr/local/lib/python3.10/site-packages/appdaemon/app_management.py", line 162, in initialize_app
    await utils.run_in_executor(self, init)
  File "/usr/local/lib/python3.10/site-packages/appdaemon/utils.py", line 304, in run_in_executor
    response = future.result()
  File "/usr/local/lib/python3.10/concurrent/futures/thread.py", line 58, in run
    result = self.fn(*self.args, **self.kwargs)
  File "/home-assistant/appdaemon/apps/pv_opt/pv_opt.py", line 372, in initialize
    while (not self.inverter.is_online()) and (retry_count < ONLINE_RETRIES):
AttributeError: 'InverterController' object has no attribute 'is_online'

@fboundy
Copy link
Owner

fboundy commented Mar 27, 2024

OK hopefully 3.14.0-ev-beta-2 fixes all these issues. Too many beta versions flying around...

@mergwyn
Copy link
Author

mergwyn commented Jul 18, 2024

@stevebuk1
Copy link
Collaborator

stevebuk1 commented Jul 19, 2024

@mergwyn, thanks for the logs, they didn't contain any further detail around the area that calculates the cost delta.

I forced an export tariff into my setup and enabled force discharging and I think I have repeated the same behaviour you've got, in that there is no forced discharge due to a cost delta of 0. I'm also getting no Low cost charging plan. You did get one of these in your logs you posted when you first raised this but not in the logs with debug switched on.

I then reverted to the latest full production release (3.15.4 from https://github.com/fboundy/pv_opt), did the same tweak to force an export tariff and have found exactly the same behaviour.

I also tried 3.14.0 and then 3.13.2, exactly the same.
(3.13.2 predates the start of any EV integration work)

It might be worth repeating the same reversion testing to see if it makes a difference, otherwise its a case of opening a new issue.

I'm afraid I can't explain why there isn't any charging and discharging going on. Uncommenting some extra logging lines in the code illicts this, which I can't explain: (@fboundy there is a fix for the * being an hour out within the existing PR, its a UTC v GMT thing)

21:10:22     INFO: 44 slots have an export price greater than the min import price
21:10:22     INFO: 43 Max export price 15.00p/kWh at 19/07 20:00   SOC:  16.0%-> 16.0% Net: 164.8 
21:10:22     INFO: 42 Max export price 15.00p/kWh at 19/07 20:30   SOC:  16.0%-> 16.0% Net: 164.8 
21:10:23     INFO: 41 Max export price 15.00p/kWh at 19/07 21:00 * SOC:  16.0%-> 16.0% Net: 164.8 
21:10:23     INFO: 40 Max export price 15.00p/kWh at 19/07 21:30   SOC:  16.0%-> 16.0% Net: 164.8 
21:10:23     INFO: 39 Max export price 15.00p/kWh at 19/07 22:00   SOC:  16.0%-> 16.0% Net: 164.8 
21:10:23     INFO: 38 Max export price 15.00p/kWh at 20/07 04:30   SOC:  25.0%-> 24.0% Net: 171.8 
21:10:23     INFO: 37 Max export price 15.00p/kWh at 20/07 05:00   SOC:  24.0%-> 23.0% Net: 170.9 
21:10:23     INFO: 36 Max export price 15.00p/kWh at 20/07 05:30   SOC:  23.0%-> 22.1% Net: 170.2 
21:10:23     INFO: 35 Max export price 15.00p/kWh at 20/07 06:00   SOC:  22.1%-> 21.5% Net: 169.6 
21:10:23     INFO: 34 Max export price 15.00p/kWh at 20/07 06:30   SOC:  21.5%-> 21.6% Net: 169.7 
21:10:24     INFO: 33 Max export price 15.00p/kWh at 20/07 07:00   SOC:  21.6%-> 22.3% Net: 170.2 
21:10:24     INFO: 32 Max export price 15.00p/kWh at 20/07 07:30   SOC:  22.3%-> 23.5% Net: 171.1 
21:10:24     INFO: 31 Max export price 15.00p/kWh at 20/07 08:00   SOC:  23.5%-> 24.5% Net: 172.1 
21:10:24     INFO: 30 Max export price 15.00p/kWh at 20/07 08:30   SOC:  24.5%-> 26.4% Net: 173.5 
21:10:24     INFO: 29 Max export price 15.00p/kWh at 20/07 09:00   SOC:  26.4%-> 28.4% Net: 175.3 
21:10:24     INFO: 28 Max export price 15.00p/kWh at 20/07 09:30   SOC:  28.4%-> 31.3% Net: 177.7 
21:10:24     INFO: 27 Max export price 15.00p/kWh at 20/07 10:00   SOC:  31.3%-> 34.3% Net: 180.3 
21:10:24     INFO: 26 Max export price 15.00p/kWh at 20/07 10:30   SOC:  34.3%-> 37.2% Net: 181.0 
21:10:25     INFO: 25 Max export price 15.00p/kWh at 20/07 11:00   SOC:  37.2%-> 40.0% Net: 181.0 
21:10:25     INFO: 24 Max export price 15.00p/kWh at 20/07 11:30   SOC:  40.0%-> 43.2% Net: 181.2 
21:10:25     INFO: 23 Max export price 15.00p/kWh at 20/07 12:00   SOC:  43.2%-> 46.5% Net: 181.3 
21:10:25     INFO: 22 Max export price 15.00p/kWh at 20/07 12:30   SOC:  46.5%-> 49.7% Net: 181.3 
21:10:25     INFO: 21 Max export price 15.00p/kWh at 20/07 13:00   SOC:  49.7%-> 52.2% Net: 180.8 
21:10:25     INFO: 20 Max export price 15.00p/kWh at 20/07 13:30   SOC:  52.2%-> 54.6% Net: 180.7 
21:10:25     INFO: 19 Max export price 15.00p/kWh at 20/07 14:00   SOC:  54.6%-> 57.0% Net: 180.7 
21:10:25     INFO: 18 Max export price 15.00p/kWh at 20/07 14:30   SOC:  57.0%-> 59.4% Net: 180.7 
21:10:26     INFO: 17 Max export price 15.00p/kWh at 20/07 15:00   SOC:  59.4%-> 60.7% Net: 180.0 
21:10:26     INFO: 16 Max export price 15.00p/kWh at 20/07 15:30   SOC:  60.7%-> 60.4% Net: 178.9 
21:10:26     INFO: 15 Max export price 15.00p/kWh at 20/07 16:00   SOC:  60.4%-> 58.0% Net: 177.0 
21:10:26     INFO: 14 Max export price 15.00p/kWh at 20/07 16:30   SOC:  58.0%-> 56.3% Net: 177.6 
21:10:26     INFO: 13 Max export price 15.00p/kWh at 20/07 17:00   SOC:  56.3%-> 55.1% Net: 178.1 
21:10:26     INFO: 12 Max export price 15.00p/kWh at 20/07 17:30   SOC:  55.1%-> 53.9% Net: 178.1 
21:10:26     INFO: 11 Max export price 15.00p/kWh at 20/07 18:00   SOC:  53.9%-> 52.7% Net: 178.1 
21:10:26     INFO: 10 Max export price 15.00p/kWh at 20/07 18:30   SOC:  52.7%-> 52.4% Net: 178.8 
21:10:27     INFO:  9 Max export price 15.00p/kWh at 20/07 19:00   SOC:  52.4%-> 51.2% Net: 178.1 
21:10:27     INFO:  8 Max export price 15.00p/kWh at 20/07 19:30   SOC:  51.2%-> 44.0% Net: 172.8 
21:10:27     INFO:  7 Max export price 15.00p/kWh at 20/07 20:00   SOC:  44.0%-> 36.7% Net: 172.7 
21:10:27     INFO:  6 Max export price 15.00p/kWh at 20/07 20:30   SOC:  36.7%-> 30.7% Net: 173.9 
21:10:27     INFO:  5 Max export price 15.00p/kWh at 20/07 21:00   SOC:  30.7%-> 25.6% Net: 173.2 
21:10:27     INFO:  4 Max export price 15.00p/kWh at 20/07 21:30   SOC:  25.6%-> 18.1% Net: 166.6 
21:10:27     INFO:  3 Max export price 15.00p/kWh at 20/07 22:00   SOC:  18.1%-> 16.0% Net: 164.8 
21:10:27     INFO:  2 Max export price 15.00p/kWh at 20/07 22:30   SOC:  16.0%-> 16.0% Net: 164.8 
21:10:28     INFO:  1 Max export price 15.00p/kWh at 20/07 23:00   SOC:  16.0%-> 16.0% Net: 164.8 
21:10:28     INFO:  0 Max export price 15.00p/kWh at 20/07 23:30   SOC:  16.0%-> 16.0% Net: 164.8 
21:10:28     INFO: 
21:10:28     INFO: Discharge net cost delta: -0.0p: > Discharge Threshold (0.0p) => Slots included
21:10:28     INFO: Iteration  1: Slots added:   0

@stevebuk1
Copy link
Collaborator

stevebuk1 commented Jul 30, 2024

Hi @mergwyn,I'm about to start the EV support under the Agile tariff. I've released Beta-8 which allows me to turn off Octopus Auto and set manual tariffs so I can progress the development.

I do however have a question though, mainly for my own benefit - I remember you saying that you tend to use IOG in spring/summer and then swap back to Agile for late Autumn/Winter.

On IOG, I can pretty much average 7p per kWh as I have sufficient battery capacity to power the house for all consumption. Looking at months of Agile, I cannot see how its possible to approach that 7p, even the regular cheapest Agile slots seem to exceed that. Granted I have seen negative pricing and plunge events, but I assume you think it is is possible to achieve 7p or less using the Agile tariff over winter. Am I correct in my assumptions? Whilst I don't have export, the price for export on IOG and Agile is the same (15p)?

@mergwyn
Copy link
Author

mergwyn commented Jul 31, 2024

@stevebuk1 Unfortunately I don't have the battery capacity to bridge a whole day. We have an ASHP which in the depths of winter can consume over 800 kWh in a month on its own (vs less than 50 kWh in the lowest summer month)! My 10kWh batteries are usually enough to bridge the 3 hour agile peak periods in the morning and afternoon.

It doesn't like you have had chance to add the fix for discharge issue in Beta-8 as per your email:

Ok I think I know what the issue is here. Its either been caused by the price change on IOG that happened the 1st of July or the change of numpy library we all had to force to a particular version, or both combined.

I think prior to the 1st of July the IOG cheap rate price was a nice round number, now it is to something like four decimal places.

Both the low cost charging and the discharging algorithm search for the cheapest slots by looking at all of them, finding the minimum, matching that slot, utilising it then removing it from the list. (Whilst IOG slots are all exactly the same price the theory is still the same). The problem is that the calculation to find the minimum isn't reporting the actual minimum, so when the matching part runs - it finds none of them, hence no forced charging/discharging.

Is that still something you are thinking of taking a look at?

@stevebuk1
Copy link
Collaborator

stevebuk1 commented Jul 31, 2024

Unfortunately I don't have the battery capacity to bridge a whole day. We have an ASHP which in the depths of winter can consume over 800 kWh in a month on its own (vs less than 50 kWh in the lowest summer month)! My 10kWh batteries are usually enough to bridge the 3 hour agile peak periods in the morning and afternoon.

Thanks for your answer - makes perfect sense. I'm planning an ASHP, but aware that for the cold months I'm averaging around 110kWh a day on gas, so even with better insulation (WIP) and a decent SCOP I'll have the same problem (I have 12kWh of batteries).

@stevebuk1
Copy link
Collaborator

Ok I think I know what the issue is here. Its either been caused by the price change on IOG that happened the 1st of July or the change of numpy library we all had to force to a particular version, or both combined.
I think prior to the 1st of July the IOG cheap rate price was a nice round number, now it is to something like four decimal places.
Both the low cost charging and the discharging algorithm search for the cheapest slots by looking at all of them, finding the minimum, matching that slot, utilising it then removing it from the list. (Whilst IOG slots are all exactly the same price the theory is still the same). The problem is that the calculation to find the minimum isn't reporting the actual minimum, so when the matching part runs - it finds none of them, hence no forced charging/discharging.

Is that still something you are thinking of taking a look at?

Ok, the reason you'll no longer find that post in the thread when viewed on the Github website is because I deleted it. Basically I'd written nonsense: although the figures reported by my debug log were very slightly different, it didnt actually affect the if/then/else statement that used them.

I'd gone on a bit further after that and I'm still not sure I understand what "Low cost charging" actually does. I think it adds a single slot of full rate charging and then sees if it saves you money, but this cost saving is calculated prior to running the forced discharge part of the algorithm. Therefore, the only way it can work is making the assumption that any spare charge added is over and above needs and can be exported. It should always be over and above needs, as the "High cost swaps" that runs prior to that deals with all your actual needs, i.e ensures the battery finishes the day at just above max DOD. So, then doing a full rate charge in the cheapest slot should mean that excess can be exported, thus netting a cost saving. When I ran it I found the same as you. It may be because the cheapest slot for IOG is all 12 overnight slots (as they are all the same price), but as it appears to do one at a time I don't think its that. Its a case of setting up some detailed logging of the power flows to get a better understanding of whats actually happenning, to ensure the algorithm that was developed for the Agile tariff works equally as well for a dual rate tariff. I feel it should, but the devil is somewhere in the detail.....

@stevebuk1
Copy link
Collaborator

stevebuk1 commented Jul 31, 2024

@mergwyn to carry on from the previous post I found this write-up on what the algorithm does from Francis in the Solax development thread.

#188 (comment)

"Low Cost Charging" is point 4, and should result in a cost saving because of "Max Export". As stated in #121 (comment), the issue we are both finding in Beta-7 and that I'm finding in all historic versions that I've tried is that Net Cost Delta is zero. Did you try out 3.14.0 and then 3.13.2?

@mergwyn
Copy link
Author

mergwyn commented Aug 2, 2024

@stevebuk1 I could not get either 3.14.0 or 3.13.2 to work for me - both got stuck at "Checking tariff prices vs Octopus Energy Integration" so I gave up on both. I did try 13.15.4 which did work but did not produce any discharge slots (I can supply logs if it helps) At the moment my export rate is 15p and overnight import is 7p so I would have expected discharge slots.

@stevebuk1
Copy link
Collaborator

stevebuk1 commented Aug 5, 2024

Regarding Discharge slots, I think I've found the source of the problem , it was introduced in 3.15.4 (3.15.3 is ok) which I keep the Dev branch up to date with. I've raised this as a separate issue (#254) so it can get fixed on the production release. I'll do another Beta release with this code fix and additional logging, and post again when its done.

Regarding low cost charging, I think I now understand how this works.

As mentioned in #188 (comment) , the algorithm runs in 3 parts:

  1. High Cost usage swaps. Basically adds charging to cover your usage

  2. Low cost charging. Looks for any slots where the import price is < export price and see if there is a reduction in net cost by charging during these Low Cost Charging slots.

  3. Force Discharging. Do the same as low cost charging but for discharging.

On a time of use tariff , 1) will basically aim to leave your battery at max DOD at 11.30pm.

The key part I've noticed is that 2) runs on each slot at a time, by simulating a maximum rate charge for 1/2 hour and see if that saves you any money. The only way this can reduce cost prior to 3) being run is that the battery reaches 100% so there is then an export that will go to grid. If the available solar in that day doesn't get your battery anywhere near 100% then adding a full charge slot isnt going to get the battery to 100%, so no export and no cost saving. All that happens for each simulation is that you end the day with a battery that it is now above max DOD. If each slot simulation in 2) doesn't save you any money then no charge is actually added, so when 3) runs the battery is sitting at max DOD at 11.30pm, so there isnt anything additional to discharge in any slot (any discharge means you'll be importing at peak rate later on).

If the algorithm instead tried one slot of low rate charging and then ran through all of the slots of discharging to see if that saved you any money, then it would find a battery that had spare charge in it at 11pm and discharge it. Both of these slots would then be locked in, so the next low rate charging slot added would give more energy in the battery that could be discharged in another discharge slot, and so on and so forth. I guess doing that might be theoretically possible, but wasn't chosen when Pv Opt was originally written and at first glance, would be an architectural rewrite from scratch.

Francis may well be able to chime in and explain more? I'm aware that he always thought that Predbat was not all that optimum for Agile, and I know Trefor (the author of Predbat) is on IOG, so its certainly possible we may well be looking at the fact that the optimal algorithm for these tariffs differs, i.e an Agile based algorithm only considers everything a slot at a time but a time of use tariff would be better considering the 6 hour cheap rate period as a full block.

@stevebuk1
Copy link
Collaborator

I'll do another Beta release with this code fix and additional logging, and post again when its done

3.16.0-Beta-9 released at:
https://github.com/stevebuk1/pv_opt/tree/3.16.0-Beta-9

Copy all of pv_opt.py, pvpy.py and solis.py.

In your config.yaml, add the following lines:

  #=======================================
  #Logging Category Control
  #=======================================
  #Defines Logging subjects to add to logfile
  #Ignored if "debug" is set to False above
  #
  # S = Startup/Initialisation Logging
  # T = Tariff loading Logging
  # P = Power consumption history Logging
  # C = Charge algorithm Logging
  # D = Discharge algorithm Logging
  # W = Charge/Discharge Windows Logging
  # F = Power Flows Logging
  # V = Power Flows debugging (extra verbose)
  # I = inverter control/commands Logging
  # E = EV debugging

  # Letters can be added to "debug_categories" in any order

  debug_categories: CDFW

And when you see charging/discharging that doesn't look right (late evening is best), change
debug: False
to
debug: True
(or add "debug: True" if not present in config.yaml

and post the log from a single optimser run.

@mergwyn
Copy link
Author

mergwyn commented Aug 6, 2024

@stevebuk1 Thanks so much for tracking this down and supplying a fix. I've installed Beta-9 this morning and discharge slots have returned! However, as you suggest above, the timing of the slots doesn't look right: I have discharge slots at 16:30 when it makes more sense to discharge immediately before the cheap rate at 23:30.

Also, given there is good solar today and the battery looks like it will be at 100% from around 10:30 this morning, there are opportunities to discharge and then recharge from solar at at 15p per kWh nominal gain.

Here are the logs for an optimiser run starting at 10:213:23 with debug on:
pv_opt.06.log
pv_opt.05.log
pv_opt.04.log
pv_opt.03.log
pv_opt.02.log
pv_opt.01.log
pv_opt.log
error.log

@mergwyn
Copy link
Author

mergwyn commented Aug 6, 2024

@stevebuk1

If the algorithm instead tried one slot of low rate charging and then ran through all of the slots of discharging to see if that saved you any money, then it would find a battery that had spare charge in it at 11pm and discharge it. Both of these slots would then be locked in, so the next low rate charging slot added would give more energy in the battery that could be discharged in another discharge slot, and so on and so forth. I guess doing that might be theoretically possible, but wasn't chosen when Pv Opt was originally written and at first glance, would be an architectural rewrite from scratch.

I don't pretend to fully understand the subtleties of what you are saying, but I think my expectation is roughly what you describe. At a high level the goal is the same irrespective of Agile or IOG, ie to use cheap rate slots to charge the battery such that battery + solar can bridge higher rate slots. As I think you explain above the devil is in the detail!

Taking the simple case of IOG, I would hope that the algorithm can can determine whether to force discharge before the cheap rate slot, as well as not charge during the cheap rate slot if there is enough battery capacity to meet the forecast demand until solar generation can take over and start filling the battery.

Whilst I don't claim to understand Predbat (and find it hugely complicated), it does produce differer very different results for the next day or so:
Screenshot 2024-08-06 at 17 32 27

Screenshot 2024-08-06 at 17 31 28

@fboundy
Copy link
Owner

fboundy commented Aug 15, 2024

HI guys - I've been busy with other things. Should I pull a latest Dev branch yet?

@stevebuk1
Copy link
Collaborator

the timing of the slots doesn't look right: I have discharge slots at 16:30 when it makes more sense to discharge immediately before the cheap rate at 23:30.

Sorry for the delay, I had a look at the logs. All thats happenning is that the discharge algorithm runs through the slots in time order and works out if disharging for that slot saves you money - if it does that slot is "banked" (set as a discharge slot) and the next slot is tried. The 16:30 slot is the first one that saves you money, it banks the next few until it works out that banking any more doesn't save you money, in this case its because after a few theres insufficient battery to then last you to 11.30pm. In other words it is ensuring your battery is discharged by 11.30pm, and whether it does that at 16:30 or 22:00 probably makes no cost difference.

@stevebuk1
Copy link
Collaborator

stevebuk1 commented Aug 15, 2024

HI guys - I've been busy with other things. Should I pull a latest Dev branch yet?

@fboundy Yes, its ready. I haven't made any changes since Beta9, and that was just to fix the error introduced in 3.15.4. Mines been left to its own devices for weeks now and its doing what I expect.

Longer term:

  1. I will do the Agile EV charging at some point but thats all in pseudocode at the moment.
  2. I will give the algorithm some more thought. A slight issue I have is that on a time of use tariff where the lowest cost slots are always a minimum of 4 contiguous hours, for the high cost swaps part of the algorithm theres quite a step change between doing no charging and setting inverter to hold for the full four hours i.e. using the grid for four hours. What would be cost optimum would be to hold for some of that time and revert to self use for the rest of the time. One way of doing that would be to artificially change the import price of the cheap slots so they are all slightly different, then it would use those slots first, just like it does on Agile. However that does also have disadvantages w.r.t inverter writes, and also the spreading the charging at a low charge rate would then be lost. I'm not sure the cost saving makes it worth the effort, and in any case I think is unique to those having no export tariff, which won't be many.

@stevebuk1
Copy link
Collaborator

@fboundy Yes, its ready. I haven't made any changes since Beta9, and that was just to fix the error introduced in 3.15.4. Mines been left to its own devices for weeks now and its doing what I expect.

@fboundy I forgot to add the PR is already raised.

@stevebuk1
Copy link
Collaborator

stevebuk1 commented Sep 1, 2024

@mergwyn: just to let you know I'm probably only a couple of weeks away from a beta release that will do car charging on Agile.

I've largely based it on how IOG works, as thats all I know:

  1. a "candidate plan" will be generated on each optimiser run, based on the car battery capacity, a "% charge to add" setting, a "ready by" time and a 'max price per slot' setting
  2. upon car plugin the candidate plan will be transfered to an 'active plan'.

The active plan remains fixed unless updated via an HA button press, or if 4pm is reached (new Agile rates are available)

For a first release I think 'charge to add%' keeps it simple and allows an external automation to set it based on battery capacity, target charge and car current SOC - this is what I currently do on IOG with two EVs and it works well.

I'm thinking that a "little and often" strategy is best for Agile, in that you'd plug the car in most nights and set a max cost. If you really needed it to be full for the next day then you'd up the max cost and let the car charge in accordance with charge to add.

Obviously all this will be integrated with current Pv_opt functionality so that any car charging doesn't discharge the house battery. In my trial runs I've found the EV wants more slots than the house, but in your situation I'd imagine its the other way around.

Given your previous experience with EV charging on Agile do you think this generally what you'd expect?

@stevebuk1
Copy link
Collaborator

stevebuk1 commented Sep 1, 2024

@mergwyn I forgot to add that for now I'm just going to quantise the EV charge plan to 1/2 hour slots, i.e for a 7kW charger the plan will be 3.5kWh or 7kWh or 10.5kWh and so on. The algorithm will always round up - i.e. it will not leave you with a lesser percentage charge than requested. Will consider something more complex based on kWh tracking and/or actively redoing the plan based on car SOC incrementing at a later stage. I think with a large car battery then a 3.5kWh resolution is an acceptable approximate.

@mergwyn
Copy link
Author

mergwyn commented Sep 5, 2024

Hi @stevebuk1

Given your previous experience with EV charging on Agile do you think this generally what you'd expect?

Yes this is just about what I'd expect. I did have a thought: I pretty much have the car plugged in every night - I think I need to comfort blanket of knowing that the car will be charged when I need it in the morning. I assume that even when increasing the max cost, pv_opt will always use the cheapest slots to achieve the charge?

I think with a large car battery then a 3.5kWh resolution is an acceptable approximate.

This sounds reasonable.

Thanks for all your work on this!

@stevebuk1
Copy link
Collaborator

stevebuk1 commented Sep 5, 2024

I pretty much have the car plugged in every night - I think I need to comfort blanket of knowing that the car will be charged when I need it in the morning.

To achieve this, set max cost to something like 40p. This then will schedule enough slots to ensure the car gets the "charge to add %" by the "Ready by" time that you set.

I assume that even when increasing the max cost, pv_opt will always use the cheapest slots to achieve the charge?

Yes, it will always use the cheapest slots (between plugin time and ready by time, to achieve the charge it needs), regardless of the max cost setting.

I tend to charge once a week and add about 40kWh, so the concept of a max cost was that I'd set a level I'd be happy with paying, and plug in every night. Some nights I'd then get maybe nothing and other nights I'd end up with a full charge. Whilst writing the code though, I've noticed that Agile is either generally expensive or generally cheap, so maybe its of limited use, in which case set it to max and forget about it.

I think with a large car battery then a 3.5kWh resolution is an acceptable approximate.

This sounds reasonable.

Thanks. Its only for the first release. I'll sort out something that measures kWh and can charge for partial slots later, once the bugs are ironed out on the interfacing. I think it will be worth it as the partial slot will always be the most expensive slot. The big unknown is the Zappi itself, which I'm certain will actually apply random delays to the start/stop commands its sent, which may then mean its not worth doing.

@stevebuk1
Copy link
Collaborator

@mergwyn I've now finished development and have started testing (which I can do by forcing the Agile tariff).

Let me know if/when you swap back to the Agile tariff and I'll find a place to upload the new code.
(As my last PR is still outstanding, it probably won't be to the Dev fork)

@mergwyn
Copy link
Author

mergwyn commented Sep 19, 2024

@stevebuk1 that's great news. Notwithstanding the price increase announced today, the Octoprice app is suggesting that it currently doesn't make sense for me to swap to agile until February 2025!

IMG_3361

@stevebuk1
Copy link
Collaborator

lol - ok no rush then!

Its relatively easy for me to test anyway, even with live EV charging - I just do a tariff override in config.yaml and turn off smart charging in the Octopus app. I note that in reality IOG smart slots tend to follow Agile relative pricing which I guess isn't a surprise. I'll carry on with testing and also add the other bits and pieces like partial slots.

@fboundy
Copy link
Owner

fboundy commented Sep 21, 2024

@stevebuk1 - testing this on my system now. There is something funny going on with the `_create_windows' method. When the current slot has (say) 3500W but there is only 15 minutes left it is scaling this up to 7000W which the inverter cannot physically do. This is down to this comment:

        # However, a factored "forced" value means that it result in a seperate charge window for the remaining time, and for IOG that will result in needless
        # writes to the inverter every 10 minutes and an increasing charge power towards the end of the night to compensate.
        # As the .flows calls are all done with now, the "forced" power can be set back to what the inverter needs it to be, which is the original value prior to factoring```

It seems to be complicating what should be a very simple method.  If this has do be done here for the IOG logic to work then I suggest you put a switch in so that it is only done for IOG and the original logic is kept for non IOG setups

@stevebuk1
Copy link
Collaborator

stevebuk1 commented Sep 21, 2024 via email

@fboundy
Copy link
Owner

fboundy commented Sep 22, 2024

To be clear the optimiser returns the optimum power for the remainder of the 1st period. If we are already in the 1st period the SOC at the end is calculated on the remaining time at that power but I'm struggling to see why the power itself would need adjusting, Maybe I'm missing something.

@stevebuk1
Copy link
Collaborator

stevebuk1 commented Sep 22, 2024

I've checked my design notes; reproduced below. (the logging excerpt is pre the mods)

image

Why when 10 mins into a slot does the forced power decrease?
The factor is a representation of how much of the slot window is left. This should not decrease the power, but it probably does it to ensure .flows doesn’t have to be time aware, so factoring power is done so that energy transfer is factored. However the inverter still needs the same power regardless of how many minutes it is into the current slot, which it gets from 'forced'. We need to factor that back up by the same amount it was reduced by.

Problem - the factoring back up will assume the same factor has been applied for each high cost swap. This is a valid assumption unless the slot is full, then reversing the factor will result in a charge rate higher than the converter can do. However - does this matter? .flows remains correct, and the inverter cannot do more than it is capable of, so should be ok? For a 3.6kW inverter its only ever going to occur for people who have more than 20kWh of batteries.

For Agile any charging slots will always tend to be full because the high cost swaps fill one 1/2 hour slot at a time (the cheapest), but for time of use tariffs with a fixed cheap rate period (Octopus Go, Octopus Intelligent Go, Eco 7 etc) the algorithm spreads the charge equally between all slots for each high cost swap. Its very unlikely that any slot is full for any of these tariffs, i.e. you'd be needing to charge at full rate all night. However, the reverse is true for Agile - its very unlikely that any slot is not full. When the slot is full, factoring back up isnt needed and causes the issue you've seen.

I'll spend some time trying to sort a better solution that suits all tariffs. One could easily solve the problem by treating the constant cheap rate tariffs like Agile in that each high cost swap gets allocated to a particular slot until its full, then another is chosen, but I do like the the fact that high cost swaps are spread across all slots of the same price because it means a constant low charge rate all night which IMHO is better for the inverter.

@fboundy
Copy link
Owner

fboundy commented Sep 23, 2024 via email

@stevebuk1
Copy link
Collaborator

stevebuk1 commented Sep 23, 2024 via email

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request long term Good idea but not a priority
Projects
None yet
Development

No branches or pull requests

5 participants