diy solar

diy solar

JK-BMS-CAN with new Cut-Off Charging Logic (open-source)

Well they are working for me now (minus the wrong SKU of the Rs485 adapter...).

I quickly tried OTA on a Raspberry Pi inside my home. That's also working. Next is try in the garage but I don't see why that should fail...

TLDR if somebody is having OTA updates issues that when triggered brick the device, causing the watchdog to continuo trigger and never reboot (as it should do when OTA fails), then it's a network issue.

Check your network routing, routes gateway. Test point to point using iperf3, ping/traceroute are NOT that accurate (because typically ping ICMP is allowed no matter what). Tcptraceroute is somewhat better but can also give false positives. Check that using iper3 you get 100-1000 Mbit/S.

Of course you cannot test the Esp32 like that. But at least all the way until the last step (AP - > ESP32).

My solution was to define static routes.

I haven't had many problems with OTA, my ESP32s are in DHCP, my DHCP server assigns them a fixed IP address.

I flash the firmware by specifying the IP of the ESP32:

Bash:
esphome run file.yaml --device 192.168.x.y
 
I haven't had many problems with OTA, my ESP32s are in DHCP, my DHCP server assigns them a fixed IP address.

I flash the firmware by specifying the IP of the ESP32:

Bash:
esphome run file.yaml --device 192.168.x.y
Me too, I have several esp32-S2lolin in remote internet address but ota and even api (initial version now I moved to MQTT ) and all working fine!
 
My hurdles with ESP's is simple. If I utilize the web server interface, they keep rebooting if I am refreshing the page.
If I exclusively use Home Assistant, everything is rock solid. Clearly, there is a difference between the relative stability of the REST and HA api
 
I haven't had many problems with OTA, my ESP32s are in DHCP, my DHCP server assigns them a fixed IP address.

I flash the firmware by specifying the IP of the ESP32:

Bash:
esphome run file.yaml --device 192.168.x.y
I'm not saying it's rocket science. Many people do NOT have the problems I had.

Probably I had something VERY wrong in my network setup (maybe even more than one thing).

The biggest clue was that disabling the OPNSense Firewall for a quick testing (pfctl -d) made iperf3 work.
You should only do this in case you have an upstream ISP Router that has a firewall enabled ! Otherwise you would expose yourself to the Internet (Dangerous) !!!

Schematics of the issue:
ESP32 Devices (172.22.x.y) <---> Rock 5B (172.22.1.1) WiFi AP Hostapd + DNSmasq <---> Rock 5B (192.168.4.x) <---> Switch <---> Main Router/Gateway (192.168.1.1) <--> Ubuntu Client (192.168.x.y)

Schematics of the solution with Static Routes:
ESP32 Devices (172.22.x.y) <---> Rock 5B (172.22.1.1) WiFi AP Hostapd + DNSmasq <---> Rock 5B (192.168.4.x) <---> Ubuntu Client (192.168.x.y) with Static Route of 192.168.4.x for the 172.22.0.0/16 Network

The most likely issue is that the OPNSense Router was getting traffic IN from one Network Interface and then having to send it OUT to the SAME Network Interface. Even with some settings (e.g. Firewall -> Settings -> Advanced -> Static route filtering: Bypass firewall rules for traffic on the same interface [CHECKED]) it just wouldn't work properly. The strange part is that it "kinda" worked with ping, traceroute and even more surprisingly tcptraceroute (since this uses TCP packets ! , whereas ping ICMP is usually allowed no matter what). The main hint was running iperf3 between the AP and the Ubuntu Client (Rock 5B/Raspberry Pi AP running server iperf3 -s while Ubuntu Client running client iperf3 -c 192.168.4.x -P 1). iperf3 would just show MISERABLE performance below 0.1 Mbit/s. As soon as you disable the firewall on OPNSense (just for testing, anyway I already have firewall enabled on the ISP Router upstream !) with pfctl -d then it would start working. Enable with pfctl -e then the iperf3 performance would go again off a cliff.

I solved with RFC3442 and DHCP Option 121 to automatically push these static routes to DHCP clients: see https://github.com/luckylinux/rfc3442-dhcp-option-121 for a script that can build the DHCP Option 121 in case you are interested (I based it on an existing Github gist in Python + some fixes based on some Web Based Javascript Calculators I found online).

But otherwise you can just define a Static Route manually for your Linux Hosts: https://www.cyberciti.biz/tips/configuring-static-routes-in-debian-or-red-hat-linux-systems.html.

Proper solution will be to have VLAN and VLAN routing, but I am currently facing many issues on that front, so it's not a day (or week) fix. Getting Mikrotik Switches work with Openwrt Switches work with Zyxel Switches working with OPNSense in a VM on Proxmox VE (with Mellanox ConnectX-2 or Mellanox ConnectX-3 card and the Linux Driver VLAN limitation of 128 VLANs) is more complicated than you might think (but will need to be done at some point) ....
 
My hurdles with ESP's is simple. If I utilize the web server interface, they keep rebooting if I am refreshing the page.
If I exclusively use Home Assistant, everything is rock solid. Clearly, there is a difference between the relative stability of the REST and HA api
IIRC @Sleeper85 or @MrPablo said that logging on the webserver is quite resource intensive and it would be enough to crash the ESP32.

I did NOT experience that (although the webserver is VERY SLOW).

Try increasing the Stack size:
YAML:
esphome:
  name: ${name}
  platformio_options:
    build_flags:
      - -DCONFIG_ARDUINO_LOOP_STACK_SIZE=32768

Although not sure if it really works, since we use the esp-idf framework, and that option is for Arduino Framework only I think.
 
I have never ever managed to keep an ESPhome device connected to my network for longer than a couple of minutes. I don't know what the hell I'm doing wrong but apparently everyone else is doing it wrong too.
It just dawned on me, May I ask what framework is everyone using? We recommend the esp-idf framework due to its stability.

IIRC @Sleeper85 or @MrPablo said that logging on the webserver is quite resource intensive and it would be enough to crash the ESP32.

I did NOT experience that (although the webserver is VERY SLOW).
I also said the same. Basically, new users should use DEBUG logging initially to debug UART or CAN pin issues. Once, everything is up and running they should return to a reasonable and lower log value.
 
It just dawned on me, May I ask what framework is everyone using? We recommend the esp-idf framework due to its stability.


I also said the same. Basically, new users should use DEBUG logging initially to debug UART or CAN pin issues. Once, everything is up and running they should return to a reasonable and lower log value.
It could be logging being set to DEBUG causing the esphome to crash.

But network could also play a thing e.g. simultaneous broadcast of 2.4GHZ and 5.0GHz networks using the same AP SSID or some wireless option only supported by more modern (and/or advanced) devices.

Only logs can tell.

@Quattrohead

I suggest you either manually monitor it via USB using
Code:
esphome logs XXX.yaml

Or automatically capture logs over several minutes/hours by spinning up a console e.g.
Code:
minicom --device /dev/ttyUSB0 -C my-log-name.txt

And then analyze it later.
Exit minicom using CTRL+A -> Click "Q" -> "Leave without reset?" -> "Yes" Selected (Default) -> Click "ENTER"

Make sure you have auto carriage return ENABLED otherwise you are just going to see a lot of tabs shifted between the lines:
In the file ~/.minirc.dfl (e.g. /home/<username>/.minirc.dfl) add:
Code:
pu addcarreturn    Yes

Make sure that AFTER "pu " and before "Yes" the whole "addcarreturn " takes EXACTLY 16 characters.
Source: https://unix.stackexchange.com/ques...manently-translate-incoming-newline-n-to-crlf

EDIT 1: If you are on Windows, Kitty, Putty or Teraterm might be an option. I am NOT using Microsoft Windows anymore. Your mileage might vary ...

- https://www.putty.org/ (I didn't know that Putty Development had restarted (when I still used Windows I would have used Kitty for SSH)
- http://www.9bis.net/kitty/index.html#!index.md
- https://github.com/TeraTermProject/teraterm
 
Last edited:
I'm not saying it's rocket science. Many people do NOT have the problems I had.

Probably I had something VERY wrong in my network setup (maybe even more than one thing).

The biggest clue was that disabling the OPNSense Firewall for a quick testing (pfctl -d) made iperf3 work.
You should only do this in case you have an upstream ISP Router that has a firewall enabled ! Otherwise you would expose yourself to the Internet (Dangerous) !!!

Schematics of the issue:
ESP32 Devices (172.22.x.y) <---> Rock 5B (172.22.1.1) WiFi AP Hostapd + DNSmasq <---> Rock 5B (192.168.4.x) <---> Switch <---> Main Router/Gateway (192.168.1.1) <--> Ubuntu Client (192.168.x.y)

Schematics of the solution with Static Routes:
ESP32 Devices (172.22.x.y) <---> Rock 5B (172.22.1.1) WiFi AP Hostapd + DNSmasq <---> Rock 5B (192.168.4.x) <---> Ubuntu Client (192.168.x.y) with Static Route of 192.168.4.x for the 172.22.0.0/16 Network

The most likely issue is that the OPNSense Router was getting traffic IN from one Network Interface and then having to send it OUT to the SAME Network Interface. Even with some settings (e.g. Firewall -> Settings -> Advanced -> Static route filtering: Bypass firewall rules for traffic on the same interface [CHECKED]) it just wouldn't work properly. The strange part is that it "kinda" worked with ping, traceroute and even more surprisingly tcptraceroute (since this uses TCP packets ! , whereas ping ICMP is usually allowed no matter what). The main hint was running iperf3 between the AP and the Ubuntu Client (Rock 5B/Raspberry Pi AP running server iperf3 -s while Ubuntu Client running client iperf3 -c 192.168.4.x -P 1). iperf3 would just show MISERABLE performance below 0.1 Mbit/s. As soon as you disable the firewall on OPNSense (just for testing, anyway I already have firewall enabled on the ISP Router upstream !) with pfctl -d then it would start working. Enable with pfctl -e then the iperf3 performance would go again off a cliff.

I solved with RFC3442 and DHCP Option 121 to automatically push these static routes to DHCP clients: see https://github.com/luckylinux/rfc3442-dhcp-option-121 for a script that can build the DHCP Option 121 in case you are interested (I based it on an existing Github gist in Python + some fixes based on some Web Based Javascript Calculators I found online).

But otherwise you can just define a Static Route manually for your Linux Hosts: https://www.cyberciti.biz/tips/configuring-static-routes-in-debian-or-red-hat-linux-systems.html.

Proper solution will be to have VLAN and VLAN routing, but I am currently facing many issues on that front, so it's not a day (or week) fix. Getting Mikrotik Switches work with Openwrt Switches work with Zyxel Switches working with OPNSense in a VM on Proxmox VE (with Mellanox ConnectX-2 or Mellanox ConnectX-3 card and the Linux Driver VLAN limitation of 128 VLANs) is more complicated than you might think (but will need to be done at some point) ....

Actually it's more network problems on your side.

Funny, I also use OPNsense and Proxmox VE.
 
But network could also play a thing e.g. simultaneous broadcast of 2.4GHZ and 5.0GHz networks using the same AP SSID or some wireless option only supported by more modern (and/or advanced) devices.

This shouldn't be a problem, I work with 2.4Ghz and 5Ghz WiFi with the same SSID without problems.

The ESP32 works in 2.4Ghz, they will never try to connect to 5Ghz, this network is not even visible to them.
 
Actually it's more network problems on your side.

Funny, I also use OPNsense and Proxmox VE.
Well you can say it's "my problem", fair enough. It doesn't really help.

That being said, I am working on improving things. It just will NOT happen overnight.
 
This shouldn't be a problem, I work with 2.4Ghz and 5Ghz WiFi with the same SSID without problems.

The ESP32 works in 2.4Ghz, they will never try to connect to 5Ghz, this network is not even visible to them.
There were several posts of GitHub issue tracker of ESPHome as well as some Home Assistant Forum Posts.

Some of them also related to Ubiquiti/UNIFI configuration.
 
Well you can say it's "my problem", fair enough. It doesn't really help.

It's true that it doesn't help 😅 , your network configuration is unnecessarily complicated.
An OPNsense router, several Rock 5Bs also acting as router and AP, it's complex and not how to build a good network (I work in this field).

You can create a multi-LAN network including a WiFi AP (with hostapd) with a single mini-PC server equipped with a WiFi card and two LAN ports.

You install Proxmox VE on the mini-PC, a dedicated Ethernet port for the WAN, a bridge for the LAN with an Ethernet Port and the WiFi card.
Then you install an OPNsense VM, OPNsense should be your only router, the ISP router is bridged to the OPNsense WAN port.

Then if you want to create VLANs you pass it through the uplink between the OPNsense LAN port and your switch supporting VLANs. No need to play with static routes OPNsense knows the different networks and takes care of routing the traffic.

This is how a corporate network is created (except for the mini-PC and the WiFi AP part of course), there is never more than a single router-firewall through which all traffic passes. ;)
 
I work with 2.4Ghz and 5Ghz WiFi with the same SSID without problems.
This seems to be router dependant and how well or how badly its firmware is programmed, in the UK the BT Smarthub router just cannot handle a SSID shared between frequencies and the first job on receipt of the router is to separate into 2 SSID's. The Smarthub is in its 5th Generation and still not fixed.
 
It's true that it doesn't help 😅 , your network configuration is unnecessarily complicated.
An OPNsense router, several Rock 5Bs also acting as router and AP, it's complex and not how to build a good network (I work in this field).

You can create a multi-LAN network including a WiFi AP (with hostapd) with a single mini-PC server equipped with a WiFi card and two LAN ports.

You install Proxmox VE on the mini-PC, a dedicated Ethernet port for the WAN, a bridge for the LAN with an Ethernet Port and the WiFi card.
Then you install an OPNsense VM, OPNsense should be your only router, the ISP router is bridged to the OPNsense WAN port.

Then if you want to create VLANs you pass it through the uplink between the OPNsense LAN port and your switch supporting VLANs. No need to play with static routes OPNsense knows the different networks and takes care of routing the traffic.

This is how a corporate network is created (except for the mini-PC and the WiFi AP part of course), there is never more than a single router-firewall through which all traffic passes. ;)
As I said I know it's not optimal 😅.

I'm not a Network Engineer, just a homelabber Sysadmin. And every time you try to add something new stuff break 😅 . Sometimes I wish I would have chosen IT and not Electrical Engineering as my Profession :rolleyes:.

I could NOT get VLANs to work. That's why, at least to have something working to monitor the Battery and Inverter for now, I set up some services on the Rock 5B SBC (Mosquitto MQTT, Home Assistant, Wifi AP, etc) which is accessible both from the ESP32 and both from the main LAN.

And in the meantime I have a lot of mess to fix in my house.

I was having a look right now for the VLAN topic based on some Proxmox Forum Discussions ...

This could NOT work due to Mellanox ConnectX-2 / ConnectX-3 Driver Limitation to 128 VLANs (I would actually say it only allows 125 VLANs !):
Code:
auto vmbr0
iface vmbr0 inet static
        address 192.168.2.13/20
        gateway 192.168.1.1
        bridge-ports enp9s0
        bridge-stp off
        bridge-fd 0
        bridge-vlan-aware yes
        bridge-vids 2-4094

This seems to work, now I'd have to test DHCP Server leases etc:
Code:
iface vmbr0 inet static
        address 192.168.2.13/20
        gateway 192.168.1.1
        bridge-ports enp9s0
        bridge-stp off
        bridge-fd 0
        bridge-vlan-aware yes
        bridge-vids 1 150-159 160-169 ...

And then of course there is something for the VM configuration regarding VLAN trunking and "native" VLAN id.
I didn't test this (of course after the latest update of the Proxmox VE Server, Grub decided to have a strike on me with ZFS, so I'm currently moving all the data and setting it up again with /boot of mdadm+ext4).

1712580047394.png

Add/Modify /etc/pve/qemu-server/XXX.conf
Code:
net1: virtio=XX:XX:XX:XX:XX:XX,bridge=vmbr0,firewall=1,tag=1,trunks=2-4094

(hopefully that doesn't trigger the same issue with RNETLINK No space left on device due to this 128 or 125 VLAN Number Limitation)

EDIT 1: Actually @Sleeper85, it's TWO OPNSense Routers (High-Availability CARP). Took a while to sort out the correct OPNSense SYNC and DHCP SYNC Options though 😅 . Maybe I'll introduce a third one, you never know ...
 
Last edited:
It's true that it doesn't help 😅 , your network configuration is unnecessarily complicated.
An OPNsense router, several Rock 5Bs also acting as router and AP, it's complex and not how to build a good network (I work in this field).

You can create a multi-LAN network including a WiFi AP (with hostapd) with a single mini-PC server equipped with a WiFi card and two LAN ports.

You install Proxmox VE on the mini-PC, a dedicated Ethernet port for the WAN, a bridge for the LAN with an Ethernet Port and the WiFi card.
Then you install an OPNsense VM, OPNsense should be your only router, the ISP router is bridged to the OPNsense WAN port.

Then if you want to create VLANs you pass it through the uplink between the OPNsense LAN port and your switch supporting VLANs. No need to play with static routes OPNsense knows the different networks and takes care of routing the traffic.

This is how a corporate network is created (except for the mini-PC and the WiFi AP part of course), there is never more than a single router-firewall through which all traffic passes. ;)
Are we saying the Atom Lite S3 might not work with a 5G network as this is what the current ESP32 dev chips are using?

Also - a simple network question since we are on this topic

Router 1 - IP: 192.168.1.1 >>Device 1
Router 2 - PI: 192.168.0.1 >> Device 2

What is the best and most straightforward way to make both device talk to each other - I know its something to do with creating a static route between the 2 routers, but not really strongly looked into how.
will the static route config be just on ne router or on both?

thanks...
 
It's true that it doesn't help 😅 , your network configuration is unnecessarily complicated.
An OPNsense router, several Rock 5Bs also acting as router and AP, it's complex and not how to build a good network (I work in this field).

You can create a multi-LAN network including a WiFi AP (with hostapd) with a single mini-PC server equipped with a WiFi card and two LAN ports.

You install Proxmox VE on the mini-PC, a dedicated Ethernet port for the WAN, a bridge for the LAN with an Ethernet Port and the WiFi card.
Then you install an OPNsense VM, OPNsense should be your only router, the ISP router is bridged to the OPNsense WAN port.

Then if you want to create VLANs you pass it through the uplink between the OPNsense LAN port and your switch supporting VLANs. No need to play with static routes OPNsense knows the different networks and takes care of routing the traffic.

This is how a corporate network is created (except for the mini-PC and the WiFi AP part of course), there is never more than a single router-firewall through which all traffic passes. ;)
And Amen to that - @silverstone looks like somebody who likes complexity... I was following his setup - at some point, he completely lost me.

it just seems there are a lot of dependencies for his Solar system to function and if one component breaks the system is down.

I like to keep the system as simple and as independent as possible - so that a few years down the line after taking a long break from the project I won't have too much to re-learn when management is required or when the system needs attention... 😜
 
Are we saying the Atom Lite S3 might not work with a 5G network as this is what the current ESP32 dev chips are using?

Also - a simple network question since we are on this topic

Router 1 - IP: 192.168.1.1 >>Device 1
Router 2 - PI: 192.168.0.1 >> Device 2

What is the best and most straightforward way to make both device talk to each other - I know its something to do with creating a static route between the 2 routers, but not really strongly looked into how.
will the static route config be just on ne router or on both?

thanks...
The Static Route is more of a "Hack" since I'm not using VLANs.

Why do you have 2 routers in your case ?

The best ? Depends if you want isolation/firewall/access control/NAT/etc between the devices or not.

The easiest would probably be to have a single router 192.168.0.0/22 which would be 192.168.0.0-192.168.3.254


As @Sleeper85 said Static Routes are NOT the proper way. it's more of a stop-gap measure until I fix everything VLAN related (needs setup on software level, switch configuration, managed switches, etc).

In your case, if the routers have 1 extra wired interface available, you could use ONE extra port on each where the "WAN" side of Router 1 is connected to the "LAN" side of Router 2 and viceversa.

So for Device 1 it would look:
192.168.1.x Device 1 -> 192.168.1.1 Router [LAN] -> 192.168.0.100 [WAN] -> 192.168.0.1 [LAN] -> 192.168.0.x Device 2

And for Device 2 it would look:
192.168.0.x Device 2 -> 192.168.0.1 Router [LAN] -> 192.168.1.100 [WAN] -> 192.168.1.1 [LAN] -> 192.168.1.x Device 1

WAN and LAN are just some labels. In this specific case they should rather be "LAN1" and "LAN2" but you get the idea.

So in one direction Router 1 is the Gateway of Router 2. In the other direction Router 2 is the Gateway of Router 1.
 
And Amen to that - @silverstone looks like somebody who likes complexity... I was following his setup - at some point, he completely lost me.

it just seems there are a lot of dependencies for his Solar system to function and if one component breaks the system is down.

I like to keep the system as simple and as independent as possible - so that a few years down the line after taking a long break from the project I won't have too much to re-learn when management is required or when the system needs attention... 😜
Wrong. The system works. The MONITORING is just not accessible in real time if something is down. And OTA was broken.

It's more difficult to troubleshoot an INTERMITTENT problem, rather than a straight "not working" situation, you know ?
 
The Static Route is more of a "Hack" since I'm not using VLANs.

Why do you have 2 routers in your case ?

The best ? Depends if you want isolation/firewall/access control/NAT/etc between the devices or not.

The easiest would probably be to have a single router 192.168.0.0/22 which would be 192.168.0.0-192.168.3.254


As @Sleeper85 said Static Routes are NOT the proper way. it's more of a stop-gap measure until I fix everything VLAN related (needs setup on software level, switch configuration, managed switches, etc).

In your case, if the routers have 1 extra wired interface available, you could use ONE extra port on each where the "WAN" side of Router 1 is connected to the "LAN" side of Router 2 and viceversa.

So for Device 1 it would look:
192.168.1.x Device 1 -> 192.168.1.1 Router [LAN] -> 192.168.0.100 [WAN] -> 192.168.0.1 [LAN] -> 192.168.0.x Device 2

And for Device 2 it would look:
192.168.0.x Device 2 -> 192.168.0.1 Router [LAN] -> 192.168.1.100 [WAN] -> 192.168.1.1 [LAN] -> 192.168.1.x Device 1

WAN and LAN are just some labels. In this specific case they should rather be "LAN1" and "LAN2" but you get the idea.

So in one direction Router 1 is the Gateway of Router 2. In the other direction Router 2 is the Gateway of Router 1.
Thanks

Why do you have 2 routers in your case ?

It's been like that for many years - I have an internet-facing router and the other has my secure stuff on it ( my ESPHOme tech for example is not internet-facing...
 
Thanks



It's been like that for many years - I have an internet-facing router and the other has my secure stuff on it ( my ESPHOme tech for example is not internet-facing...
Make sure you understand the differences between and the different aspects of networking:
- Routing
- Switching / Bridging
- NAT including OUTBOUND NAT
- Firewall (Input Rules, Forward Rules, Passthrough Rules, Output Rules, etc)

As well as Layer 2 vs Layer 3 stuff.

I myself have NOT gotten all of those sorted out.

Essentially unless you take good care of all of the above (plus others I'm sure) you risk sending some IP address to a subnet/interface that is NOT supposed to see them. Or you might be able to send traffic in one direction but not being able to get back the response (although that should be a bit better with statefull firewalls w.r.t. stateless firewall) because for instance one of the router rejected the request (or dropped the connection since it didn't know where it should go to).

In OPNSense this would trigger a default state rejection for instance as it doesn't make sense from a networking point of view to have that situation occur. But that was also one of the things I WAS doing.
 
Back
Top