PLEASE NOTE: If you had an account with the previous forum, it has been ported to the new Genetry website!
You will need to reset the password to access the new forum. Click Log In → Forgot Password → enter your username or forum email address → click Email Reset Link.

Notifications
Clear all

Firmware Version 1.1r6

61 Posts
6 Users
0 Likes
1,054 Views
(@deeezz)
Eminent Member
Joined: 4 years ago
Posts: 21
 

Yes it is still on rev. Before and after I reboot.


   
ReplyQuote
(@deeezz)
Eminent Member
Joined: 4 years ago
Posts: 21
 

It was on rev. To begin with.  I only switched back and forth to see if it would help find the update. Each time I switched I exited out of the wifi to save the settings and rebooted. It acts like it sees the update but then just goes back to saying up to date. 


   
ReplyQuote
(@sid-genetry-solar)
Member Admin
Joined: 4 years ago
Posts: 2883
Topic starter  
Posted by: @deeezz
It was on rev. To begin with. I only switched back and forth to see if it would help find the update. Each time I switched I exited out of the wifi to save the settings and rebooted. It acts like it sees the update but then just goes back to saying up to date.

Sounds like the MQTT part and file path is working correctly.  It's failing on validation of the file--which can sometimes indicate a file access issue.

Does your WiFi router/firewall allow downloading of unsecured files (i.e. regular http, not https), of "unknown" extensions?  Can an Internet device (phone/computer) with the same WiFi access privileges as the inverter download the following file?  [ wifi.genetrysolar.com/U1646490204.gsf ]


   
ReplyQuote
(@deeezz)
Eminent Member
Joined: 4 years ago
Posts: 21
 

Just checked and my android phone downloaded it just fine. I even shut off the cellular data. 


   
ReplyQuote
(@sid-genetry-solar)
Member Admin
Joined: 4 years ago
Posts: 2883
Topic starter  
Posted by: @deeezz
Just checked and my android phone downloaded it just fine. I even shut off the cellular data.

Well, can you confirm whether it downloaded HTTP or HTTPS?

The inverter is only able to do an HTTP access.  If the WiFi system/router (or even ISP) is forcing redirect to HTTPS, it will fail.

We can try a Tech update just to see if that makes a difference (as version control won't apply then).  Unfortunately, I can't "add diagnostics" to this...if we can't get it update in the first place.

 

P.S. I guess I'm expecting that you're pretty technosavvy in the Internet department if your router has DD-WRT in it 😉.  I could be making a very wrong assumption though.


   
ReplyQuote
(@deeezz)
Eminent Member
Joined: 4 years ago
Posts: 21
 

Id like to think I'm tech savvy know more than most but unfortunately I don't know it all I've worked in communication/ telecommunications since 1998. Ok so I logged into my dsl modem checked settings. I did not see anything that looked off. So i thought maybe I I turn the firewall off. And now I 1.1r6. Thank you Sid. You are the best.


   
ReplyQuote
(@sid-genetry-solar)
Member Admin
Joined: 4 years ago
Posts: 2883
Topic starter  
Posted by: @deeezz
Id like to think I'm tech savvy know more than most but unfortunately I don't know it all I've worked in communication/ telecommunications since 1998. Ok so I logged into my dsl modem checked settings. I did not see anything that looked off. So i thought maybe I I turn the firewall off. And now I 1.1r6. Thank you Sid. You are the best.

Looks like I need to add a feature for 1.1r7 where the inverter will show the HTTP error code in the event it can't access the specified update file.  Make it easier to diagnose this sort of issue in the future.


   
ReplyQuote
(@thebutcher)
Reputable Member
Joined: 4 years ago
Posts: 355
 
Posted by: @deeezz
Id like to think I'm tech savvy know more than most but unfortunately I don't know it all I've worked in communication/ telecommunications since 1998. Ok so I logged into my dsl modem checked settings. I did not see anything that looked off. So i thought maybe I I turn the firewall off. And now I 1.1r6. Thank you Sid. You are the best.

It looks like you have dd-wrt?  If so you can use tcpdump to snoop the GS' traffic since it isn't https.  Look into tcpdump.


   
ReplyQuote
(@sid-genetry-solar)
Member Admin
Joined: 4 years ago
Posts: 2883
Topic starter  
Posted by: @thebutcher
It looks like you have dd-wrt? If so you can use tcpdump to snoop the GS' traffic since it isn't https. Look into tcpdump.

Inverter is just trying to do a standard HTTP GET to the URL provided via the update channel.  If the router is forcing an HTTP redirect to HTTPS protocol, the inverter will fail to download the file--as it doesn't support HTTPS.  However, a smart Internet device (phone/tablet/computer/refrigerator) generally supports HTTPS and will accept the redirect and download the file as HTTPS.

I probably should mod the server settings so said files can ONLY be downloaded non-HTTPS.  That would make it easier to diagnose errors of this sort.


   
ReplyQuote
(@thebutcher)
Reputable Member
Joined: 4 years ago
Posts: 355
 

The router shouldn't be doing that, pretty terrible if it does.  This sort of thing should be left up to the requesting device alone.  tcpdump would show what is happening in any case.


   
ReplyQuote
(@notmario)
Reputable Member
Joined: 3 years ago
Posts: 314
 
On 3/6/2022 at 7:20 PM, TheButcher said:

The router shouldn't be doing that, pretty terrible if it does.  This sort of thing should be left up to the requesting device alone.  tcpdump would show what is happening in any case.

I agree.
Naturally, that means firewall designers do not.

It is unfortunate that the inverter doesn't support https://. Is that because openssl is impractical to fit? There is wolfssl now, which i think is supposed to be small - but of course small is relative in the embedded world.


   
ReplyQuote
(@sid-genetry-solar)
Member Admin
Joined: 4 years ago
Posts: 2883
Topic starter  
9 hours ago, NotMario said:

It is unfortunate that the inverter doesn't support https://. Is that because openssl is impractical to fit? There is wolfssl now, which i think is supposed to be small - but of course small is relative in the embedded world.

SSL opens up a huge quagmire of dependencies (and yes, lack of space).  Not only does the inverter then have to contact an NTP server (to get date/time) at every system startup (as there is no internal RTC), it also has to be able to download updated SSL certs on a periodic basis--which means that they then have to be hosted somewhere.  And if they're hosted somewhere, then anyone could download them and mimic the GS system--if not outright nullify any supposed security gains. 

And if the inverter can't access BOTH an NTP server AND a cert host, the WiFi connection is basically useless...which I guess might be a pretty lovely SHTF situation.

 

Fitting the current firmware into the WiFi board is being an extreme challenge due to the memory layout in the ESP32 (particularly the 2 RAM banks which are not contiguous).  This is without SSL: the project will fail compilation if I adjust the RAM distribution slightly one way (dynamic allocation vs compile-assign block).  And it'll crash/bootloop (stack fault) if I adjust it too far the other way.

I would expect SSL to be flaky on the ESP32 on a good day with nothing else going on.  Just for reference comparisons, the WiFi board firmware will not fit on an ESP32 with the standard FLASH partition table.  And that's without SSL.  Mind you, I don't have any huge multi-thousand-line bloated "off the shelf" libraries in the firmware either...the ones I am using are extremely small (and I've rewritten them to remove even more bloat).

 

I just don't see any pertinent benefit to SSL when there isn't anything of critical secrecy being downloaded/shared.  At some point here, I plan to be providing firmware update file download links (so remote users can "upload" a fimware file into the inverter from a phone/tablet/laptop if the inverter is installed in a remote location without Internet access)--so there's no "secrecy" point there.  And you can turn off the MQTT control function, removing any remote control access.


   
ReplyQuote
(@notmario)
Reputable Member
Joined: 3 years ago
Posts: 314
 
12 hours ago, Sid Genetry Solar said:

SSL opens up a huge quagmire of dependencies (and yes, lack of space).  Not only does the inverter then have to contact an NTP server (to get date/time) at every system startup (as there is no internal RTC), it also has to be able to download updated SSL certs on a periodic basis--which means that they then have to be hosted somewhere.  And if they're hosted somewhere, then anyone could download them and mimic the GS system--if not outright nullify any supposed security gains. 

And if the inverter can't access BOTH an NTP server AND a cert host, the WiFi connection is basically useless...which I guess might be a pretty lovely SHTF situation.

Oh, no integrated RTC... that sucks. Though, i think most routers serve NTP now so default gateway could be used as a fallback as well. On the other hand, you could in that situation just ignore time verification for the SHTF case. Very nasty indeed.

12 hours ago, Sid Genetry Solar said:

I just don't see any pertinent benefit to SSL when there isn't anything of critical secrecy being downloaded/shared.  At some point here, I plan to be providing firmware update file download links (so remote users can "upload" a fimware file into the inverter from a phone/tablet/laptop if the inverter is installed in a remote location without Internet access)--so there's no "secrecy" point there.  And you can turn off the MQTT control function, removing any remote control access.

So, it's not necessarily a matter of secrecy. It's a matter of security.
I make no assumption about how the device validates downloaded firmware, but unless you're using a "shared secret" (or, even better, PK) style crypto to validate the authenticity of the firmware, you could be leaving people open to MITM firmware manipulation. This probably isn't very likely, but if your inverters became very popular, it could become a target. The largest botnet in the world was made up of CCTV IP-cameras with compromised firmware that people didn't properly secure on their networks.

Given your pragmatic situation with SSL, you'd probably be better off with a custom designed solution, which shouldn't be too hard if you're good at type of math involved. The super fast, nasty fix would be a signature based on a hash of the binary with a shared secret. Without knowing the secret, an attacker can't compute the correct signature when they try to inject their modified version. Of course it might be possible to JTAG the device and get the secret off of it, but at least it becomes considerably less trivial.

Wasn't sure if you'd considered this problem. Just some food for thought.


   
ReplyQuote
(@sid-genetry-solar)
Member Admin
Joined: 4 years ago
Posts: 2883
Topic starter  
10 hours ago, NotMario said:

Given your pragmatic situation with SSL, you'd probably be better off with a custom designed solution, which shouldn't be too hard if you're good at type of math involved. The super fast, nasty fix would be a signature based on a hash of the binary with a shared secret. Without knowing the secret, an attacker can't compute the correct signature when they try to inject their modified version. Of course it might be possible to JTAG the device and get the secret off of it, but at least it becomes considerably less trivial.

Let's just say that if a single bit is changed in the entire firmware file, the inverter will fail the update process.  The firmware update file is definitely not a "plain vanilla BIN file", that's for sure--so hacking it to spoof things will be pretty nigh impossible.  There is also no capability to remotely initiate an update--for simple security reasons (not the least being that the inverter has to shut down to install the update...so there goes your power if you aren't prepared!)

 

10 hours ago, NotMario said:

On the other hand, you could in that situation just ignore time verification for the SHTF case. Very nasty indeed.

I'd just as soon ignore time verification...but if the SSL cert is expired, it won't work for decoding the incoming datastream--bringing any communications to a grinding halt.


   
ReplyQuote
(@notmario)
Reputable Member
Joined: 3 years ago
Posts: 314
 
3 hours ago, Sid Genetry Solar said:

Let's just say that if a single bit is changed in the entire firmware file, the inverter will fail the update process.  The firmware update file is definitely not a "plain vanilla BIN file", that's for sure--so hacking it to spoof things will be pretty nigh impossible.  There is also no capability to remotely initiate an update--for simple security reasons (not the least being that the inverter has to shut down to install the update...so there goes your power if you aren't prepared!)

That's a pretty good point. At least the attack window is minimized to when the user "hits the big red button." And since i literally can see you tell me there's a new update, that window is pretty darn small.
Hackers are pretty smart though -- if there is enough interest, they'll do it. Even to the point of physical analysis of the hardware to figure out critical details to enable them. Like i said, probably not a problem right now -- but when you sell a few million inverters... that may change.

Basically the only point i'm making is that in the long-run, it may really be worth your time. A CVE posted publicly with your product name on it wouldn't be the greatest thing in the world.

3 hours ago, Sid Genetry Solar said:

I'd just as soon ignore time verification...but if the SSL cert is expired, it won't work for decoding the incoming datastream--bringing any communications to a grinding halt.

An expired cert can still decode the datastream... the date is only used for authentication of the cert, not for the actual encryption. This doesn't matter anyway when it's impractical to implement in your hardware, but wasn't sure why you thought the expiration affected the encryption. Here's an example:

owner$ curl -I  https://expired.badssl.com/ 
curl: (60) SSL certificate problem: certificate has expired
More details here:  https://curl.haxx.se/docs/sslcerts.html 

curl failed to verify the legitimacy of the server and therefore could not
establish a secure connection to it. To learn more about this situation and
how to fix it, please visit the web page mentioned above.
owner$ curl -I  https://expired.badssl.com/  -k
HTTP/1.1 200 OK
Server: nginx/1.10.3 (Ubuntu)
Date: Thu, 10 Mar 2022 16:47:03 GMT
Content-Type: text/html
Content-Length: 494
Last-Modified: Thu, 10 Feb 2022 02:30:44 GMT
Connection: keep-alive
ETag: "620478d4-1ee"
Cache-Control: no-store
Accept-Ranges: bytes

   
ReplyQuote
Page 3 / 5