Thumbnail image

Ingesting Solar Panel Production From a SunPower PVS to InfluxDB

  • Sun, Sep 29, 2024
  • 8-minute read

Table of Contents

I’ve had a SunPower system and 14, 400W solar panels installed on my house since May of 2019. It has always frustrated me that SunPower does not allow you to check the production and status of each panel out of the box. The app they give owners access to only displays the total amount of energy generated by the system over a given period of time. For most people this is probably fine and any additional displays would probably confuse the average owner. However, I (and probably you reading this) want to see all the data the system generates because I purchased the system and like hoarding data. I’m also concerned with SunPower’s recent Bankruptcy filing, I will lose monitoring services through SunPower all together.

After some googling and reddit rabbit holes, I came across the sunpower-pvs-exporter GitHub repo, Scott Gruby’s fantastic blog post on how to use a Raspberry Pi Zero (WH) as a bridge to call the PVS API, and this PDF by Dolf Starreveld which goes into significant detail about monitoring different SunPower products.

Sunpower and Monitoring Hardward Overview

Hardware Setup

SunPower System

  1. SunPower PVS6
  2. 14 SunPower 400W Solar Panels

Monitoring System

  1. Raspberry Pi Zero WH
  2. Ethernet/USB HUB HAT Expansion Board for Raspberry Pi
  3. Raspberry Pi 4B
  4. InfluxDB v2 running on a Synology NAS

The Ethernet/USB HUB HAT Expansion Board for Raspberry Pi is installed on the Raspberry Pi Zero WH to add an ethernet port to the RPi Zero. Cheaper options exist, like a mini-USB to Ethernet Adapter, but I had one of these left over from a previous project and used it.

There are a million ways to store the data captured from the PVS6. In my case I chose to use an InfluxDB instance because I wanted to learn the technology. Others have used HomeAssistant as their data store, or a simple text file.

Data Ingestion System Overview

Identical to Scott Gruby and Dolf Starreveld’s setups, I used a RaspberyPi Zero (WH) and an Ethernet Adapter as a network bridge to allow me to call the PVS API from any client on my network. I then wrote a simple python script to ingest data from the PVS every minute to an InfluxDB instance I host locally.

Data Ingestion System

Setup Instructions:

Some customers have consumption monitors and storage installed, my system does not. This post will only cover how to ingest Inverter, PSV, and Power Meter data into an InfluxDB instance.

Setup the Raspberry Pi as a Bridge

  1. Download the Raspberry Pi Imager
  2. Select the Raspbian Lite image
  3. Using the advanced setting:
    • Add your wifi setting
    • Assign a username and password
    • Enable SSH
  4. Write the image to an SD card
  5. Boot Pi and ensure it connects to your network by finding it on your router’s DCHP table
  6. Verify you can connect to the Pi by SSHing into it
  7. Assign a static IP address mapping on your router for the Pi
  8. Reboot the Pi
  9. After the Pi finishes it reboot and reconnects to your networt; Update the OS using
    sudo apt-get update
  10. Reboot the Pi
  11. Install ha-proxy
    sudo apt-get install haproxy
  12. Add the following to /etc/haproxy/haproxy.cfg: \
 1frontend http
 2    bind *:80
 3    default_backend backend_servers
 4
 5backend backend_servers
 6    server sv1 172.27.153.1:80
 7
 8listen stats
 9    bind *:8080
10    stats enable
11    stats uri /
12    stats refresh 10s
13    stats admin if LOCALHOST \
  1. Shutdown the Pi.
  2. Connect the Pi to PVS6 Ethernet Port LAN(1) port via ethernet
  3. Connect the Pi to any PVS6 USB to supply power to the Pi Zero
  4. Now when you issue HTTP calls to the Pi, they’ll go to the PVS6.

Setup the InfluxDB Bucket

Create an InfluxDB Bucket called solarPanelProduction, and generate an API token you will use to post data to the bucket. Remember to copy the API token to a text file as you can only view it once in InfluxDB.

Ingest Data to InfluxDB

I’m using a Raspberry Pi 4B running Ubuntu to pull data from the PVS6 (via the RPi Zero Bridge) and write the data to my InfluxDB instance. I wrote the following Python script to grab data from the PVS6 and write the data to my InfluxDB instance. This script is being run every minute via a cron job, and prints are output to a text file. You will need to install influxdb_client and modify the script to include your RPi IP Address, InfluxDB URL, and API Token (see the highlighted lines).

 1import influxdb_client, os, time, requests, pprint
 2from influxdb_client import InfluxDBClient, Point, WritePrecision
 3from influxdb_client.client.write_api import SYNCHRONOUS
 4
 5token = "<token>"
 6org = "<InfluxDB Org>"
 7bucket="solarPanelProduction"
 8influxUrl = "<influxDB URL and Port>"
 9sunpowerUrl = "http://<RPiStaticIP>/cgi-bin/dl_cgi?Command=DeviceList"
10
11r = requests.get(sunpowerUrl)
12data = r.json()
13
14#pprint.pprint(data)
15
16measurements = []
17
18for device in data["devices"]:
19
20    pvs = {}
21    powerMeter = {}
22    inverter = {}
23
24    if device["DEVICE_TYPE"] == "PVS":
25    
26        pvs["measurement"] = "PV Supervisor"
27        pvs["tags"] = {}
28        pvs["tags"]["Device Type"] = device["DEVICE_TYPE"]
29        pvs["tags"]["Model"] = device["MODEL"]
30        pvs["tags"]["Hardware Version"] = device["HWVER"]
31        pvs["tags"]["Software Version"] = device["SWVER"]
32        pvs["tags"]["Serial Number"] = device["SERIAL"]
33        pvs["tags"]["Operational State"] = device["STATE"]
34        pvs["fields"] = {}
35        pvs["fields"]["Comm Error"] = int(device["dl_comm_err"])
36        pvs["fields"]["CPU Load"] = float(device["dl_cpu_load"])
37        pvs["fields"]["Uptime"] = int(device["dl_uptime"])
38        pvs["fields"]["Error Count"] = int(device["dl_err_count"])
39        pvs["fields"]["Untransmitted"] = int(device["dl_untransmitted"])
40        measurements.append(pvs.copy())
41
42    if device["DEVICE_TYPE"] == "Power Meter" and device["TYPE"] == "PVS5-METER-P":
43        
44        powerMeter["measurement"] = "Power Meter"
45        powerMeter["tags"] = {}
46        powerMeter["tags"]["Device Type"] = device["DEVICE_TYPE"]
47        powerMeter["tags"]["Model"] = device["MODEL"]
48        powerMeter["tags"]["Serial Number"] = device["SERIAL"]
49        powerMeter["tags"]["Operational State"] = device["STATE"]
50        powerMeter["tags"]["Type"] = device["TYPE"]
51        powerMeter["tags"]["Software Version"] = device["SWVER"]
52        powerMeter["fields"] = {}
53        powerMeter["fields"]["Frequency (Hz)"] = float(device["freq_hz"])
54        powerMeter["fields"]["Total Net Energy (kWh)"] = float(device["net_ltea_3phsum_kwh"])
55        powerMeter["fields"]["Real Power [Avg] (kW)"] = float(device["p_3phsum_kw"])
56        powerMeter["fields"]["Reactive Power (kVAr)"] = float(device["q_3phsum_kvar"])
57        powerMeter["fields"]["Real Power (kVA)"] = float(device["s_3phsum_kva"])
58        measurements.append(powerMeter.copy())
59
60    if device["DEVICE_TYPE"] == "Inverter":
61        inverter["measurement"] = "Inverter"
62        inverter["tags"] = {}
63        inverter["tags"]["Device Type"] = device["DEVICE_TYPE"]
64        inverter["tags"]["Model"] = device["MODEL"]
65        inverter["tags"]["Model Serial Number"] = device["MOD_SN"]
66        inverter["tags"]["Serial Number"] = device["SERIAL"]
67        inverter["tags"]["Operational State"] = device["STATE"]
68        inverter["tags"]["Type"] = device["TYPE"]
69        inverter["tags"]["Software Version"] = device["SWVER"]
70        inverter["tags"]["Hardware Version"] = device["hw_version"]
71        inverter["fields"] = {}
72        inverter["fields"]["Frequency (Hz)"] = float(device["freq_hz"])
73        inverter["fields"]["AC Current (amps)"] = float(device["i_3phsum_a"])
74        inverter["fields"]["DC Current (amps)"] = float(device["i_mppt1_a"])
75        inverter["fields"]["Total Energy (kWh)"] = float(device["ltea_3phsum_kwh"])
76        inverter["fields"]["AC Power (kW)"] = float(device["p_3phsum_kw"])
77        inverter["fields"]["DC Power (kW)"] = float(device["p_mppt1_kw"])
78        inverter["fields"]["DC Voltage (V)"] = float(device["v_mppt1_v"])
79        inverter["fields"]["AC Voltage (V)"] = float(device["vln_3phavg_v"])
80        inverter["fields"]["Heat Sink Temp (C)"] = float(device["t_htsnk_degc"])
81        measurements.append(inverter.copy())
82
83pprint.pprint(measurements)
84
85write_client = influxdb_client.InfluxDBClient(url=influxUrl, token=token, org=org)
86write_api = write_client.write_api(write_options=SYNCHRONOUS)
87write_api.write(bucket=bucket, org=org, record=measurements)
88
89print("Posted")

Visualize the Data in InfluxDB

Assuming everything is working correctly, you should see data begin to populate in your solarPanelProduction Bucket. I created an InfluxDB dashboard to visual my production data over time.

InfluxDB Dashboard

Generated Power Graph

The Generated Power Graph is a Band type graph that will display the Max, Min, and Avg power generated for a time window over a time range. When the Min and Max deviate from the average that generally means power generation was not consistent across all your panels during a time window, usually due to weather or shadow. If your min is constantly at zero, a panel is not functioning.

Generated Power Graph

InfluxDB Query Script

 1from(bucket: "solarPanelProduction")
 2  |> range(start: v.timeRangeStart, stop: v.timeRangeStop)
 3  |> filter(fn: (r) => r["_measurement"] == "Power Meter")
 4  |> filter(fn: (r) => r["_field"] == "Real Power [Avg] (kW)")
 5  |> aggregateWindow(every: 15m, fn: mean, createEmpty: false)
 6  |> yield(name: "mean")
 7
 8from(bucket: "solarPanelProduction")
 9  |> range(start: v.timeRangeStart, stop: v.timeRangeStop)
10  |> filter(fn: (r) => r["_measurement"] == "Power Meter")
11  |> filter(fn: (r) => r["_field"] == "Real Power [Avg] (kW)")
12  |> aggregateWindow(every: 15m, fn: max, createEmpty: false)
13  |> yield(name: "max")
14
15from(bucket: "solarPanelProduction")
16  |> range(start: v.timeRangeStart, stop: v.timeRangeStop)
17  |> filter(fn: (r) => r["_measurement"] == "Power Meter")
18  |> filter(fn: (r) => r["_field"] == "Real Power [Avg] (kW)")
19  |> aggregateWindow(every: 15m, fn: min, createEmpty: false)
20  |> yield(name: "min")

Yesterday and Today’s Production

The Yesterday and Today’s Production gages show a sum of how much power the system is generating today and generated yesterday. You will need to update your Timezone information In the query below. Yoy will also need to swap out today() for yesterday() to get yesterday’s data.

Yesterday and Today’s Production Gages

InfluxDB Query Script for Today’s Production

1import "timezone"
2option location = timezone.fixed(offset: -8h) 
3from(bucket: "solarPanelProduction")
4  |> range(start: today())
5  |> filter(fn: (r) => r["_measurement"] == "Power Meter")
6  |> filter(fn: (r) => r["_field"] == "Total Net Energy (kWh)")
7  |> spread()
8  |> yield(name: "spread")

Min and Max Instantaneous Production Gages

These gages will tell you what your best and worst performing panels are currently producing. I used this as a quick way to tell if I’m having an issue. If these values are disconnected more than expected (like 200W max vs 0W min), I know a panel is offline. The query below is for Max production, simply swap out max for min to get the minimum production panel value.

Min and Max Gages

InfluxDB Query Script for Max Single Panel Production

1from(bucket: "solarPanelProduction")
2  |> range(start: v.timeRangeStart, stop: v.timeRangeStop)
3  |> filter(fn: (r) => r["_measurement"] == "Inverter")
4  |> filter(fn: (r) => r["_field"] == "DC Power (kW)")
5  |> filter(fn: (r) => r["Operational State"] == "working")
6  |> map(fn: (r) => ({r with _value: r._value * 1000.0}))
7  |> group(columns: ["_field"])
8  |> aggregateWindow(every: v.windowPeriod, fn: max, createEmpty: false)
9  |> yield(name: "max")

Panel Specific Instantaneous Production Gages

These gages will tell you what each panel is producing. Setting these gages up is tedious since you have to set each one up individually, but worth the effort to quickly see which panel is offline or under performing. You will need to change the Serial Number filter for each gage you need to create.

Panel Gages

1from(bucket: "solarPanelProduction")
2  |> range(start: v.timeRangeStart, stop: v.timeRangeStop)
3  |> filter(fn: (r) => r["_measurement"] == "Inverter")
4  |> filter(fn: (r) => r["Serial Number"] == "E00121903036844")
5  |> filter(fn: (r) => r["_field"] == "DC Power (kW)")
6  |> map(fn: (r) => ({ r with _value: r._value * 1000.0 }))
7  |> aggregateWindow(every: v.windowPeriod, fn: last, createEmpty: false)
8  |> yield(name: "last")
Home Lab influxdbrpisolarsunpoweringestproject