Aug 23 2021
RPI, ESP, MQTT for WATT Meter
Posted by kll · In RPI · 1596 Reads ·
Arduino IDE 2 for ESP
DC Watt Meter
& coding
& hardware
RPI4 MQTT broker
RPI3 MQTT test client
RPI4 MQTT store to own mongoDB
RPI4 MQTT or store by python to CSV file
RPI4 mosquitto 1.5.7 Python client and save CSV
update IDE / ESP boards
my code
wait, there still one more thing in the last shipment box:
ESP32 LCD
ESP32 with LCD, 3 buttons, no LED
( i needed a replacement for a old ESP with OLED where i fail to get the screen running again from Arduino IDE )
now while i am also interested to test this with MicroPython,
here we talk about using Arduino IDE 2
from a win10 PC.
i see the link to the board basics
a video about using it with Arduino IDE and the DHT sensor
or use it to make games , more info.
i now usually make my own PINOUT for learning / compact info / little connection diagrams...
unpack:
verify, same version v1.1 as in the above website.
connecting USB -C cable for power i see
+ + the TTGO logo picture
+ + and after some seconds a screen color test sequence.
+ + on the right side is a black push button you can reset that show..
after some time the screen went's black ( power save )
+ + on the right ( but on the backside of the board ) is a fast blinking blue mini LED / ?WIFI activity?
+ + on the right white operation button see first a voltage info ( here 4.88V )
on second press text " press for wake-up " ( from power save )
+ + on the left white operation button "scan network"
one second later see the list of wifi networks ( with dB )
but in a very small font.
a very good demo!
first not see that related code in the GitHub docu?
check out here
possibly it is at https://github.com/Xinyuan-LilyGO/TTGO-T-Display/tree/master/TFT_eSPI/examples/FactoryTest ( what includes a SD card code too?? )
? ? is that a touch screen display? wow, need to dig into that, i did see a example code?
but not in the DEMO, possibly not?
Arduino IDE 2 for ESP
as mentioned in the main article i have Arduino IDE 2.0.0 beta 10 nightly arduino-ide_nightly-20210808_Windows_64bit.zip
( still get the warning about Visual Studio Code is unable to watch for file changes in this large workspace" (error ENOSPC)
but we are to ignore that until it is fixed in 'theia' )
with https://dl.espressif.com/dl/package_esp32_index.json in preferences
UPDATE:
https://github.com/espressif/arduino-esp32/releases/download/2.0.0/package_esp32_index.json
and install esp32 by Espressif Systems
i can select board: TTGO LoRa32-OLED V1 ( just found in web that i should select that one?? )
but for the display need add ( tools / manage libraries / TFT_eSPI by Bodmer )
need some MODs see \Arduino\libraries\TFT_eSPI\
Open the file User_Setup_select.h
Comment the line #include User_Setup.h with //
and uncomment
#include User_Setups/Setup25_TTGO_T_Display.h // Setup file for ESP32 and TTGO T-Display ST7789V SPI bus TFT
and for the firmware add need 'Button2.h' 'Button2.cpp' from here
( manually install ) OR just copy the full TTGO-T-Display with download ZIP.
he tested the graph lib on this board: here,
to add more tools to arduino IDE like to use pictures from a board directory and display them... here
now follow a new idea:
DC Watt Meter
RPI / ESP / MQTT / spreadsheet /
small and different from old article here
what was a bigger project, RPI, CMS, PCS...
on a ESP TTGO T-display use 2 Ain to sample
like [V] and [A] from a DC Power Supply
+ + show on board TFT screen Volt / Amp / Watt
+ + print to USB ( if PC with IDE Monitor is connected ) ( use CSV line )
+ + send via MQTT to BROKER
that will be a Raspberry Pi running Eclipse Mosquitto 2.0
( and added test a speadsheet tool Cedalo for streaming data )
but lets start slowly, as we come directly from UNPACKING of the new board
( not try any download ( upload? always get confused.. ) to it )
but already setting up Arduino IDE 2.0 for ESP32 and Display lib.
-a- as i did it with the ESP8266 OLED can start with a intro graph ( at reboot for 3 sec )
-b- i want try a little menu system ( 5 color rectangle with text / if scroll 6 .. more )
+ right PB ( OP1) select next ( and scroll and jump at end to one again / menu down only NO up )
+ left PB ( OP2 ) select that menu entry.
+ ( there could be a timeout ?20sec? jump to default ( like menu entry 1 ) )
+ from where ever you are can call menu again by PB ( OP2 )
-c- in case of a simple data acquisition you could send the data "in real time" to a
data collection computer, where you add a time stamp...
but,
as we use a WIFI ESP we could get network time first ( at boot )
and give the sample ( data record ) a timestamp right here and send it incl ( USB / MQTT )
-d- menu entry 1
+ show reading of 2 A-in ( as Volt Ampere ( of a DC power supply test ) )
+ calculate / show Watt
+ try a idea of data reduction ( actually needed for file (DB) storage only )
- as a A-in sampler it should run as fast as possible..
- then a filtering or averaging could take place
- map to calibrated Volt Ampere range
- calculate Watt from it
- now need a rounding to ONE decimal ( for Volt Amp Watt )
- here i want a memory of last current ( and the minute of that sample )
- and a trigger logic, like:
+++ only if the rounded number of current changed send the data set
( decades ago i learned that concept as "swinging door" data reduction )
actually its only a dead-band of 0.1 Amp as in 1.15 to 1.249 rounded to 1.2A
+++ but also if the minute value changed.. to be sure you are LIVE (if there is no change ))
+ and local display show at a reasonable interval ?0.5 sec?
at change only ( above logic )
+ also send to USB as CSV line ( add date time )
24.8.2021 10:40:54,12.4,31.8,494.2,[date time][V][A][W],
+ + optional show it also on its own web server
( as ESP must have a fix IP on our LAN anyhow to do MQTT later )
-e- install MQTT and send to BROKER (RPI )
https://www.arduino.cc/reference/en/libraries/espmqttclient/
-f- there save to DB (last time used phyton paho sqlite3?) or spreadsheet
coding
first need to have my software workspace ready...
start with compile upload the example / demo code to see if IDE / settings / libs.. all fit.
verify OK
connect bord USB, select port 'COM13' here, keep default settings, upload OK
see unreadable small infotext. ( so yes, it is little different from DEMO )
now copy that as my own project: 'PowerMeter'
and from here start with above projects steps:
-a- my own intro picture
simple text
and lines and rectangles ( drawn 3 times with 1 pix offset )
and try also with FSB "FreeSerifBold" (Free_Fonts.h)
-b- button menu
start with 3 menu entries landscape ( with RED border and number later selected by button )
and have a Button operation ( [right] select next / [left] activate that menu entry/job ) loop to select a JOB:
example ( 1..5 prepared )
as a job1(); every loop ( like used for data collection / filter / range map...)
and a job1_screen(); depending on a refresh timer (5000) update displays dynamic data.
this version
-c- WIFI login and get network time
first start with the original factory wifi scan ( in landscape and FSB9 font ) under menu/job/4
i have old project for ESP about wifi login and web server...
( and many compiler switches also for ESP 32 / ESP8266 auto adjust )
so i copy that into and make the option to run a web server.. ( its for free! )
or you can disable it with
//#define useSERVER
not forget to adjust
+ + fix IP (STA mode ) ( or disable that )
+ + LAN WIFI 'SSID' 'password'
so here come the bad thing... and i think same happened to me already..
when i use from Files/Examples/ ESP32/Time/SimpleTime
it works fine..
when i copy that into my project it fails...
so lets put that NTP in HOLD
-d- [V]olt,[A]mp reading, [W]att calculation, send to display and USB (CSV line)
( as not even order hardware for current reading ?analog?I2C? later )
now i still run a simulation and have no data preprocessing...
so slowly go into hardware IO. ( even i not know what current sensor i will order ? analog?link? )
the last job1() code had a own timer 5sec and send simulation already to MQTT
now i want work on that loop.
-a- lets put the simulation to a next level: ESP32 DAC
make a analog output for simulation can jumper with the analog input for 'amp'
from Arduino IDE it is easy:
dacWrite(25,200); // 0 .. 255
later i changed that to a sin output,
again for filter test added a sin 10th harmonic / with 0.1 amplitude
here i have no idea how to show that filter action as a graph
** would be needed to tune my numeric filter F(x) = 0.9 * F(x-1) + 0.1 * In
(check rule: 0.9 + 0.1 == 1.0 )
done already:
also operation possible via web page:
-b- use the real analog input for 'volt' & 'amp'
i see / installed / a library
ESP32AnalogRead Author: Kevin Harrington arduino code
and want use GP32 A4 / GP33 A5
here we could use a
-c- use calibration linear spline to convert hardware input ( 0 .. 4095 )
function MAP need to make a own float version
to get real Volt and Amp
//___ lets make a good signal range ( from that volt input )
Volt = map_f(volt_raw, 0.0, 3.0, 0.0, 48); // use a 15k / 1k divider give 3V at 48V
Amp = map_f(amp_raw, 0.0, 3.3, 0.0, 100); // use HALL sensor board / with pre amplifier and analog output
on display with jumper ( but no Volt input ( bad floating / like follow next channel ) )
when i fix Volt to 1.0 and jumper A5 to DAC1 ( simulate the Amp with a sinus from DAC ) and look MQTT spreadsheet
( still calculate Watt in display loop AND again in spreadsheet only)
now should think about that special data reduction function:
here have in job1() a timing of 0.5sec analog read...
to send that to MQTT might be not too much, but storing the data to file with that rate is.
( esp. for a RPI )
but sending it with the 5sec ( display refresh timer ) might not make more sense?
so here we do a check for "one decimale change" of current.
( like when the Volt was changed or the ohm / process changed )
as now have no good time must use the millis() / 60000 as minutes runtime additionally to the ON CHANGE of current;
shows the DIAG true prints and the 0.1 dAmp rule skips the MQTT
( where i have a jumper from amp input to GND )
shows that the minute rule works
one more remark to that concept:
while any continuous sampling and storing has the advantage of easy plotting ( equidistant timing = isochronal )
it might cost drive space and so usually you restrict yourself with the sampling time.
this here is a gamble... and requires thinking and tuning of code ( also filter ! )
but it is dynamic.. so in one minute you can have between 1 and 119 samples
depending if the signal is active.
as you get it with timestring you have to plot it volt,amp over time ( scatter graph?)
over samples would be possible misunderstood.
yes sure, there are very good other concepts, like 1 min avg , min , max
but that depends on your requirements, well, built in already ( but show on webserver page only )
but the ESP ADC bad NULL is a real problem
like for amp range 0 .. 100A the 0 ( jumper GND ) is read as 0.14V makes 4.3A
that is too much, need some ? low range suppress ?
now ZeroDrop is enabled and set to 0.15V / below will be set to 0.0 V/
#define AinZeroDrop 0.15
-e- MQTT to Broker on RPI
but already can try this:
* get from /Files/Examples/EspMQTTClient/SimpleMQTTClient/ after install LIB:
* EspMQTTClient by Patrick Lapointe
* see also https://github.com/plapointe6/EspMQTTClient
now try to build this in my project, but as that combines WIFI and MQTT broker login,
might have bigger problems, if i start with this, i might loose the fixIP thing for the ESP...
-a- i could just erase the recent work about wifi / login / webserver and drop this code in to see if it would do 'the job'
yes, it does!
Arduino IDE: code
Mosquitto 2 / Streamsheets: export
little bit tricki,
when the ESP starts it connects to broker ( RPI4 )
the data generation ( now simulated ) is still inside the job1_screen() ( what later will come from hardware )
the job1(); gets a new timer ( but also 5sec ) to use the data and it creates
the MQTT_JSON string from it and sends it to Broker/ Streamsheets
so, first on ESP must select menu1 / job1 ( [right] to select menu1, [left] to activate and see "power meter" screen
only then starts the sending of data to Broker.
p.s. any activation of the Arduino IDE (serial) Monitor resets the ESP ( intro screen / menu )
( yes might be good idea to default job1 )
works better.
so, i see some problems with that tool when the Broker is not running it messes up the timing of screen update
because it tries anyhow to reconnect and send data.
ok, at first the now running MQTT version gets a disable switch
#useMQTT // can disable with //
like for local use only
also need to check the libraries for timing settings ( for wifi or broker fails )
-b- i could dive deep into esp mqtt lib and its using pubsub lib, first by bringing it into the open ( as .h .cpp files ) inside the example project
here first have to pull the 5 files, edit filenames and all content by using a '_OL' behind filename and all procedure name...
add i code in again a little simulation and test it to RPI4
i add the zip here as that was lots of edit work, but only interesting for you,
if you also want dig into / mod / that two libs...
code
see all files?
that was a good start, but now it would mean lots of detailwork / learning about that libraries...
like to find where need that fixIP patch
-1- ok, first i check about the usage of pubsub in/from EspMQTTClient and found
that never i would need a mod of pubsub.. ( so delete that _OL version and call orignal again
-2- found the place where i would need the fixIP change: EspMQTTClient.cpp after 'WiFi.begin(' about line 537
there add a
if ( _webServerIp != IPAddress(0,0,0,0) )
WiFi.config( _webServerIp, _webServerGateway, _webServerSubnet ); //________________________ add fix IP
and in main SimpleMQTTClient(_OpenLib).ino
// fix IP
IPAddress STAip(192,168,1,217);
IPAddress STAgateway(192, 168, 1, 1);
IPAddress STAsubnet(255, 255, 255, 0);
void setup() {
Serial.begin(115200);
//______________________________________________________________ Optionnal functionnalities of EspMQTTClient :
client.enableDebuggingMessages(); //____________________________ Enable debugging messages sent to serial output
//client.enableHTTPWebUpdater(); //_____________________________ Enable the web updater. User and password default to values of MQTTUsername and MQTTPassword. These can be overrited with enableHTTPWebUpdater("user", "password").
client.enableLastWillMessage("ESP32/lastwill","bye bye" ); //___ You can activate the retain flag by setting the third parameter to true
client.enableFixIP(STAip,STAgateway,STAsubnet); //_ KLL lib mod fixIP
}
sounds good already.
-3- now look into my old webserver where i add show the power meter measurement
( when you like to just see the data on your smart TV, instead on the small local screen on the ESP meter )
there is already that ?firmware? update server
you can call: client.enableHTTPWebUpdater();
and a file ESP32HTTPUpdateServer(_OL).h
that i could use as template for my server....
There i failed... with a "core 1 panic'ed" and give up after a day
now i can make a copy of the original library, to *_OL .*
edit some library settings and put the modified files in.
so can call that modified library and add the new procedure call.
+ for fix IP
currently i use the fix IP mod of the lib ( so you find it in the zip. )
and a manual webserver start ( with a tricky timer )
and put in my original web server page with the job1 measurement.
a other small change, now use http://192.168.1.217:8888
and show it to the world ( no, some student.. )
how is that possible? a 6$ ESP32 ( ok mine with LCD was 20$ / but that is not the point here )
as a worldwide web server?
testing a tunnel
-f- RPI: mosquito / spreadsheet / optional database storing
see below about the work on RPI4
status: streamsheets work, database storing NOT
i will explore the old-fashioned way without the streamsheet thing.
manually install mosquitto on ( new setup ) RPI4 and use python tp cache stream and save to RAM DISK file.
hardware
1 TTGO T-display ESP32______ 10$
1 plastic box_________________ 3$
4 connectors ( 2 red 2 black )____ 6$ ( power type or speaker type ( high amp ) )
2 resistors for voltage divider____ 1$ ( measuring type required ? 48VDC to 3V3 ?)
1 hall sensor ? max 30A________ 3$
( picture from Sandeep-Nagar )
( ACS712 is 5V, 3V3 types ACS723? are rare )
if i use USB power unstable 5V for hall sensor
and a 3V3 ESP board for ADC
i could reverse the current and use / see
2.5V as 0A and 0.5V as 30A...( what normally would be -30A )
so i not need to read over 3V3, what is out of range for the ESP board.
check resolution:
the 30A sensor has Sensitivity: 66mV/A
the ESP has a range of 3V3
and a ADC 12bit resolution aka 4096 steps, so see 0.0008V resolution
or here as 82 steps / A or 12mA steps ?
good enough for a ONE DECIMAL current indication please check that!
here a 50A example:
for higher amp hall not only getting expensive, also there is a add problem like they require +-15VDC supply.
if using a shunt resistor need ?3V3 instrument amplifier,
i see a ready breakout board for Volt Current / amplify, sample
+ + and link I2C
INA219 High Side DC Current Sensor Breakout 26V 3.2A Max / I2C
but could experiment with own shunt resistors ( i need to look into "Manganin Copper Shunt" )
while i checked more the internet about breakout boards
( for the 2 involved jobs / current measurement / instrument operational amplifier / )
here 2 interesting:
a better HALL sensor WCS1700 (70ADC) is a wire through design
here with breakout with op amp and analog output,
p.s for small current ( wire ) can loop/coil the wire
ADS1115 a I2C bus ADC , 4 channel / 2 differential) , 16 bit
has a internal programmable amplifier so range select from 0.2 to 6 V ( datasheet say +- for each ??? )
might be good with the above WCS1700 sensor or shunt ( low side )
so i order : ?form china need 2 weeks?
* ACS712 HALL 5A ( 5V )
* ACS724 HALL 50A ( 5V )
* ACS758 HALL 50A ( 3 .. 5.5V )
* INA219 I2C 3.2A shunt high-side 26V amplifier ( 3 .. 5V )
* ADS1115 I2C 4 channel 16bit ADC ( 2 .. 5V )
today here already:
here i have several challenges:
-a- the ESP board and several sensor boards need header soldered.
-b- the analog ones what come for 5V use must be tested if operational 3v3
and esp. about HALL where is the ZERO point 2v5 or VCC/2 ( 1.65 for 3v3 )
OR could these be powered ( questionable ) 5V and current reversed ( 2.5 .. 0.5 V measuring range aka 0 .. -5(0)A ) to ESP?
-c- I2C linked sensors to ESP32 is most interesting.. lets start with INA219 on ESP
check out: https://diyi0t.com/ina219-tutorial-for-arduino-and-esp/
it states default address 0x40 , wiring for many boards, incl ESP32
default adafruit lib use example,
also circuits show that the board has 10k pull up's already.
-c++- here we have to learn if we can add a additional shunt to increase the range..
idea is to leave the original in ( ?0.1ohm, 3.2A max >> 0.32 dV )
and add || 0.003ohm shunt with still < 3A through the original onboard shunt
and 97A through the add external shunt
( sorry i have one for only 20A ( called 20A 60mV ( they not say the ohm?? should be 0.003ohm ) ) )
the small error from the 0.1 || 0.003 we do with the calibration.
-d- but i am stuck on many points:
also i have no power supply:
reuse a old PS of a PC 3V3 , 5V, 12V ?each? 20A
and no LOAD to test..
play with 5 resistors 1 Ohm 100W
after soldering headers try it on a breadboard
with:
INA219 _______ ESP32
VCC _________ 3V3
GND _________ GND
SCL _________ GP22
SDA _________ GP21
Vin- _ nc
Vin+ _ nc
in 2 examples online i see a confusion about the actual power loop
regarding wiring and indication ( measuring via INA219 board )
but lets start dry:
install the library
#include < Wire.h >
#include < Adafruit_INA219.h >
Adafruit_INA219 ina219;
ina219.begin();
and see the example running ( sorry, no power circuit here )
the default I2C address the device can be found on is 0x40 ( 2 soldering jumper pads on board 0x41, 0x44, 0x45, so max 4 boards )
but is default / empty in library.
what is even more confusing is that it runs without any Pin assignment, lets see in:
c:\Users\< user >\Arduino\libraries\Adafruit_INA219\Adafruit_INA219.cpp
use
#include < Wire.h >
means?
c:\Users\< user >\AppData\Local\Arduino15\packages\esp32\hardware\esp32\2.0.0\libraries\Wire\src\Wire.cpp
there find:
bool TwoWire::begin(int sdaPin, int sclPin, uint32_t frequency)
and in:
c:\Users\< user >\AppData\Local\Arduino15\packages\esp32\hardware\esp32\2.0.0\variants\ttgo-lora32-v1\pins_arduino.h
i found:
static const uint8_t SDA = 21;
static const uint8_t SCL = 22;
YES, there i connected the INA219 board. i am so lucky...
i did a little test, jumper
USB 5V OR board 3v3 OR GND to Vin
and see that as 'bus' voltage correctly, while on open connectors see 0.89V there
now back to the PowerMeter code..
while i want build in the tested modified INA219 example code
i also need to rearrange a few things..
like put sensor IO each in a extra TAB, that includes the now unused Analog In gets also a TAB
here all the analog things incl filter and minute values are disabled,
but i build in a detail info into the webserver
also i checked, boot RPI4 Mosquitto 2 / stream sheets /
reboot [reset] TTGO T-Display board
see connection to broker,
see data in streamsheets APP data ( follow the delta_I > 0.1A OR new minute value rule)
ok, that was a other big step.
now need take care of DC power source and load
sure i will start with testing the INA219 (I2C) first, but any test costs setuptime...
why not run same current serial through 2 sensors to check faster.
so i could include the ACS712 Hall 5A model (analog) in range 0 .. 3A
( powered 5V or 3V3 / check function and zero point (also 4.5 ..5.2V USB power problem ) / reverse current ++ 2.5 ..0.5 signal idea )
it must have same timing so called from INA219 code, but analog code in a extra TAB 'test_parallel'
i use GP27 A17 pin
and just make a String print to USB
Serial.println( test_analog() ); //________________________ temp test from TAB test_parallel
shows:
INA219 Current: -0.30 [mA]
____ test_parallel: Board Voltage : 4.43 [V] , GP27_A17 : 0.13 [V]
yes, from my "good" desktop PC the USB voltage is always low???
even drops down to 4.1 regularly at ?ESP WIFI sending?
but running on a normal phone charger ( add with USB doctor ) i can't see that.
max 40mA, no voltage dips?
#KLL in_work
after that i might look for the WCS1700 board ( add using ADS1115 for I2C )
WCS1700 70A
WCS1800 35A
for a Power Supply Watt need Volt and Amp
the Volt we assume in the safe range of 0 .. 24 ( or max 48 ) VDC
the voltage can bring down to measuring range ( 0 .. 3V3 ) just with 2 resistors as voltage divisor like:
R1 15k
R2 _1k
would bring a voltage range of 48VDC down to 3.0V
( with 3mA and 0.14W loss )
( for the Amp over 30A better buy professional equipment )
using a HALL sensor is isolated ( so does not matter where build into the loop )
BUT
a shunt in PLUS would be high side means you want measure a voltage difference example:
of 24 v.s. 23.85 V ( delta 0.15V ) ( with 30A over 0.005ohm shunt)
if you build it into the MINUS your ADC & added pre op amp only need to see 0 .. 0.15V, more easy.
RPI MQTT broker
on a Raspberry Pi 4
test Eclipse Mosquitto 2.0 ( and added test a speadsheet tool Cedalo for streaming data )
Installation:
update:
Raspbian GNU/Linux 10
DEBIAN buster
Linux version 5.10.52-v7l
Raspberry Pi 4 Model B Rev 1.2
now from install
curl -fsSL raw.githubusercontent.com/cedalo/streamsheets-installer/master/pi-installer.sh -o get-streamsheets.sh && sh get-streamsheets.sh
[enter]
[Y]
[Y]
ends in ERROR
( also reboot and manually use in /home/pi/cedalo_platform/ sudo sh start.sh )
( install Raspbian OS on SD card again. )
retry on a RPI3
what to do? i not even know how to uninstall it so i need to burn and setup the SD again.
but i put it in the forum, lets see how is the response...
ok, actually try 5 times..
now seem to have a running version:
+ setup from 2021-05-07-raspios-buster-armhf.zip and copy ssh wifi files
+ boot enable vnc and fixIP
BUT NO UPDATE
install MQTT
curl -fsSL raw.githubusercontent.com/cedalo/streamsheets-installer/master/pi-installer.sh -o get-streamsheets.sh && sh get-streamsheets.sh
full update
Linux version 5.10.60-v7l
now instead of a manual start try a autostart at boot
cd .config
mkdir autostart
cd autostart
nano mqttserver.desktop
cd /home/pi/
nano MQTTstart
chmod +x MQTTstart
reboot and see open terminal window with server running,
do not close that window
type [ctrl][c] for shutdown
and [ctrl][c] again for close
try remote login ( LAN PC ) to management and spreadsheet:
default logins
not understanding what i am doing..
but read that a connection from 'outside' this server is not possible until adjust
for example 'Unauthenticated access'
cd /home/pi/cedalo_platform/mosquitto/config/
sudo nano mosquitto.conf
here i add a line 2 with
allow_anonymous true
ok but a reboot shows same start... PLEASE NOT TRY, IN HOLD
more about authentication here
RPI3 MQTT test client
but can not test anything until i have a other device sending data...
so on other RPI (RPI3) try a test via PYTHON
pip3 install paho-mqtt
nano mqtt_client.py ( or better use THONNY )
mqtt_client.py
i see in log:
mosquitto_1 | 1630638927: New connection from 192.168.1.203:52901 on port 1883.
mosquitto_1 | 1630638927: New client connected from 192.168.1.203:52901 as RPI3 (p2, c1, k60).
also in
http://rpi4:8088/system/status
i see "clients" goes from 4 to 5
but i have no idea where i can see the data / or what to do now?
in the old way i could use the terminal to subscribe to the stream
( the management center has a terminal / i can not even use the help ??)
but i did follow the "getting started"
* with streams in out &
* 2 spreadsheets talk to each other MQTT
* and topic 'dogsandcats' WORKS
but when i use CHROME
new APP MQTTlens
i could not connect to broker / that topic /
so might be all about security settings
follow: https://docs.cedalo.com/mosquitto/2.0/broker-dynamic-security
run cats and dogs tutorial: http://rpi4:8081/dashboard
add: http://rpi4:8088/security/clients
make a group and a client for MQTTlens and configure that in MQTTlens app
and subscribe to 'cedalo/catsanddogs'
p.s. that config ends up in
/home/pi/cedalo_platform/mosquitto/data/dynamic-security.json
( backup b4 new install? )
following the docu/tutorial 'catsanddogs' sheet app,
* one sheet generates a MQTT stream with random numbers in a JSON '{ "cats":4,"dogs":23}'
* other sheet catch that data and show it ( here i added already a time graph )
now from PC Chrome browser app MQTTlens
OK, we are getting somewhere.
again i did find a cryptic info:
You may wish to allow anonymous access, but still make use of the dynamic security plugin, and this is supported through the automatic anonymous group. If allowed, anything connecting without a username will be assigned to a group that you define. By assigning roles to that group, you can control what anonymous devices can access.
still i have no idea how to use that?
now try to use that for the RPI3 python publish test:
make a new user PowerMeter
mod the python code
pls see:
client.username_pw_set(username="PowerMeter",password="< password >")
start and find connect rc '0'
and check if data come?
good, but now i am stuck in the sheet app stream config ( no connection )
when i try my own connector / consumer.
so i try under existing connector 'MQTT_connector'
a new consumer 'PowerMeter'
but i not get data into the sheet... well i needed to send ( by RPI3: python test ) to topic 'cedalo/power_meter_1/amp'
good,see it "data" 3.2,
but not the volt...
so next change is to change to send volt amp as a JSON thing
like the catsanddogs:
{
"Cats": 45,
"Dogs": 8
}
new code
RPI4 MQTT store to own mongoDB
i try to follow this video https://www.youtube.com/watch?v=xtxOWX2YCQg
and this manual https://docs.cedalo.com/streamsheets/2.4/functions/streams/mongostore
but the MONGO.STORE command give ERROR because the producer is disabled,
enable not much use..
not clear if can connect to the internal mongo db at
/home/pi/cedalo_platform/mosquitto/data/mosquitto.db
( actually i doubt that on raspberry pi that is a mongo db ) wrong wrong
update: this is a internal db, not mongo, but
/home/pi/cedalo_platform/streamsheets/data/mongodb/streamsheets.0
is a mongo db
and behind runs a experimental 32bit mongo special for raspberry pi OS
if / how i can make a new db ( but using the internal mongo )
or i have to even install a external mongo??
well, there is a FILE WRITE command to CSV, so i would be sure i can use the data later everywhere.
i asked forum and they answered very fast: check
https://docs.cedalo.com/streamsheets/2.4/addons#mongodb
sudo docker run --name mymongodb --net cedalo-platform andresvidal/rpi3-mongodb3
-a- i have tried but failed about the path options ( would be nice to know where the files are.
-b- usually there is a password login to a db required, that whole part i missed too.
sudo docker start mymongodb i have to do every boot
sudo docker ps tells you if running
looks good
still error, when i recheck the connector producer show red ( not connected )
need learn more
docker https://docs.docker.com/engine/reference/commandline/container/
mongo https://github.com/andresvidal/rpi3-mongodb3
RPI4 MQTT or store by python to CSV file
now as i have no big idea how to progress with that tool,
so i actually want go back to a own python tool to cache and save the records...
so on RPI4 ( where mosquitto 2 & streamsheets is running ) but could be also any other LAN computer
begin with python to subscribe / read JSON / convert to CSV "12.0,30.4" for volt amp
save to file in RAM DISK '/run/shm/' and every hour move that to a own directory ( on SDcard )
that saves thousands of writes to SDcard and keeps the file in RAM small ( for fast open/write/close ).
while it sounds like i give up on the streamsheet thing, no
but when i use a RPI and install not the streamsheet, only the mosquitto broker,
( ? must make the client user password there CLI? )
then i need something to send the data to anyhow...
that will be the next test ( of same python code.. )
but i will not go full way and make it a service now, as i still have hope on the mongo db.
anyhow python code
here i run it in THONNY, later should ( chmod +x mqtt_sub_power_meter.py ) run it from python3 in terminal
the hourly csv file is about 16kB only
ESP powered, inputs floating, but inbetween i jumper AIN5 to simulation pin DAC1 so the data not too boring.
here i try with RPI libre office calc a bad graph over the samples
RPI4 mosquitto 1.5.7 Python client and save CSV
now i do a test without the new mosquitto 2 / docker container streamsheet / thing
setup a new system on RPI4
setup from 2021-05-07-raspios-buster-armhf.zip and copy ssh wifi files
+ 'RPI4'
+ boot enable vnc and fixIP 104/204
NO UPDATE
Linux version 5.10.17-v7l
Raspberry Pi 4 Model B Rev 1.2
12.9.21
-1- copy files from RPI4 mosquitto-streamsheets to 'share'
-2-
pip3 install paho-mqtt
-3- install normal mosquitto
follow https://pimylifeup.com/raspberry-pi-mosquitto-mqtt-server/
sudo apt update
sudo apt upgrade ( not dist-upgrade )
reboot
Linux version 5.10.60-v7l
Raspberry Pi 4 Model B Rev 1.2
try dist-upgrade / 0
sudo apt install mosquitto mosquitto-clients
//______________________
mosquitto -v
mosquitto version 1.5.7 starting
sudo systemctl status mosquitto
terminal1
mosquitto_sub -h localhost -t "mqtt/pimylifeup"
terminal2
mosquitto_pub -h localhost -t "mqtt/pimylifeup" -m "Hello world"
see in terminal1
Hello world
//______________________
cat /etc/mosquitto/mosquitto.conf
ls /usr/share/doc/mosquitto/examples/
cd /etc/mosquitto/
sudo nano pw.txt
PowerMeter:voltandamp
sudo mosquitto_passwd -U pw.txt
cat pw.txt
PowerMeter:$6$TfBq8ffT3f1jlqff$nJbW3GdXEX3rY9ob5zhoSkgnbcbPgah7AH4NC7TT81XHzD4XfUJWhpeq1y8uOh8afoyIvc8WkKd7pPRaJkd8dA==
sudo nano mosquitto.conf
add at end 2 new lines:
allow_anonymous false
password_file /etc/mosquitto/pw.txt
reboot
sorry, i use
Desktop/share
for coding python here,
it is a dir shared by samba in/to my LAN
very easy exchange as i work headless from win10 PC
( but sure, SFTP works great too )
Terminal1
cd Desktop/share
python3 mqtt_client.py
Publish cedalo/power_meter_1 { "volt":12.0, "amp":3.1 }
Connected with result code 0
Publish cedalo/power_meter_1 { "volt":12.0, "amp":3.2 }
Publish cedalo/power_meter_1 { "volt":12.0, "amp":3.3 }
Terminal2
cd Desktop/share
python3 mqtt_sub_power_meter.py
/run/shm/ESP_PowerMeter.csv
rc: 0
Subscribed: 1 (0,)
12/09/2021 23:23:35,12.0,3.7,
12/09/2021 23:23:40,12.0,3.8,
12/09/2021 23:23:45,12.0,3.9,
12/09/2021 23:23:50,12.0,4.0,
Terminal3
ls /run/shm/
ESP_PowerMeter.csv
update IDE / ESP boards
13.9.2021 read that now arduino also can do ESP32 models like C3...
so i try a big UPDATE
+ + Arduino IDE 1.8.16
+ + Arduino IDE 2.0.0 beta 11
with /files/Preferences/
add with ','
https://github.com/espressif/arduino-esp32/releases/download/2.0.0/package_esp32_index.json
now can ON THE ALREADY INSTALLED BOARD ESP32
update to v2.0.0 ( of ESP32 tools )
info
first verify give a minor error about a redefine line i needed to disable ( so i am sure some things are new... )
check the available boards:
my code
for TTGO T-display ESP32
with / LOGO / MENU ( here landscape ) 5 entries / JOB1 DC Power Meter /
with WEB SERVER ( on fix IP )
with MQTT //
to Raspberry Pi / Mosquitto 2 / streamsheets / save to DB not work
or to Raspberry Pi / Mosquitto 1.5.7 / python service to CSV hourly file /
current code
now in HOLD until i have measuring hardware tested.
