Decoding 'len' field in LUA in AM new_message function HW_Messageport ?

Support for Arduino in combination with Air Manager and Air Player

Moderators: russ, Ralph

Message
Author
User avatar
jph
Posts: 2846
Joined: Fri Apr 10, 2020 12:50 pm
Location: Somewhere over the rainbow..

Decoding 'len' field in LUA in AM new_message function HW_Messageport ?

#1 Post by jph »

Hi all,
Maybe someone can advise here ? - perhaps @Corjan please ?

When sending multiple values from HW to AM in a single message the 'len' field (a string) needs ' parsing / decoding' to be of any use. This seems very strange. Am I missing something (probably :D ) or is it a case of examining the string return value and stripping out the appropriate section. ?

For example - using (in LUA in AM)

Code: Select all

function new_message(id, payload,len)  
  print("received new message with id " .. id)
  print("len ".. len)
then 'len' returned by the function 'new_message' is a string - as in "BYTE[4]" which needs to be parsed - I have no idea why it is done like this ? this is, as far as I can see, not mentioned anywhere at all in the documentation - in fact, the use of the third value after id and payload is not mentioned anywhere at all. I found it by trial as there has to be some way of telling AM the type and quantity of the values sent from the MessagePort device. I am not really understanding why there are not 2 values, one an enum for type (Float, INT, Byte) and another purely for 'len' ?

If I send, for example,
4 uint8_t values then len is "BYTE[4]" (string)
4 int32_t values then len is "INT[4]" (string)
4 floats values then len is "FLOAT[4]" (string)

To decode the intended sent 'type' and 'quantity' - (to use math with the quantity for addressing the individual elements) from the string I can of course use string examination and pick out the type and quantity and then use math on the quantity but that seems rather cumbersome. Is that the preferred method ? - or am I missing something really obvious :o


Many thanks,
Joe

edit - typo corrected.
Last edited by jph on Sun Jul 10, 2022 9:36 am, edited 1 time in total.
Joe. CISSP, MSc.

JackZ
Posts: 2262
Joined: Mon Feb 22, 2016 1:02 pm

Re: Decoding 'len' field in LUA in AM new_message function HW_Messageport ?

#2 Post by JackZ »

Hi

I think that using the Lua string.match() function, with the proper pattern, it could be as simple as that:

Code: Select all

  
function new_message(id, payload,len)  
  print("received new message with id " .. id)
  print("len ".. len)
  print("length of payload "..tonumber(string.match(len, '%d+')))
end
Not tested on my side, as I still have my PC out for repairs, but this should extract whatever digits are in the string. You’ll then have to convert the string result into an integer using tonumber()

Jacques
My YouTube Chanel on the A320 (Real SOPs by an Airline Pilot IRL):
https://www.youtube.com/playlist?list=P ... 0Q6SBASRqJ

User avatar
Ralph
Posts: 7878
Joined: Tue Oct 27, 2015 7:02 pm
Location: De Steeg
Contact:

Re: Decoding 'len' field in LUA in AM new_message function HW_Messageport ?

#3 Post by Ralph »

Corjan is on holiday until the 17th, so it can take a while for him to answer.

User avatar
jph
Posts: 2846
Joined: Fri Apr 10, 2020 12:50 pm
Location: Somewhere over the rainbow..

Re: Decoding 'len' field in LUA in AM new_message function HW_Messageport ?

#4 Post by jph »

Hello guys and many thanks for the replies.
@JackZ Hi Jacques, yes, that does indeed offer a workaround. Nice one. A very useful function. I had tried a simple string.sub but the problem was that the length of the string is not constant. Your solution is perfect. Very much appreciated. Thank you.
I am now using the following as a basis for extracting the information and it certainly seems to work.

Code: Select all

-- This function will be called when a message is received from the Arduino.
function new_message(id, payload,len)  -- NOTE ! (Joe) 'len' as I have called it - is undocumented and I see no other way of getting the data ?? 
  print("received new message with id " .. id)
  print("len ".. len)
  local x,y                                                       -- temp storage for test
  
  --x = tonumber(string.sub(len,5,5)) -- wont work due to variable length of string - do not use.

  x = tonumber((string.match(len, '%d+')))                        -- ref: Jacques %d is digit, %d+ is digit(S)
  y = string.sub(len,1,1)                                         -- Get first character of 'len' to determine type


  print("Type = (B = BYTE, I = INT, F = FLOAT     )  "..y)        -- test of intended type extraction from HWMP 
  print("Number of elements (len) = "..x)                         -- test of 'len' or number/quantity of the 'type' intended to be sent from HWMP  
I must confess I am still totally baffled as to what @Corjan intended / is intending for this. As said, at no place can I find ANY reference to anything that AM does in response to a new message from a HWMP device apart from function new_message(id, payload)
I added the third argument and called it 'len' just for the sake of it and was surprised to find the string returned in such a format.
There is plenty of documentation for sending multiple values from the HWMP device so there HAS to be appropriate data for the multiple values and intended types from the AM new_message function - but absolutely nothing about the reception is documented (that I can find). In the past I have just used single values or just an ID but wanted to use multiple values as the decoding of the data in code is far faster than sending the data over a slooooooooooooooow serial link.
Very interesting. I am sure I am missing something but I simply cannot find any reference at all in this area. At least with your help I can actually get the data. I need to think of another name instead of 'len' as it is not just 'len'

@Ralph Understood and no problem. He deserves the break :D . I am very much looking forward to hearing his thoughts on this when he returns to work.

Thanks again guys.

Oh, and have you guys seen the new ESP32-S3 ! ? :o a little stunner. FAR more I/O than the ESP32 and on chip flash. Very very nice.
Joe
Joe. CISSP, MSc.

User avatar
Ralph
Posts: 7878
Joined: Tue Oct 27, 2015 7:02 pm
Location: De Steeg
Contact:

Re: Decoding 'len' field in LUA in AM new_message function HW_Messageport ?

#5 Post by Ralph »

Yes! I was just looking at this one: https://eu.mouser.com/ProductDetail/Esp ... ekcA%3D%3D

I might order one to see how it performs.

User avatar
jph
Posts: 2846
Joined: Fri Apr 10, 2020 12:50 pm
Location: Somewhere over the rainbow..

Re: Decoding 'len' field in LUA in AM new_message function HW_Messageport ?

#6 Post by jph »

Ralph wrote: Sun Jul 10, 2022 12:23 pm Yes! I was just looking at this one: https://eu.mouser.com/ProductDetail/Esp ... ekcA%3D%3D

I might order one to see how it performs.
Looking at the specs yesterday I had more or less decided to go for https://eu.mouser.com/ProductDetail/356 ... S3DVKTC1N8 as opposed to the units with external flash / quad or octal /psram as it frees up 5 GPIO pins (EDIT _ it is only the units with >2MB PSRAM - Any OCTAL units that use the extra gpios. It would appear the best compromise is probably the N8R2 unit with 8MB flash (QSPI) and 2MB PSRAM) - https://eu.mouser.com/ProductDetail/Esp ... XFsw%3D%3D . However, to get free postage at Mouser I need to spend 50 euros :? so I will have to get more than 1 to experiment :lol: . The IDE supports octal psram ok, BUT, it is worth noting the IDE (Arduino that is) doesn't support more than 16MB flash on the current esp32 board lib so this may be better. (the one you selected has 32MB flash and is octal so 5 gpio pins less and - not supported 'yet' in Arduino IDE but I am sure it will be at some point)

I am still checking the actual support in the IDE and the terminology just to confirm this but it definitely seems that ANY Octal flash unit uses 5 gpio pins extra. That is a lot. The 8MB flash with 2MB PSRAM N8R2 looks a really good unit. I would rather use the pins for an SPI ethernet port to be honest but all depends on use scenario.

What IDE do you use Ralph ?
Joe. CISSP, MSc.

User avatar
Ralph
Posts: 7878
Joined: Tue Oct 27, 2015 7:02 pm
Location: De Steeg
Contact:

Re: Decoding 'len' field in LUA in AM new_message function HW_Messageport ?

#7 Post by Ralph »

Corjan does the ESP32 software, I'm not sure what he uses.
It looks like the S3 does not have Ethernet, only WiFi or Ethernet through SPI, which I do not prefer. Maybe the S2 would be better?

User avatar
jph
Posts: 2846
Joined: Fri Apr 10, 2020 12:50 pm
Location: Somewhere over the rainbow..

Re: Decoding 'len' field in LUA in AM new_message function HW_Messageport ?

#8 Post by jph »

The S2 doesn't either Ralph, neither did the original. You can use an SPI unit or a direct connect unit as per the Lan8710 or 20 such as
https://www.aliexpress.com/item/3292009 ... f8f2397857
Thats the same unit as used on the olimex boards.
I dont mind the SPI solutions as they use at least 8 less gpio pins. The olimex boards, for example, use 12 gpios. Yikes. If you look at any of the olimex boards that air player mini is supposed to run on then there are virtually none of the gpio pins listed on the wiki available as they are all used by the 87 unit and 4 of the others are input only.
The beauty of a modular add on board is that you are not limited to just an esp32 s3 etc, you can use a pico, which btw runs happily at 240 mhz overclock.
Also the teensy 4.1 and any of the upcoming 32 bit boards.
Using SPI you can run some of the ETH units at 80 mhz on an ESP giving you a comfortable ethernet throughput approaching full duplex 10 Mbit ethernet. I think its not good to design for a specific board these days, if possible, when there are so many to chose from. All the suppported 32 bit boards in the Arduino IDE can run the same ethernet libs and stack. We are getting to the point now where processing speed isnt an issue and the ram on the S3 ..the 2 MB P stuff with 8 MB flash is just crazy. I just bought 10 pi picos for less than 45 euros delivered. Things like the uno and mega are simply obsolete imho. Even the leonardos, micros and due etc etc, they are not worth buying. The prices are going up all the time to silly money as virtually no fab plant will waste time on the silicon. Some may never start again unless some crazy person is willing to pay ridiculous money for them. The chip crisis has meant that the large wafer fabs are simply not producing the bottom end now and never will again in a lot of cases . There is more money and profit in modern 32 bit small size wafers. I very much doubt the prices will ever come down again... but its not an issue as far far better options are about.. the standard arduino stuff is history basically and will never recover. They will just be unobtanium... imho of course. ;) . The speed of the tft lcd screens and touch screens, the cheap ones, even the size of a mega, those running spi at 80mhz on an esp, or even better DMA parallel on a pico at 240 ,MHz.. amazing. Some real quality gear around and more appearing as we speak. A Genuine PI Pico is half the price of a cheap Chinese nano. Wtf? A genuine expressif ESP32 s3 is less than half the price of a mega or DUE, mind you, 2 picos can replace a mega if its pin count etc. For 8 or 9 euros!. Crazy. The chip shortage is forcing the hand of change and the old 8 and 16 bit stuff is toast. It is history. Think of the possibilities for AM? A whole new world. :D
I just ordered 2 of the N8R2 units a few hours ago so they should be here in a day or so. Forgive any typos I am using the tablet and its a begger to type lol.

Oh, and you should get the 1637 6 digit in a day or so .. hopefully. .unless you moved to the Highlands :lol:
Joe
Joe. CISSP, MSc.

User avatar
Ralph
Posts: 7878
Joined: Tue Oct 27, 2015 7:02 pm
Location: De Steeg
Contact:

Re: Decoding 'len' field in LUA in AM new_message function HW_Messageport ?

#9 Post by Ralph »

The WT32-ETH01 that we use at the moment has a LAN8720A. It uses RMII as interface, I have no clue what that is exactly :) Preferably I would use the same for the S3. Do you know if that is possible?

User avatar
jph
Posts: 2846
Joined: Fri Apr 10, 2020 12:50 pm
Location: Somewhere over the rainbow..

Re: Decoding 'len' field in LUA in AM new_message function HW_Messageport ?

#10 Post by jph »

Hi Ralph, absolutely yes.

already supported in Arduino IDE with the ESP32/S3 with the 'eth.h' lib. Supports the ETH LAN8720 and also the ETH_TLK110 and many others in fact.

Have you actually tried the SPI ETH versions ? - they are pretty good. Not as fast as RMII but still very respectable.

Joe
Joe. CISSP, MSc.

Post Reply