Integer number from Lua calculation needed

Help creating logic scripts for Air Manager Instruments

Moderators: russ, Ralph

Message
Author
User avatar
Keith Baxter
Posts: 4674
Joined: Wed Dec 20, 2017 11:00 am
Location: Botswana

Re: Integer number from Lua calculation needed

#21 Post by Keith Baxter »

Hi,
@Corjan

Running this simple code produces the same error as the OP.

bad argument #2 to 'format' (number has no integer representation)"

Code: Select all

local input = 0.5
local output = string.format("%x", input * 255) -- "7F"
print(output)
However running this code does not produce an error

Code: Select all

local input = 0.4
local output = string.format("%x", input * 255) -- "66"
print(output)
Looks like there is a format "style" bug

On another topic. Right clicking on a console report "copy to clip board " produces this in AM4.2 beta 6

DevelopmentObjectLogMessage: 0x4B62D5A7

It should produce what is in green above.

Keith
AMD RYZEN 9 5950X CPU, Corsair H80I cooler, ASUS TUF GAMING B550-PLUS AMD Ryzen Mother Board,  32Gb ram Corsair Vengeance 3000Mh, MSI GTX960 4G graphics card 

User avatar
jph
Posts: 2846
Joined: Fri Apr 10, 2020 12:50 pm
Location: Somewhere over the rainbow..

Re: Integer number from Lua calculation needed

#22 Post by jph »

Keith Baxter wrote: Tue Nov 22, 2022 2:47 pm Hi,
@Corjan

Running this simple code produces the same error as the OP.

bad argument #2 to 'format' (number has no integer representation)"

Code: Select all

local input = 0.5
local output = string.format("%x", input * 255) -- "7F"
print(output)
However running this code does not produce an error

Code: Select all

local input = 0.4
local output = string.format("%x", input * 255) -- "66"
print(output)


Keith
0.5 * 255 is NOT an integer result whereas 0.4 x 255 IS an integer result......... ergo...... sem problemas;)
Joe. CISSP, MSc.

User avatar
Keith Baxter
Posts: 4674
Joined: Wed Dec 20, 2017 11:00 am
Location: Botswana

Re: Integer number from Lua calculation needed

#23 Post by Keith Baxter »

Hi,

As I already requested @Corjan . A AM <var_hex_convert()> helper would be a great help.

I do not know of a lua decimal to hex code rounder.
interesting conversation..

For what it is worth. I created a table for hex values that I use to control color and opacity. You can view this in my color picker instrument code somewhere in the forum abyss.
So instead of doing complicated math and coding the table is loaded and available at call.

Code: Select all

--Create a Hex Table to provide (33x33x33) color palate 
ht={"ff","f8","f0","e8","e0","d8","d0","c8","c0","b8","b0","a8","a0","98","90","88","80","78","70","68","60","58","50","48","40","38","30","28","20","18","10","08","00"}
Yes you could do the same for 0 ->255 hex codes.
I will produce a table that peeps can use. However with more simmers wanting to use color manipulation and shading. This might be something to look towards.

Keith
AMD RYZEN 9 5950X CPU, Corsair H80I cooler, ASUS TUF GAMING B550-PLUS AMD Ryzen Mother Board,  32Gb ram Corsair Vengeance 3000Mh, MSI GTX960 4G graphics card 

Detlef
Posts: 304
Joined: Mon Nov 16, 2020 9:43 am
Location: Bavaria

Re: Integer number from Lua calculation needed

#24 Post by Detlef »

Keith Baxter wrote: Tue Nov 22, 2022 2:47 pm Running this simple code produces the same error as the OP.

bad argument #2 to 'format' (number has no integer representation)"

Code: Select all

local input = 0.5
local output = string.format("%x", input * 255) -- "7F"
print(output)
However running this code does not produce an error

Code: Select all

local input = 0.4
local output = string.format("%x", input * 255) -- "66"
print(output)
Looks like there is a format "style" bug

On another topic. Right clicking on a console report "copy to clip board " produces this in AM4.2 beta 6

DevelopmentObjectLogMessage: 0x4B62D5A7

It should produce what is in green above.

Keith
@Keith,
thank you for that! At least its not me here. The %x formatter should work but does not.


I will try again with this. That has helped in all my other instruments:

Code: Select all

function toint(n)
  -- needed because math.floor() returns floats
    local s = tostring(n)
    local i, j = s:find('%.')
    if i then
        return tonumber(s:sub(1, i-1))
    else
        return n
    end
end

    gColorPrint = string.format("#%02X%02X%02XFF",
      toint(math.floor(255*brightness_lines)),
      toint(math.floor(255*brightness_lines)),
      toint(math.floor(255*brightness_lines)))

If that also turns out to be a problem, I will find another solution :)

Thank you all,
Detlef

User avatar
Corjan
Posts: 2936
Joined: Thu Nov 19, 2015 9:04 am

Re: Integer number from Lua calculation needed

#25 Post by Corjan »

Hi,

Might have been opted before, but did someone test %a? That is the float -> hex thingy.

Corjan

User avatar
Keith Baxter
Posts: 4674
Joined: Wed Dec 20, 2017 11:00 am
Location: Botswana

Re: Integer number from Lua calculation needed

#26 Post by Keith Baxter »

Corjan wrote: Tue Nov 22, 2022 8:05 pm Hi,

Might have been opted before, but did someone test %a? That is the float -> hex thingy.

Corjan
Corjan,

Yup %a produces no errors.

Code: Select all

local input = 0.5
local output = string.format("%a", input * 255) -- "7F"
print(output)
returns a value of
0x1.fep+6

Now to round/convert to a 2 digit hex?

Is there another trick to do that?

We should be looking for a hex value of 7F for a 10th input of 5 (0.5)

string.format(%x) is hexadecimal
string.format(%a) is hexadecimal with binary exponent

I am not clever enough to convert this to a two digit color hex.
Need some help here please.

Keith
Last edited by Keith Baxter on Tue Nov 22, 2022 9:11 pm, edited 1 time in total.
AMD RYZEN 9 5950X CPU, Corsair H80I cooler, ASUS TUF GAMING B550-PLUS AMD Ryzen Mother Board,  32Gb ram Corsair Vengeance 3000Mh, MSI GTX960 4G graphics card 

Tetrachromat
Posts: 236
Joined: Sun Feb 14, 2021 6:55 pm

Re: Integer number from Lua calculation needed

#27 Post by Tetrachromat »

Anybody ever considered to use integer division? It always returns an integer.

Code: Select all

gColorPrint = string.format("#%02X%02X%02XFF", 255*brightness_lines // 1, 255*brightness_lines // 1, 255*brightness_lines // 1 )
Paul

User avatar
Keith Baxter
Posts: 4674
Joined: Wed Dec 20, 2017 11:00 am
Location: Botswana

Re: Integer number from Lua calculation needed

#28 Post by Keith Baxter »

Tetrachromat wrote: Tue Nov 22, 2022 9:10 pm Anybody ever considered to use integer division? It always returns an integer.

Code: Select all

gColorPrint = string.format("#%02X%02X%02XFF", 255*brightness_lines // 1, 255*brightness_lines // 1, 255*brightness_lines // 1 )
Paul
Hi Paul

NO.
But thank you for the pointer. Learned something today.

Keith
AMD RYZEN 9 5950X CPU, Corsair H80I cooler, ASUS TUF GAMING B550-PLUS AMD Ryzen Mother Board,  32Gb ram Corsair Vengeance 3000Mh, MSI GTX960 4G graphics card 

Tetrachromat
Posts: 236
Joined: Sun Feb 14, 2021 6:55 pm

Re: Integer number from Lua calculation needed

#29 Post by Tetrachromat »

LUA 5.3 Reference Manual is your friend. Actually it calls that 'floor division'.
Floor division (//) is a division that rounds the quotient towards minus infinity, that is, the floor of the division of its operands.

User avatar
Keith Baxter
Posts: 4674
Joined: Wed Dec 20, 2017 11:00 am
Location: Botswana

Re: Integer number from Lua calculation needed

#30 Post by Keith Baxter »

Tetrachromat wrote: Tue Nov 22, 2022 9:23 pm LUA 5.3 Reference Manual is your friend. Actually it calls that 'floor division'.
Floor division (//) is a division that rounds the quotient towards minus infinity, that is, the floor of the division of its operands.
Paul,

I did some playing around with what you say. I think there is a bit more formatting required.
The goal is to get 4 two digit hex codes pairs. 00 -> FF from a variable input of 0 to 1. Each pair is to represent RBGA (Red,Blue,Green,Opacity) channels in the color hex code.

I found your // division does not round hex values. If you run this code you will see that the string does not round per RGBA Chanel. You get more than the 8 digit string length.
#FFFF1FE1FE
A good suggestion but not quite the answer


Basic Indications ...

hex color #00000000 is transparent
hex color #000000ff is black.
hex color #0000007f is 50% black. GREY.


When running this code you will note that the _fill() #0000007F is a fixed 8 character, hex color code format. The last pair determines the opacity. What the OP wants is to change the 0 to 1 value to a valid hex value. Like 7F or 66 or some other hex value .
Oh I changed the last two characters in the OP string from FF to %02x else FF will be fixed.

Code: Select all

brightness_lines =1
brightness_lines2 =1
gColorPrint = string.format("#%02X%02X%02X%02X", 255*brightness_lines // 1, 255*brightness_lines // 1,255*brightness_lines // 0.5, 255*brightness_lines2 //0.5)

print(gColorPrint)

canvas_add(10,10,400,400,function()
   _rect(10,10,200,200)
   _fill("#0000007f")
 end)
Keith
AMD RYZEN 9 5950X CPU, Corsair H80I cooler, ASUS TUF GAMING B550-PLUS AMD Ryzen Mother Board,  32Gb ram Corsair Vengeance 3000Mh, MSI GTX960 4G graphics card 

Post Reply