Feature/add yocto light sensor

[ThomasMonkman]

Hello! First off this has been a great library to use, thank you very much for making it.

I've added the ability to set the brightness levels based off the ambient light levels from an yocto light sensor

It works exactly like phones or laptops where it will just adjust the light levels based on how bright the room is.

You set a brightness mapping per screen, setting the 0% and 100% brightness at "X" Lux, it then linearly interpolates between that based on sensor readings.

I've been using it for the last few weeks, and it's like automagic :) as I found my self always having to adjust my screens, especially now its getting dark so fast in winter.

It can be toggled per screen, as well light levels set per screen.

Translations

I've completely left out translations as I wasn't sure how new translations are integrated, I do use a fair amount of information inside strings, so I'm not sure how that would work. But I'm happy to add that in.

I've tired my best to make minimal changes to the repo, and kept in the style you currently have everything.

Settings

I've also tried to make the settings ui as informative as possible, so hopefully someone can set it up without help

User Flow

1. install yocto virtual hub service on pc
2. Set password and user name in browser using the virtual hub ui
3. plug in sensor
4. enter connection password and username in the settings
5. adjust brightness of screens to match what you want

Error handling

I've added error handling as much as I could, here are some screenshots show that off

Everything ok

No sensor found

No connection to virtual hub

[xanderfrangos]

Very cool feature! Thanks for the PR!

Light sensor support has come up many times over the years, but the sensor requested hasn't been consistent. If Twinkle Tray were to offer light sensor support, I would like for it to be as vendor agnostic as possible. For example, I would prefer the settings tab be called "Light Sensor", with an on-page option to select the type of sensor the user has (if supported). From there, users could input the settings relevant to their light sensor type.

I don't have a light sensor myself, so I would love to hear what other users have to say about this.

I'm not going to merge this PR as-is, but I have activated the workflow so that users can play around with the feature:
https://github.com/xanderfrangos/twinkle-tray/actions/runs/19647041391

[xanderfrangos]

Just to clarify, I don't expect you to add support for other light sensors. The idea is that if/when support for other light sensors are contributed, they can share the same settings tab/UI.

[ThomasMonkman]

Ok, thanks I'm going to go ahead and make those changes, hopefully before Christmas.

I'll change it so it's more extendable, although of course it will be a bit of guess work as I don't know how other sensors will work.

[psxvoid]

Hi,

@xanderfrangos Were there any attempts to integrate laptops sensors? Many laptops have ambient light sensors, including mine.

I was thinking that when a laptop has a built-in ambient light sensor and it's connected to an external monitor, it can control it's brightness automatically.

[ThomasMonkman]

Hi @psxvoid, So while revisiting the pr, I've looked in to the ambient sensors built in to a laptop, as i figured how hard can it be?

What I have found is there is a simple windows UWP api for it.
https://learn.microsoft.com/en-us/uwp/api/windows.devices.sensors.lightsensor?view=winrt-26100

however that is an UWP api, and im not really sure how to consume that in to electron, I've tired edge-js, but that api isn't avabile that way (although i didn't spend much time on it)

here was the code I quickly tired (just make a bare bones node project with edge-js installed and run this as a script)

var edge = require('edge-js');

var getLight = edge.func({
    source: function() {/*
        using System.Threading.Tasks;
        using Windows.Devices.Sensors;

        public class Startup
        {
            public async Task<object> Invoke(dynamic input)
            {
                var sensor = LightSensor.GetDefault();
                if (sensor == null)
                {
                    return "No light sensor available";
                }

                var reading = sensor.GetCurrentReading();
                return reading.IlluminanceInLux;
            }
        }
    */}
});

getLight(null, function (error, result) {
    if (error) throw error;
    console.log("Light sensor output (lux):", result);
});

another option is nodeRT but I found that requires installing visual studio 2013, although there are many versions of the package built for different version of windows sdk here
and im not sure how that versioning is handled, can you just use the oldest? or do you have to ship multiple versions? I assume the oldest works forwards. (but then does the ci need vs2013 installed??)

What I am going to do is add in a fake ui sensor, that lets you mimic having a sensor plugged in.

This also lets me test what its like to extend the sensor page with multiple sensors.

I will focus on the changes requested (more generic ui, support for multiple sensors in theory), then maybe look at how to add the ambient light sensor.

[psxvoid]

Hi @ThomasMonkman,

I've seen this UWP API as well, though I was planning to use an older Win32 Sensor API, seems like it can be consumed similarly to how it's already done in other native modules in this repo. But nodeRT might be a better option indeed, haven't tried it.

I will focus on the changes requested (more generic ui, support for multiple sensors in theory), then maybe look at how to add the ambient light sensor.

Yes, sure, don't want to put any pressure on you :)

I was thinking that your feature might intersect a bit with what I want, but I'm not completely sure. Seems like your feature request is related to a remote light sensor - there is "yokto virtual hub" on your screenshots. And probably a UI for a built-in into a laptop ambient light sensor might be slightly different.

[ThomasMonkman]

@psxvoid Thanks, ohh yes that looks like it could work.

So from what I've seen the ambient light sensor returns Lux, which is the same as the yocto sensor, so hopefully it should all just work "flawlessly" :D

haha they even have an article talking through light aware ui

I'm going to change the ui, so it should be simple to slot in another sensor.

and yes it would be great if it worked with the ambient light sensor of a laptop, i went down the yocto route as I wanted it on my pc.

[xanderfrangos]

@ThomasMonkman It's a real pain in the ass, but you can access the UWP APIs through C++. I had to write a few Node modules for Twinkle Tray that do this. For example:
https://github.com/xanderfrangos/twinkle-tray/blob/master/src/modules/tt-windows-utils/windows_media_status.cc
and
https://github.com/xanderfrangos/twinkle-tray/blob/master/src/modules/tt-windows-utils/windows_app_startup.cc

[ThomasMonkman]

@xanderfrangos @psxvoid Ok I've got windows ambient light sensor working in the project using the old COM api you linked above, I need some time to wrap up the ui for it, but I think it shouldn't be much more work.

Then hopefully this should work for a wide range of people :)

I'm not sure if I should move this pr in to draft? But I need to push some in progress as I'm travelling and don't want it to disappear in to the ether. But it is not ready to review, I will ping when it is.

[ThomasMonkman]

Ok @xanderfrangos @psxvoid I've updated the PR with both a fake Sensor for testing, and a real windows ambient sensor.

Which works on my laptop.

I've done the work in spits and spurts, but I've tired to keep the style consistent.

Typescript would have made this a million times easier, many a curse word was used :P
I considered adding partial typescript support but figured it would be better as a separate PR.

In the future I think some kind of log, bucket, or fan curve style graph would be better for mapping the Lux to screen brightness.

I did notice a rare bug where the monitors just don't update, feels like some kind of race condition with the monitors field which I get by reference from electron.js, but I didn't have the heart to dig in to the massive electron.js file.

I've kept the monitor mapping field common between all the sensor types, although I don't really like how much text it has, but wanted it to be clear.

Everything is now split out between files and a new sensor should be easy to add.

The window ambient sensor technically supports multiple sensors, although I'm not sure how you would have this on a device.
Currently it just picks the first one, although the UI does show multiple, if they exist, but doesn't allow you to select which one is used.
I just wanted to get this pr over the line, I think selection could be added at a later date, along with not really being able to test it.

[ThomasMonkman]

I have just updated to also handle damaged/missing settings file, as well as putting the settings menu back to default...

[ThomasMonkman]

Hello, @xanderfrangos this pr is ready for review. Let me know if you have any questions?

[xanderfrangos]

@ThomasMonkman Sorry for the delay! This looks great from what I've been able to review.

I need to rummage around for a device that might have an ambient light sensor, but the "Fake" option worked as expected in my test. There were a few issues with building the EXE installer that I've resolved. I'll clean up the UI a little, too. You've got all of the pieces there, so it just needs some adjustments.

In the meantime, anyone who wants to can try out this feature using the build artifact here: https://github.com/xanderfrangos/twinkle-tray/actions/runs/23117199584

[ThomasMonkman]

Cool thank you!

I think most modern laptops should have an ambient light sensor. @psxvoid do you have one? can you test?

@xanderfrangos What changes did you have in mind for the ui? I was never really happy with the description of the light sensor lux mapping.

One issue I've noticed at least on my dell monitor is it stops me be able to access the on screen menu (it closes it very quickly), I'm guessing it closes when ever it receives a command, perhaps it shouldn't change the brightness if there is not real change, although this would have to be based off the screen brightness percentage and not the lux, as lux changes easily.

--

I also think a good guide for setting the mapping is getting your monitors lux/nits output and setting that as the 100%
then setting the minimum to be something like 0% at 5 lux.

here is a lux to nix convertor.
https://calculator.academy/lux-to-nits-calculator/

What you will find is lux goes up crazy fast, a room with almost no lights on is 5 lux
where as outside is somewhere between 20k-100k lux

I'm in medium shade with no direct light on my laptop inside, with only ambient light bouncing off the floor through a window and I'm at 2.2k lux.

With my dell s2421H 5-250 is good as well as my dell u2515h, but I don't have any more modern brighter HDR screens to test.