RE: Video Game Changer?26 Jan 2020 15:05
I have been reviewing SEE’s HIGH FRAME RATE IMAGE PRE-PROCESSING SYSTEM AND METHOD patent.
Frame rate is the number of pictures taken over a period of time. Cinema was 24 frames per second. TV UK TV is 60 Hz (frames per second). If you speak to gamers they will complain if the refresh rate is lower than 60 Hz and are happiest with 100 or 144 frames per second, a more natural feeling and allows them to react faster to on screen events.
So why is this patent important?
Computation is expensive. It takes time and it takes power. The more computation you add the longer it takes. The more short cuts that you can take the more effective performance you can get in a short period of time and also it will save power if that is important.
I just did a test, while driving in traffic through town, I glanced at the side mirror and back to the road ahead. I got partway through the “W” of “One Mississippi” say 1/6 of a second. At 30Hz that is 5 pictures.
An aside: when the eye moves suddenly, often we blink. If not, then the brain “blinks” as it can’t process the fuzzy image, so in that dart to the side, the brain already knew the forward scene, shut down until the eye stabilised on the mirror, then shut down while it processed the mirror view as it returned to the front. The snapshot of the mirror was a tiny fraction of the 1/6s This means that at 30Hz, the camera may not have seen my eye reach the mirror, just blurred eye or blink as I scanned past the bottom right of the windshield. That is why 60Hz + is required for safety
If we want to analyse our images at say 60 Hz, we need to have every image perfect. We can do this by taking images at say 300 Hz and selecting only best images to pass on to the next stage. We could vary the brightness of the LED, or the sensitivity of the camera and pick the images with the best glints from the LED. That will allow us to better compensate for bright sunlight or dark glasses. Only relatively simple computation is required to do this and it can be done at the higher frequency. The resulting “lower rate” images are then processed using more complicated algorithms to get gaze angles and detect drowsiness.
Based on the requirements the analysis that you do at the high frame rate could be looking for different things – good quality glints, a specific number of glints in set locations (based on the number and locations of the currently illuminated LEDs and where the glints were last seen and expecting extra reflections from glasses that were there on previous scan.) It could be to get a good image of the Iris in some frames for identification, or to maximise the contrast between an image with the IR on and off so it can subtract the glare from the sun.
Without this patent, we don’t get to select the best images or everything has to be done at the output frequency which then drops frames that don’t work.
Sign In
Follow the stocks
