Night Vision and Drowsiness Alerts in the Modern Car
Features

Car Night Vision Dates to 2000 and EU Law Now Demands Drowsiness Alerts

Photo: Robert Basic from Germany, CC BY 2.0, via Wikimedia Commons · Source · Licence

Infrared cameras that spot pedestrians and animals in the dark, and the sensors that watch steering, lane position and eyelids for fatigue.

FeaturesBy 9 min read
The short answer

Car night vision uses infrared cameras to show pedestrians and animals beyond headlight range, either by lighting the road with invisible infrared or by reading the heat objects give off. Drowsiness systems judge fatigue from steering, lane position or the driver's face. The EU requires them in all new vehicles from 2024.

From 2024 the European Union requires every new car to carry a system that can tell when its driver is getting drowsy. The rule lands on a problem with large numbers attached: various studies suggest that around 20 percent of all road accidents are fatigue-related, and up to 50 percent on certain roads.

Fatigue is one half of this story. The other half is darkness, and the infrared cameras that let a car see heat that headlights miss. Some systems throw out their own invisible light and read the reflection, others simply read the heat that things give off, and a different set of sensors decides, without asking, that the person at the wheel is fading.

How car night vision sees what headlights miss

An automotive night vision system uses a thermographic camera to extend a driver's perception and seeing distance in darkness or bad weather, beyond where the headlights reach. The sensors are generally infrared, which means they pick up energy on the infrared spectrum. A pedestrian or an animal gives off heat, so it shows up even when it is far from any lamp.

It is optional equipment on certain premium vehicles, and it adds to the complexity of the modern car. Electronic control units have to process the raw data, and the result must be woven into the car's existing data networks so that it works reliably alongside the engine and powertrain.

The technology rests on night vision devices, a term for electronically enhanced optics that work in three modes: image enhancement, thermal imaging and active illumination. A car's complete system blends infrared cameras with other sensors such as GPS, lidar and radar. More recently, thermal imaging has moved beyond the driver's display and into the sensor suites of autonomous vehicles and driver assistance systems.

Active night vision lights the road with infrared; passive night vision captures heat already given offActiveLights the road with infrared invisible to humansInfrared lightReflection150 to 200 metresPassiveCaptures heat already given off by objectsHeatAbout 300 metres
Active night vision lights the road with infrared and reads the reflection, while passive systems capture heat that objects already give off.
Fatigue is blamed for around 20 percent of road accidents, and up to 50 percent on certain roads.

Active and passive night vision compared

Night vision systems come in two types, passive and active, and each has strengths the other lacks.

FeatureActivePassive
How it seesLights the road with infrared invisible to humansCaptures heat already given off by objects
Typical range150 to 200 metres (500 to 650 ft)About 300 metres (1,000 ft)
ImageHigher resolution, better for inanimate objectsGrainier, lower resolution, higher contrast for living things
WeatherBetter in warmer conditions, weaker in fog or rainWorks poorly in warmer weather
Sensor sizeSmaller, can sit by the rear-view mirrorLarger

There is more than one kind of active system. A gated design pairs a pulsed light source with a synchronised camera, which allows ranges of 250m and strong performance in rain and snow. The wavelength matters too. Near infrared light has a wavelength of 0.75 to 1.4 micrometres, small enough to pass between the typical particles in fog, which measure 10 to 15 microns. Far infrared runs from 15 micrometres to 1 mm.

Where the picture appears

Makers have shown the image in three places. The earliest was the windshield, using a head-up display whose brightness can be reduced with a dimmer knob and which sits closest to the driver's line of sight. The least expensive is a navigation or information screen, farther from the line of sight; BMW has used it exclusively, along with the W212 E-Class. The newest is a high-resolution liquid-crystal display in the instrument cluster.

Who built night vision first

Cadillac gets the credit for the first series production system, on the 2000 Deville. It was developed with Raytheon and used a passive infrared camera behind the grille. The windshield head-up display showed a black-and-white image, with warm objects in white and cooler ones in black. Cadillac dropped it in 2004. Because the system outputs a standard NTSC composite video signal and its parts are fairly easy and inexpensive to find, it became a popular way to add thermal night vision to other vehicles. After a long hiatus, the 2016 flagship CT6 returned with an improved version, with the picture inside the instrument panel rather than reflected on the windshield.

Toyota followed in 2002 with the first worldwide series production active system, offered on the Landcruiser Cygnus and the Lexus LX470. Near-infrared projectors beside the fog lights lit the road like high beams. A CCD camera behind the rear-view mirror caught the reflection, and a computer turned it into a monochrome picture projected above the gauge cluster.

Honda's 2004 Legend introduced pedestrian detection, using far infrared and a head-up display that enclosed any human-like figure in a red box and sounded a tone. BMW introduced its Night Vision on the 2005 7 Series (E65), processing far infrared to put the emphasis on pedestrians and animals, with a range of 300 metres and no dazzle from headlights or road lights.

Mercedes-Benz showed a Night View Assist prototype on the F500 concept in 2003, then in 2005 put the series version on the redesigned S-Class (W221), the first to use the instrument cluster's LCD. Audi's Night Vision Assistant arrived in 2010 on the A8, and DS Automobiles added its version on the DS 7 Crossback, with a camera in the grille looking up to 100 m ahead. Peugeot offers an optional system on the 508.

  • 2000: Cadillac Deville, first series production night vision.
  • 2002: Toyota Landcruiser Cygnus and Lexus LX470, first active system.
  • 2004: Honda Legend, first worldwide system with pedestrian detection.
  • 2005: BMW 7 Series (E65) and Mercedes-Benz S-Class (W221).
  • 2010: Audi A8.

From seeing a pedestrian to protecting one

The next step was to make the car react. Toyota's Crown Hybrid, in February 2008, was the first active system to highlight pedestrians, placing them in a box on an LCD in front of the driver. BMW's 2008 update on the redesigned 7 Series (F01) flashed a caution symbol on the navigation screen and head-up display, scanning up to 100 yards ahead once the car reached 25 mph and tinting pedestrians yellow.

Audi's A8 system marks humans in yellow, and the computer judges whether the person is moving in a way that could lead to a collision. If so, the pedestrian turns red and the driver hears an audible warning. A 2013 update added animal detection, and Matrix LEDs that flash briefly three times to alert the pedestrian.

Mercedes-Benz brought pedestrian detection to the E-Class (W212) and refreshed S-Class in 2009 with Night View Assist Plus. In 2011 the CL-Class (C216) became the first series car with night-vision-guided pedestrian spotlighting, flashing at anyone it detects in a way that avoids dazzling traffic ahead and oncoming. In 2013 animal detection arrived on the W222 S-Class.

BMW's 2013 Dynamic Light Spot goes further. When the infrared camera spots pedestrians or large animals on a collision course in the dark, the headlights direct two separately controlled spots at them without causing glare. In an acute risk, an acoustic warning sounds and the brakes are set to maximum standby. For the 2014 model year the 5 Series received the same features.

What happened to some early systems

Not every attempt lasted. Lexus launched Night View on the LS in Japan only in 2009, and went worldwide in 2012 on the LS and GS. That version combined a windshield camera and near-infrared projectors in the headlamps with a dedicated control unit, and put the image on the centre console to give a larger display and an easier viewing angle. The projectors switched on above 10 MPH, and the display showed an area up to 65 feet ahead. The system was discontinued for 2014 in the US.

Car fatigue alerts and driver attention alerts: how detection works

Drowsiness impairs a driver's mental stability and decision-making, with risk of physical harm and financial loss for driver and passengers. Carmakers use several methods, each with its own limits.

  • Steering pattern monitoring. The car reads input from the electric power steering. It only works while the driver is actively steering rather than relying on lane-keeping.
  • Position in the lane. A lane camera watches drift, with the same limitation.
  • Eye and face monitoring. Computer vision watches the driver's face through a built-in camera.
  • Physiological measurement. Body sensors record brain activity, heart rate, skin conductance, muscle activity and head movements.

Impairment detection takes this furthest: it assesses a person's functional state in real time by monitoring cognitive performance or physiological responses.

Driver attention alert systems, maker by maker

Volvo launched the world's first driver drowsiness detection system in 2007, called Driver Alert Control. It watches how the car moves and decides whether it is being driven in a controlled or uncontrolled way. If the risk of drowsiness is high, it sounds an audible signal and shows a coffee cup symbol. The driver can check a score that starts at five bars and drops as the driving gets less consistent.

Mercedes-Benz unveiled Attention Assist in 2009. It judges fatigue from the driver's inputs, issues a visual and audible alarm, and uses the navigation system to say where coffee and fuel can be found. Bosch's own system draws on the steering angle sensor, a front-mounted lane assist camera, vehicle speed and the turn signal stalk. BMW's Attention Assistant analyses driving behaviour and shows a rest recommendation on the Control Display.

Others watch the driver directly. DS Driver Attention Monitoring uses an infrared camera above the steering wheel to follow blinking, face and head movement, and the course steered. Cadillac's Super Cruise on the 2018 CT6 relies on FOVIO technology from Seeing Machines, a gumdrop-sized infrared camera on the steering column that measures head orientation and eyelid movement day and night, even with sunglasses.

Citroen takes a physical route: sensors ahead of the front wheels, or a windscreen camera, watch the lane markings and the seat vibrates on the side where the car is drifting. It started with the 2005 C4 and followed on the 2008 C5 and 2013 C4 Picasso.

Some systems make a point of being persistent. In Jaguar Land Rover's, a TAKE A BREAK! message stays on screen for 1 minute with a chime. If the driver keeps going for more than 15 minutes, a further warning appears and continues until the OK button is pressed. Mazda's activates above 65 km/h (40 mph), learns the driver's behaviour at the start of the ride and compares it later on.

MakerSystemFirst car named
VolvoDriver Alert Control2007
Mercedes-BenzAttention Assist2009
FordDriver Alert2011 Focus
NissanDriver Attention Alert2014 Qashqai
MazdaDriver Attention Alert2015 CX-5
Renault and DaciaTiredness Detection Warning2016 Megane
HondaDriver Attention Monitor2017 CR-V
HyundaiDriver Attention Alert2017 i30
KiaDriver Attention Warning2018 Stinger

Aftermarket devices exist as well. The Anti Sleep Pilot from Denmark fits any vehicle and combines accelerometers with reaction tests, and the Vigo headset reads eye and head motion and alerts the wearer with light, sound and vibration. Skoda has offered its iBuzz Fatigue Alert on most models since 2013.

What the rules say about drowsiness warnings

In the European Union, regulation (EU) 2019/2144, the General Safety Regulation, set high-level requirements for vehicle safety systems, including driver monitoring. It defines a driver drowsiness and attention warning as a system that assesses alertness through vehicle systems analysis and warns the driver when needed.

The rules also protect privacy. These systems must not continuously record or retain any data beyond what their purpose needs, the data must never be available to third parties, and it must be deleted immediately after processing. They must avoid overlap, too, and must not prompt the driver separately and at the same time, or in a confusing way. Since 2021 these requirements have been fully defined in an EU implementing act.

What this means for someone buying a car

Night vision remains optional equipment on certain premium vehicles, shown on a head-up display, an instrument cluster or a central screen depending on the model. Fatigue alerts, by contrast, no longer need to be sought out. In EU markets they are fitted to all new vehicles from 2024.

Where this goes next

Outside the EU, a UNECE working group on Driver Drowsiness and Distraction Warning Systems intends to develop draft proposals for two new UN regulations, one on driver drowsiness and one on driver distraction, to prevent crashes caused by the driver not paying attention. On the camera side, thermal imaging has already begun to feed the sensors that self-driving and driver assistance technology depend on.

Frequently asked questions

What is the difference between active and passive night vision?
Active systems light the road with infrared that humans cannot see and read the reflection. Passive systems capture the heat that people, animals and objects already give off. Passive reaches about 300 metres (1,000 ft), against 150 to 200 metres (500 to 650 ft) for active.
Which car had the first night vision system?
The 2000 Cadillac Deville, developed with Raytheon, was the first series production car with night vision. Cadillac discontinued it in 2004. Toyota's 2002 Night View was the first worldwide series production active system.
How does a car know the driver is drowsy?
Different systems watch different signals: steering input, position in the lane, the driver's eyes and face, or body measurements such as heart rate. Volvo's 2007 Driver Alert Control judged how controlled the car's movements were, while DS and Cadillac use infrared cameras aimed at the driver.
Is drowsiness detection compulsory?
In the European Union it is. From 2024 the bloc mandates drowsiness detection systems in all new vehicles, under the General Safety Regulation, regulation (EU) 2019/2144. A UNECE working group intends to draft UN regulations on driver drowsiness and distraction.
Does drowsiness monitoring record the driver?
EU rules say these systems must not continuously record or retain data beyond what their purpose needs. The data must not be made available to third parties and must be deleted immediately after processing.

Sources

  1. Automotive night vision - Wikipedia
  2. Driver drowsiness detection - Wikipedia
Next story / Profiles

Audi A8: Four Generations of Aluminium Flagship Saloon