Despite the wide-scale use of conventional CMOS detectors, a possibility exists for the development of image sensors that offer enhanced capabilities regarding increased temporal resolution, better dynamic range, reduced influence of scattering, and more.
Conventional image sensors are well-established. And it is expected that in future these will be upgraded with new sensor technologies that will allow us to acquire image based data having more details. This change is mainly influenced by the growing adoption of machine learning for which complex image analysis needs to be performed by computational algorithms. So, the question arises, what will the new image sensing technologies offer?
The answer to this question lies in the following points:
Existing technologies, especially indium-gallium-arsenide (InGaAs) alloy for short-wave infrared (SWIR), are quite expensive and restricted to niche applications. To integrate them into an autonomous vehicle system, the costs need to be considerably brought down, which the new image sensors promise.
The availability of increased computer processing power will require more accurate data for analysis and correlation. This is again possible with future high-end image sensors.
- Advertisement -
On a related note, more advanced sensors working together will surely improve object classification. By comparing the outputs from different sensor types, one will receive accurate insights. Take for example an advanced driver-assistance systems (ADAS) vehicle, which requires lidar, radar, and IR cameras for emergency braking, blind-spot detection, surround view, parking assistance, and much more. Besides this, the advanced sensors’ other applications include agriculture, land surveying, industrial monitoring, medical imaging, etc.
So, let’s take a look at these emerging image sensor technologies and the specific benefit each delivers.
SWIR image sensing
SWIR is an infrared technology whose wavelength range is 1,000nm to 2,000nm. It can only be detected by InGaAs sensors. In comparison, silicon CMOS image sensors are capable of detecting wavelengths of 400nm to 1,000nm (consisting of visible + near-infrared).
A key benefit of the SWIR image sensor is that it successfully helps in reducing light scatter.
From the graph in Fig. 1, we can deduce that scattering declines as the wavelength of light increases. This is beneficial for autonomous vehicles, ADAS systems, and drones.
Fig. 1: Declining scattering with increase in wavelength (Credit: IDTechEx)
The image in the left side of Fig. 2 depicts a person walking through a road on a foggy day from the perspectives of a visible camera and a SWIR camera. A similar comparison has been made in the right side, which shows a car travelling through dust. These examples show the differences of an improved image range due to reduced scattering. SWIR also has many industrial applications for inspection purposes.
Fig. 2: Comparison of images from SWIR and visible-light cameras (Credit: TriEye)
Options that will make SWIR technology feasible while replacing the incumbent InGaAs technology include:
Extended Silicon. Absorption spectrum can be increased by shaping silicon
Germanium
Quantum Dots (QD)-on-CMOS. By changing the diameter of the dots, the wavelength size for absorption can be changed
These can be considered to be solar cells that operate under reverse bias rather than forward bias.
An organic photodetector (OPD) consists of an electrode, wherein the bulk heterojunction (BHJ) is sandwiched between the electron extraction layer (EEL) and hole extraction layer (HEL). When illumination falls on the top electrode, the excited electrons travel to the BHJ to combine with the holes, producing a signal.
Fig. 4: Organic photodetector (OPD) architecture (Credit: Simone et al)
The advantage of thin-film photodetector over a silicon detector is that it can be flexible and printable. Since this type is suitable for covering large areas, applications could be biometric imaging (fingerprint detection), thin pulse oximeter, touchless control, and position and object detection.
Similar to the organic photodetectors, perovskite thin-film photodetectors use simpler/cheaper light-absorbing materials that can produce low-dark currents for increased detectivity or dynamic range and thus give more contrast to images.
Hyperspectral imaging
Although a bit established due to the use of conventional sensors, this technology has applications that are linked to emerging SWIR sensors. Hyperspectral imaging can be used outside the visible spectrum.
The basics of hyperspectral imaging technology lie in the availability of the entire spectrum wavelength at every pixel. So, while in a conventional image each pixel has red, green, and blue values, in hyperspectral imaging an entire spectrum is present. This can be achieved through point scan, line scan, wavelength scan, and snapshot, with each having a high spectral resolution.
Fig. 5: Point scan, line scan, wavelength scan, and snapshot used for hyperspectral imaging (Credit: Wang, Yu et al)
The line scan approach is very well suited for industrial imaging or land scanning via drones. The snapshot has been recently commercialised and is used in astronomy.
Other applications of hyperspectral imaging include:
Agriculture
Infrastructure
Oceanography
Surveying and extraction
Chemical identification
Identification of food degradation products
Environmental monitoring
Defence
Event based imaging
It is a relatively new approach to data acquisition. Event based imaging, also known as dynamic vision sensing (DVS), only records changes that occur rather than the entire frame at specified intervals.
A conventional image sensor in a camera reads the intensity of each pixel at a pre-defined frame rate (for example, 25fps). Here, the data readouts are equally spaced in time at a constant frequency.
Comparatively, in event based imaging, each pixel is read independently when the received intensity changes. So, rather than having data readouts at a constant frequency, you receive data having different frequency time-stamps based on intensity change.
To understand the advantage of image based sensing technology, let’s take the help of graphs shown in Fig. 6 and Fig. 7, which depict a clear blue sky being recorded by an HD camera.
Fig. 7: Image based sensing behaviour (Credit: IDTechEx)
When using conventional sensing techniques, for most of the time, a camera records scenes where no relevant change occurs in the sky (see over-sampled in Fig. 6). And when a change does occur for a brief period, say an aeroplane passes by, the camera records that along with additional scenes of no change (see under-sampled in Fig. 6). All this happens because the data readouts have a fixed frequency.
Now, the same scenario recorded using image based sensing technology would focus on capturing only the relevant scenes related to the flying aeroplane (see Fig. 7). Since the data readouts have a flexible frequency here, scenes of no change will be ignored.
This is highly beneficial for machine vision applications as seen from the comparison in Fig. 8.
Fig. 8: Images from frame based and image based cameras (Credit: Prophesee)
Only changing subjects such as moving vehicles and people relative to immovable backgrounds such as buildings get recorded.
Other advantages include:
Higher dynamic range
Greater temporal resolution
Less data production, making data transmission much easier
Simple data processing (as static image areas get ignored by the hardware)
However, the technology has some drawbacks such as difficulty in interpreting datasets without specialist software, greyscale image, and less extensive analysis algorithms than conventional image analysis.
Applications include:
Autonomous vehicles
Drones
Occupant tracking in smart buildings
Iris-tracking in AR/VR goggles
Flexible X-ray sensors
Conventional X-ray sensing works when the X-ray wavelengths reflected from an object hit a digital X-ray image detector. This image detector consists of a layer known as a scintillator that converts the X-rays into visible light, which is further refined by a silicon photodetector. The emerging amorphous silicon X-ray detector takes this one step ahead by allowing devices to become flexible and lightweight enough to be wrapped around curved body parts.
Another exciting innovation in this field is the perovskite based X-ray sensor in which a very thin (200µm) layer of perovskite absorbs the X-rays and creates charges. The sensor is beneficial for generating high-resolution images.
Fig. 9: Perovskite based X-ray sensor (Credit: Holst Center)
The road ahead
Most of the sensors discussed here are suited for machine vision and image analysis across many applications from cameras to ADAS and drones. These will also serve as a cheaper alternative to current InGaAs sensors. While some of them are nearing the end of the development phase, others are still in it. Nevertheless, most of them would be rolled out into the markets in the next few years.
Hyperspectral imaging based sensors are quite established. Among the SWIRs, the extended silicon is going to be the first one to arrive in markets by 2022. Event based imaging is expected to follow a year after.
This article is compiled by Vinay Prabhakar Minj, a technology journalist at EFY and is based on an IDTechEx webinar.
