Choosing the right image sensor for embedded and IoT applications

Wednesday, September 15, 2021
e-con Systems
e-con Systems

Cameras and processing platforms are constantly evolving to provide reliable and cost-effective embedded vision solutions across different markets. Imaging solutions are deployed across industries, from manufacturing, retail, and medical to smart cities, auto farming, and sports broadcasting. Considering their ability to enable business transformation, the design of an imaging system can be a complex journey - with several key points to cross off in the checklist.

Firstly, it is important to understand that engineering such an imaging system depends upon the type of objects you want to capture, the lighting conditions, operating temperature, available environmental space, and of course – the costs involved. This article discusses various factors to consider while choosing an image sensor across some of the most popular vision applications. We hope to bring you up to speed on selecting the one that perfectly fits these seven innovative applications.

Smart City
Smart City

Smart cities are increasingly adopting embedded vision systems to enhance citizen experiences. They use imaging solutions such as smart parking lot and traffic management, intruder detection, and infrastructure monitoring systems. One of the biggest challenges these solutions face is environmental factors such as varying and challenging outdoor lighting conditions. However, new-gen embedded vision systems have the imaging power to overcome this. Let’s look at the factors that affect the selection of sensors for smart city applications.

HDR (High Dynamic Range)
In varying lighting conditions, outdoor illuminance levels depend on the weather or the objects in the surrounding environment. Since most cameras are placed outdoors in smart city applications, lighting varies in the target scene. For instance, it is difficult to cover the brightest spots such as the sun or the sky, and the darkest shadows under the trees, all in a single frame with a normal camera. With an image sensor that can operate in a very high dynamic range, the brightest and the darkest regions can be captured in a single frame.

To understand this better, have a look at the below comparison of images captured using a normal camera and an HDR camera.

HDR (High Dynamic Range)

You can clearly observe that an HDR camera is able to capture the exact scene, whereas, in the image output delivered by a normal camera, bright areas are masked or bleached by sunlight.

NIR (Near Infrared) Lighting
Applications such as intruder detection require cameras that operate 24X7 - day and night – in challenging lighting conditions. For such applications, reliable performance can be achieved by using NIR lighting to assist cameras. It equips cameras with night vision capabilities since their image sensors are designed to be highly sensitive to this NIR spectrum.

For instance, have a look at the below illustration to understand how an image captured at night with the help of IR (infrared) illumination offers visibility which is almost as good as the daylight image.

NIR (Near Infrared) Lighting

Hence, it is crucial to choose an image sensor that comes with the capabilities to adjust to varying lighting conditions.

High signal-to-noise ratio (SNR)
Another important factor to consider is the level of accuracy needed, given that there’s a possibility of challenging outdoor lighting conditions creating noise in the output image. Capturing colors accurately in dark conditions is even more difficult. To ensure effective monitoring, surveillance or monitoring cameras used in smart cities should produce images with greater color accuracy and lesser noise. In such a scenario, cameras with large pixel sizes and better SNR are recommended.

Given below is a comparison of images captured by a low SNR camera and a high SNR camera in dark lighting conditions.

High signal-to-noise ratio (SNR)

Try this camera selector to find the right camera for your application

Try this Processor platform selector to find the right processing platform for your application.

Automated Number Plate Recognition

Automated Number Plate Recognition (ANPR) applications use imaging solutions to tackle problems in law enforcement and vehicle management. A major challenge faced in such applications is the quick recognition of license plate characters when the vehicle is in motion. Choosing an image sensor with capabilities to capture high-quality images even when the target object is in motion can be a make or break moment. Let’s look at the factors to be considered during the selection of an image sensor for ANPR.

Shutter artifacts
Cameras used in law enforcement are expected to capture images without any distortion or skew. A typical rolling shutter sensor produces artifacts in the image, which can make the characters in the number plate unrecognizable. But with a global shutter camera, producing images of moving vehicles without any distortion is possible since it does not expose a frame row-by-row as a typical rolling shutter camera would. Instead, global shutter exposes a complete frame simultaneously enabling it to freeze on a fast-moving scene.

The below figure illustrates how a rolling shutter camera delivers the output with shutter artifacts while capturing a rotating fan. Whereas a global shutter camera is able to capture the fan as it is without any distortion.

Global and Rolling Shutter

High sensitivity (Visible and NIR)
Choosing a global shutter sensor is only half the job done. For accurately capturing fast-moving vehicles, a low exposure time is to be set. Due to this low exposure time, image sensors must be capable of capturing a scene with speed in low lighting conditions. This can be addressed by a camera that has high sensitivity in both visible and NIR regions.

The following figure illustrates the difference between images captured by a low sensitivity camera and a high sensitivity camera when the ambient light is low.

Low and High Sensitivity

Sufficient HDR
While capturing a license plate, bright headlights tend to bleach the image, thereby making the license plate less recognizable. But if the sensor comes with sufficient dynamic range, it can ensure good visibility of the plate making the characters on it recognizable for OCR purposes.

The below image is a good illustration of this scenario. You could clearly see that an HDR camera ensures proper visibility and prevents headlights from bleaching the license plate.

Dynamic Range Camera

Try this camera selector to find the right camera for your application

Try this Processor platform selector to find the right processing platform for your application

Industrial Handhelds
Industrial Handhelds

Industrial handheld applications use imaging solutions for mission-critical processes such as barcode scanning, OCR, and image capture for documentation, inventory tracking, and quality assessment purposes. A major challenge faced in such applications is the minuscule details to be captured that are often far away from cameras. It is pertinent to choose an image sensor with the right capabilities to capture high contrast and high-resolution images. Let’s now look at the factors involved in choosing the right image sensor for industrial handhelds.

High contrast
Data collection tablets are intended to detect barcodes and process data with contrast-based algorithms. These algorithms target only the gray-scale contrast in images. Monochrome image sensors or high-resolution color sensors can bring out a very high contrast image output. The following figures show the comparison of low contrast imaging and high contract imaging outputs.

Contrast Imaging

Image detail
Barcodes can be detected from a distance with cameras that are not close to the subject so that they can cover a wide region. The number of pixels available in the sensor accounts for the ability of the system to detect defects or barcodes from a distance. Nowadays, the size of barcodes is getting smaller, which makes detection difficult. But a high-resolution image sensor can be used to overcome this by delivering even the minutest of the details.

Handheld devices are often not at a fixed distance from the target. The camera’s autofocus capabilities ensure that the target is always within the focus area. Camera modules that come equipped with autofocus ensure that the output from the camera is consistent in terms of sharpness. Below is a sample of a well-focused image captured using an autofocus camera.

Well Focused Image

Try this camera selector to find the right camera for your application

Try this Processor platform selector to find the right processing platform for your application.

Quality Management Systems
Quality Management Systems

Quality management systems use imaging solutions to tackle problems on the manufacturing floor – from quality assurance to testing and production. The most significant challenge faced in any quality management system in an industrial setup that uses embedded vision systems is varied lighting conditions. The solution to overcome this challenge is to find an image sensor with capabilities to adjust to random and varying lighting conditions. Let’s look at the factors that influence the selection process of the image sensor.

Fast capture
With automation in activities such as freight movement, quality inspection systems have to now capture moving objects on a conveyor belt or production line. This often results in a phenomenon called motion blur, which makes the output image blurry and different from the target scene. The below figure illustrates the difference between images with and without motion blur.

Images with and without motion blur

In such a scenario a camera with low exposure time or fast shutter speed is recommended. Such a camera would be capable of capturing any discrepancy or fault and report it on the go.

Color reproduction
An industrial setup can have varied lighting that could affect brightness or color temperatures. Even at low light, the images must produce consistent colors almost as good as those captured in good lighting. Otherwise, tasks such as anomaly detection and documentation may be inconsistent. The below figure shows a comparison of image outputs with good and inaccurate color reproduction.

Color reproduction

Try this camera selector to find the right camera for your application

Try this Processor platform selector to find the right processing platform for your application

Document Readers

Document readers are becoming increasingly popular as OCR (Optical Character Recognition) algorithms have become quick and efficient in their ability to accurately identify letters, symbols, images, graphs, etc. Document readers with OCR capability can be used for applications such as low vision, automated document verification, text to audio conversion, etc. With algorithms getting progressively smarter, there’s a need for high-quality cameras that can deliver images with letters and characters clearly identifiable in them. Let’s have a look at the key factors to be considered before selecting an image sensor for document readers.

High resolution
Any typical contrast algorithm-based OCR application would demand a very high-resolution image sensor. It enables the camera to read very fine prints that even a person with normal vision would find difficult to read. A high-resolution camera also would mean fewer errors. Below is an image captured using a high resolution camera where the letters are clearly visible.

High resolution image

Sharp focus
A document reader needs to capture sharp images. Whether it is books or boxes of varying dimensions, the camera must set the perfect focus every time, which is why an autofocus camera module would be the best-suited solution. Have a look at the below image to better understand how lack of focus could impact the image output.

Color reproduction

Try this camera selector to find the right camera for your application

Try this Processor platform selector to find the right processing platform for your application

Digital Microscope (Medical)
Digital Microscope - Medical

Digital microscopes are used to tackle problems in manual reporting which tend to cause human errors and inaccuracies. It is of paramount importance to ensure high-quality images to overcome this. Unwanted noise is possibly the biggest challenge faced in such applications. Let’s look at the factors that would affect the decision-making process of finding the best-fit image sensor.

High SNR (Signal to Noise Ratio)
Being a medical application, the camera in a digital microscope should be able to eliminate unnecessary image noises. Noise is an unwanted attribute in microscopy that could negatively impact a test result or diagnosis. High SNR ensures that the image captured consists of valid signals with very minimal noise.

We had seen a comparison of low SNR and high SNR image outputs in a previous section. Here is another example.

SNR image output

True Colors
Microscopes with monochrome sensors are limited to shape or size-based detection. It is crucial to detect true color to make accurate decisions or create reports. The below image shows a comparison between images with accurate and inaccurate color reproduction.

True Color reproduction

Try this camera selector to find the right camera for your application

Try this Processor platform selector to find the right processing platform for your application

Robotic Arms and Autonomous Mobile Robots (AMR)
Robotic Arms and Autonomous Mobile Robots

Robotic arms use embedded cameras to carry out various tasks across auto farming, warehousing, and manufacturing. As for autonomous mobile robots, they offer personal assistance and security solutions - enabled by embedded vision systems. A major challenge faced in both these applications is the quick recognition of surrounding objects which are either stationary or in motion. Therefore, the camera solution should have a high frame rate, fast shutter speed, and multi-camera synchronization (to provide a 360-degree view of the robot’s surroundings). Now, let’s look at the primary factors that affect the image sensor selection process when it comes to robotics arms or AMRs.

High frame rate
The constant motion of the target object would mean that the camera must quickly respond while interacting with the main system. The camera’s fast response time ensures that the images are delivered at a faster rate. Thus, a high frame rate camera and camera interface become necessary in AMRs and robotic arms.

Fast shutter speed
Even if the cameras come with high frame rates, images must be devoid of motion blur. It can be achieved by capturing the images at fast shutter speed (low exposure time), which ensures that the target object has not moved much during image capture – thereby providing images with close to zero motion blur.

Try this camera selector to find the right camera for your application

Try this Processor platform selector to find the right processing platform for your application

Choosing the right ISP

The focus of this article has been on choosing the right sensor. However, since building the right imaging solution is incomplete without picking the right ISP and lens, it is worthwhile to spare a few words about the two. Let us start with ISP.

Choosing the right Image Signal Processor (ISP) for your imaging solution is as important as choosing the image sensor itself. This is because images from sensors are usually in raw format, and an ISP is required to convert the picture into a usable format. Converting a picture can be HW intensive and has a major impact on the quality of the image. This means you will need to understand where this is happening. Some sensors are available with integrated ISP, there are dedicated ISP chips and some SoCs have the ISP integrated such as the NXP® i.MX 8M Plus is used on the Toradex Verdin System on Modules. Also, in some use cases it can be fine to do the conversion on the CPU or GPU, sometimes called Soft ISP. All these different solutions have pros and cons affecting performance, hardware and software complexity. If ISP solutions are incorrectly chosen, it completely neutralizes the advantage of choosing a high-quality sensor. For example, a high frame rate sensor paired with an ISP with an old data transfer technology can completely ruin the benefits of getting faster frames.

Choosing the right lens

Another factor that may be taken for granted is the lens in a camera system. A lens can make or break the imaging system. So, basic lens criteria such as FoV (Field of View) and aperture must be considered while deciding the right lens for your application.

A higher FoV may cover a wider region. But the resolution should be good enough to bring out the details, especially in a large scene. A smaller aperture means a higher depth of field, which in turn means that more objects in the field of view are in focus. But it also limits the light entering the camera, which might result in a lower signal-to-noise ratio. The resolution capability of the lens plays a key role too. After all, high-resolution image sensors demand high-resolution lenses.


It is important to choose the right imaging system for your application after considering all the design requirements. The right combination of sensor, ISP and optics will help enhance the performance of your vision-based product. Working with a specialized camera partner who has a wide portfolio of camera products with experience and expertise in integrating cameras smoothly into end products is highly recommended.

Toradex and e-con Systems India Pvt. Ltd.

e-con Systems offers camera boards for the i.MX 8 based family of SoMs that can be interfaced directly with the CSI-2 MIPI interface on the Toradex's Ixora carrier board for the Apalis iMX8QM SoM (product IDs 0037xxxx and 0047xxxx).

Arun Asokan
, Lead, Camera products, e-con Systems
Share this on:

Leave a comment

Please login to leave a comment!
Have a Question?