03/15/12
Expanding The Dynamic Range of Short Wave Infrared (SWIR) Imagery
Photonics Online The Magazine, photonicsonline.com
by Marc Hansen and Mark C. Stern, Sensors Unlimited Inc.
Advances have been made in short wave infrared (SWIR) imaging technology to address the most demanding imaging and surveillance applications. Multiple techniques have been developed and deployed in Sensors Unlimited SWIR indium gallium arsenide (InGaAs) cameras to increase the dynamic range performance of standard, commercial off-theshelf (COTS) products. New developments have been implemented on multiple levels to give these cameras the unique ability to automatically compensate for changes in light levels of up to five orders of magnitude,while improving intra-scenic dynamic range through raw data acquisition techniques, and a proprietary, patent-pending, image-enhancement algorithm.
Autonomously imaging from intense daylight conditions to low light level nighttime conditions requires that the imaging system can automatically adjust for changes in light levels over many orders of magnitute. Compensating for a change in the average scene intensity and adjusting for a high dynamic range within a single scene (where together there are very bright and very dark objects) are both of high interest to the surveillance community. Sensors Unlimited SWIR cameras are designed to address these challenges through every step of the imaging process, from signal acquisition to the management and enhancement of raw data for display.
Acquiring The Raw Data
The pixel architecture used to acquire data over this large range of scene intensities is unique to Sensors Unlimited SWIR cameras. The focal plane array (FPA), or lightsensing chip, is composed of an InGaAs chip bumpbonded to a silicon read-out integrated circuit (ROIC) that has the ability to continuously change the in-pixel analog gain without the limitations of the fixed integration capacitors found in capacitive transimpedance amplifier (CTIA) ROICs used by other camera manufacturers. Commonly, CTIA pixel architecture is limited to two capacitors, and therefore only two different gain settings due to pixel size constraints. The gate-modulated (GMOD) pixel architecture ROIC can change the sensitivity of the focal plane array by modifying internal biases, resulting in a change in effective capacitance. The flexibility of an infinitely variable capacitance results in an infinitely variable full well capacity for the GMOD ROIC. This allows the camera to adjust its sensitivity and full well capacity to compensate for large changes in scene intensity without the integration capacitor limitations of a CTIA ROIC.
Figure 1: Simplified schematic of GMOD pixel architecture.
A simplified GMOD pixel schematic is shown in Figure 1. Each pixel of the FPA contains a gate-modulated input circuit for converting current to voltage with continuously adjustable gain. The photodiode current flows through M0 with a proportional amount of current mirrored in M1 while the ratio of the currents through M1 and M0 is controlled though the externally set BIAS and GAIN voltages. The camera internally provides all bias voltages necessary for operation of the focal plane array.
Programmed into each camera is an automatic gain control (AGC) that analyzes the image and adjusts the camera sensitivity and integration time, maximizing the information available in the raw data. The result is an imager with the ability to acquire data over a very large range of scene intensities. The camera will change its gain, based on the scene content, to increase sensitivity in low-light-level scenarios or increase full well capacity to avoid saturation in highintensity daytime scenes.
Digitizing For Maximum Dynamic Range
It seems there are two sides to every dynamic range story. Compensation for large variations of the average scene intensity is successfully demonstrated through the use of the GMOD pixel architecture described above, but a large dynamic range contained within a single scene requires further techniques to maximize the information displayed. Before any data from the FPA can be processed or enhanced by the camera electronics, the data must be converted from the analog domain to digital counts. Once the analog voltage data from the FPA is converted to digital counts by an analog-to-digital converter (ADC), the data can be easily manipulated mathematically.
One type of manipulation, often referred to as digital gain, is to simply multiply all of the pixel data by a value greater than one. Sensors Unlimited SWIR cameras use digital gain during the conversion of the analog signal from the FPA to a 12-bit digital signal, fine-tuning the data for increased inter-scenic dynamic range. Within a single scene, the camera can automatically analyze and adjust the raw data to best fit the analog FPA data to the entire range of available 12-bit digital data values. The result is that the information from the scene is adjusted to accurately fit the 12 bits of available digital values for later enhancement with the highest available fidelity.
Figure 2: Nighttime SWIR imaging in an urban environment. Unenhanced, raw data image (left); enhanced data image (right).
A New Way To Enhance Images
At this point in the signal chain, the analog signal from the FPA has been converted to digital counts and the digital data has been adjusted to fit the 12-bit ADC range. Now the data must be displayed on a monitor for observation. Sensors Unlimited SWIR cameras supply both a digital output and an analog RS-170 video output simultaneously, but these two different data display methods are not equivalent. The digital output of the camera will deliver a 12-bit image that is capable of producing 4096 different gray scales. The analog output of the camera must conform to the RS-170 video standard for display on common video monitors, but these monitors only have the ability to display 256 different gray scales. To ensure that the most information possible is displayed to the user (regardless of what output is used), a new image enhancement algorithm has been developed to deliver the maximum achievable amount of scene data. The image enhancement algorithm not only adjusts the data for the best display, it also modifies and interprets the data to maximize the viewable information in high dynamic range scenarios. This patentpending form of histogram equalization expands lowcontrast data and applies digital gain to low signal areas of the scene while compressing the high-intensity data. The resulting image delivered to the display device contains much more information than what is possible to view in the raw, unenhanced data as seen in Figure 2.
Figure 3: Effects of different POWER function values on the enhancement look-up table.
More POWER To The User
A user-accessible control parameter, called the POWER function, has been developed by Sensors Unlimited to give the user direct control over the general shape of the enhancement look-up table while still observing the positive benefits of the proprietary, histogram equalization enhancements. The look-up table receives image data as an input and then converts that data to output according to the look-up table curve. Figure 3 shows a simplified representation of a look-up table and what effects the POWER function will have on the shape of the lookup table curve. The look-up table accepts the 12-bit data from the image on the X-axis, which ranges in intensity from 0 to 4095 counts, then outputs or maps the data in 12 bits to the Y-axis according the curve dictated by the POWER function. A POWER value of less than 1 will accent data in the shadowed or low-light-level areas of an image. As shown in Figure 3, a power value of less than 1 has a large (more vertical than horizontal) slope for the lower input values on the X-axis. A large slope will take a narrow range of input values on the Xaxis and stretch them over a larger output range on the Y-axis. The result is a stretching of the information in the shadowed areas of the image to produce more detail as shown in the left image in Figure 4.
Figure 4: Effects of the POWER function on SWIR urban night imagery. From left to right, a POWER value less than 1, equal to 1, and greater than 1.
A POWER value of greater than 1 will accent data in the brighter areas of the image to reduce the effects of saturation and expand the available information and detail in the high-light-level areas of the scene. A power value of greater than 1 has a large slope for the higher input values which translates a narrow range of high intensity data on the Xaxis to a larger range on the output Y-axis, resulting in an image that contains more detail in the brighter areas of the scene. The result of a POWER value greater than 1 is shown in the right image in Figure 4. A POWER value of 1 will not specifically accent either the dark areas or bright areas of a scene beyond what the proprietary histogram equalization enhancement algorithm automatically produces. A power value of 1 is shown by the center image in Figure 4. Through a combination of smart pixel design, electronics, and onboard enhancements, Sensors Unlimited SWIR InGaAs cameras address some of the most difficult imaging problems for high dynamic range scenarios. Past difficulties of passively imaging over a large range of situations are no longer an obstacle. The new cameras are able to function automatically to display as much information from the scene as possible. For most applications, this occurs without the need for user control, helping reduce operational costs and training time.
Authors
Marc Hansen, Applications Engineer for Sensors Unlimited Inc., part of UTC Aerospace, has over eight years of experience studying spectral phenomena in the fields of optical imaging, spectroscopy, and mechano-optical systems. He is now concentrating his research in the near infrared (NIR) and short wave infrared (SWIR) spectral regions to support the development of future InGaAs detector applications.
Mark C. Stern, P.E., Senior Electronics Design Engineer Sensors Unlimited – part of UTC Aerospace,
Mr. Stern has extensive experience in electronics design and new product development. He has designed a variety of successful products in the communications, medical instrumentation, and electro-optics industries. Since he joined Sensors Unlimited in Princeton in 2005, he has designed hardware and software for short-wave infrared cameras. He is the chief developer of the DR1 High Dynamic Range control algorithm. Mr. Stern holds several patents and has one patent pendingthan 1, equal to 1, and greater than 1. He has a B.S. degree in Electrical Engineering (City College of NY) and an M.S. degree in Electrical Engineering and Computer Science (Columbia University).