In hyperspectral remote sensing, rich spectral information is acquired for each pixel of an image collected over a region of interest. This spectral information is extremely valuable as it can be used to identify and characterize the materials from each collected pixel. Before analyzing hyperspectral imaging data, it is important to geometrically correct the data. In the geometric correction, each pixel of the collected imagery is located in a real-world coordinate space. Due to factors such as lens distortion, sensor movement and rugged terrains, the pixels from raw hyperspectral imaging data are not uniformly spaced over the analyzed scene. To generate a square pixel raster image, the geometrically corrected data must be spatially resampled to a north-oriented regular grid, typically with a nearest neighbor approach to preserve spectral data integrity. Although the raster data model is the standard end product for hyperspectral imaging efforts, it misrepresents hyperspectral imaging data for two important reasons: 1) pixels are not square; and 2) pixels are not uniformly distributed in the easting and northing directions.

Pixels Are Not Square

Hyperspectral applications implicitly rely on the assumption that pixels are square (i.e., the spectra from each pixel is equally representative of the materials within the square spatial boundaries of the pixel, as defined by the spatial resolution in the final raster end product). This assumption is invalid for real hyperspectral imaging data. Due to technological limitations, the spatial contribution to each pixel is non-uniform and extends past the square spatial boundaries defined by the pixel resolution. The spatial response of a pixel can be described by the net point spread function, which gives the relative spatial contribution of a material as a function of its position within the pixel field of view. In a recent study conducted by our lab, we emphasized the importance of point spread functions, highlighting the invalidity of the square pixel assumption. In this work, we derived a point spread function (see figure below) for real hyperspectral imaging data. The grid in the x-y plane of the figure corresponds with the actual raw pixels sizes, which are typically defined by the full width at half maximum of the sensor point spread function. Using the derived point spread function, we showed that there is often a discrepancy in the spatial response, and raw pixel size by extension, between the across track and along track directions. We also showed that materials near the center of the pixel contribute more to the pixel spectrum than materials near the edge. With this information in mind, pixels are more ovular than square. Further analysis of the studied point spread function showed that less than 60% of the signal to each pixel originated from materials within its spatial boundaries as defined by the raw pixel resolution. Although this may seem relatively small, it is important to recognize that this value is standard for imaging spectrometers in general. For instance, only ~55.5% of the signal to each Landsat 8 Operational Land Imager pixel originates from the materials within its square spatial boundaries.

From (Inamdar et al. 2020): Example point spread function from CASI-1500 visible–near-infrared hyperspectral imaging data. The points spread function gives the relative spatial contribution to the spectra from a single pixel as a function of across track and along track displacement from the center of the pixel. The grid in the x-y plane corresponds with actual pixel sizes.

Pixels Are Not Uniformly Distributed Across Imaged Scenes

When using the raster data model, end-users also implicitly assume that pixels are uniformly distributed across imaged scenes. This is primarily due to the nearest neighbor spatial resampling methodology that is applied to the geometrically corrected data. It is critical to recognize that the nearest neighbor resampling theoretically shifts, duplicates and eliminates pixels from the geometrically corrected hyperspectral imaging data so that it can conform to the raster data structure. As such, the raster model compromises spatial data integrity. In another study from our lab, we numerically quantified pixel loss, pixel duplication and pixel shifting using four different hyperspectral imaging datasets that varied in acquisition sensor, field site and spatial resolution. These images were processed using conventional techniques, generating georeferenced raster end products. All of the raster end products were characterized by either substantial pixel loss (∼50–75%) or pixel duplication (∼35–75%). Additionally, pixels were shifted by 0.33-1.95 pixels in the raster data products. These results suggest that the raster data model may be inadequate for hyperspectral imaging efforts.

A Novel Data Representation for Hyperspectral Imaging

As part of the same study, our lab proposed a novel point cloud data format, the Directly-Georeferenced Hyperspectral Point Cloud (DHPC), as an alternative to conventional raster formats. An example DHPC is shown in the video below. The DHPC is generated through a data fusion workflow that utilizes pre-existing processing protocols. The DHPC was assessed using the same hyperspectral imaging datasets and data quality metrics described in the previous section. Impressively, the DHPC has zero pixel loss, pixel duplication or pixel shifting. These results suggest that the DHPC preserves spatial-spectral data integrity more effectively than raster end products. Even with the additional elevation data calculated for each spectrum, the DHPC data storage requirements are up to 13 times smaller than raster data products. In various applications (i.e. classification, spectra geo- location and target detection), the DHPC also consistently outperforms raster data products. The DHPC data fusion workflow has actively been implemented as part of the Canadian Airborne Biodiversity Observatory (CABO) project funded by the National Sciences and Engineering Research Council of Canada Discovery Grant.

From (Inamdar et al. 2021a): Directly-Georeferenced Hyperspectral Point cloud (R=640.8 nm, G=549.9 nm, B=459.0 nm) derived from CASI-1500 visible–near-infrared hyperspectral imaging data collected over the Parc National du Mont- Mégantic. The displayed bands are linearly stretched between 0 and 12% reflectance.

For more indepth information about the DHPC, a summary video of the full paper is given below:

Introduction video to the DHPC data format by AppliedRemoteSensingLab on Vimeo.

Implementing the Directly-Georeferenced Hyperspectral Point Cloud

To make the DHPC more approachable for end users and data providers, a methods article was published, providing all the required tools for the implementation of the DHPC data fusion workflow in pre-existing processing protocols. In this article, we highlight the ideal file format for the DHPC: a comma-delimited text file. This file format is ideal as it is approachable for end-users from fields outside of remote sensing. The work also highlights the advantages of the DHPC for site exploration via virtual (VR) and augmented (AR) reality. See the link below for an example VR-ready data product from the Mer Bleue Peatland in Ottawa, Ontario, Canada:

From (Inamdar et al. 2021b): 3-band DHPC example AR/VR ready by Applied Remote Sensing Lab, McGill University on Sketchfab.

The corresponding .ply file for the displayed example VR-ready data product can be downloaded from the supplementary material of the described methods article for site exploration using a virtual reality headset such as an Oculus Quest 2 or HTC Hive Pro 2. The full DHPC can be downloaded from https://zenodo.org/record/4694950#.YUOooYhKiUk.

By navigating a DHPC in VR or AR, researchers can study the field conditions of remote sites in a cost-effective (no travel, food or lodging costs) and repeatable manner. Without the structural information provided by the elevation data, the same level of immersion is not possible with conventional rasters.

Although it may seem counterintuitive, hyperspectral imaging data end products should not be rasters. Pixels are not square or uniformly distributed across the imaged scene. Interestingly, it is possible to preserve the spatial and spectral integrity of hyperspectral imaging data using a point cloud end product such as the DHPC. Based on the findings from our lab, the presented DHPC data format has the potential to push the boundaries of hyperspectral imaging data analysis, allowing for applications that may not be possible using conventional raster end products.

Mentioned Studies:

Inamdar, D., Kalacska, M., Leblanc, G., & Arroyo-Mora, J.P. (2020). Characterizing and Mitigating Sensor Generated Spatial Correlations in Airborne Hyperspectral Imaging Data. Remote Sensing, 12, 641 Inamdar, D., Kalacska, M., Arroyo-Mora, J.P., & Leblanc, G. (2021a). The Directly-Georeferenced Hyperspectral Point Cloud: Preserving the Integrity of Hyperspectral Imaging Data. Frontiers in Remote Sensing, 2 Inamdar, D., Kalacska, M., Leblanc, G., & Arroyo-Mora, J.P. (2021b). Implementation of the directly-georeferenced hyperspectral point cloud. MethodsX, 8, 101429