Operational Risk Factors in Computer Vision#

Operational risk factors in computer vision refer to real-world conditions and system-level variables that can degrade the performance of vision algorithms once deployed. These risks can stem from environmental conditions, sensor limitations, data mismatches, or system integration challenges.

Note: Some operational risks arise from system-level input issues (such as metadata mismatches, labeling errors, or other integration defects) that do not fall directly under the five root-cause categories shown in the figure above.

The following table provides a summary of risk factors. Where applicable, a T&E guide providing a detailed example is listed as well as functionality from NRTK that can be used to simulate the operational risk.

Some of these perturbation functions are not implemented in NRTK directly but can be simulated using the AlbumentationsPerturber which provides a wrapper around functionality of the Albumentations library. The Apply Albumentations Perturbations notebook shows how to use Ablumentations with NRTK.

Some of the risk factors listed don’t have any associated functionality or documentation in NRTK but may be covered in the future.

Interactive Risk Matrix#

Operational Risk	NRTK Perturbation(s)	Key Parameters	Severity	dropdown
Extreme Illumination	Brightness	factor	low	Photometric
High Frequency Vibration	Jitter	s_x, s_y (jitter amplitudes)	high	Optical
Lens Water Droplet	WaterDroplet	size_range, num_drops	high	Environment
Mist, Fog, or Snow	Haze	factor, depth_map	high	Environment
Noise	Scikit-image	seed, amount	low	Optical
Radial Distortion	RadialDistortion	k (distortion coefficients)	low	Optical
Resolution/Optics	pyBSM	f, D, p (in sensor)	low	Optical
Target out of Focus	Defocus	w_x, w_y (strength)	low	Optical
Turbulence	TurbulenceAperture	altitude, D, int_time	high	Optical

Photometric Risk Factors#

Extreme (Low / High) Illumination#

Lighting conditions and camera settings result in excessive or insufficient illumination.

Impact	Image has low contrast or dynamic range, reducing usefulness.
Root Cause	Target
Affected Domains	All
NRTK Perturbation	`BrightnessPerturber`
Learn More	Extreme Illumination Simulation Module

../_images/illumination-1.jpg — doers-brc@kitware.com#

../_images/illumination-2.jpg — doers-brc@kitware.com#

Shadows#

Strong shadows are cast in the target area due to direct illumination.

Impact	Features of interest in shadows may be undetectable.
Root Cause	Target
Affected Domains	All

Glint / Glare#

Bright reflections due to lighting, target materials, or angles.

Impact	Can obscure targets and skew autoexposure or detection.
Root Cause	Target
Affected Domains	All

../_images/glare.png — “A data set for airborne maritime surveillance environments”, Ribeiro et al., IEEE Trans. Circuits & Systems for Video Technology, 2017#

Night Mode / Low-Light Behavior#

In low light, camera may switch to monochrome or a different capture mode.

Impact	Color data lost; resolution may be reduced slightly.
Root Cause	Sensor
Affected Domains	Ground, Sea

../_images/night-mode-1.jpg — mevadata.org#

../_images/night-mode-2.jpg — mevadata.org#

Geometric Risk Factors#

Look Angle Different from Training Data#

Operational viewpoint differs from viewpoints in training data.

Impact	Model performance degrades due to lack of viewpoint coverage.
Root Cause	Inferencing
Affected Domains	UAV, WAMI, Satellite

No sample available.

Environment Risk Factors#

Mist / Fog / Snow / Etc#

Reduced visibility conditions such as fog, mist, or blowing snow decrease contrast and obscure scene details. These effects commonly degrade computer-vision performance in outdoor environments by making targets harder to distinguish from the background.

Impact of Risk Factor	Lower contrast and partial occlusion of features.
Root Cause	Optical Path
Affected Domains	Ground, Sea
NRTK Perturbation	`HazePerturber`
Learn More	Haze Simulation Module

../_images/mist.png — *Source: mevadata.org*#

Water Droplets on Lens#

Water droplets create localized refraction patterns that distort image regions, making targets harder to detect and track. These effects are particularly problematic in outdoor and maritime environments where lens contamination is common.

Impact	Obscured or out-of-focus image regions; specularities may confuse algorithms.
Root Cause	Sensor
Affected Domains	Ground, Sea
NRTK Perturbation	`WaterDropletPerturber`
Learn More	Water Droplets Simulation Module

../_images/droplets-2.png — mevadata.org#

Dirt / Specularities on Lens#

Obstructions or smudges on the lens surface can block parts of the scene or create bright reflective artifacts, especially in infrared (IR) imaging or Pan-Tilt-Zoom (PTZ) dome cameras.

Impact	Obscured or out-of-focus image regions; specularities may confuse algorithms.
Root Cause	Sensor
Affected Domains	Ground, Sea

../_images/droplets.png — *Source: mevadata.org*#

Clouds#

Clouds obscure targets, and may be transient or unpredictable.

Impact	Targets not visible or have reduced contrast.
Root Cause	Optic Path
Affected Domains	UAV, WAMI, Satellite

Optical Risk Factors#

High-Frequency Vibration#

Vibrations in the sensor platform (e.g. from wind) induce jitter and blurring.

Target Out of Focus#

Target is out of focus (due to sensor optics settings, rather than atmospheric / environmental issues.)

../_images/out-of-focus.png — mevadata.org#

Sensor Noise#

The sensor data exhibits noise as a result of poor lighting, high ISO settings, or overheating.

../_images/sensor_dark_current_sample.png

Atmospheric Turbulence#

Localized distortion due to atmospheric conditions.

../_images/turbulence.gif — doers-brc@kitware.com#

Radial Distortion / Fisheye Artifacts#

Wide-angle lenses cause distortion at the image periphery.

../_images/radio-distortion.png — mevadata.org#

Data Integrity Risk Factors#

Metadata Incorrect#

Metadata stream is out of sync or contains incorrect values.

Impact	Algorithms may use incorrect models or misinterpret data.
Root Cause	Labeling / Operating input
Affected Domains	All

No sample available.

Burned-in Metadata#

Metadata is overlaid directly on pixels instead of provided separately.

Impact	Obscures target pixels and confuses detection or stabilization algorithms.
Root Cause	Sensor
Affected Domains	All

../_images/metadata-burn.png — Example UAV frame from FFMPEG project#

Video Codec Artifacts#

Compression errors from overloaded camera processors or poor settings.

Impact	Visual glitches such as smearing or pixel corruption.
Root Cause	Inter-frame
Affected Domains	Ground, Sea, UAV

../_images/video-artifacts.gif — mevadata.org#

Video Feed Failures#

Hardware or transmission issues interrupt video feed.

Impact	Causes disruption of object tracking or pipeline shutdown.
Root Cause	Inter-frame
Affected Domains	Ground, Sea, UAV

../_images/overheat.jpg — Camera overheating, doers-brc@kitware.com#

Unstable Frame Rates#

Feed is encoded at inconsistent rates, often due to overload.

Impact	May drop or duplicate frames, confusing motion-based algorithms.
Root Cause	Inter-frame
Affected Domains	Ground, Sea, UAV

../_images/frame-rate.gif — mevadata.org#

Shot Boundary#

Sudden camera motion creates a new view, invalidating prior context.

Impact	Detectors and trackers need to restart.
Root Cause	Inter-frame
Affected Domains	Ground, Sea

../_images/shot-boundary.gif — mevadata.org#