Testing & Evaluation (T&E) Guides

The following Testing & Evaluation (T&E) guides demonstrate how NRTK-provided perturbations can be applied to test imagery and their impact on model performance measured via the MAITE evaluation workflow. Each guide is provided in the form of a notebook and follows a similar structure that demonstrates how a T&E engineer might configure an end-to-end natural robustness evaluation to assess a particular operational risk.

Below are a list of notebooks with a short description:

• nrtk_brightness_perturber_demo modifies illumination conditions using NRTK's BrightnessPerturber.

• nrtk_focus_perturber_demo modifies visual focus using NRTK's DefocusOTFPerturber.

• nrtk_sensor_transformation_demo modifies the resolution and noise of a sensor using NRTK's PybsmPerturber.

• nrtk_translation_perturber_demo performs random translations of the image using NRTK's RandomTranslationPerturber.

• nrtk_turbulence_perturber_demo modifies atmospheric turbulence using NRTK's TurbulenceApertureOTFPerturber.