Pixel Calibration

Overview

The pixel calibration component in daolite (Durham Adaptive Optics Latency Inspection and Timing Estimator) handles the preprocessing of raw camera data before wavefront sensing. This preprocessing is essential for accurate wavefront measurements and includes dark frame subtraction, flat fielding, bad pixel correction, and other operations to prepare the raw pixel data for centroiding.

Key Calibration Operations

Dark Subtraction: Removal of dark current and bias
Flat Fielding: Correction for pixel sensitivity variations
Bad Pixel Correction: Handling of defective pixels
Background Estimation: Subtraction of background flux
Thresholding: Application of intensity thresholds
Normalization: Scaling of pixel intensities

Using Calibration Components

daolite makes it easy to add pixel calibration stages to your AO pipeline:

from daolite import Pipeline, PipelineComponent, ComponentType
from daolite.pipeline.calibration import PixelCalibration
from daolite.compute import hardware
import numpy as np

pipeline = Pipeline()

# Define a CPU resource
cpu = hardware.amd_epyc_7763()

# Define pixel agenda (how many pixels per iteration)
n_pixels = 1024 * 1024
n_groups = 10
pixel_agenda = np.ones(n_groups, dtype=int) * (n_pixels // n_groups)

# Add pixel calibration component
pipeline.add_component(PipelineComponent(
    component_type=ComponentType.CALIBRATION,
    name="Pixel Calibration",
    compute=cpu,
    function=PixelCalibration,
    params={
        "pixel_agenda": pixel_agenda,
        "bit_depth": 16,  # 16-bit camera data
        "n_workers": 1
    },
    dependencies=["Camera"]
))

Calibration Configuration

The pixel calibration component accepts the following parameters:

params={
    # Required parameters
    "pixel_agenda": pixel_agenda,  # Processing agenda (np.ndarray)

    # Optional parameters
    "bit_depth": 16,       # Bit depth of pixel data (default: 16)
    "n_workers": 1,        # Number of parallel workers (default: 1)
    "flop_scale": 1.0,     # FLOP scaling factor (default: 1.0)
    "mem_scale": 1.0,      # Memory scaling factor (default: 1.0)
    "debug": False         # Enable debug output (default: False)
}

Understanding Pixel Calibration

The PixelCalibration function models the computational latency of preprocessing raw camera data before wavefront sensing. This includes operations like dark frame subtraction, flat fielding, and other pixel-level corrections.

The function uses an agenda-based API where you specify how many pixels to process in each iteration through the pixel_agenda parameter. This allows modeling of pipelined or batched processing patterns.

Key Factors Affecting Performance

The computational cost of pixel calibration depends on:

Number of Pixels: Scales linearly with pixel count
Bit Depth: Higher bit depths require more memory bandwidth
Memory Bandwidth: Calibration operations are typically memory-bound
Computational Resource: CPU vs GPU implementation affects throughput

Practical Example

Complete Example: Full Pipeline with Calibration

from daolite import Pipeline, PipelineComponent, ComponentType
from daolite.pipeline.camera import PCOCamLink
from daolite.pipeline.calibration import PixelCalibration
from daolite.compute import hardware
import numpy as np

# Create pipeline
pipeline = Pipeline()

# Define compute resources
cpu = hardware.amd_epyc_7763()

# Camera parameters
n_pixels = 1024 * 1024
camera_groups = 10

# Pixel calibration agenda
pixel_agenda = np.ones(camera_groups, dtype=int) * (n_pixels // camera_groups)

# Add camera
pipeline.add_component(PipelineComponent(
    component_type=ComponentType.CAMERA,
    name="Camera",
    compute=cpu,
    function=PCOCamLink,
    params={
        "n_pixels": n_pixels,
        "group": camera_groups,
        "readout": "rolling"
    }
))

# Add pixel calibration
pipeline.add_component(PipelineComponent(
    component_type=ComponentType.CALIBRATION,
    name="Pixel Calibration",
    compute=cpu,
    function=PixelCalibration,
    params={
        "pixel_agenda": pixel_agenda,
        "bit_depth": 16,
        "n_workers": 1
    },
    dependencies=["Camera"]
))

# Run pipeline
results = pipeline.run()
print(f"Calibration time: {results['Pixel Calibration'].duration:.2f} µs")

GPU Acceleration

You can model GPU-accelerated pixel calibration by using a GPU compute resource:

from daolite.compute import hardware

# Use GPU instead of CPU
gpu = hardware.nvidia_rtx_4090()

# Add GPU-accelerated calibration
pipeline.add_component(PipelineComponent(
    component_type=ComponentType.CALIBRATION,
    name="GPU Calibration",
    compute=gpu,  # Use GPU resource
    function=PixelCalibration,
    params={
        "pixel_agenda": pixel_agenda,
        "bit_depth": 16,
        "n_workers": 1
    },
    dependencies=["Camera"]
))

Tuning Performance

You can tune the performance model using the flop_scale and mem_scale parameters:

# Scale computational intensity
pipeline.add_component(PipelineComponent(
    component_type=ComponentType.CALIBRATION,
    name="Tuned Calibration",
    compute=cpu,
    function=PixelCalibration,
    params={
        "pixel_agenda": pixel_agenda,
        "bit_depth": 16,
        "flop_scale": 1.2,  # Increase FLOPs by 20%
        "mem_scale": 0.8,   # Reduce memory ops by 20%
        "n_workers": 4      # Use 4 parallel workers
    },
    dependencies=["Camera"]
))

API Reference

For complete API details, see the Calibration API Reference section.

Chunked Processing

For very large pixel arrays, chunked processing can be more efficient:

# Configure chunked processing
pipeline.add_component(PipelineComponent(
    component_type=ComponentType.CALIBRATION,
    name="Chunked Calibration",
    compute=cpu,
    function=PixelCalibration,
    params={
        "n_pixels": 4096*4096,  # 16 megapixel array
        "operations": ["dark_subtract", "flat_field"],
        "use_chunking": True,
        "chunk_size": 1024*1024  # Process in 1 megapixel chunks
    },
    dependencies=["Camera"]
))

Customizing Calibration Models

daolite allows you to create custom calibration operations with your own timing models:

def custom_calibration(compute, n_pixels, extra_param=1.0, debug=False):
    """
    Custom calibration function with timing model.

    Args:
        compute: Compute resource
        n_pixels: Number of pixels to process
        extra_param: Custom scaling parameter
        debug: Enable debug output

    Returns:
        Numpy array with timing information
    """
    # Model timing based on compute resource
    if compute.hardware == "GPU":
        # GPU timing model
        bytes_per_pixel = 4  # Float32
        total_bytes = n_pixels * bytes_per_pixel * 2  # Read + write
        mem_time = total_bytes / compute.memory_bandwidth

        # GPU kernels typically need some launch overhead
        kernel_overhead = 5.0  # microseconds
        compute_time = n_pixels * 0.5 / compute.flops + kernel_overhead
    else:
        # CPU timing model
        bytes_per_pixel = 4  # Float32
        total_bytes = n_pixels * bytes_per_pixel * 2  # Read + write

        mem_bandwidth = (compute.memory_channels *
                       compute.memory_width *
                       compute.memory_frequency / 8)  # Bytes/sec
        mem_time = total_bytes / mem_bandwidth

        # CPU model with core utilization
        ops_per_pixel = 10  # Example number of operations per pixel
        total_ops = n_pixels * ops_per_pixel
        compute_time = total_ops / (compute.cores *
                                  compute.core_frequency *
                                  compute.flops_per_cycle)

    # Scale by custom parameter
    compute_time *= extra_param

    # Memory or compute bound?
    processing_time = max(mem_time, compute_time)

    if debug:
        print(f"Memory time: {mem_time} µs")
        print(f"Compute time: {compute_time} µs")
        print(f"Total processing time: {processing_time} µs")

    # Create timing array
    timing = np.zeros([1, 2])
    timing[0, 1] = processing_time

    return timing

# Use custom calibration in a pipeline
pipeline.add_component(PipelineComponent(
    component_type=ComponentType.CALIBRATION,
    name="Custom Calibration",
    compute=cpu,
    function=custom_calibration,
    params={
        "n_pixels": 1024*1024,
        "extra_param": 1.2,
        "debug": True
    },
    dependencies=["Camera"]
))

Real-World Applications

Example: High-speed AO System

Calibration configuration for a high-speed adaptive optics system:

# High-speed system with optimized calibration
pipeline.add_component(PipelineComponent(
    component_type=ComponentType.CALIBRATION,
    name="Fast Calibration",
    compute=gpu,  # Use GPU for speed
    function=PixelCalibration,
    params={
        "n_pixels": 128*128,  # Small format sensor
        "operations": ["dark_subtract"],  # Minimal processing
        "use_gpu_kernels": True,
        "use_async": True  # Use asynchronous processing
    },
    dependencies=["Camera"]
))

Example: High-precision System

Calibration configuration for a high-precision AO system:

# High-precision system with comprehensive calibration
pipeline.add_component(PipelineComponent(
    component_type=ComponentType.CALIBRATION,
    name="Precision Calibration",
    compute=cpu,
    function=PixelCalibration,
    params={
        "n_pixels": 2048*2048,
        "operations": ["dark_subtract", "flat_field", "bad_pixel",
                      "background", "threshold", "normalize"],
        "window_size": 5,  # Larger window for better correction
        "bad_pixel_fraction": 0.02,
        "threshold": 50
    },
    dependencies=["Camera"]
))

API Reference

For complete API details, see the Calibration API Reference section.