0
# HAP Processing
1

2
Advanced processing capabilities for Hubble Advanced Products (HAP), including sophisticated catalog management, source detection, automated quality assessment, and standardized product generation. The HAP framework provides enhanced algorithms and quality control for systematic HST data processing.
3

4
## Capabilities
5

6
### Product Management
7

8
Hierarchical product classes for managing different types of HAP outputs with standardized metadata and quality metrics.
9

10
```python { .api }
11
class HAPProduct:
12
    """
13
    Base class for HAP product management and mosaicing.
14

15
    Parameters:
16
    - prop_id: str, proposal ID
17
    - obset_id: str, observation set ID
18
    - instrument: str, HST instrument name
19
    - detector: str, detector identifier
20
    - aperture_from_poller: str, aperture from poller system
21
    - filename: str, output filename
22
    - filetype: str, product file type
23
    - log_level: int, logging level
24

25
    Methods:
26
    - generate(): Create product from input data
27
    - validate(): Perform quality validation
28
    - write(): Save product to file system
29
    """
30

31
class TotalProduct(HAPProduct):
32
    """
33
    Total detection products combining all filters.
34

35
    Attributes:
36
    - total_exposure_time: Combined exposure time
37
    - filters: List of contributing filters
38
    - n_exposures: Number of input exposures
39
    """
40

41
class FilterProduct(HAPProduct):
42
    """
43
    Filter-specific products for individual filters.
44

45
    Attributes:
46
    - filter_name: HST filter identifier
47
    - central_wavelength: Filter central wavelength
48
    - bandwidth: Filter bandwidth
49
    """
50

51
class ExposureProduct(HAPProduct):
52
    """
53
    Single exposure products for individual observations.
54

55
    Attributes:
56
    - rootname: HST observation rootname
57
    - exptime: Exposure time
58
    - date_obs: Observation date
59
    """
60

61
class GrismExposureProduct(HAPProduct):
62
    """
63
    Grism-specific exposure products for spectroscopy.
64

65
    Attributes:
66
    - grism: Grism identifier
67
    - orient: Grism orientation angle
68
    - dispersion: Dispersion solution
69
    """
70

71
class SkyCellExposure(HAPProduct):
72
    """
73
    Sky cell exposure products for wide-area surveys.
74

75
    Attributes:
76
    - skycell_id: Sky cell identifier
77
    - ra_center: Right ascension center
78
    - dec_center: Declination center
79
    """
80

81
class SkyCellProduct(HAPProduct):
82
    """
83
    Sky cell combination products.
84

85
    Attributes:
86
    - cell_size: Angular size of sky cell
87
    - n_visits: Number of contributing visits
88
    - coverage_map: Exposure coverage information
89
    """
90
```
91

92
### Advanced Catalog Management
93

94
Comprehensive catalog system for source detection, photometry, and quality assessment across multiple HST instruments and observing modes.
95

96
```python { .api }
97
class HAPCatalogs:
98
    """
99
    Main catalog management class for HAP processing.
100

101
    Methods:
102
    - identify(): Identify sources in image
103
    - measure(): Perform aperture photometry
104
    - filter(): Apply quality filters
105
    - annotate(): Add metadata and flags
106
    - write(): Save catalog to standard formats
107
    """
108

109
class HAPCatalogBase:
110
    """
111
    Base class for HAP catalogs with common functionality.
112

113
    Attributes:
114
    - catalog_type: Type of catalog (point, segment, etc.)
115
    - n_sources: Number of detected sources
116
    - columns: List of catalog column names
117
    """
118

119
class HAPPointCatalog(HAPCatalogBase):
120
    """
121
    Point source catalog management for stellar objects.
122

123
    Methods:
124
    - detect_sources(): Find point-like sources
125
    - measure_aperture(): Aperture photometry
126
    - measure_psf(): PSF fitting photometry
127
    - compute_colors(): Calculate color indices
128
    """
129

130
class HAPSegmentCatalog(HAPCatalogBase):
131
    """
132
    Extended source catalog management for galaxies.
133

134
    Methods:
135
    - segment_image(): Create segmentation map
136
    - measure_morphology(): Morphological parameters
137
    - measure_photometry(): Extended source photometry
138
    - deblend_sources(): Separate blended objects
139
    """
140

141
class CatalogImage:
142
    """
143
    Image processing for catalog generation.
144

145
    Methods:
146
    - compute_background(): Background estimation
147
    - detect_threshold(): Compute detection threshold
148
    - create_segmap(): Generate segmentation map
149
    - measure_properties(): Extract source properties
150
    """
151
```
152

153
### Astrometric Processing
154

155
Enhanced astrometric capabilities using external catalogs and advanced fitting algorithms for improved positional accuracy.
156

157
```python { .api }
158
def create_astrometric_catalog(inputs, catalog="GAIAedr3", output="ref_cat.ecsv",
159
                              existing_wcs=None, **kwargs):
160
    """
161
    Create reference catalogs from external astrometric sources.
162

163
    Parameters:
164
    - inputs: list, input images for astrometric processing
165
    - catalog: str, reference catalog ('GAIADR2', 'GAIADR3', 'GSC241', etc.)
166
    - output: str, output reference catalog filename
167
    - existing_wcs: WCS, existing WCS for catalog extraction
168
    - **kwargs: dict, catalog query parameters
169

170
    Returns:
171
    str, path to created reference catalog
172
    """
173

174
def build_reference_wcs(inputs, sciname='sci'):
175
    """
176
    Build reference WCS from input images.
177

178
    Parameters:
179
    - inputs: list, input image list
180
    - sciname: str, science extension name
181

182
    Returns:
183
    WCS, combined reference world coordinate system
184
    """
185

186
def get_catalog(ra, dec, sr=0.1, epoch=None, catalog='GSC241'):
187
    """
188
    Retrieve astrometric catalogs from external sources.
189

190
    Parameters:
191
    - ra: float, right ascension center (degrees)
192
    - dec: float, declination center (degrees)
193
    - sr: float, search radius (degrees)
194
    - epoch: float, catalog epoch for proper motion correction
195
    - catalog: str, catalog identifier
196

197
    Returns:
198
    Table, astrometric source catalog
199
    """
200

201
def extract_sources(img, dqmask=None, fwhm=3.0, threshold=None, **kwargs):
202
    """
203
    Extract sources from images for astrometric alignment.
204

205
    Parameters:
206
    - img: ndarray, input image data
207
    - dqmask: ndarray, data quality mask
208
    - fwhm: float, expected source FWHM in pixels
209
    - threshold: float, detection threshold
210
    - **kwargs: dict, source detection parameters
211

212
    Returns:
213
    Table, extracted source catalog
214
    """
215

216
def find_fwhm(psf, default_fwhm, log_level=None):
217
    """
218
    Determine FWHM values for source detection.
219

220
    Parameters:
221
    - psf: ndarray, point spread function image
222
    - default_fwhm: float, default FWHM if measurement fails
223
    - log_level: int, logging level
224

225
    Returns:
226
    float, measured or default FWHM value
227
    """
228

229
def perform_align(input_list, catalog_list, num_sources, archive=False,
230
                  clobber=False, debug=False, update_hdr_wcs=False,
231
                  result=None, runfile="temp_align.log",
232
                  print_fit_parameters=True, print_git_info=False,
233
                  output=False, headerlet_filenames=None, fit_label=None,
234
                  product_type=None, **alignment_pars):
235
    """
236
    Perform a posteriori astrometric fits to input images.
237

238
    Parameters:
239
    - input_list: list, input image filenames for alignment
240
    - catalog_list: list, reference catalog filenames
241
    - num_sources: int, number of sources for fitting
242
    - archive: bool, archive headerlets to database
243
    - clobber: bool, overwrite existing files
244
    - debug: bool, enable debug output
245
    - update_hdr_wcs: bool, update WCS in image headers
246
    - result: dict, results container for fit statistics
247
    - runfile: str, alignment log filename
248
    - print_fit_parameters: bool, print fitting parameters
249
    - print_git_info: bool, print software version information
250
    - output: bool, save detailed alignment products
251
    - headerlet_filenames: list, custom headerlet filenames
252
    - fit_label: str, label for the fit solution
253
    - product_type: str, type of product being aligned
254
    - **alignment_pars: dict, additional alignment parameters
255

256
    Returns:
257
    AlignmentTable, object containing alignment results and fit statistics
258
    """
259
```
260

261
### Image Processing and Alignment
262

263
Advanced image processing capabilities with multiple alignment algorithms and quality assessment.
264

265
```python { .api }
266
class HAPImage:
267
    """
268
    HAP image representation and processing.
269

270
    Attributes:
271
    - filename: str, image filename
272
    - instrument: str, HST instrument name
273
    - detector: str, detector identifier
274
    - filter: str, filter name
275
    - exptime: float, exposure time
276

277
    Methods:
278
    - load(): Load image data and metadata
279
    - calibrate(): Apply instrument calibrations
280
    - align(): Perform astrometric alignment
281
    - combine(): Combine with other images
282
    """
283

284
class SBCHAPImage(HAPImage):
285
    """
286
    SBC-specific image processing class for ACS Solar Blind Channel.
287

288
    Methods:
289
    - correct_distortion(): Apply SBC-specific distortion corrections
290
    - handle_low_counts(): Special processing for low count rate images
291
    """
292

293
class AlignmentTable:
294
    """
295
    Manage alignment solutions and transformations.
296

297
    Methods:
298
    - add_fit(): Add alignment fit results
299
    - get_fit(): Retrieve fit parameters for image
300
    - apply_fit(): Apply transformation to coordinates
301
    - write(): Save alignment table to file
302
    - read(): Load alignment table from file
303
    """
304
```
305

306
### Quality Assessment and Flagging
307

308
Comprehensive quality flagging system for automated source and image quality assessment.
309

310
```python { .api }
311
def run_source_list_flagging(drizzled_image, flt_list, param_dict,
312
                           log_level=None):
313
    """
314
    Main interface for source quality flagging.
315

316
    Parameters:
317
    - drizzled_image: str, combined image filename
318
    - flt_list: list, input FLT files used in combination
319
    - param_dict: dict, flagging parameters
320
    - log_level: int, logging verbosity level
321

322
    Returns:
323
    dict, flagging results and statistics
324
    """
325

326
def ci_filter(drizzled_image, catalog_name, catalog_data, **kwargs):
327
    """
328
    Concentration index filtering for stellar classification.
329

330
    Parameters:
331
    - drizzled_image: str, reference image
332
    - catalog_name: str, catalog identifier
333
    - catalog_data: Table, source catalog
334
    - **kwargs: dict, concentration index parameters
335

336
    Returns:
337
    ndarray, boolean mask for point-like sources
338
    """
339

340
def hla_saturation_flags(drizzled_image, flt_list, **kwargs):
341
    """
342
    Saturation flagging for sources.
343

344
    Parameters:
345
    - drizzled_image: str, combined image
346
    - flt_list: list, contributing exposure list
347
    - **kwargs: dict, saturation flagging parameters
348

349
    Returns:
350
    ndarray, saturation flags for sources
351
    """
352

353
def hla_swarm_flags(drizzled_image, catalog_name, **kwargs):
354
    """
355
    Swarm source identification and flagging.
356

357
    Parameters:
358
    - drizzled_image: str, reference image
359
    - catalog_name: str, catalog identifier
360
    - **kwargs: dict, swarm detection parameters
361

362
    Returns:
363
    ndarray, boolean flags for swarm sources
364
    """
365

366
def hla_nexp_flags(drizzled_image, flt_list, **kwargs):
367
    """
368
    Exposure count flagging for source reliability.
369

370
    Parameters:
371
    - drizzled_image: str, combined image
372
    - flt_list: list, contributing exposure list
373
    - **kwargs: dict, exposure count parameters
374

375
    Returns:
376
    ndarray, flags based on exposure coverage
377
    """
378
```
379

380
## Usage Examples
381

382
### Basic HAP Processing
383

384
```python
385
from drizzlepac.haputils import catalog_utils, product
386

387
# Create HAP product for filter-specific processing
388
filter_product = product.FilterProduct()
389
filter_product.configobj_pars.input = ['image1_flt.fits', 'image2_flt.fits']
390
filter_product.configobj_pars.filter = 'F606W'
391

392
# Generate product
393
filter_product.generate()
394

395
# Create point source catalog
396
point_catalog = catalog_utils.HAPPointCatalog(filter_product.drizzle_filename,
397
                                            catalog_type='point')
398
point_catalog.identify()
399
point_catalog.write()
400
```
401

402
### Advanced Astrometric Processing
403

404
```python
405
from drizzlepac.haputils import astrometric_utils
406

407
# Create reference catalog from Gaia
408
ref_catalog = astrometric_utils.create_astrometric_catalog(
409
    inputs=['obs1_flt.fits', 'obs2_flt.fits'],
410
    catalog='GAIADR3',
411
    output='gaia_reference.ecsv',
412
    mag_limit=20.0
413
)
414

415
# Extract sources for alignment
416
source_catalog = astrometric_utils.extract_sources(
417
    image_data,
418
    fwhm=2.5,
419
    threshold=5.0,
420
    deblend_nthresh=32
421
)
422
```
423

424
### Quality Assessment Workflow
425

426
```python
427
from drizzlepac.haputils import hla_flag_filter
428

429
# Run comprehensive quality flagging
430
flag_results = hla_flag_filter.run_source_list_flagging(
431
    'combined_drz.fits',
432
    ['exp1_flt.fits', 'exp2_flt.fits'],
433
    param_dict={
434
        'ci_lower_limit': 1.3,
435
        'ci_upper_limit': 2.0,
436
        'sat_limit': 60000.0
437
    }
438
)
439

440
# Apply individual quality filters
441
ci_flags = hla_flag_filter.ci_filter('combined_drz.fits',
442
                                   'point_catalog.ecsv',
443
                                   catalog_data)
444

445
sat_flags = hla_flag_filter.hla_saturation_flags('combined_drz.fits',
446
                                               flt_list)
447
```
448

449
### Multi-Visit Processing
450

451
```python
452
from drizzlepac.haputils import product, align_utils
453

454
# Process multiple visits for sky cell
455
skycell_product = product.SkyCellProduct()
456
skycell_product.input_list = ['visit1', 'visit2', 'visit3']
457
skycell_product.skycell_id = 'sc_001'
458

459
# Align all visits to common reference frame
460
alignment_table = align_utils.AlignmentTable()
461
for visit in skycell_product.input_list:
462
    fit_results = align_utils.match_default_fit(visit, ref_catalog)
463
    alignment_table.add_fit(visit, fit_results)
464

465
# Generate combined sky cell product
466
skycell_product.generate()
467
```
468

469
## HAP Configuration
470

471
### Processing Parameters
472

473
Key parameters for HAP processing:
474

475
```python
476
hap_params = {
477
    'catalog_types': ['point', 'segment'],
478
    'aperture_ee': [70, 80, 85],  # Encircled energy apertures
479
    'ci_lower_limit': 1.3,       # Concentration index limits
480
    'ci_upper_limit': 2.0,
481
    'quality_control': True,
482
    'write_catalogs': True
483
}
484
```
485

486
### Quality Control Thresholds
487

488
```python
489
quality_params = {
490
    'min_sources': 10,           # Minimum sources for processing
491
    'max_rms': 0.1,             # Maximum alignment RMS (arcsec)
492
    'sat_limit': 60000.0,       # Saturation threshold (DN)
493
    'coverage_limit': 0.5        # Minimum exposure coverage
494
}
495
```
496

497
## Output Products
498

499
### Catalog Formats
500

501
HAP catalogs are saved in multiple formats:
502
- **ECSV**: Enhanced CSV with metadata
503
- **FITS**: Binary table format
504
- **ASCII**: Human-readable text format
505

506
### Standard Columns
507

508
#### Point Source Catalogs
509
- **X_IMAGE, Y_IMAGE**: Pixel coordinates
510
- **RA, DEC**: Sky coordinates (degrees)
511
- **MAG_AUTO**: Automatic aperture magnitude
512
- **MAGERR_AUTO**: Magnitude error
513
- **CI**: Concentration index
514
- **FLAGS**: Quality flags
515

516
#### Segment Catalogs
517
- **KRON_RADIUS**: Kron radius
518
- **ELLIPTICITY**: Source ellipticity
519
- **THETA_IMAGE**: Position angle
520
- **A_IMAGE, B_IMAGE**: Semi-major/minor axes
521

522
## Error Handling
523

524
### HAP-Specific Exceptions
525

526
- **ProductError**: Product generation failures
527
- **CatalogError**: Catalog processing failures
528
- **AlignmentError**: Astrometric alignment failures
529
- **QualityError**: Quality assessment failures
530

531
### Recovery Strategies
532

533
```python
534
from drizzlepac.haputils import product
535

536
try:
537
    hap_product = product.FilterProduct()
538
    hap_product.generate()
539
except product.ProductError as e:
540
    print(f"Product generation failed: {e}")
541
    # Implement fallback or retry logic
542
```
543

544
## Best Practices
545

546
### HAP Processing Workflow
547
1. **Validate input data** before processing
548
2. **Use appropriate catalog types** for science goals
549
3. **Apply quality filters** consistently
550
4. **Document processing parameters** for reproducibility
551
5. **Validate output products** before distribution
552

553
### Performance Optimization
554
- **Process in chunks** for large datasets
555
- **Use parallel processing** when available
556
- **Monitor memory usage** for large images
557
- **Cache intermediate results** for repeated processing