AuxTel Image Processing¶

Overview¶

The image processing of Rubin Observatory images is highly complex. This document is intended to give a high level overview for users of the Auxiliary Telescope so they can understand the basic steps and assist with troubleshooting.

Overview of the main image processing steps¶

../../_images/Block_diagram.png — Block diagram of the image processing steps.¶

This figure shows the path of images taken by the Auxiliary telescope. After being commanded by the script queue, the CCS code under the control of ATCamera takes the requested image. The CCS code writes a FITS file (which contains the image data) and a json file which contains the camera related data for the FITS header. These two files are then passed to ATHeaderService, which places the data from the json file, as well as observatory data, into the FITS header of the final FITS file. The ATOODS CSC then places this FITS file in a storage location at the summit. These FITS files are only stored at the summit for about 1 month, after which they are deleted. A separate piece of code also copies this FITS file to a permanent storage location at USDF (the US Data Facility, in California). At both locations, these FITS files are “ingested”, which means they are read into a piece of code called the Butler, which makes it possible to quickly access the data you want from the huge amount of data that will eventually be stored.

For quick feedback to the summit, a set of Rapid Analysis code performs operations on the images before displaying them in RubinTV. The most important operation is ISR (Instrument Signature Removal). This removes deficiencies in the raw image and generates an image that more accurately reflects the sky. The Monitor image in RubinTV has been thru ISR. The ISR process is quite complex, and more detail of the ISR is available in this pdf file pdf The Rapid Analysis also performs some quick, very basic analysis of the spectrum which is displayed in RubinTV. RubinTV also displays mount errors that occur and in the tables it displays the “Mount motion image degradation” in arcseconds. More detail of the different types of mount errors is available at: https://sitcomtn-125.lsst.io/

The team analyzing the AuxTel spectra is in France. They perform offline analysis of the images. An example of their analysis is shown in this pdf file pdf

Troubleshooting image analysis problems.¶

There are two ways to check the image processing when problems occur. At point A in the block diagram, the images are stored at auxtel-fp01.cp.lsst.org:/data/ats/ccs-ipa. It is possible to log into auxtel-fp01 and verify that the files are being written there, as below. You should see a directory for the current day (like the below 20250401). In that directory, there should be a directory for each exposure. If they are not there, that indicates a problem with ATCamera or with CCS. This is not usually a problem, but if you find the images are not there, contact a camera expert like Tony Johnson or put a note in #auxtel-computing.

ssh <your login>@auxtel-fp01.cp.lsst.org
cd /data/ats/ccs-ipa
ls
cd <20250401>
ls

A more common problem is that images are failing to ingest. This can happen for a number of reasons, like a problem with ATOODS or ATHeaderService, or the ingest code is failing. RubinTV uses the ingested data, so if the images fail to ingest at the summit, they will not display in RubinTV. At point B in the block diagram, we can check whether the images have been ingested. Below is a code snippet to run on the summit-rsp. The output will only display ingested images. If the images have not ingested, contact an expert like Michael Reuter.

To use the summit RSP, do the following:

(1) Access the summit-rsp with Nublado
    (see https://obs-ops.lsst.io/Observing-Interface-Setup/getting-started-nublado.html)
(2) Create a new jupyter notebook, paste in the snippet below and run it.

import lsst.summit.utils.butlerUtils as butlerUtils
butler = butlerUtils.makeDefaultButler("LATISS")
dayObs = 20250319
exposureList = []
for record in butler.registry.queryDimensionRecords("exposure", where="exposure.day_obs=%d"%dayObs):
    exposureList.append([record.id, record])
exposureList.sort(key=lambda x: x[0])
for [id,record] in exposureList:
    print(f"{record.id}\t{record.observation_type}\t{record.exposure_time}\t{record.physical_filter}")

Another way to check on the status of the ingestion is to go to the “LATISS OODS Ingest Status” Chronograf dashboard.