ATHexapod Position Out of Limits

Overview

When running scripts that require ATAOS corrections (e.g., atpneumatics_checkout.py or enable_ataos_corrections.py, ATHexapod may fail to apply corrections due to its position being outside of the limits.

Error diagnosis

  1. ATHexapod CSC produces the following error in its message log:

    ATHexapod CSC Error Message
    Position out of limits (7)

  2. Should the ATAOS CSC go into FAULT, the following error will display in from the current script in ATQueue:

    ATAOS Script Failure Message
        File
"/opt/lsst/software/stack/conda/envs/lsst-scipipe-9.0.0/lib/python3.11/site-
packages/lsst/ts/salobj/topics/remote_command.py", line 191, in next_ackcmd
    raise
base.AckError(msg="Command failed", ackcmd=ackcmd) lsst.ts.salobj.base.AckError: msg='Command
failed', ackcmd=(ackcmd private_seqNum=493863606, ack=<SalRetCode.CMD_FAILED: -302>, error=1,
result="Failed: disableCorrection not allowed in Fault state: errorCode=8103,
errorReport='Correction loop died.'")

    And the ATAOS CSC produces the following error in its message log:

    ATAOS CSC Error Message
    Correction loop died.

Procedure Steps

Case 1: Only ATHexapod Faults

  1. Reset the position of the ATHexapod loading a run_command.py SAL script into ATQueue with the following configuration:

    component: ATHexapod
cmd: moveToPosition
parameters:
  x: 0
  y: 0
  z: 0
  u: 0
  v: 0
  w: 0

  2. Load enable_ataos_corrections.py to re-enable the ATAOS corrections.

    • Run this script only at high elevations \((\geq 70^\circ)\).

  3. Verify ATHexapod is operating properly by running atpneumatics_checkout.py.

Case 2: ATHexapod & ATAOS Fail

  1. Send ATAOS CSC to OFFLINE using the set_summary_state.py script with the following configuration:

    data:
  - [ATAOS, OFFLINE]

  2. Reset the ATAOS CSC on ArgoCD by following the instructions for Restoring a CSC from the OFFLINE state.

  3. Send ATAOS CSC back to ENABLED with another set_summary_state.py script:

    data:
  - [ATAOS, ENABLED]

  4. Load enable_ataos_corrections.py to re-enable the ATAOS corrections.

    • Run this script only at high elevations \((\geq 70^\circ)\).

  5. Verify ATHexapod is operating properly by running atpneumatics_checkout.py.

Post-Condition

  • The ATHexapod and ATAOS CSCs are successfully in the ENABLED state.

  • ATAOS corrections are enabled without causing faults to the CSCs.

  • The ATCS system is fully operational and ready to proceed with observations or further procedures.

Contingency

If the procedure was not successful, report the issue in #summit-auxtel and/or activate the Out of hours support.