Cassini Spacecraft-DSN Communications, Handling Anomalous Link Conditions, and Complete Loss-of-Spacecraft Signal

Abstract

Once spacecraft are launched, it is impossible for engineers to physically repair anything that breaks onboard the vehicle. Instead, remote solutions must be employed to address spacecraft anomalies and fault conditions. To achieve this goal, telemetered data from the spacecraft are collected and assess by ground personnel to resolve problems. However, if the ground-to-spacecraft communication system breaks down, or the vehicle delivers an anomalous signal, a rigorous protocol must be employed in order to re-establish or fix the telecommunications link. There are several factors that can contribute to link problems, such as malfunctions or mishandling of the ground station equipment, onboard failures of the spacecraft’s flight software coding, or even mishaps caused by the space environment itself. This chapter details the anomaly recovery protocols developed for the Cassini Mission-to-Saturn project, to resolve anomalous link problems as well re-acquisition of the spacecraft should a complete Loss of Signal (LOS) condition occur.

1. Introduction

Despite the vast distance between remote-controlled interplanetary spacecraft launched from earth and the Deep Space Network (DSN) ground stations that operate them, the communications link to the spacecraft is very reliable, thanks to the extraordinary telecommunication capabilities built into NASA’s DSN antennas around the world and the spacecraft’s own system design. For the Cassini Mission-to-Saturn spacecraft (Figure 1), it takes nearly an hour and a half for commands from the Spacecraft Operations Flight Team (SOFS) here on earth to reach Cassini, where the orbiter is touring the Saturnian system (~8.5 AU). Yet, an anomalous downlink (D/L) signal condition can occur (or complete LOS) from several sources: environmental effects such as bad weather conditions at the DSN station or station problems (broken equipment), erroneous ground commands uplinked (U/L) to the spacecraft by the SOFS team, errors in the onboard running sequence, spacecraft pointing errors, internal FSW errors, or computer platform failures can cause problems when attempting to acquire the spacecraft’s D/L signal. The space environment itself can also contribute to an LOS condition, since cosmic ray bombardment on the spacecraft’s systems can cause spurious Solid State Power Switch (SSPS) trip-off of the spacecraft’s Radio Frequency System (RFS) units, as well activations of the onboard Fault Protection (FP) routines which will reconfigure to redundant backup RFS units, so that reconfiguration by the ground is required in order to lock-up on the spacecraft’s D/L signal.

Figure 1.

The Cassini-Huygens spacecraft.

To safeguard against these DSN-spacecraft link problems, troubleshooting methods have been developed by the Cassini SOFS team to diagnose and resolve conditions that inhibit spacecraft signal acquisition. A “Loss of Downlink Signal Recovery” protocol was developed for the SOFS team to follow in the event of an anomalous D/L signal (or completed LOS), as well as special FP which is implemented into Cassini’s onboard FSW. This algorithm will monitor for prolonged absence of ground commanding, eventually invoking a “Loss of Commandability” FP (FP which is typically implemented into most deep space missions to safeguard against these undetected, sometimes waived or ground-induced failure conditions). Called “Command Loss FP” (from the perspective of the spacecraft since it’s no longer receiving ground commands), this “catch-all” type of autonomous monitor-response algorithm will observe the absence of ground commands for a predetermined (programmable) period of time, using a “countdown timer” which decrements until it is reset by a ground command or reaches “0” (which triggers the response). An extended series of actions are then commanded by FP to re-establish ground commandability by configuring various telecom arrangements and spacecraft attitudes in an attempt to find a viable U/L path. Each attempt by the response to command a new path is separated by an appropriate ground response interval for the SOFS team to re-acquire the spacecraft via U/L command.

In all anomalous spacecraft D/L cases, it is desirable to re-establish spacecraft communications before the Command Loss Response activates in order to avoid the autonomous commanded actions of the FP: termination of the onboard running sequence (lost science opportunities), device swaps, propellant consumption via commanded turns, etc. Therefore, an expedient method for identifying possible anomalous/LOS causes is highly desirable before the FP activates, if possible. To aid in this goal, an Excel tool was developed to supplement the LOS Recovery Protocol in “timeline” format. Described herein are the optimized solutions implemented on Cassini for re-acquisition of the spacecraft’s signal during anomalous D/L and LOS events, as well as an expedient method for recovery from the actions of the Command Loss Response, if activated.

2. The Cassini mission

NASA’s Cassini Mission-to-Saturn spacecraft is the first robotic mission ever to orbit the planet Saturn. Managed by the Jet Propulsion Laboratory (JPL) in Pasadena, this flagship-class mission is composed of 11 operating scientific instruments which study many intriguing features of Saturn, its moons, and ring system. The Cassini Program is an international cooperative effort involving primarily NASA, the European Space Agency (ESA), and the Italian Space Agency (Agenzia Spaziale Italiana, ASI). Cassini is the fourth spacecraft to visit the Saturnian system (but is the first vehicle to enter its orbit), and is composed of the NASA/ASI Cassini orbiter and the ESA-developed Huygens probe. Cassini launched on October 15, 1997, arriving at Saturn in 2004, after performing scientific observation of Earth’s moon, Venus, and Jupiter (as well as participating in several scientific experiments) during its 6.7 year cruise period. Cassini’s suite of (currently operating) science instruments consists of the following (Figure 2):

1. Composite Infrared Spectrometer (CIRS)

2. Ion & Neutral Mass Spectrometer (INMS)

3. Visible & Infrared Mapping Spectrometer (VIMS)

4. Ultraviolet Imaging Spectrograph (UVIS)

5. Imaging Science Subsystem (ISS)

6. Magnetospheric Imaging Instrument (MIMI)

7. Dual Technique Magnetometer (MAG)

8. Cosmic Dust Analyzer (CDA)

9. Radio & Plasma Wave Science instrument (RPWS)

10. Radio Science Subsystem (RSS)

11. Radar

Figure 2.

Cassini’s instrument suite.

Also included onboard Cassini is the Huygens Probe; an atmospheric laboratory designed to collect data in the Titan Moon atmosphere and its surface. Deployed in January 2005, the probe consisted of six scientific instruments which performed experiments in aerosol collection, descent imaging & spectral radiometry, gas chromatography & mass spectrometry, atmospheric sampling, and surface science. The entire Cassini mission consists of seven phases:

• Launch and initial acquisition of the spacecraft (October 15, 1997)

• Inner cruise (beginning October 20, 1997)

• Outer cruise (beginning February 2000)

• Science cruise (starting July 2002)

• Saturn Orbit Insertion (SOI; July 2004)

• Huygens Probe Release (January 2005)

• Saturn Tour continues (2004–2017)

During the cruise portion of the journey to the Saturnian system, two gravity assist maneuvers were required from Venus, one from Earth, and one from Jupiter. Until Cassini reached 2.7 AU from the sun (during the inner cruise phase), communications between earth and the spacecraft were accomplished via the Low Gain Antenna (LGA), since the 4-m diameter High Gain Antenna (HGA) must be used to shield the spacecraft from the sun’s heating (i.e. used as a sunshade). After reaching this distance (begin Outer cruise phase), communications begin on the earth-pointed HGA.

Cassini’s “Prime Tour Mission” began in 2004, where planet/moon science investigation activities continued until 2008. Two mission extensions were granted: the “Equinox Mission” from 2008 to 2010, and the “Solstice Mission” from 2010 to 2017 (Figure 3, [1]). The spacecraft’s 20 year mission ends with 42 orbits around the main ring system (Figure 4, [2]). Beginning on November 30, 2016, Cassini’s orbit reoriented the spacecraft to the outer edge of the main rings to perform a series of 20 F-Ring orbits; a region of Saturn’s rings which look like an odd “interwoven” structure. The last time that Cassini observed these rings close-up was at Saturn arrival in 2004, which allowed observation of only the dim, backlit side. But in November of 2016, numerous opportunities became available to examine the F-Ring’s structure, with high-resolution observation of both sides of the F-Ring. The final mission phase called “The Grand Finale” began in April 2017 with a close flyby of Saturn’s giant moon Titan, which provided re-orientation of the spacecraft’s trajectory, allowing it to pass through the gap between Saturn and the D-Ring; the closest ring to the planet. With only a 1500 mile-wide corridor to fly through, Cassini will investigate this unexplored region of the Saturnian system, making the closest observations of Saturn to date. During these last 22 (D-Ring) orbits of the Cassini mission, the planet’s magnetic and gravity fields will be mapped with high precision, and extremely close views of the atmosphere will be observed. New insights into Saturn’s interior structure, the precise length of a Saturnian day, and the age and total mass of the rings will also be evaluated. On September 15, 2017, Cassini will end its 20 year mission with a fiery plunge into Saturn, providing valuable data about the planet’s chemical composition as the friction forces (from the atmospheric entry) cause the vehicle to burn up, thus satisfying Planetary Protection requirements [3].

Figure 3.

Cassini’s prime, equinox XM, & solstice XXM tours.

Figure 4.

Cassini’s end-of-Mission F & D Ring Orbits.

3. The Cassini radio communications system

Cassini’s onboard telecommunications system consists of three antennas: a High-Gain Antenna and two Low-Gain Antennas (LGA-1 & LGA-2); all which interface with the RFS system (which performs command, telemetry, and radio-metric communications) and Radio Frequency Instrument Subsystem (RFIS); Figure 5. Cassini’s 4-m Cassegrain HGA communicates with earth on X-band, and on S-Band with the Huygens probe (and radio-science). It also communicates on Ka-band to support radio science activities, and Ku-band for the imaging RADAR subsystem. The two LGA antennas operate on X-band only, with LGA-1 mounted on the top of the HGA (giving it an unobstructed field of view of 112°), and LGA-2 which is mounted on a boom below the Huygens probe near the bottom of the vehicle, yielding a 120° field of view. The LGA antennas were used for communication with the ground when the HGA could not be configured on earth-point due to thermal constraints (when in close proximity to the sun). In this case, the spacecraft had to be shielded by the HGA, leaving the LGA antennas to transmit and receive data at very low delivery rates. The LGA antennas are configured when FP executes.

Figure 5.

The Cassini spacecraft.

Spacecraft are typically equipped with transmitters of relatively low radiating power for communication with earth (20 Watts for Cassini). This telecommunications link must bridge the distance of over a billion kilometers (earth-Saturn distance), which is achieved by employing frequencies in the microwave range using reflectors onboard the spacecraft to concentrate all available power into a narrow beam pointed precisely towards earth. Cassini’s HGA is used to achieve this goal (as opposed to the LGA antennas which sacrifice gain but provide relatively uniform coverage over a wide range of spacecraft orientation angles). At the DSN station, large aperture Cassegrain reflectors are used to pick up the spacecraft’s signal. These radio antennas use cryogenically cooled (low-noise) amplifiers to first amplify the faint spacecraft signal, followed by sophisticated receivers and decoders which can lock onto and extract the data with virtually with no errors at all.

The signal delivered from the spacecraft to earth’s ground station is called a “downlink,” and the transmission of commands and sequences from the ground to the spacecraft is called an “uplink.” When a D/L signal is received from the spacecraft, the communication is called “one-way” (or if the D/L signal is generated onboard the spacecraft itself, the communication is also called “one-way”). When the U/L signal is being received by the spacecraft at the same time a D/L is being received by the ground station, the communication is called “two-way.” Both U/L and D/L consist of a pure Radio Frequency (RF) tone which is called a “carrier.” In order to carry information to or from the spacecraft, the carrier signal must be “modulated.” A modulated signal may be sent from the ground station to transmit commands to the spacecraft. Likewise, the modulated signal is generated by the spacecraft to transmit science and engineering data to earth on its D/L carrier. The spacecraft’s carrier signal is also used for tracking and navigation (as well as some types of science experiments such as radio science or gravity field mapping). Each DSN complex uses a hydrogen-maser-based frequency unit which is maintained in an environmentally controlled room (in the basement), sustained by an uninterruptable power supply. The maser serves as the reference for generating a precisely known U/L frequency. When an U/L signal is received by the spacecraft, it can choose to use the received U/L carrier to control its D/L carrier transmission (called 2-way coherent transmission). This ground-generated reference frequency is multiplied by a predetermined constant (1.1748999 for Cassini) and the transmitted D/L signal is phase coherent with the U/L signal (this multiplier prevents the D/L signal from interfering with the U/L signal which is being received from the ground). Precise tracking of the spacecraft is accomplished through this method, as well as the ability to carry out high precision science experiments onboard the orbiter.

Cassini carries its own Ultra-Stable Oscillator (USO). During the one-way phase when the spacecraft transmits its signal to the ground (before two-way communication is established), the spacecraft must generate its own D/L signal using the on-board USO. Once the ground’s U/L signal is acquired by the vehicle, it will abandon its own D/L signal to regenerate the D/L, thus changing the frequency. During this time, the ground station will “lose lock” on the spacecraft and must tune in the new frequency. This “out-of-lock” condition is predetermined by the ground (on the order of a minute or two), so that data delivery to the ground is temporarily halted during this transition period, in order to preserve the precious science data. The USO device is quite reliable in generating a stable D/L signal, more so than the 2-way method with the ground, because the ground U/L signal phase is subject to corruption by atmospheric effects, solar wind, etc. Therefore, the USO is more desirable than the hydrogen maser. However, the USO frequency cannot be precisely known if the D/L frequency changes due to relative motion of the spacecraft (as well as vehicle drifting). Since ranging is fundamentally a phase measurement, the ground must use the hydrogen maser referenced U/L along with phase coherent receivers on the spacecraft and on the ground to determine the correct measurement.

4. Cassini mission telecommunications operations in flight

NASA’s DSN is a part of JPL, consisting of a worldwide network of US spacecraft communication facilities. Placed approximately 120° apart around the Earth, three deep-space telecommunications stations are located in Goldstone, California (US), Madrid, Spain, and Canberra, Australia. The placement of these ground stations permits constant observation of spacecraft like Cassini as the Earth rotates. Unlike near-earth orbiters which move quickly round the earth, few ground stations are required to support deep space missions since they are visible for long periods of time. As mentioned before, these earth-based DSN ground stations contain steerable, high-gain, parabolic reflector antennas, providing a two-way communications link that tracks robotic interplanetary spacecraft like Cassini, acquiring telemetry data, transmitting commands, uploading software modifications, tracking spacecraft position and velocity, measuring variations in radio waves to support radio science experiments, and collecting science & engineering data. Interplanetary spacecraft such as Cassini, require huge DSN antennas with ultra-sensitive receivers and powerful transmitters in order to transmit/receive information over the vast earth-planet distances, with the largest antennas of the DSN often called upon during spacecraft emergencies. Nearly all spacecraft are designed to use the smaller DSN antennas (e.g. 34 m diameter) for nominal operations, but for a spacecraft emergency, the largest antennas are typically used (e.g. 70 m diameter) since the onboard FP typically configures low transmitter power, so that recovering any available telemetry is crucial to assessing the spacecraft’s health in preparation for recovery actions. In the case of Cassini, the LGA is configured by FP with very low U/L & D/L rates.

Ground commands from earth travel at the speed of light (referred to as “One-Way Light Time;” OWLT), reaching Cassini from approximately 1 hour. 15 minute to 1 hour 30 minute, depending on the relative distance between earth and Saturn, given the change in relative distance due to the earth’s rotation around the sun and the spacecraft’s motion around the Saturnian system. Therefore, the majority of commands sent to the spacecraft for operations and science investigations must be uplinked to the Command & Data processing System (CDS) computers in large “command sequences,” which consist of several weeks of planned commanding. These sequences typically consist of commanded turns to point Cassini’s 11 operating instruments towards specific targets, providing high precision (down to the sub-milliradian) via two Attitude, Articulation, & Control System (AACS) computers. Captured science data is recorded on two Solid-State Recorders (SSR) during off-earth observation periods. These science activities (e.g. moon and ring encounters) are paused typically once each day (or two) for approximately 9 hours to establish communication with earth (via a scheduled DSN station) to downlink the science & engineering (housekeeping) data.

Once Cassini’s earth-pointed attitude is stabilized, its D/L signal is received by the DSN station. Ten minutes later, the ACE initiates the U/L signal for commanding and navigational purposes. The data is transmitted from the spacecraft in the format of “symbols” which are “wiggles” in Cassini’s radio signal’s phase. The DSN receives the symbols and decodes it into “0” and “1 seconds” in order to reconstruct the telemetry data (engineering housekeeping data, science digital images, etc.). After the 9 hours of telemetry data have been downlinked to earth’s DSN ground station, the spacecraft reduces its data rate, suspends its data playback (from the SSRs), and turns to the next science target via the onboard running sequence to collect new science data [4].

4.1. Nominal S/C acquisition

Prior to spacecraft acquisition at JPL’s Space Flight Operations building in Pasadena, California, the “Cassini ACE” Real-time Operations Engineer must prepare to receive the data transmission stream from Cassini, and is in voice contact with the DSN station staff (in California, Australia, or Spain). The Cassini ACE provides their station operator with a 2 minute briefing to review the expected events for the day, before the DSN pass starts (any planned Reaction Control System (RCS) burns or Main Engine (ME) maneuvers, Flight Software (FSW) patches or uploads, etc.) and provides any pertinent updates. The DSN station operator, in turn, provides a weather report (clear skies or rain, plus wind conditions) and that all equipment is in working order (green), or has suffered a system breakdown (red). The designated (34 m or 70 m) antenna at the DSN station for the day’s 9 hour pass has already been pointed precisely towards Saturn where Cassini’s faint signal will be received. Once the spacecraft’s signal has been acquired, the DSN station operator reports to the Cassini Ace that the station’s receiver is “in lock.” The Cassini Ace then acknowledges that the telemetry at his/her workstation is being received and looks nominal. From this point, the 9 hour DSN pass is in progress with Cassini transmitting its telemetry data. Thousands of engineering telemetry measurements (i.e. temperatures, voltages, pressures, computer statuses detailing the vehicle’s health and status) are interleaved with the science data.

4.2. Anomalous D/L conditions

An “out of lock” condition can occur suddenly if Cassini’s signal strength drops out (LOS condition). This can be caused by rain at the DSN station from too many water molecules in the vicinity of the antenna which give off an abundance of radio noise that can literally drown out the spacecraft’s signal. In this case, the “DSN Receiver Status” on the Cassini ACE’s console will light up with an “OUT OF LOCK” reading. The measured system operating noise temperature on the console should rise high enough to indicate that rain is the reason for the signal loss. But if bad weather is not the cause of the LOS condition, or caused by an unforeseen problem in the ground system equipment itself, the ACE will contact the Operations Chief (who is concurrently working with the Cassini ACE at JPL), to request that a second DSN antenna look for the spacecraft’s signal, if available. If no signal is detected, the Cassini ACE will declare a “LOS condition” and proceed to follow the “LOS/Anomalous Downlink Contingency Plan” Procedure which requires that he/she contact the appropriate SOFS team members. These are spacecraft subsystem experts who must evaluate the situation and concur with the Cassini ACE that there is no earth-based problem causing the LOS condition (ground station or weather). In this case, the most likely explanation is that an onboard RFS-related FP routine has triggered. Numerous fault monitors are installed into Cassini’s FSW that are constantly running to detect faults in spacecraft systems. Upon fault detection, a “canned” response routine(s) is executed autonomously to fix the problem, which is typically followed by an activation of the Safing Response. This response places the spacecraft in a predictable state, configuring lower power consumption with low U/L and D/L rates on LGA, commanding the HGA to sun-point (off earth-point). In the case of a RFS FP routine activation, the RFS device states might be altered, as a swap to a redundant RFS unit is commanded which changes the telecommunications configuration for D/L signal acquisition.

The ACE knows that Cassini will have transitioned from the HGA to the LGA antenna, should the FP activate. The LGA provides an extremely weak D/L signal since its beamwidth is much larger than the HGA beamwidth. At Saturn, the spacecraft’s signal is so weak that telemetry delivery is only possible at 5 bps, requiring nearly 18 hours to receive all 30 decks of telemetry data that are needed for the SOFS team members to verify the spacecraft’s health and determine its post-fault states. Recovery from any fault is extremely slow, but if no attitude control system problems are present and spacecraft attitude knowledge is preserved (no faults in the AACS computers), a second FP routine called the “High Gain Antenna Swap (HAS) Response” will automatically activate 1 hour after the Safing Response concludes. This FP will increase the U/L and D/L rates (D/L = 1896 bps), followed by a turn of the spacecraft’s HGA to earth-point. In this configuration, all 30 decks of telemetry data are delivered to the ground in approximately 10 minutes, making recovery from the fault much more expedient. For typical FP activations, the SOFS team will examine the spacecraft telemetry for off-nominal conditions, sometimes reading out additional sections of Cassini’s computer memory to confirm the diagnosis, and then prepare commands for the ACE to send which will recover the spacecraft from the FP activation, and restart the onboard running sequence once again.

In certain cases, complete LOS can occur. Resolution of a LOS fault may require extra DSN coverage, depending on the difficulty in determining the fault cause. As mentioned previously, the Cassini ACE also looks for other DSN tracks that can be borrowed from other flight projects or scheduled maintenance for the next few days. If the LOS condition persists, a “spacecraft emergency” will be declared to guarantee continuous DSN coverage to support spacecraft recovery efforts.

4.3. No spacecraft signal acquisition (LOS)

Unlike most faults that trigger the onboard FP, a fault causing total LOS means no acquisition of the spacecraft’s signal at all (i.e. no lock-up on the expected or post-FP RFS configuration) by the DSN station. There are several reasons why a LOS condition can occur. These include DSN station breakdowns, misconfigured lock-up parameters, or even faults which are not detected by the FP design. Unfortunately, not every spacecraft fault case can be precluded by the onboard FP. In spite of the best efforts of pre-launch designers to identify all possible fault scenarios and produce a FP system to support them (detect, isolate, & resolve), certain failure modes are sometimes missed or are very difficult to avoid. Most JPL projects like Cassini strive to meet a “Single Point Failure” (SPF) policy [5], but certain failures cannot be easily detected, or are not identified during the design phase, and some failures can actually occur even though they have been exempted or waived [6]. Other LOS fault possibilities are problems that occur in devices which are intentionally not protected by the onboard FP. These devices include the HGA or LGA antennas, Waveguide Transfer Switches (WTS), and the USO on Cassini. Multiple faults are also a possibility, since they do not fall under FP design guidelines due to the SPF policy.

Hence, LOS can occur from several sources: erroneous ground-generated commands uplinked to the spacecraft, onboard sequence failures, multiple failures which are not typically required to be addressed by the onboard FP, spacecraft pointing errors, failed telecom configurations (via ground commanding), internal FSW errors, computer platform failures, bad weather, or DSN ground equipment failures. Also, not only can RFS FP swap to redundant units due to device faults and malfunctions, thus inhibiting the ground from locking up on Cassini’s signal (since the RFS D/L signal path has changed), but environmental effects can also cause a LOS condition. SSPS trip-off of RFS units (caused by cosmic ray bombardment) can also cause temporary loss of the spacecraft’s signal. To address this condition, the Cassini ACE must perform several “uplink sweeps” on different variations of the RFS units in an attempt to re-acquire the spacecraft’s D/L signal. Once ground problems and weather are ruled out as an LOS cause, the assumption is that hopefully the onboard FP has executed and commanded a RFS device swap to a redundant unit. Otherwise, determination of the fault cause becomes increasingly difficult to diagnose.

5. Cassini LOS experiences

Cassini has experienced several LOS events during its mission lifetime. Some events have been caused by relatively minor problems, but two events are of significance. The first occurred on May 1, 2006. At the beginning of the DSN track, the DSN station was unable to acquire the spacecraft’s “one-way” carrier signal (i.e. the ground-received spacecraft signal), which in turn, initiated the anomaly response process. However, after Round-Trip-Light-Time (RTLT; twice OWLT) had elapsed, the DSN station was able to lock up on the “two-way” carrier signal and the spacecraft’s data. Telemetry indicated that Cassini’s USO had suffered an SSPS trip event [7].

Cassini’s power system consists of power control boards which contain 192 SSPS. SSPS trip events occur spuriously and without warning, on average 2–3 times per year due to the unforeseen environmental effects of galactic cosmic ray bombardment [8]. This condition is thought to be caused by one or more photon hits on the voltage comparator of the device, resulting in a false indication that the current load is anomalously high, thus tripping off the switch. Because of this phenomenon, a new “SSPS Trip FP” monitor & response algorithm was uploaded to the Cassini spacecraft’s FSW. The monitor examines one SSPS switch state per second, (starting with switch number 1), and proceeds through all 192 SSPS switches. If a SSPS trip is detected, the response contains a table of appropriate actions for FP to act upon, based upon the specific SSPS switch and its function. The actions of the original SSPS FP response table for the USO (uplinked prior to 2006) only recorded the USO trip event (USO SSPS is #68) and cleared the tripped condition by commanding the unit OFF. However, after this USO trip event occurred, the response table was augmented (via uplink command) to command the device on (see Table 1).

Table 1.

SSPS trip FP for USO trip (post-2006).

Five years later on December 23, 2011 at the Beginning of the DSN Track (BOT), once again, no D/L signal was seen from the Cassini spacecraft. The DSN station at Canberra was supporting Cassini at the time. Following ACE direction, additional tracking was obtained using a Canberra station antenna, as well as a Goldstone station antenna, but without successful acquisition of the spacecraft’s signal. The SOFS Anomaly Team was called together to diagnose the problem. At RTLT, Cassini was once again acquired in 2-way mode, confirming that the problem was with the spacecraft’s USO. Commands were sent on Christmas Day to inhibit the USO and swap to the Auxiliary Oscillator as the frequency source for the D/L signal until the fault within the USO device could be evaluated.

The next step for the SOFS team was to evaluate whether one-way operation of the USO was functioning properly (the two-way U/L must be halted in this case). Once configured, the DSN station was unable to lock onto Cassini’s one-way signal which indicated that the USO was not operating properly. After a second attempt to establish the one-way link failed, a command was sent to inhibit the USO, allowing the Auxiliary Oscillator to take over again for spacecraft operations. Further tests conducted in January of 2012 confirmed that normal USO operation could not be re-established. After consulting with Radio Science and Applied Physics Laboratory (the builder of the USO), it was decided that the USO would be power cycled in an effort to “reset” the unit, although it was thought unlikely to work since the USO is an analog device. On January 9, 2013 the USO was powered OFF permanently and the Auxiliary Oscillator has been in operation ever since.

6. LOS protocol

For Cassini, addressing an “anomalous downlink” or LOS condition starts with the RFS Subsystem’s “LOS/Anomalous Downlink Contingency Plan” Procedure to help identify possible reasons for the abnormal (or absence of) the spacecraft’s D/L signal. This procedure describes possible troubleshooting methods and recovery actions needed for both off-nominal signal levels (e.g. carrier power is too low or too high) as well as partial lock-up conditions (e.g. no subcarrier, symbol, telemetry, or frame lock-up), and complete LOS. The procedure provides diagnoses & recovery actions in the form of flowcharts for the ACE and SOFS Anomaly team members to follow. Five partial signal loss/LOS candidate faults are considered when determining required anomaly resolution actions:

1. Spacecraft is not on earth-point when expected due to an incomplete turn, a fault in the AACS system, or FP activation.

2. DSN ground-station problem: station is not tracking the spacecraft properly, station receiver is down, breakdowns, weather, etc.

3. Spacecraft telecom problem: there is a problem in the telecommunications system (error caused by the onboard sequence commanding, ground U/L commanding, or the FP has executed)

4. Loss of the CDS (most likely a multi-fault condition)

5. Multiple faults or a catastrophic failure

RFS FP response actions are also noted in the recovery strategy flowcharts of the procedure and specify the expected post-fault RFS device states. Any attempt to re-acquire the spacecraft on the newly commanded RFS configuration is directly dependent on when the FP response has concluded. Attempted spacecraft recovery actions continue through each branch of the flowcharts until re-acquisition of the vehicle is successful (if possible).

For a complete LOS condition, the Cassini ACE must perform the “uplink sweep” on the correct RFS device configuration to re-acquire the spacecraft [9]. The assumption is that the activation of a RFS FP response will have swapped to its counterpart unit, possibly changing the polarity of the D/L signal. Depending on the failure (or number of failures), several RFS device combinations are possible with variations on the following components, depending on which FP has activated and what the current RFS prime units are:

• DST-A/CDU-A or DST-B/CDU-B (Deep Space Transponder; Command Detector Unit)

• TWTA-A or TWTA-B (Traveling Wave Tube Amplifire)

• TCU-A or TCU-B (Telemetry Control Unit)

• WTS-A or WTS-B (Waveguide Transfer Switch)

• LGA-1 (LGA-2 is no longer in use) or HGA antenna

• Auxiliary Oscillator or DST VCO (Voltage-Controlled Oscillator)

Figure 6 depicts the RFS Functional diagram for Cassini, whose prime RFS units are: DST-A/CDU-A, TCU-B, and TWTA-B; WTS-A used for U/L, WTS-B used for D/L. The use of these devices are listed below:

• DST: is used for both the U/L and D/L function

• CDU: is part of the DST and used for the U/L function

• TWTA: is an amplifier used in the D/L function

• TCU: controls the RFS system.

• WTS: provide switching capability for transmitting or receiving the signal through the HGA, LGA-1, or LGA-2 antennas.

• Auxiliary Oscillator: provides 1-way D/L carrier frequency reference.

• VCO: is part of the DST and provides 2-way D/L carrier frequency reference.

Figure 6.

Cassini’s RFS functional diagram.

Also included in certain RFS FP response actions is a Power-on-Reset (POR) of the prime TCU and/or the Power subsystem where selected devices are turned off, reset, or reconfigured, which will select spacecraft components according to their own FP protocols. Further complicating the anomalous/LOS condition is the fact that RFS FP algorithms are multi-tiered (address several different fault types), and can activate at any time per their persistence counters (unique for each FP algorithm) which can range from seconds to minutes, further reconfiguring these device states after spacecraft re-acquisition is attempted, so that it is difficult to know which RFS combinations for the ACE to try (or which combinations should be re-tried or eliminated). Therefore, it is very important to keep track of when RFS related FP responses have timed out.

7. Command loss FP

An unresolvable LOS condition where the ground is no longer able to deliver commands to the spacecraft will eventually lead to the activation of a LOS FP response. The actions of this response can help to re-establish the U/L. In Cassini’s FP design, loss of D/L fault coverage is not protected in an “end-to-end” manner since the D/L is not considered to be a critical spacecraft function which requires autonomous restoration. But restoration of the U/L however, is considered crucial to mission success and is therefore allocated “end-to-end” protection through a “Loss of Commandability” algorithm [10]. Although several other (higher priority) FP routines are installed into Cassini’s FP suite to protect against these same type of failures in the U/L path (which provide more timely action), the Loss of Commandability algorithm provides a “safety net” type of FP which has the potential to restore both U/L and D/L. With this scheme in place, multiple levels of FP defense are provided (covering up to 3 faults).

This catch-all type of FP is referred to as a “Command Loss FP” (from the perspective of the spacecraft since it is no longer receiving ground commands) and is typically an “endless-loop” response. The Command Loss Monitor aboard Cassini will detect an extended period of time during which no commands have been received by the spacecraft from the ground. The Command Loss Monitor is configured with a timer which counts down from a programmable value (usually days) until it reaches “0” seconds or is reset via ground command (on Cassini, this “Command Loss Timer” (CLT) is currently set to 115 hours). The receipt of a valid U/L command by the spacecraft will reset the timer to its original value and restart the countdown. This provides an end-to-end check on command functionality between the vehicle and the ground. If triggered (timer reaches “0”), the Command Loss Response will initiate an extended series of actions which are designed to re-enable ground commandability onboard the spacecraft. The response will attempt to command various telecom configurations and spacecraft attitudes in an attempt to find a viable uplink path. Each reconfiguration of a new uplink path is separated by an appropriate ground response interval for the SOFS team to re-acquire the spacecraft.

Figure 7 illustrates Cassini’s Command Loss Response chain. Once triggered, it progresses through a series of “Command Groups” divided by multi-hour “Command Pauses” which allow the SOFS team to react by sending an U/L command to halt the response. The Command Groups consist of actions to reconfigure redundant hardware and re-command spacecraft attitude and antennas. Each Command Pause allow several hours for the SOFS team to attempt re-acquisition of the spacecraft upon the newly commanded spacecraft configuration (the pause durations are set to a minimum of two RTLT periods). As shown in the figure, the first Command Group will select the Auxiliary Oscillator and execute the Safing Response which will turn off non-essential spacecraft loads, place the spacecraft in a lower power state, and re-direct the spacecraft’s High Gain Antenna to sun-point, placing the spacecraft in a low U/L & D/L state through the LGA-1 antenna. After the first Command Group has executed, a 15 hour wait period (Command Pause) allows sufficient time for the SOFS Anomaly team to assemble at JPL and attempt re-establishment of the U/L, if possible, before RFS hardware swaps begin in successive Command Groups. If the re-acquisition attempt fails after Command Group #1 execution, the response will proceed with the next course of actions specified in Command Group #2, which starts the series of RFS hardware unit swaps. Seven hour Command Pauses are installed between each subsequent Command Group to allow the SOFS team sufficient time to re-acquire the spacecraft on the newly commanded configuration. If the SOFS Anomaly team is able to re-acquire the vehicle within the first 71 hours (during the RFS unit swap phase), it is permissible for the HAS Response FP to execute (1 hour after the Command Loss Response has been terminated) via the selected (6NOP) U/L command which halts the response. Faults resolved during this first 71 hours are deemed to be “non-severe,” since they are associated with RFS device failures. The HAS Response will increase the post-Safing U/L & D/L rates and swap from LGA-1 to the HGA antenna. However, if the Command Loss Response proceeds to Command Groups #5, it must be halted using the HAS FP “disable” command to keep the spacecraft on LGA-1 with the lower U/L & D/L rates, since the fault is considered to be too severe to transition to the higher rates. At the end of the Command Loss Response chain (approx. 7 days 15 hours), a swap to the redundant CDS is commanded and the Command Loss Response will start all over again on the redundant backup computer. The response will run endlessly until an U/L command is received by the ground. Once the spacecraft receives a ground command which restores the uplink successfully, the response will terminate and reset its Command Loss Timer, thus leaving the spacecraft on the last (successfully) commanded RFS/antenna configuration.

Figure 7.

Cmdloss response actions.

8. The LOS/Cmdloss timeline EXCEL tool

In all cases, it is desirable to re-acquire the spacecraft before the Command Loss algorithm times out and triggers its response, if at all possible, since this FP routine will configure the LGA antenna, which yields extremely slow data delivery. Should this response trigger, the Command Group actions (device swaps, etc.) most likely cannot be confirmed in telemetry with the very slow D/L rate of 5 bits per second. Therefore, it was determined that two timelines were needed to provide visibility into fault possibilities and to supplement the LOS/Anomalous Downlink Contingency Plan Procedure recovery efforts: 1) a pre-Command Loss Response “LOS Timeline” containing FP expiration times (and the corresponding RFS configurations) to eliminate the numerous fault possibilities, 2) a timeline to track the Command Loss Response actions if activated. This goal was accomplished through the development of an EXCEL tool which receives minimal user inputs, utilizing the Space Flight Operations Schedule (SFOS) file which is used daily by both the ACE and SOFS teams. The “LOS/Cmdloss EXCEL Tool” provides the following:

• Sheet #1: instructions for using the EXCEL Tool & required inputs taken from the SFOS file

• Sheet #2: Timeline #1 starting from LOS occurrence = > CLT = 0 seconds (Command Loss Response trigger time)

• Sheet #3: Timeline #2 detailing the Command Loss Response actions from CLT = 0 seconds through one entire CDS response cycle

• Sheet #4: all corresponding end conditions for each FP response activation in Timeline #1 with the required recovery actions

9. EXCEL tool example: 2011 USO Failure

Experience gained from the failed USO/LOS event on December 23, 2011 at BOT led to the development of this LOS/Cmdloss Timeline EXCEL Tool. To demonstrate its use, an example is provided here for this USO failure event.

Once no signal was detected from Cassini on Day of Year (DOY) 357 of 2011, the ACE proceeded to follow the “LOS/Anomalous Downlink Contingency Plan” Procedure, performing sweeps of the spacecraft on different RFS configurations to attempt re-acquisition of the vehicle. A second DSN station was requested and confirmed no acquisition of Cassini’s signal (ruling out weather and station configuration problems). Had the EXCEL tool been available at the time, the following data would have been collected from the SFOS file as noted in Figure 8:

1. Time of LOS = > 17:15:00 UTC

2. OWLT = > 1 hour 23 minute 51 seconds

3. Year = > 2011

4. Last time CLT was reset = > DOY357 @ 02:15:00 UTC

5. Command Loss Defaults = > 115 hours

6. Prime RFS Devices set to: DST-A, TCU-B, TWTA-B

Figure 8.

SFOS file for USO failure event.

Once these data had been collected per the instructions listed in Sheet #1, EXCEL Sheet #2 inputs would be entered in the YELLOW spaces as shown in Figure 9, which in turn, will cause Sheet #2 through Sheet #4 to be populated with desired timing/post-fault configuration data. Copies of the SFOS and Sheet #2 though Sheet #4 would then be printed and distributed to each subsystem once the Anomaly team gathered to determine the cause and resolution of the LOS condition. As the group followed along with the SFOS file in LOS Timeline #1, spacecraft recovery efforts would have been coordinated with the Cassini ACE via telecom. All system-level FP responses are included in the LOS timeline for completeness (RFS-related responses are shown in red). These are the LATEST times that the FP responses would conclude, assuming that each activation started at BOT. Fault cases would be eliminated by the SOFS Anomaly team once re-acquisition for each completed FP response failed to re-establish the earth-spacecraft link.

Figure 9.

LOS timeline of SFP response expiration times.

In the figure, each completed response notes whether a RFS POR occurs, as well as RFS device swap occurrences. The end of the timeline calculates when the Command Loss Timer will decrement to “0” seconds. For each response case, the resulting antenna selected (LGA or HGA if the HAS response is executed for that particular response) is noted in the timeline. Corresponding RFS post-response states and end conditions of interest are listed in Sheet #4 (Table 2).

Table 2.

Post-response concluding end conditions (sheet #4).

Although there are eight possible RFS combinations (see Table 3), there are only three DST/TCU/TWTA combinations of interest due to the selection of RFS prime units in the FP (i.e. the FP will never command the alternate combinations). Also, telemetry delivery on the post-Safing commanded LGA is minimal at best, so that the recommendation to the Cassini ACE would be to attempt re-acquisition with the FP commanded RFS combinations after the HAS response had concluded (since all RFS-related responses will execute the HAS response to swap to the HGA antenna and increase the D/L rate). According to the LOS timeline, no new RFS configurations will be commanded after 3 hours 20 minute (so that the nominal DST-A/TCU-B/TWTA-B arrangement is assumed), since all RFS-related FP responses will have executed. Problems to focus on from this point forward would be an onboard sequencing error, an activation of the AACS FP, undetected RFS failures not protected by FP, a LGA-1 or HGA antenna failure, WTS-B failure, multiple faults, or possibly a waived failure; all which will most likely leave the spacecraft on the LGA-1 antenna (note: for a USO failure, the DST’s VCO will take over the D/L delivery once 2-way communication is established).

Table 3.

Possible RFS combinations.

9.1. Command loss response activation

If the SOFS Anomaly team was unable to re-acquire the spacecraft before the Command Loss Timer decremented to “0” seconds, the Command Loss Timeline in Figure 10 would have been followed in synchrony with the SFOS file. In Sheet #3, the event times are listed in UTC (Universal Time Coordinated), which is consistent with the SFOS file timeline (in successive pages to DOY357 which are not shown in this article). The timeline is also quoted in terms of DOY and elapsed time since the Command Loss Response triggered, showing each upcoming Command Group execution time. As mentioned before, the Command Groups consist of actions which reconfigure redundant hardware, eventually commanding spacecraft attitude and antennas in later Command Groups. Once a ground command is successfully received by the spacecraft, the response will be terminated, the CLT reset (to 115 hours), leaving the vehicle on the successfully commanded configuration.

Figure 10.

One command loss response cycle (sheet #3).

The Command Loss Timeline is listed for one “CDS cycle” of the response. If all attempts to re-acquire the spacecraft have failed on the first response cycle of Command Groups on the prime CDS unit, the backup CDS computer will take over at the end of this response chain (after 7 days 15 hours 29 minute), so that the cycle is repeated on the redundant computer. As stated above, the Command Loss Response is an endless loop algorithm; below are the actions of the response cycles:

• 1st Response Cycle: The Prime CDS uses its RAM load; it is then re-booted with a FSW load stored on the SSR (at the end of the response cycle).

• 2nd Response Cycle: The BU CDS takes over immediately using its RAM load; it is re-booted with a FSW load stored on the SSR (at the end of the response cycle).

• 3rd Response Cycle: The Prime CDS uses the default SSR FSW load from the previous reset; the Command Loss Timer is set to the FSW default value of CLT = 5 days; at the end of this cycle, the CDS is re-booted with the same FSW load stored on the SSR (at the end of the response cycle), but must wait 5 days before continuing the response.

• 4th Response Cycle: The BU CDS uses the default SSR FSW load from the previous reset; the Command Loss Timer is set to the FSW default value of CLT = 5 days; at the end of this cycle, the CDS is re-booted with the same FSW load stored on the SSR (at the end of the response cycle), but must wait 5 days before continuing the response.

• 5th Response Cycle - ∞: Repeat cycles 3 & 4 above indefinitely.

For the 2011 USO failure event, the EXCEL LOS/Cmdloss Tool would have been used to generate the supporting data needed for trouble-shooting the anomaly for the SFOS Anomaly team, with recommendations included for the Cassini ACE as shown in Figure 11.

Figure 11.

LOS/Cmdloss response info for SOFS team & ACE.

10. Other uses for the Excel tool

Cassini also relies upon the Command Loss Response to protect events of significant importance should a loss of U/L occur during science experiments and other selected spacecraft activities. Figure 12 provides an example of this type of “Command Loss Response strategy” used to support the RSS LGA Gravity Experiment performed in 2015, where the HGA must be swapped to LGA-1 and then back again to HGA. The risk associated with this experiment was commanding the WTS switch during the HGA/LGA-1/HGA antenna swap series, where if a malfunction occurred on WTS-A, the U/L capability would be permanently lost (since there is no WTS FP on Cassini). In this case, the Command Loss Timer default of 115 hours would cause the Orbital Trim Maneuver (OTM) #407 to be missed on DOY078 should WTS-A fail (as well as the planned OTM backup opportunity on DOY079). To protect against loss of U/L after the WTS switch is commanded, the EXCEL tool was used to predict actions from the Command Loss Response which can provide a different U/L path through DST-B/WTS-B should WTS-A fail. The strategy shown in the figure depicts a reduced Command Loss Timer default of CLT = 72 hours with a “command moratorium” period implemented (no commanding allowed), which allows a controlled decrementation of the CLT timer during the RSS LGA Gravity Experiment. Once the test is complete on DOY075, an attempt to verify the telecom state by uplinking the original CLT default value of 115 hours is performed on DOY076. Should this U/L command fail to execute on the spacecraft, the command moratorium will continue until the CLT clocks down to “0” seconds, allowing the Command Loss Response to execute through to Command Group #2 which swaps DST-A= > DST-B, placing the U/L and D/L on WTS-B, just before the DSN track starts. The spacecraft would then be acquired on this new RFS configuration. The OTM would then proceed on the backup DSN pass. For Cassini, a failure of WTS-A would have meant that WTS-B must be used for the remainder of the mission, since the WTS-A switch is henceforth unusable. The actual execution of the RSS Gravity Experiment was successful without the need for FP intervention.

Figure 12.

EXCEL tool support of 2015 RSS LGA gravity experiment.

11. Conclusions & lessons learned

Overall, anomalous D/L and LOS occurrences are very challenging and can be difficult for the SOFS Anomaly team to diagnose and resolve. Once the spacecraft’s D/L signal is lost, an expedient, accurate resolution process is needed for quick re-acquisition of the vehicle. Identification of FP responses, their conclusion times and corresponding end states, as well as plausible LOS causes, is extremely helpful in eliminating fault cases systematically, thus allowing the SOFS Anomaly team to focus on the actual cause of the LOS problem. Unfortunately, pre-launch FP analyses do not always protect against all LOS-related fault possibilities since design oversites, lack of schedule or funding in implementing FP algorithms, errors within the FSW, or even false assumptions made during the pre-launch testing phase (waived failures) can occur. In all cases, it is highly desirable to address a LOS condition before the Command Loss FP response activates. But if not, a concise timeline of this response and its actions is essential in order to coordinate team efforts in attempting to re-acquire the vehicle; especially since the LGA-1 antenna is commanded, configuring the very low D/L rate which must be delivered through Cassini’s very noisy Auxiliary Oscillator (backup device used since the primary USO failed). Therefore, the “LOS/Anomalous Downlink Contingency Plan” Procedure in combination with “LOS/Cmdloss” EXCEL tool is expected to be very useful when supporting this challenging class of faults during the remainder of Cassini’s highly successful 20 year mission, until its final plunge into Saturn’s atmosphere on September 15, 2017.

Acknowledgments

This research was carried out at the Jet Propulsion Laboratory, California Institute of Technology, under a contract with the National Aeronautics and Space Administration.

Nomenclature

AACS	Attitude, articulation, & control system
ASI	Agenzia Spaziale Italiana (Italian space agency)
BOT	Beginning of (DSN) track
CDA	Cosmic dust analyzer
CDS	Command & data processing system
CIRS	Composite infrared spectrometer
D/L	Downlink
DOY	Day of year
DSN	Deep space network
ESA	European space agency
FP	Fault protection
FSW	Flight software
HAS	High gain antenna swap (algorithm)
HGA	High gain antenna
INMS	Ion & neutral mass spectrometer
JPL	Jet propulsion laboratory
LGA	Low gain antenna
LOS	Los of signal
MAG	Dual technique magnetometer
ME	Main engine
MIMI	Magnetospheric imaging instrument
OTM	Orbital trim maneuver
OWLT	One-way light time
RCS	Reaction control system
RF	Radio frequency
RFIS	Radio frequency instrument subsystem
RFS	Radio frequency system
RPWS	Radio & plasma wave science instrument
RSS	Radio science subsystem
RTLT	Round trip light time
SOFS	Spacecraft operations flight team
SOI	Saturn orbit insertion
SPF	Single point failure
SSPS	Solid state power switch
SSR	Solid-state recorder
U/L	Uplink
USO	Ultra-stable oscillator
UTC	Universal time coordinated
UVIS	Ultraviolet imaging spectrograph
VIMS	Visible & infrared mapping spectrometer
WTS	Waveguide transfer switch