‘‘Missions go through several stages, from initial analysis, identification, feasibility, definition to the design, and finally operations. Spacecraft operations refer precisely to this latter stage, where the mission goals and requirements are carried out. Machine learning allows computers to predict future events or make informed decisions without being explicitly programmed to do so but rather by learning from past data and relevant features.’’ Joao Guerreiro

‘‘When you consider that spacecraft operations collect extensive amounts of data, machine learning can work very well, allowing you to train models using such data, to make autonomous decisions on various problems quickly. In contrast, manual processes take significantly more effort to get comparable results than when combined with machine learning or when compared to a fully independent machine learning agent.’’ William Jones

Artificial intelligence (AI), especially the subset machine learning (ML), is hype at the moment, and sometimes we get a bit carried away while thinking about what these technologies can do. Soon we are talking about Sci-Fi movies with robots taking over the world. However, AI and ML are not that new to the space industry, and they are used to improve our daily lives in simple things, such as better weather forecasting and GPS guidance.  

To better understand how the space industry uses artificial intelligence and how we do it at VisionSpace, we interviewed João Guerreiro and William Jones, our Machine Learning Engineers. 

To start, what are spacecraft operations?  

William: Spacecraft operations are a part of the mission’s life cycle that deals with controlling the spacecraft during the operational phase. From the time the spacecraft separates from the launch vehicle, during launch and early orbit phase (LEOP), passing through commissioning, routine operations, to decommissioning and end of life, there are different functional segments. For example, communication, flight dynamics, navigation, and others, and you must control and synchronize them to ensure that the mission behaves according to its requirements and fulfills its goals.

Joao: Missions go through several stages, from initial analysis, identification, feasibility, definition to the design, and finally operations. Spacecraft operations refer precisely to this latter stage, where the mission goals and requirements are carried out. There are many different missions, each with a goal, ranging from simple tasks, such as weather forecasting and telecommunications, to taking pictures of the Earth to study environmental causes, such as deforestation or the melting of polar caps. Other spacecraft carry more elaborated science missions such as scientific experiments on Earth’s magnetosphere, such as ESA’s mission Cluster, and other planets, such as the Mars Rover, all supported by ground systems and data processing centers. 

What is machine learning? Why is it necessary for spacecraft operations?  

Joao: Machine learning allows computers to predict future events or make informed decisions without being explicitly programmed to do so but rather by learning from past data and relevant features. Using it for space is particularly exciting because there’s still so much we don’t know about space, even though we have data we’ve been gathering for the past century. Combining that with recent computational progress, the increasing investment in the space industry from the private sector, and the excitement about such a mysterious environment, it comes as no surprise as machine learning surges in space operations and even the space domain in general.  

William: I agree with that. Several of these new use cases have emerged due to the increasing efficiency and power of computer chips and their decreasing size. The more powerful and smaller they are, the more you can do in a useful setting. Twenty years ago, you would’ve needed a huge computer running for days to train some of the models you can now train on a laptop in a few hours.  

Machine learning algorithms can extract information from data and find patterns, allowing the algorithm’s designer to understand relationships between unique features within the dataset. The most exciting ML applications use deep learning, which incorporates artificial neural networks inspired by information processing in biological systems, allowing for the processing of vast amounts of data. For spacecraft operations, we aim for full autonomy, which will require a thorough understanding of relationships between all the operational details. When you consider that spacecraft operations collect extensive amounts of data, machine learning can work very well, allowing you to train models using such data, to make autonomous decisions on various problems quickly. In contrast, manual processes take significantly more effort to get comparable results than when combined with machine learning or when compared to a fully independent machine learning agent.

What are the challenges machine learning help to overcome?  

Joao: The advent of machine learning has been increasingly pronounced in several industries, and the space sector is no different. ESA itself has hosted several competitions, particularly the Mars Express Power Challenge (2017), where contestants looked to predict the Mars Express spacecraft’s thermal subsystem’s power consumption. And the Spacecraft Collision Avoidance Challenge (2020), in which contestants looked to predict the final collision risk between orbiting objects. The results were overwhelmingly in favor of machine learning and deep learning solutions, showing once more the viability of these technologies in the space sector.  

Of course, these two challenges are a small drop in the ocean of possibilities for employing machine learning solutions in the space sector. If there is enough data, one can devise a vast number of use cases for machine learning in space. One would be creating predictive models for spacecraft telemetry, which would aid in anomaly monitoring and overall spacecraft maintenance. Many deep learning solutions, particularly reinforcement learning (RL), will contribute to increased autonomy during operations by automating spacecraft guidance and reducing human error. Guidance is just one example; tracking and docking missions, collision avoidance, and object retrieval, areas that can use machine learning and deep learning to innovate and bring new paradigms to spacecraft control.

William: The primary benefit of machine learning and deep learning for spacecraft operations comes from the fact you can automate almost anything you want. A constant problem in spacecraft operations is the vast space between the ground station and the satellite. If you have extensive data to transfer, it costs a significant amount of energy and time to do so. If we have machine learning agents on board doing the work that would be done on the ground, we can drop a significant percentage of the data bandwidth requirement. The ML agent not only has an impact on bandwidth but also on spacecraft’s design as you don’t need as much hardware to store and transfer data. As they don’t need to be as powerful, spacecraft can be smaller, lighter, and less complex, making them cheaper to design, build, and operate.

For example, data from Earth Observation satellites are often used in classification. What is in this photo? Has it changed over time? Deep learning classifiers can answer these questions on board the spacecraft without transferring substantial amounts of data back to the ground, which increases the efficiency of operation. However, it is not just about data transfer but also about these machine learning agents’ performance. Because they are so fast at working on data they are given, machine learning models open the potential to do reactive operations. A ML model’s output is used to make operational decisions without delay due to the time it takes to transfer data to the ground and receive a response. Also, new research has shown that relatively low-quality pictures can be classified with high accuracy, removing the need to use high-resolution cameras, reducing spacecraft’s complexity and operation. 

This form of reactive operation is not confined to the spacecraft, however. When we consider data transfer between the ground and spacecraft, some issues arise due to adverse weather conditions. Data loss due to signal interaction with rain can be mitigated by merging predictive ML tools onto current physics-based weather models by significantly reducing the time it takes to generate an accurate forecast. The same approach can be extended to problems around ground station antenna pointing due to wind levels. Spatio-temporal machine learning models can be trained to predict local wind strength and direction, thus allowing for pre-emptive decisions about signal transmission, as the quality of signal received by the spacecraft is extremely sensitive to antenna pointing. In other words, if the machine learning model says it’s going to be very windy, we can decide to delay signal transmission until the wind is calm again.  

Another machine learning technique named reinforcement learning takes inspiration from the natural world, which concerns how animals learn control from interaction with an environment and experience of a change in the environment state. Currently, spacecraft are controlled by a team of operators working around the clock, sending commands, reading the spacecraft’s pose and state, adjusting trajectory, ensuring mission requirements are met. Integrating a reinforcement learning model onboard the spacecraft will enable automatic control based on the spacecraft state. High-level mission goals that define the RL model’s control outputs are built within what is known as the reward function, which provides the RL agent with a stimulus to execute actions that maximize reward.

Adopting automatic ML is also useful for mission scenarios with high uncertainty, like scouting an area with a high density of dangerous objects, such as in the asteroid belt. Imagine a scenario where a previously undetected asteroid is moving towards the spacecraft. There may not be enough time to adjust trajectory with traditional control techniques, in which the spacecraft state is to be sent to the ground, and a control command returned. In this case, an ML agent provided with the asteroid’s state information could automatically react, adjusting spacecraft trajectory away from a collision. This autonomous, in-situ decision-making is an enabling technology for many high-risk missions.  

ML applications go beyond direct spacecraft operations and can be used in indirect operations, such as machine learning agents like Karel. Karel is a project developed by VisionSpace, which uses the spacecraft’s information history and communication between spacecraft operators. The machine learning agent can collate this data to answer operators’ questions about the state of the spacecraft and what may happen in the future, rather than having the operators go through the database manually to find the answer. The ML agent searches all the databases and returns with a quick suggestion, such as how you can counter some situations and what might happen if you take determined actions. It simplifies and reduces the amount of work that spacecraft operators must do on the ground to ensure that the mission requirements are met.

Machine learning can also help generate the infrastructure used to define the whole operational model. The high complexity inherent in spacecraft operations means the models developed to describe how the systems should behave are also extraordinarily complex. Not only this, but the collaborative nature of design, testing, and operations of the entire system can result in an interwoven development environment. Different contractors work on their part of the system at different rates, with different tools, and sometimes in different languages. Machine learning tools such as natural language processing can act as an intermediary between contractors’ models. They can clarify how each contractors’ model is represented relative to the core model, transforming the global working environment into one that is clearer and easier to understand for everyone. As previously discussed, spacecraft operations are high cost and high risk, so it follows that any increase in clarity in the road to successful launch and operation is no doubt highly valuable. 

What is your experience work with machine learning?  

Joao: In VisionSpace, I worked on four different projects related to machine learning. The first one was Karel, which William mentioned. Karel is a conversational assistant supporting spacecraft operations, and one of the models I worked on was predictive analytics. We gathered telemetry data from the spacecraft, such as the temperature from a specific system, voltage, and power. We take all these values from each subsystem and component of the spacecraft and run them through the machine learning algorithm to extract knowledge and patterns and predict these values in the future.  

The second project is one I am working on with William, which is the OPS-SAT Autonomy experiment. Our role is to analyze pictures taken by satellite cameras and understand whether they are cloudy or usable for processing, which is particularly important because, as William said, the satellite is far from the ground segment, and downloading these images has a considerable cost. It is essential to ensure that the image is of acceptable quality and is useful for processing. In this project, we have a separate model that aims to classify interesting sites for images, such as cities, volcanos, and forests. The idea is to contribute to an autonomous experiment that takes superior quality pictures, not cloudy ones, and then classify the interest in these pictures. All should be done autonomously, and this is how machine learning is contributing to this project. 

Thirdly, I am also working on a project with ESOC (European Space Operations Center) in which we are updating how the voice loop system works. The voice loop system is the basis for all communication between ESA Spacecraft Controllers (SPACONs) and ESTRACK (ESA’s tracking station network). The communication between them is logged as audio files. We are using machine learning to automatically transcribe these audio files to the written form and index them in a database. Therefore, people can use keywords to search for the information and find out what was discussed on the audio exchanges in written texts. Instead of having these voice loops stored away, we can extract knowledge from them. 

Finally, I’m involved in a project called MBSE (Model-Based Systems Engineering): Enabling continuity from Design to Operations. Here the goal is to provide a seamless transition between the design and the operations phase of a mission. In particular, we propose a model-centric approach containing all the information needed to build the artifacts necessary to run the mission. This model comes in opposition to the typical document-based system where all data is spread across different documents. My role in this project is twofold: to understand how MBSE could open the door for machine learning artifacts and understand how machine learning can help build the model-based hub itself. 

William: I started working at VisionSpace in November 2020, and since then, I have been involved in the same OPS-SAT and ESOC automatic speech recognition projects that Joao mentioned. Before starting at VisionSpace, my exposure to machine learning was through research on deep reinforcement learning applications for motion control and path planning for legged climbing robots. When given control of the position of the robot’s legs, the RL agent can learn how to move through the terrain, such as walking on flat ground, moving up and down slopes, or climbing up walls.  

I see many use cases in the future for RL in spacecraft operations. Instead of having a climbing robot being controlled by the RL agent, we have satellites. Instead of the neural network’s outputs being the robot’s legs position, it could be the thrust of the spacecraft propulsion system, the angular velocity of ACS (Attitude Control System), or logic commands within the control system. By designing the reward function to give the agent a reward aligned with your goals, you can train the spacecraft to behave as you want. Imagine you wanted to ensure a spacecraft kept a constant orbit around the Earth, using minimal energy to keep trajectory. In this case, the reward function could be composed of positive reward for keeping a predefined course, minus some cost for deviating from this trajectory and using up its energy reserves. It is RL’s general nature that I believe will be a crucial reason for its uptake in future spacecraft operations.

To get to know more about João Guerreiro and William Jones, visit their LinkedIn profile, and don’t forget to follow us on Linkedin, Twitter, or Facebook to get our updates.

This article was produced by Juliane Verissímo - Marketing Department of VisionSpace.