Adaptive operator selection (AOS) is a high-level controller for an optimization algorithm that monitors the performance of a set of operators with a credit assignment strategy and adaptively applies the high performing operators with an operator selection strategy. AOS can improve the overall performance of an optimization algorithm across a wide range of problems, and it has shown promise on single-objective problems where defining an appropriate credit assignment that assesses an operator's impact is relatively straightforward. However, there is currently a lack of AOS for multiobjective problems (MOPs) because defining an appropriate credit assignment is nontrivial for MOPs. To identify and examine the main factors in effective credit assignment strategies, this paper proposes a classification that groups credit assignment strategies by the sets of solutions used to assess an operator's impact and by the fitness function used to compare those sets of solutions. Nine credit assignment strategies, which include five newly proposed ones, are compared experimentally on standard benchmarking problems. Results show that eight of the nine credit assignment strategies are effective in elevating the generality of a multiobjective evolutionary algorithm and outperforming a random operator selector.
Consistently collecting the earth's climate signatures remains a priority for world governments and international scientific organizations. Architecting a solution requires transforming scientific missions into an optimized robust 'operational' constellation that addresses the needs of decision makers, scientific investigators and global users for trusted data. The application of new tools offers pathways for global architecture collaboration. Recent (2014) rulebased decision engine modeling runs that targeted optimizing the intended NPOESS architecture, becomes a surrogate for global operational climate monitoring architecture(s). This rule-based systems tools provide valuable insight for Global climate architectures, through the comparison and evaluation of alternatives considered and the exhaustive range of trade space explored. A representative optimization of Global ECV's (essential climate variables) climate monitoring architecture(s) is explored and described in some detail with thoughts on appropriate rule-based valuations. The optimization tools(s) suggest and support global collaboration pathways and hopefully elicit responses from the audience and climate science shareholders.
This paper proposes a set of six canonical classes of architectural decisions derived from the tasks described in the system architecture body of knowledge and from real system architecture problems. These patterns can be useful in modeling architectural decisions in a wide range of complex engineering systems. They lead to intelligible problem formulations with simple constraint structures and facilitate the extraction of relevant architectural features for the application of data mining and knowledge discovery techniques. For each pattern, we provide a description, a few examples of its application, and briefly discuss quantitative and qualitative insights and heuristics. A few important theoretical properties of the corresponding set of patterns are discussed, such as completeness, degradedness, and computational complexity, as well as some practical guidance to be taken into account when applying them to real-life architecture problems. These patterns are intended to be a useful tool for researchers, practitioners, and educators alike by facilitating instruction and communication among system architects and with researchers from other fields such as combinatorics, computer science and operations research; and fostering reuse of domain-independent knowledge necessary to develop architecture decision support tools (and thus development time and cost reductions for such tools).
In 2007, The National Research Council released a report known as the Earth Science Decadal Survey. This report lays out an architecture for a holistic Earth Observation Program consisting of 17 missions to be flown in a decade for a total cost of about $7B. Six years after, mission cost estimates have grown by 70% on average, and at the current levels of funding for NASA Earth Science, it would take about 40 years to fly these missions. Furthermore, missions that played central roles in satisfying the needs of the Earth science community have not materialized, due to launch failures, mission cancellations, severe delays or descoping processes. The Earth Science community is in desperate need of novel architectures for Earth observation missions that can satisfy at least part of the scientific requirements at a fraction of the cost of the Decadal Survey missions. Cubesats have the potential to become an important component of such novel architectures by providing low-cost opportunities to fly advanced miniature instruments such as GNSS receivers in radio occultation and reflectometry modes, visible and near-infrared imagers, short-wave infrared spectrometers, millimeter-wave radiometers, microbolometers, and so forth. While Cubesats have hitherto mostly been used for technological demonstration and educational purposes, there has been some emphasis lately in developing Cubesats capable of satisfying demanding scientific requirements. In a recent paper, a survey and assessment of the capabilities of Cubesats as a platform for Earth observation instruments of high scientific value, was presented. This paper takes that work a step further by analyzing, in terms of both performance and cost, several constellations of Cubesats carrying such instruments. The performance of an architecture (i.e., a certain mix of constellations of Cubesats) is computed by assessing its potential to satisfy the Decadal Survey scientific requirements. This is done leveraging prior work on the development of a rule-based expert system for assessing the relative merit of Earth observing system architectures. Different constellation designs carrying different mixes of payloads are analyzed using performance and cost models. Non-dominated architectures in the Pareto sense are identified, and one preferred architecture is analyzed in more detail. A preliminary mission analysis is conducted for this preferred architecture, and its cost-effectiveness is compared to that of the original Decadal Survey architecture. The paper shows how, while Cubesats still suffer from serious limitations in terms of their performance and capabilities for Earth science, they are a very cost-effective way of satisfying a relatively large portion of the Decadal Survey requirements.
In this paper, we present an integrated tool to support the high level design decisions concerning the assignment of instruments to satellites in an Earth Observation Program. This integrated tool features a rule-based expert system to model scientific synergies between measurements and engineering incompatibilities between instruments. We introduce two matrices: the science Design Structure Matrix (DSM) and the engineering DSM. Both matrices are calculated using the expert system, and provide insight into how to decompose the initial set of instruments into smaller tractable clusters. Then, each cluster of instruments is efficiently explored using metaheuristic algorithms (i.e. non-exact optimization algorithms that use heuristics to search the tradespace). We apply the methodology to the Earth Science Decadal Survey (DS) and identify a few architectures that are different from the baseline architecture and potentially better.
When designing Earth observation missions, it is essential to take into account the programmatic context. Considering individual missions as part of a whole enables overall program optimization, which may bring important cost reductions and scientific and societal benefits. Several implementation trade-offs arise in the architecting process of an Earth Observation program such as NASA's National Polar-orbiting Operational Environmental Satellite System (NPOESS) or ESA's Earth Explorers. Such tradeoffs include choosing between large satellites and small satellites, standard buses and tailored buses, or centralized architectures versus clusters or trains of satellites. This work focuses on packaging problems, i.e. the assignment of instruments to satellites. More precisely, we study the tradeoff between multi-instrument platforms satellites that carry more than one instrument versus dedicated satellites carrying a single instrument. Our approach to the problem takes a systems engineering perspective and consists of three steps: first, a historical review of past Earth observation programs was done in order to gain insight into how decision makers have solved this trade-off in the past; second, we performed a qualitative analysis in which the most important issues of the trade-off were identified; third, a quantitative analysis was done based on an architecting model. The architecting model is multi-disciplinary because it takes a holistic view of the problem by considering at the same time scientific, engineering and programmatic issues. This exhaustive and multi-disciplinary exploration of the architectural tradespace can be very useful in the early steps of program architecting and could be a valuable tool to support decision making. The model is applied to ESA's Envisat satellite as an example. Finally, some general insights on the architecture of an Earth Observation Program that we gained by developing and applying this methodology are provided.
In this paper, we investigate how technical complexity affects the decision to collaborate or combine. First, we discuss a model that mimics the system architecting process that was used on the National Polar Orbiting Environmental Satellite System (NPOESS). Next, we present a metric that can be used to assess technical complexity and risk during the early system architecting phase of a program. We combine our technical complexity metric and additional measures of lifecycle cost and requirement satisfaction to evaluate a large tradespace of joint and single-agency, single-mission system architectures that we generate with our NPOESS model. Finally, we illustrate how agencies' decisions to collaborate and combine depend highly on agency preference, on the cost of collaboration, and on the final system architecture that is selected.
This paper proposes a method for leveraging large language models (LLMs) to improve the question-answering capabilities of artificial intelligence (AI) assistants for tradespace exploration. The method operates by querying an information space composed of fused data sources encompassing the tradespace exploration process and responding based on the gathered information. The information retrieval process is modeled as an internal dialog where an LLM-based dialog agent converses with a subquery answering agent. A case study is conducted on a next-generation soil moisture mission (SM-NG), and a generative AI assistant (named Daphne-G) is configured on it. The effect of the dialog agent and the choice of LLM are assessed by comparing the performance of three different system configurations on a validation question set. A second validation effort is conducted, comparing Daphne-G’s responses to those of a baseline template-based AI assistant, Daphne-VA. Results show that the dialog-based system is necessary for answering complex questions requiring multiple documents. Furthermore, results show that Daphne-G can correctly answer all the questions Daphne-VA can answer, while simultaneously being able to answer a greater number of questions than Daphne-VA. The results suggest that LLMs could significantly improve the outcomes of the tradespace exploration process, which may result in better and more cost-effective mission concepts being implemented.