This diagram illustrates the difference between static and dynamic prompt generation in the field of computer science. The examples in the image are as follows: 1. Static Prompt Example (Multi-hop Question Answering Task Scenario): For the multi-hop knowledge question answering task "Briefly describe the core differences and historical impact of the Tang Dynasty's imperial examination system compared to the Song Dynasty's imperial examination system," the static prompt uses a manually preset fixed template. The content is "Please answer the following question: {question}. Please elaborate in points and use concise language." This prompt template does not differentiate between the depth of historical knowledge involved in the question or the complexity of the reasoning steps (this question requires at least two steps of reasoning: first, sort out the core content of the Tang/Song Dynasty imperial examination system, then compare the differences and analyze the impact). It also does not adapt to the capability level of the large language model used (e.g., it does not supplement basic concept explanations for basic-level models, nor does it add in-depth analysis guidance for advanced models). It always acts on the model with a unified expression, which can easily lead to basic models being unable to complete reasoning due to a lack of conceptual groundwork, and advanced models outputting superficial content due to insufficient guidance. 2. Dynamic Prompt Example (Same Multi-hop Question Answering Task Scenario): For the same question above, the dynamic prompt generation process needs to go through four adaptive stages: "task analysis - knowledge retrieval - model adaptation - prompt generation." First, the task analysis module identifies that the question belongs to a multi-hop reasoning task in the field of history, requiring a 2-3 step logic chain. Second, the RAG mechanism retrieves core knowledge points of the Tang Dynasty's imperial examination system (such as the status of the Jinshi subject and the non-implementation of the anonymity system) and the Song Dynasty's imperial examination system (such as the implementation of the anonymity transcription system and the institutionalization of the palace examination). Then, combined with the characteristics of the model used (assuming it is an intermediate-level model), it is determined that basic concept transitions need to be supplemented, but oversimplification is not necessary. Finally, a dynamic prompt is generated: "Please complete the question analysis based on the following historical knowledge, and answer in three steps of reasoning: 1. First, clarify the core characteristics of the Tang Dynasty's imperial examination system (reference: Jinshi subject as the core, no anonymity system, small number of admissions); 2. Then, sort out the key changes in the Song Dynasty's imperial examination system (reference: implementation of the anonymity transcription system, normalization of the palace examination, expansion of the scale of admissions); 3. Compare the core differences between the two and elaborate on their historical impact on social class mobility and the construction of the bureaucratic system at that time. Question: Briefly describe the core differences and historical impact of the Tang Dynasty's imperial examination system compared to the Song Dynasty's imperial examination system." This prompt can dynamically adjust the expression and reasoning guidance logic according to the complexity of the question, the retrieved knowledge details, and the model's capabilities.
Please generate a clear block diagram illustrating the archi...
![Please generate a clear block diagram illustrating the architecture of the following neural network model: Model Name: SAT-MoE. The left side has two input blocks: 1) Process features `opes` with shape [batch, num_opes, in_size_ope]; 2) Machine features `mas` with shape [batch, num_mas, in_size_ma]. First, apply two linear projections to obtain hidden representations of the same dimension, then add type embeddings (process type = 0, machine type = 1), and additionally add positional encoding to the process sequence. Subsequently, apply LayerNorm to both paths separately, and then concatenate them along the sequence dimension into a long sequence, which is fed into a stacked Transformer Encoder (self-attention, multi-head attention, feed-forward network). A Mixture-of-Experts (MoE) module can be connected in the middle: a gating network outputs weights for multiple MLP experts, and the expert outputs for each token are weighted and summed, followed by a residual connection and LayerNorm. The Transformer output is then split back into two paths according to the original number of processes and machines: process representation and machine representation. Then, pass them through two cross-attention layers sequentially: The first "Process → Machine" attention: process features serve as Query, and machine features serve as Key/Value, to obtain the updated process representation; The second "Machine → Process" attention: machine features serve as Query, and process features serve as Key/Value, to obtain the updated machine representation. The final outputs are the updated process features `out_opes` and machine features `out_mas`. Please use a left-to-right data flow layout, connect the modules with arrows, label the name of each module (such as Input Projection, Type Embedding, Positional Encoding, Transformer Encoder, MoE, Ope-to-Machine Attention, Machine-to-Ope Attention, etc.), and make the diagram style concise and suitable for paper presentation.](/_next/image?url=https%3A%2F%2Fpub-8c0ddfa5c0454d40822bc9944fe6f303.r2.dev%2Fai-drawings%2FQrRrIgUwJnufqN4Havc2b4pFYV84PSTW%2F763c7d61-fe20-46a0-af4e-ddec53f57135%2F26fdf95b-f6f5-47a1-bf67-976f398ab221.png&w=3840&q=75)
![The core of each iteration lies in generating offspring individuals that satisfy constraints and completing the fitness evaluation of the entire population. This provides a complete basis for selecting the new generation of the population. This stage is divided into three sub-steps: generating crossover offspring, generating adaptively mutated offspring, and population merging and batch evaluation. In crossover offspring generation, the offspring size is set to 10,000 to ensure the efficiency of population iteration and update. The tournament selection strategy is used for parent selection, selecting two parent individuals with better fitness from the current population to improve the overall quality of the offspring. At the same time, the crossover logic must meet constraint conditions. The specific steps are to separate the absolute value and sign of the parent parameters to avoid destroying the constraint conditions during the crossover process; perform linear interpolation crossover on the absolute values (α∈[0,1] is randomly selected) to generate the absolute value of the offspring parameters; perform non-negative clipping on the crossed absolute values, and then normalize them to ensure that the sum of the absolute values is 1; randomly inherit the parent parameter signs and reconstruct the complete offspring parameter vector. In adaptive mutation offspring generation, the mutation offspring size is set to 1,000 to balance population diversity and computational efficiency. The parent is selected from the current population through tournament selection to select a high-quality parent individual and clone it as a mutation template. At the same time, the mutation intensity dynamically decays from 0.05 to 0.005 with the number of iterations, taking into account early exploration and late refinement. Non-negative clipping and normalization are performed on the mutated absolute values to ensure that the constraint conditions are met. Finally, the parameter sign is randomly flipped with a probability of 10% to further enhance the population's exploration ability. In population merging and batch evaluation, the current population (10,000), crossover offspring (10,000), and mutation offspring (1,000) are merged into a temporary full population (21,000). Individuals with invalid fitness in the temporary population are extracted, and batch weighted comprehensive error calculation is performed on the invalid individuals, and assigned as fitness values to complete the fitness update of the full population. The minimum fitness (minimum error) and average fitness (average error) of the current generation are recorded to provide data support for iteration monitoring.](/_next/image?url=https%3A%2F%2Fpub-8c0ddfa5c0454d40822bc9944fe6f303.r2.dev%2Fai-drawings%2FXncgWQfEIJ7VYt2PJn6a3h2aOyb9nOUf%2F77e1142f-8028-491e-905e-fa27ec6c0e30%2Fb70b81d0-4503-4e08-983f-a0d0c89ad3e8.png&w=3840&q=75)