Prompt Description
This experiment aims to investigate the impact of data distribution heterogeneity on the performance of federated learning algorithms. The experiment uses a fixed number of 3 clients, each representing a survey point. Seventeen heterogeneity levels (0-16) are designed to simulate data distribution scenarios ranging from completely homogeneous to extremely heterogeneous. The data allocation strategy assigns a target client to each species category. For example, 'zhaoshui' is assigned to client 2, 'xiaotuan' to client 0, and 'nizi' to client 1. Heterogeneity level 0 represents a completely even distribution, with each species category accounting for approximately 33% in each of the three clients. Heterogeneity levels 1-6 indicate that the proportion of the target category in the target client gradually increases from 43% to 90%, with the remaining samples evenly distributed to the other two clients. Heterogeneity levels 7-15 indicate that the proportion of the target category in the target client gradually increases from 91% to 99%, where only 8-10 samples of the other two species remain in each client besides the main species. Heterogeneity level 16 represents complete heterogeneity, with the target category accounting for 100% in the target client and 0% in the other two clients. The 'noise' category remains evenly distributed across all heterogeneity levels. The evaluation results for each heterogeneity level on an independent test set are shown in the table.
Related Illustrations
![Please generate a clear block diagram illustrating the architecture of the following neural network model: Model Name: SAT-MoE. The left side has two input blocks: 1) Process features `opes` with shape [batch, num_opes, in_size_ope]; 2) Machine features `mas` with shape [batch, num_mas, in_size_ma]. First, apply two linear projections to obtain hidden representations of the same dimension, then add type embeddings (process type = 0, machine type = 1), and additionally add positional encoding to the process sequence. Subsequently, apply LayerNorm to both paths separately, and then concatenate them along the sequence dimension into a long sequence, which is fed into a stacked Transformer Encoder (self-attention, multi-head attention, feed-forward network). A Mixture-of-Experts (MoE) module can be connected in the middle: a gating network outputs weights for multiple MLP experts, and the expert outputs for each token are weighted and summed, followed by a residual connection and LayerNorm. The Transformer output is then split back into two paths according to the original number of processes and machines: process representation and machine representation. Then, pass them through two cross-attention layers sequentially: The first "Process → Machine" attention: process features serve as Query, and machine features serve as Key/Value, to obtain the updated process representation; The second "Machine → Process" attention: machine features serve as Query, and process features serve as Key/Value, to obtain the updated machine representation. The final outputs are the updated process features `out_opes` and machine features `out_mas`. Please use a left-to-right data flow layout, connect the modules with arrows, label the name of each module (such as Input Projection, Type Embedding, Positional Encoding, Transformer Encoder, MoE, Ope-to-Machine Attention, Machine-to-Ope Attention, etc.), and make the diagram style concise and suitable for paper presentation.](/_next/image?url=https%3A%2F%2Fpub-8c0ddfa5c0454d40822bc9944fe6f303.r2.dev%2Fai-drawings%2FQrRrIgUwJnufqN4Havc2b4pFYV84PSTW%2F763c7d61-fe20-46a0-af4e-ddec53f57135%2F26fdf95b-f6f5-47a1-bf67-976f398ab221.png&w=3840&q=75)