This tutorial details building an advanced federated learning experiment with NVIDIA FLARE, comparing FedAvg and FedProx on a non-IID CIFAR-10 dataset. To simulate realistic label imbalance across clients, data is partitioned using a Dirichlet distribution. The experiment leverages NVFlare's Job API for task definition and launching, and the Client API for local training, model exchange, and communication.
The process begins with installing required libraries (e.g., nvflare, torch). Experiment parameters are set, including NUM_SITES=3, NUM_ROUNDS=5, LOCAL_EPOCHS=1, ALPHA=0.3 for non-IID control, BATCH_SIZE=64, and LR=0.01. CIFAR-10 is downloaded once for all simulated clients.
A core client script defines a small CNN model, Net, tailored for CIFAR-10, notably designed without batch normalization for cleaner FedAvg state dictionaries. Crucially, the script includes a dirichlet_partition function. This function creates deterministic non-IID label-skewed data splits using the Dirichlet distribution. By utilizing a consistent random seed across all client processes, it ensures an agreed-upon global data split, effectively simulating varying class distributions per client. This setup allows for direct visualization of global model accuracy evolution for both FedAvg and FedProx across communication rounds on the same partitioned data.
