A Game-Theoretic Federated Deep Reinforcement Learning Framework with Nash Equilibrium Optimization for Secure Resource Allocation in Wireless Sensor Networks.

Abstract

A wireless sensor network deployed for industrial monitoring and sensing, smart infrastructure, precision agriculture, battlefield surveillance and 6G edge services needs to allocate sensing, spectrum, routing, transmission power, computing and security resources in severe energy, latency, privacy, and adversarial constraints. The approaches such as conventional optimization, single-agent reinforcement learning, and centralized deep learning are no longer sufficient due to the assumption of a stable topology, trusted coordination, a data that is globally available to everyone, or static node behaviour. This paper presents the Federated Nash Deep Q-Network framework (a Federated Multi-Agent Markov Decision Process) (FN-DQN) which combines federated deep reinforcement learning, multi-agent Markov decision process, non-cooperative Nash equilibrium search, Stackelberg/Bayesian security games, trust-weighted aggregation, gradient compression, and optionally, blockchain assisted auditability for the secure allocation of resources in heterogeneous WSNs. The energy-security-aware action-value function is learned locally by each sensor/cluster head from private observations; edge servers obtain parameters of the action value function with trust-weighted robust federation. The resource-allocation game views nodes as rational agents that gain a utility from the throughput, rest of energy and reliability/attack-exposure while incurring costs in terms of delay, contention and interference. Compact convex sets of strategies and continuous quasi-concave utilities are assumed in order to show the existence of a mixed-strategy Nash equilibrium set and a learning scheme based on distributed best responses is found to converge towards an ε-Nash equilibrium if the pseudo-gradient is monotone and if the local policy updates are contractive. Secure allocation is formulated as a constrained optimization problem (C-OPT) in the form of a Lagrangian/KKT system with constraints on energy, delay, packet-loss, trust and bandwidth. A reproducible simulation-study benchmarking of FN-DQN against DQN, PPO, MADDPG, Q-learning, centralized RL, and greedy allocation in Sybil, DoS, Byzantine, and false-data-injection attacks was provided on more than a hundred to five hundred heterogeneous sensor nodes. The proposed framework results in a lower amount of energy usage, a longer lifespan, and higher throughput and packet delivery, lowers latency and maximizes fairness in the generated benchmark, and helps to reduce communication overhead via compact federated updates. The study proposes a novel resource-allocation architecture for next-generation intelligent WSNs that is mathematically grounded, attack aware and edge deployable.