ML models and Intelligent Automation: Key to Decentralized Learning

April 13, 2022

3 mins read

Collaborative intelligence has been critical to our learning success and is now proving to be critical for ML models.

Human technological growth has depended on collaborative intelligence throughout time. We didn’t just gain from the data around us; we shared our discoveries with others, worked together to solve challenges, and even became picky about who we would learn from or exchange our knowledge and expertise with. These have been critical to our learning success and are now proving to be equally important in ML models for mobile networks. Next-generation autonomous cellular operators will be complex ecosystems comprised of a huge number of decentralized and smart network devices and nodes network elements enabling decentralized learning, that is capable of simultaneously producing and distributing data using ML models and intelligent automation.

Distributed and Decentralized Learning Techniques

Distributed ML approaches are deemed the most appropriate in a complex ecosystem of network parts and devices, where data is intrinsically distributed and can be confidential and high-volume. These strategies enable collaborative learning algorithms without the need for raw data exchange and can be used to incorporate all local learnings from intrinsically decentralized local datasets into a single unified ML model. This jointly-trained machine learning model, in turn, can assist staff in operating more effectively through proactive fault-handling methods, ultimately enhancing both quality of experience and operator revenue.

Decentralized learning and collaborative artificial intelligence (AI) will allow rapid training with less computing and network resource allocation and also improved efficiency – reducing network footprint, communication overhead, exchange of knowledge, and energy usage.

Addressing Heterogeneity

Decentralized datasets in distributed learning contexts are diverse because they are acquired from multiple nodes and devices, which are frequently heterogeneous themselves. They can have diverse features and content, and they can be sampled from various distributions. A base station, for example, may measure transmit power and network throughput at the cellular levels, whereas a device may measure its own position, pace, and background, and also the presentation quality of the implementation it is running. Because all of this information is useful and necessary for accurate early forecasts, optimizations, and proactive fault prevention, it must be integrated into the jointly-trained worldwide ML model.

Some networks or devices may also report negative inputs that lower model performance, either intentionally, as in the instance of attacks, or unintentionally, as in the case of information distribution shifts or sensor mistakes. Such scenarios may have an effect on global model accuracy while increasing time for training, energy usage, and network link usage. The problem of data heterogeneity, on the other hand, can be handled by autonomous and adaptable orchestration of the learning experience.

Horizontal federated learning (HFL)

HFL allows the training of a combined global model composed of diverse data samples on the same observation variables – in this case, ML characteristics. We can do a local simulation of the workers based on their own local datasets because they have both the input features (X) and the output labels (y). Because all workers and the master share the same system model, the entire model is sharable and aggregable.

Split learning (SL)

When decentralized network nodes have distinct ML characteristics on the same time data instance, SL allows for the building of a global model. In this case, worker nodes, for example, can only hold the input characteristics (X). Only the master server has access to the appropriate label (y). As a result, the worker networks may only have a portion of the neural network model. Furthermore, the worker models are not needed to have the same layers of neurons.

MAB Agents in Distributed Machine Learning

It might be difficult to predict which workers in a federation will benefit from the global model and which will jeopardize it. In the use case, we investigated, one of the worker nodes communicated erroneous data to the master server — values that differed greatly from those of the majority of worker nodes in the union. This scenario may have a severe influence on the jointly-trained modelling framework, hence a detection mechanism at the master server is required to identify the malicious worker and prevent it from joining the federation. In this way, we hope to maintain the global model’s performance, in which the bulk of workers continues to profit from the federation despite malevolent input from certain workers.

As a result, when there is at least one employee in the federation who has a detrimental impact on the federation, it is critical to exclude that worker from global model changes. The previous methods are dependent on pre-hoc clustering, which does not allow for near-real-time modification. In this situation, we use MAB-based support to assist the master server in removing any rogue worker nodes.