Building Artificial Intelligence Industry Clusters

18 Breakthroughs in AI technology and solutions to the problems of technology itself

18.1 Breakthroughs in AI technology

The release of ChatGPT by OpenAI at the end of 2022 is a major breakthrough in the AI ​​industry. Since then, generative AI has begun to come into contact with a large number of non-technical users. Most people call this revolution strong AI, or AI2.0. Here is a brief introduction to the breakthroughs in AI technology and the problems of generative AI itself. The following chapters of Volume 5 are the possible developments of various AI industries.

Generative AI has been briefly introduced before, but the Taiwan Executive Yuan believes that 2.0 should also work hard on MLOps, federated learning, transfer learning, and explanatory AI (XAI) in the future. We will only state joint learning and transfer learning here, MLOps will be stated in the Web3.0 cloud computing company, and explanatory AI will be stated in the hallucination problem solving method of generative AI.

For information on what federated learning and transfer learning are, please refer to Wikipedia’s definition. Here we will introduce the role of these two types of learning in blockchain technology. If you don’t know what blockchain is, you can watch my YouTube video introducing blockchain:

https://studio.youtube.com/video/vCM4qrcI7n0/edit

18.2 Federated learning as the consensus mechanism of blockchain

We can use federated learning as the consensus mechanism of the blockchain. Federated learning is a distributed machine learning approach that allows various parties to jointly train machine learning models while retaining data privacy. This is consistent with the decentralization and security goals of blockchain.

Federated learning can be used to implement the following blockchain consensus mechanisms:

• Asynchronous federated learning: Each participant trains at different points in time and sends model updates to other participants.

• Synchronous federated learning: All parties train at the same point in time and send model updates to other parties.

• Federated learning with secure multi-party computation: Protect data privacy using secure multi-party computation.

Federated learning can solve some problems of the blockchain consensus mechanism, such as:

• Security: Federated learning can protect data privacy and prevent data from being stolen or misused.

• Efficiency: Federated learning can improve the efficiency of model training because various participants can share computing resources.

• Scalability: Federated learning can scale to a larger size of participants because data does not need to be stored centrally.

Of course, there are also some challenges in federated learning, such as:

• Model accuracy: Federated learning may reduce model accuracy because individual parties may provide incomplete or inaccurate data.

• Coordination: An effective coordination mechanism needs to be developed to ensure that various parties can effectively conduct model training.

Overall, federated learning is a technology that has the potential to become a blockchain consensus mechanism. As technology develops, federated learning will be able to overcome current challenges and become more widely used in the blockchain field.

Regarding the question of whether the old blockchain technology can accommodate federated learning, my opinion is that federated learning can be combined with existing blockchain technology, but some improvements are needed. For example, new data exchange and coordination mechanisms need to be developed to ensure the safety and efficiency of federated learning. Additionally, new blockchain hardware will need to be developed to support the computational needs of federated learning.

Here are some potential applications of federated learning combined with blockchain:

• Finance: Federated learning can be used to develop more accurate fraud detection models and improve the security of financial transactions.

• Healthcare: Federated learning can be used to develop new drugs and treatments and improve patient outcomes.

• Manufacturing: Federated learning can be used to develop new products and services and improve production efficiency.

• Retail: Federated learning can be used to improve the accuracy of customer service and product recommendations.

As federated learning technology develops, we can expect to see more innovative applications combining federated learning with blockchain.

18.3 The relationship between federated learning and transfer learning

Federated learning and transfer learning are both machine learning techniques, but they have different application scenarios and goals.

Federated learning is a distributed machine learning approach that allows various parties to jointly train machine learning models while retaining data privacy. Transfer learning is a machine learning method that uses knowledge learned from one task to improve performance on another task.

In federated learning, each participant has its own data set, but they are unwilling to share the data with other participants. Federated learning allows various parties to jointly train machine learning models while retaining data privacy.

In transfer learning, we already have a trained model that we can use to improve the performance of another task. Transfer learning can be used to solve the following problems:

• Insufficient data: If we do not have enough data to train a new model, we can use the knowledge learned from a task to improve the performance of the model.

• Data imbalance: If our dataset suffers from data imbalance, we can use the knowledge learned from a task to improve the model's recognition of minority classes.

• Data Noise: If our dataset suffers from data noise, we can use the knowledge learned from a task to improve the robustness of the model.

Federated learning and transfer learning can be used together to improve machine learning performance. For example, we can use federated learning to train a general model and then use transfer learning to apply this model to different tasks.

Here are some commonalities and differences between federated learning and transfer learning:

common ground

• They are all machine learning techniques.

• All involve training of machine learning models.

• Both can be used to improve the performance of machine learning.

the difference

• Federated learning is a distributed machine learning method, while transfer learning is a centralized machine learning method.

• The purpose of federated learning is to protect data privacy, while the purpose of transfer learning is to improve the performance of the model.

• Federated learning requires coordination among all participants, but transfer learning does not.

Overall, both federated learning and transfer learning are important technologies in the field of machine learning. They can be used to solve different problems and improve the performance of machine learning.

18.4 The role of transfer learning in blockchain technology

If federated learning is the consensus mechanism of blockchain, then transfer learning can be used in the following aspects of blockchain technology:

• **Improve model performance: In federated learning, various participants can share computing resources to train the model. Transfer learning can be used to improve the performance of these models. For example, we can use federated learning to train a general model and then use transfer learning to apply this model to different blockchain tasks.

• **Reduce data costs: In federated learning, various participants can share data to train the model. Transfer learning can be used to reduce these data costs. For example, we can use federated learning to train a model and then use transfer learning to apply the model to the other parties’ datasets.

• **Improving privacy protection: Transfer learning allows us to improve model performance while retaining data privacy. This can be used to solve the problem of data privacy in blockchain. For example, we can use transfer learning to train a model and then use this model to identify fraud in blockchain transactions.

Here are some specific examples:

• **In the financial field, transfer learning can be used to improve the performance of fraud detection models. For example, we could use federated learning to train a general fraud detection model and then use transfer learning to apply this model to different banks’ datasets.

• **In the medical field, transfer learning can be used to improve the performance of disease diagnosis models. For example, we can use federated learning to train a general disease diagnosis model, and then use transfer learning to apply this model to data sets from different hospitals.

• **In the manufacturing domain, transfer learning can be used to improve the performance of quality control models. For example, we can use federated learning to train a general quality control model, and then use transfer learning to apply this model to data sets from different factories.

Overall, transfer learning can be used to improve the performance and efficiency of blockchain technology and reduce data costs and privacy costs. As transfer learning technology develops, we can expect to see more applications of transfer learning in blockchain technology.

18.5 The hallucination problem and solution of generative AI

One problem with generative AI is hallucination. The importance of each academic field for improving the hallucination problem of generative artificial intelligence will vary depending on the specific situation, and may require a combination of approaches. The following is a possible priority list, but please note that this is only a guide and may vary based on specific needs:

1. Knowledge graph and semantic understanding: Ensure that the model can understand the context and meaning of the text, especially in a specific domain. This helps generate content that is more reasonable and contextual.

2. Algorithm combination: Combine multiple different algorithms, such as generation models and screening models, to ensure the rationality and quality of generated content.

3. Language model improvements: Improve the quality and training methods of language models, including more diverse training data and more refined model adjustments.

4. Manual review and response mechanism: Introduce a manual review and response mechanism to ensure that the generated content is reasonable and does not contain false information.

5. Explanatory Artificial Intelligence (XAI): Provides explanatory text and visual tools to help users understand the process of model-generated content and identify problems. The Executive Yuan specifically proposed XAI as an important means of trustworthy AI, which is divided into three parts: (a) ensuring the fairness of input data, (b) making the parameters of the AI ​​model and the model itself transparent, and being able to explain the decision-making process of the AI, ( c) Be able to reasonably explain the output results. Only with these three things can we comply with internationally recognized AI ethics.

6. Predictive artificial intelligence (Predictive AI): Predict the rationality of generated content to avoid generating false information, but it may only be applicable in some cases.

7. Participation of human experts: Involve domain experts and artificial intelligence experts in the design and improvement of the model to ensure that the generated content conforms to professional knowledge and common sense.

8. Multi-modal data fusion: Fusion of data from different modalities to understand content from multiple angles and reduce reliance on a single modality.

9. Priorities will vary based on specific application scenarios and needs, and may require a balance and combination of different approaches to achieve optimal illusion improvement.

18.6 Inbreeding and quality collapse problems of generative AI and their solutions

If large language models such as ChatGPT, MS New Bing, or Google BARD capture the data used to learn from the Internet, which is actually data generated by artificial intelligence, there will be quality collapse and inbreeding problems in generative AI. This issue is indeed a challenge worthy of attention. These issues can result in generated content that lacks variety, is inconsistent in quality, and may introduce incorrect information. Here are some possible ways to resolve these issues:

1. Diverse data training: Ensure that the data set used to train the generative model is diverse, including data from different topics, styles, and sources. This helps reduce duplication of generated content.

2. Filtering mechanism: Introduce a filtering mechanism to detect and filter the generated content to exclude inappropriate or low-quality content. This can be accomplished through automated algorithms or human review.

3. Hybrid generative models: Use different types of generative models, including different architectures or training methods, to increase generative diversity.

4. Adjustment of preset settings: Adjust the preset settings of the generated model so that it can generate more diverse content and avoid over-reliance on fixed patterns.

5. Filtered generation: Introducing manual intervention or review to ensure that the generated content meets specific standards and quality requirements.

6. Use domain knowledge: Based on knowledge and common sense in specific fields, guide the generative model to generate more reasonable and high-quality content.

7. Use an ensemble approach: Combine multiple generative models and techniques to reduce the limitations of a single model.

8. Review feedback: Collect and analyze user feedback and feedback to continuously improve the quality and diversity of generated models.

In short, solving the quality collapse and inbreeding problems of generative AI requires the comprehensive application of multiple methods and continuous improvement of the model training and generation process to ensure that the generated content is diverse, high-quality, and reasonable. At the same time, as technology advances, these issues will continue to receive attention for research and improvement.

Note: Some people may think that the sixth method to solve the hallucination problem mentioned above, Predictive AI, is a tool to solve the quality problem. But in fact, Predictive AI is not a solution to the above-mentioned inbreeding and quality problems. Predictive AI (predictive artificial intelligence) mainly focuses on predicting events or outcomes and does not necessarily solve the inbreeding and quality issues in generative AI. These problems are usually related to the training and generation process of generative models, while Predictive AI prefers to use existing data and patterns to make predictions.

Addressing the inbreeding and quality issues of generative AI requires specific methods, such as diversity data training, screening mechanisms, hybrid generative models, etc., that more directly address the quality and diversity of generative models. Predictive AI can be used in some cases to predict the plausibility of generated content, but it is not necessarily the first choice for addressing inbreeding and quality issues.

In summary, addressing the quality and diversity issues of generative AI requires specific generative model improvement methods, while Predictive AI is more suitable for other predictive tasks. The two differ in goals and applications. The "diversity requisite variety" mentioned here has a solution in the problem solver of the Quantum Intelligence Association.