Lecture 12

What is Embedding?

Embedding is a means of representing objects like text, images and audio as points in a continuous vector space where the locations of those points in space are semantically meaningful to machine learning (ML) algorithms.

Embedding is a critical tool for ML engineers who build text and image search engines, recommendation systems, chatbots, fraud detection systems and many other applications. In essence, embedding enables machine learning models to find similar objects.

Unlike other ML techniques, embeddings are learned from data using various algorithms, such as neural networks, instead of explicitly requiring human expertise to define. They allow the model to learn complex patterns and relationships in the data, which would otherwise be impossible for humans to identify.

For example, OpenAI’s embedding implementation makes it possible for ChatGPT to easily understand the relationships between different words and categories instead of just analyzing each word in isolation. With embeddings, OpenAI’s GPT models can generate more coherent and contextually relevant responses to user prompts and questions.

Visualizations of Embeddings

I submitted my first paper on AI in 1990 to a small, local conference — the “Midwest Artificial Intelligence and Cognitive Science Society.” In those days, the AI field was entirely defined by research into “symbols.” This approach was known as “Good, Old-Fashion AI” or GOFAI (pronounced “go fi” as in “wifi”). Those of us working in what is now known as “Deep Learning” had to really argue that what we were researching should even be considered as AI.

Being excluded from AI was a double-edged sword. On the one hand, I didn’t agree with most of the basic tenets of what was defined as AI at the time. The basic assumption was that “symbols” and “symbol processing” must be the foundation of all AI. So, I was happy to be working in an area that wasn’t even considered to be AI. On the other hand, it was difficult to find people willing to listen to your ideas if you didn’t package it as at least related to AI.

Unsupervised word embeddings capture latent knowledge

The overwhelming majority of scientific knowledge is published as text, which is difficult to analyse by either traditional statistical analysis or modern machine learning methods. By contrast, the main source of machine-interpretable data for the materials research community has come from structured property databases, which encompass only a small fraction of the knowledge present in the research literature. Beyond property values, publications contain valuable knowledge regarding the connections and relationships between data items as interpreted by the authors. To improve the identification and use of this knowledge, several studies have focused on the retrieval of information from scientific literature using supervised natural language processing which requires large hand-labelled datasets for training. Here we show that materials science knowledge present in the published literature can be efficiently encoded as information-dense word embeddings (vector representations of words) without human labelling or supervision.
read more

Understanding the basics of Beta Distribution

Imagine you’re a chef experimenting with a new recipe. You want to create a sauce that strikes the perfect balance between sweetness and tanginess. The Beta Distribution allows you to model the distribution of probabilities over a continuous interval, making it ideal for scenarios where outcomes are bounded and diverse.

read more

Sustaining Value from an Advanced Digital Supply Chain

Supply chains act as our connective tissue, supporting growth and sparking innovation. But increasingly, they are the source of challenges and constraints. Rising costs, shifting demand, critical disruptions—many companies are in a continuous cycle of assessing obstacles, identifying fixes, and hoping for the best.

Supply Chain AI, our digital supply chain offering, lets companies return to creating value. A combination of digital assets, supply chain experts, and data science and engineering talent, Supply Chain AI takes a holistic approach to transforming a company’s supply chain, from strategy and design through integrated planning and optimized execution. Backed by a unique set of AI solutions from BCG X, we enhance seven key supply chain capabilities: network design and optimization, emissions baselining, risk management, end-to-end planning transformation, digital twin representations, control tower visibility, and logistics and distribution automation.

Page updated

Google Sites

Report abuse