AstroLLM System and LLM Topics

AstroLLM Assistant System Architecture

The AstroLLM personal assistant system was developed as RAG (Retrieval Augmented Generation) system to avoid substantial hallucinated responses from LLMs (high-end or low-end). The system architecture below should be self-explanatory.

The AstroLLM system has been developed with a combination of Python and Go.

Query-Document Similarity

Sentence Transformer all-mpnet-base-v2 is used for embedding vectors for (fact) documents and questions. The sentence transformer models generate high-dimensional vectors (that carry semantic information of facts and questions). In this particular case vector with 768 dimensions are generated. In order to explain query (question) and fact document similarity match we will have to reduce the dimension to two using principal component analysis (PCA), which will still retain the semantic information required. Similarity scores with reduced dimension vectors are relatively similar to the high-dimension ones.

Embedding Vector Dimension: (21, 768)

Original Embedding Vectors:

[[ 1.37842568e-02 -6.18561963e-03 1.14011876e-02 ... 2.62478665e-02

-7.06621930e-02 -1.67261492e-02]

[ 1.79039109e-02 -1.95299871e-02 6.97290670e-05 ... 3.71213295e-02

-5.00483252e-02 -1.45573560e-02]

[-2.21214015e-02 -7.48479068e-02 3.36945392e-02 ... 5.93995349e-03

-7.84644336e-02 -1.14610004e-04]

...

[ 1.18139470e-02 -1.09140456e-01 -1.38330832e-02 ... 6.39955327e-03

-5.22478782e-02 8.48692004e-03]

[ 1.13899019e-02 -9.98319313e-02 -4.29645739e-03 ... 8.16540048e-03

-4.03986573e-02 8.49309377e-03]

[ 1.72868520e-02 -1.25080496e-01 -1.06524182e-02 ... 6.65995711e-03

4.96427016e-03 1.06874108e-02]]

Reduced Dimension Embedding Vector Dimension: (21, 2)

Reduced Dimension Embedding Vectors:

[[-0.19467829 -0.21774453]

[-0.13463238 -0.17876521]

[-0.19947277 -0.3613095 ]

[-0.00110182 0.4231356 ]

[-0.17985937 0.11165416]

...

[ 0.31391576 -0.18153249]

[ 0.28542688 -0.16978772]

[ 0.53031737 0.2073408 ]]

In the original database there are about 400 astronomy fact text chunks (each chunk with 500 words with 100 word overlaps). See above architecture for the whole process. In addition to reducing the vector dimensions, I'll also work with a very limited set of fact texts and queries. This is shown below, which is pretty self explanatory. Notice how the queries and facts are clustered. As a result closely clustered fact and queries will have high similarity score.

Multi-modal/Vision Models

Can vision models recognize astronomy images and output the tidbits about them? Answer is generally no. The models have to be trained to do that, which, IMO would be a daunting task (one of these days vision models will be [AGI] Astronomer, Astrophysicists! Dreaming! Doubt it!).

This is one of the examples of response from Llama-3.2 11B Vision model when I input astrophotos to the model ( (high-end versions of the model, for example, online service offered by Nvidia behaves similar way) ... yeah, so much of astronomy, astrophysics info there!:

This image presents a stunning, vibrant nebula in space, with a striking blue center and a vibrant orange ring-like structure surrounding it. The nebula's core is a brilliant blue, surrounded by a ring of orange gas and dust, which gradually fades into a lighter hue towards the outer edges. The nebula's central region is characterized by a bright blue color, punctuated by a few small, white dots that appear to be stars.

The nebula's ring is surrounded by a black background, dotted with numerous small, white stars. The overall effect is one of breathtaking beauty and complexity, with the vibrant colors and intricate details of the nebula creating a sense of awe and wonder.

The image is likely a photograph taken by a telescope or camera, capturing the stunning beauty of the nebula in all its glory. The image's composition and color palette evoke a sense of cosmic wonder, inviting the viewer to contemplate the vastness and beauty of the universe.<|eot_id|>.

Hallucinated Answer Examples

The model used for AstroLLM Assistant is an Llama3 8 bit quantized 8B parameter (GGUF) version (model size: 8 GB-->8B parameter, each parameter represented with 8 bit (1 byte)). It will obviously suffer from accuracy compared to other higher-end models.

Here is an example of hallucinated response from this model when prompted with no context for Little Dumbbell Nebula (Barbell Nebula), which is wrong:

"Distance from Earth in light-years": 7,500,

"Size or Diameter in light-years": 1.5,

"Constellation": "Scorpius",

"NGC number": 6888,

"Formation": "Supernova remnant",

"Nebula type": "Emission"

High-end models or high-end online Gen-AI systems hallucinate substantially too. More examples: astronomy example, network/system design example.

Images (except GAIA image), system/development architecture are personal

Astronomy information sources (for RAG): Wiki, NASA, ESA

Title Image: Dumbbell Nebula snapped by MZH

Page updated

Google Sites

Report abuse