5 STAR AI.IO
TOOLS
FOR YOUR BUSINESS
HELLO & WELCOME TO THE
5 STAR AI.IO
TOOLS
FOR YOUR BUSINESS
PrivateGPT
Generate Your First Professional PrivateGPT PROJECT & Get Your BUSINESS 2 Another Level.
How To Install PrivateGPT - Chat With PDF, TXT, and CSV Files Privately! (Quick Setup Guide)
How To Install PrivateGPT - Chat With PDF, TXT, and CSV Files Privately! (Quick Setup Guide)
Transcript
PART 1
0:00
today we're going to take a look at
0:01
private GPT which is a new project that
0:04
is currently the number one trending
0:05
project on GitHub and it allows you to
0:08
load up documents text files PDF files
0:11
and ask questions about them using a
0:13
large language model the best part is
0:15
it's completely private hence the name
0:17
you can install it locally on your
0:19
machine it's completely open source and
0:21
it doesn't require you to send your
0:22
information to any external sources and
0:25
it also means you could do this with no
0:27
internet connection required it uses one
0:29
of the recent GPT for all models and it
0:31
works quite well let's take a look I'm
0:33
going to drop the GitHub repo Link in
0:35
the description below but here's the
0:37
instructions and I'm going to walk you
0:39
through step by step so the first thing
0:40
you're going to do is open a visual
0:42
studio code then you're going to click
0:44
file in the top left new window once you
0:47
have that open you're going to click the
0:49
toggle panel button up in the top right
0:51
and that opens up a terminal in Visual
0:53
Studio code give it a second to load up
0:55
let's switch back to the GitHub repo and
0:57
in the GitHub repo you're going to click
0:59
this little green code button and you're
1:01
going to click the copy button to get
1:03
the URL of the GitHub repo then we're
1:06
going to switch back to visual studio
1:08
code we're going to change directories
1:10
to our desktop CD desktop then once we
1:14
have that URL we're going to paste it in
1:15
here with Git clone so git clone and
1:18
then the URL
1:19
that will clone it to our desktop now
1:22
we're going to navigate to that folder
1:24
CD
1:25
private GPT
1:27
enter and now we're in the folder once
1:30
we have the private GPT folder open in
1:32
Visual Studio code the next thing we're
1:34
going to do is install the requirements
1:36
and that's done by typing pip install
1:38
Dash R requirements.txt and then hit
1:41
enter
1:43
now I've already gone through this so
1:45
all of the requirements are already
1:47
satisfied but this may take you a little
1:49
while if you haven't already done it
1:50
next on the left side there's a file
1:52
called example.nf and we're going to
1:55
right click we're going to go down to
1:56
rename and we're going to rename that
1:58
just dot m so delete the example text
2:02
hit enter let's take a look
2:05
this sets up the environment variables
2:07
that we need to actually run private GPT
2:10
now I don't need to change anything but
2:13
if you did want to change things this is
2:15
where you would do it we're going to be
2:16
using all of the default settings the
2:18
next thing you're going to do is
2:19
download the models and to get those
2:21
models look about halfway down the
2:23
GitHub repo page and there are two links
2:26
you need both of them the ggml GPT for
2:30
all J version 1.3 groovy and the ggml
2:34
model Q4 underscore zero dot bin file
2:37
download them both I've already done
PART 2
2:39
that there are a few gigabytes each so
2:41
once you've done that you need to create
2:43
a folder called models so I'm going to
2:45
come here on the left side I'm going to
2:47
right click new folder
2:49
and type models enter and that creates
2:52
that new folder called models then I'm
2:55
going to take the two models that I
2:56
downloaded and move them into the models
2:58
folder and then they are they appear
3:00
right there now once you're done with
3:01
that you can load up all the files that
3:04
you want Into The Source documents
3:06
folder so that's right here right now
3:09
when you download it it comes with a
3:10
file called State of the Union dot text
3:12
which is the recent State of the Union
3:14
Address that's where you're going to put
3:16
any documents that you want to ask
3:18
questions about you can right now use
3:20
text files PDF files and CSV files the
3:23
next thing we're going to do is ingest
3:25
the files now this is basically taking
3:28
those files chunking them up storing
3:30
them in a database so that we can use
3:32
the GPT for all model to actually ask
3:34
questions about it so to do that you
3:36
want to click the ingest.pi file on the
3:39
left side
3:41
and then in the top right there's a
3:42
little play button and go ahead and
3:44
click play
3:46
now this takes a really long time
3:48
especially depending on if you have an
3:50
older machine so I have a brand new
3:52
Macbook Pro and it took a couple minutes
3:55
so it's running now on my machine I'm
3:57
going to fast forward through this now
3:58
this puts a heavy strain on your CPU so
4:01
if you hear your fan turn on that's why
4:03
okay it just finished it took about five
4:05
minutes the last thing we need to do is
4:08
actually run the application so on the
4:09
left side we're going to click
4:11
privategpt.pi and then in the top right
4:13
we'll click play and it says enter a
4:16
query what did the president say about
4:17
Russia now this takes a little while but
4:20
not too long here it is
4:22
the president has spoken out against
4:24
Russia's actions in Ukraine stating that
4:26
they are a threat to peace and stability
4:28
in Europe and that the United States and
4:30
its allies will defend NATO countries in
4:32
the event of further aggression gives us
4:35
the question we asked
4:36
gave us the answer and it actually gave
4:39
sources for the answer so that's it
4:42
that's how to install private GPT and
4:44
again you can run this completely
4:45
locally no internet connection required
4:47
it's pretty amazing now when I was
4:50
installing this I ran into a couple
4:51
issues around versioning and python
4:54
modules but that only really has to do
4:55
with my environment and you wouldn't run
4:57
into the same thing but if you need help
4:59
debugging anything remember I just
5:01
launched a Discord and you can jump in
5:03
and ask for help link will be in the
5:05
description below to that Discord if you
5:07
like this video please consider giving a
5:09
like And subscribe and I'll see you in
5:10
the next one
Taking Your Existing Business With PrivateGPT
imartinez Merge pull request #224 from imartinez/feature/sentence-transformers-…
…
Git stats
Files
Type
Name
Latest commit message
Commit time
15 hours ago
2 weeks ago
2 weeks ago
2 weeks ago
10 hours ago
fix persist db directory at ingestion
5 days ago
Update code to use sentence-transformers through huggingfaceembeddings
11 hours ago
10 hours ago
Update code to use sentence-transformers through huggingfaceembeddings
11 hours ago
pypandoc-binary replacing pandoc-binary
3 hours ago
privateGPT
Ask questions to your documents without an internet connection, using the power of LLMs. 100% private, no data leaves your execution environment at any point. You can ingest documents and ask questions without an internet connection!
Built with LangChain, GPT4All, LlamaCpp, Chroma and SentenceTransformers.
Environment Setup
In order to set your environment up to run the code here, first install all requirements:
pip install -r requirements.txt
Then, download the LLM model and place it in a directory of your choice:
LLM: default to ggml-gpt4all-j-v1.3-groovy.bin. If you prefer a different GPT4All-J compatible model, just download it and reference it in your .env file.
Rename example.env to .env and edit the variables appropriately.
MODEL_TYPE: supports LlamaCpp or GPT4All
PERSIST_DIRECTORY: is the folder you want your vectorstore in
MODEL_PATH: Path to your GPT4All or LlamaCpp supported LLM
MODEL_N_CTX: Maximum token limit for the LLM model
EMBEDDINGS_MODEL_NAME: SentenceTransformers embeddings model name (see https://www.sbert.net/docs/pretrained_models.html)
Note: because of the way langchain loads the SentenceTransformers embeddings, the first time you run the script it will require internet connection to download the embeddings model itself.
Test dataset
This repo uses a state of the union transcript as an example.
Instructions for ingesting your own dataset
Put any and all your files into the source_documents directory
The supported extensions are:
.csv: CSV,
.docx: Word Document,
.enex: EverNote,
.eml: Email,
.epub: EPub,
.html: HTML File,
.md: Markdown,
.msg: Outlook Message,
.odt: Open Document Text,
.pdf: Portable Document Format (PDF),
.pptx : PowerPoint Document,
.txt: Text file (UTF-8),
Run the following command to ingest all the data.
python ingest.py
It will create a db folder containing the local vectorstore. Will take 20-30 seconds per document, depending on the size of the document. You can ingest as many documents as you want, and all will be accumulated in the local embeddings database. If you want to start from an empty database, delete the db folder.
Note: during the ingest process no data leaves your local environment. You could ingest without an internet connection, except for the first time you run the ingest script, when the embeddings model is downloaded.
Ask questions to your documents, locally!
In order to ask a question, run a command like:
python privateGPT.py
And wait for the script to require your input.
> Enter a query:
Hit enter. You'll need to wait 20-30 seconds (depending on your machine) while the LLM model consumes the prompt and prepares the answer. Once done, it will print the answer and the 4 sources it used as context from your documents; you can then ask another question without re-running the script, just wait for the prompt again.
Note: you could turn off your internet connection, and the script inference would still work. No data gets out of your local environment.
Type exit to finish the script.
How does it work?
Selecting the right local models and the power of LangChain you can run the entire pipeline locally, without any data leaving your environment, and with reasonable performance.
ingest.py uses LangChain tools to parse the document and create embeddings locally using HuggingFaceEmbeddings (SentenceTransformers). It then stores the result in a local vector database using Chroma vector store.
privateGPT.py uses a local LLM based on GPT4All-J or LlamaCpp to understand questions and create answers. The context for the answers is extracted from the local vector store using a similarity search to locate the right piece of context from the docs.
GPT4All-J wrapper was introduced in LangChain 0.0.162.
System Requirements
Python Version
To use this software, you must have Python 3.10 or later installed. Earlier versions of Python will not compile.
C++ Compiler
If you encounter an error while building a wheel during the pip install process, you may need to install a C++ compiler on your computer.
For Windows 10/11
To install a C++ compiler on Windows 10/11, follow these steps:
Install Visual Studio 2022.
Make sure the following components are selected:
Universal Windows Platform development
C++ CMake tools for Windows
Download the MinGW installer from the MinGW website.
Run the installer and select the gcc component.
Disclaimer
This is a test project to validate the feasibility of a fully private solution for question answering using LLMs and Vector embeddings. It is not production ready, and it is not meant to be used in production. The models selection is not optimized for performance, but for privacy; but it is possible to use different models and vectorstores to improve performance.
Google Colab Version (very very slow)
# Clone the repo
!git clone https://github.com/imartinez/privateGPT.git
Cloning into 'privateGPT'...
remote: Enumerating objects: 128, done.
remote: Counting objects: 100% (77/77), done.
remote: Compressing objects: 100% (34/34), done.
remote: Total 128 (delta 57), reused 47 (delta 43), pack-reused 51
Receiving objects: 100% (128/128), 55.94 KiB | 7.99 MiB/s, done.
Resolving deltas: 100% (64/64), done.
[ ]
# go to the new directory
%cd privateGPT/
/content/privateGPT
[ ]
# Install requirements
!pip install -r requirements.txt
Looking in indexes: https://pypi.org/simple, https://us-python.pkg.dev/colab-wheels/public/simple/
Collecting langchain==0.0.166 (from -r requirements.txt (line 1))
Downloading langchain-0.0.166-py3-none-any.whl (803 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 803.5/803.5 kB 11.3 MB/s eta 0:00:00
Collecting pygpt4all==1.1.0 (from -r requirements.txt (line 2))
Downloading pygpt4all-1.1.0.tar.gz (4.4 kB)
Preparing metadata (setup.py) ... done
Collecting chromadb==0.3.22 (from -r requirements.txt (line 3))
Downloading chromadb-0.3.22-py3-none-any.whl (69 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 69.2/69.2 kB 4.8 MB/s eta 0:00:00
Collecting llama-cpp-python==0.1.48 (from -r requirements.txt (line 4))
Downloading llama_cpp_python-0.1.48.tar.gz (1.1 MB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 1.1/1.1 MB 37.8 MB/s eta 0:00:00
Installing build dependencies ... done
Getting requirements to build wheel ... done
Preparing metadata (pyproject.toml) ... done
Collecting urllib3==1.26.6 (from -r requirements.txt (line 5))
Downloading urllib3-1.26.6-py2.py3-none-any.whl (138 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 138.5/138.5 kB 14.3 MB/s eta 0:00:00
Collecting pdfminer.six==20221105 (from -r requirements.txt (line 6))
Downloading pdfminer.six-20221105-py3-none-any.whl (5.6 MB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 5.6/5.6 MB 89.2 MB/s eta 0:00:00
Requirement already satisfied: PyYAML>=5.4.1 in /usr/local/lib/python3.10/dist-packages (from langchain==0.0.166->-r requirements.txt (line 1)) (6.0)
Requirement already satisfied: SQLAlchemy<3,>=1.4 in /usr/local/lib/python3.10/dist-packages (from langchain==0.0.166->-r requirements.txt (line 1)) (2.0.10)
Collecting aiohttp<4.0.0,>=3.8.3 (from langchain==0.0.166->-r requirements.txt (line 1))
Downloading aiohttp-3.8.4-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (1.0 MB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 1.0/1.0 MB 63.7 MB/s eta 0:00:00
Collecting async-timeout<5.0.0,>=4.0.0 (from langchain==0.0.166->-r requirements.txt (line 1))
Downloading async_timeout-4.0.2-py3-none-any.whl (5.8 kB)
Collecting dataclasses-json<0.6.0,>=0.5.7 (from langchain==0.0.166->-r requirements.txt (line 1))
Downloading dataclasses_json-0.5.7-py3-none-any.whl (25 kB)
Requirement already satisfied: numexpr<3.0.0,>=2.8.4 in /usr/local/lib/python3.10/dist-packages (from langchain==0.0.166->-r requirements.txt (line 1)) (2.8.4)
Requirement already satisfied: numpy<2,>=1 in /usr/local/lib/python3.10/dist-packages (from langchain==0.0.166->-r requirements.txt (line 1)) (1.22.4)
Collecting openapi-schema-pydantic<2.0,>=1.2 (from langchain==0.0.166->-r requirements.txt (line 1))
Downloading openapi_schema_pydantic-1.2.4-py3-none-any.whl (90 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 90.0/90.0 kB 10.0 MB/s eta 0:00:00
Requirement already satisfied: pydantic<2,>=1 in /usr/local/lib/python3.10/dist-packages (from langchain==0.0.166->-r requirements.txt (line 1)) (1.10.7)
Requirement already satisfied: requests<3,>=2 in /usr/local/lib/python3.10/dist-packages (from langchain==0.0.166->-r requirements.txt (line 1)) (2.27.1)
Requirement already satisfied: tenacity<9.0.0,>=8.1.0 in /usr/local/lib/python3.10/dist-packages (from langchain==0.0.166->-r requirements.txt (line 1)) (8.2.2)
Requirement already satisfied: tqdm>=4.48.0 in /usr/local/lib/python3.10/dist-packages (from langchain==0.0.166->-r requirements.txt (line 1)) (4.65.0)
Collecting pyllamacpp (from pygpt4all==1.1.0->-r requirements.txt (line 2))
Downloading pyllamacpp-2.1.3-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (273 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 273.3/273.3 kB 23.9 MB/s eta 0:00:00
Collecting pygptj (from pygpt4all==1.1.0->-r requirements.txt (line 2))
Downloading pygptj-2.0.3-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (246 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 246.0/246.0 kB 20.1 MB/s eta 0:00:00
Requirement already satisfied: pandas>=1.3 in /usr/local/lib/python3.10/dist-packages (from chromadb==0.3.22->-r requirements.txt (line 3)) (1.5.3)
Collecting requests<3,>=2 (from langchain==0.0.166->-r requirements.txt (line 1))
Downloading requests-2.30.0-py3-none-any.whl (62 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 62.5/62.5 kB 7.5 MB/s eta 0:00:00
Collecting hnswlib>=0.7 (from chromadb==0.3.22->-r requirements.txt (line 3))
Downloading hnswlib-0.7.0.tar.gz (33 kB)
Installing build dependencies ... done
Getting requirements to build wheel ... done
Preparing metadata (pyproject.toml) ... done
Collecting clickhouse-connect>=0.5.7 (from chromadb==0.3.22->-r requirements.txt (line 3))
Downloading clickhouse_connect-0.5.24-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (922 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 922.6/922.6 kB 33.8 MB/s eta 0:00:00
Collecting sentence-transformers>=2.2.2 (from chromadb==0.3.22->-r requirements.txt (line 3))
Downloading sentence-transformers-2.2.2.tar.gz (85 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 86.0/86.0 kB 9.3 MB/s eta 0:00:00
Preparing metadata (setup.py) ... done
Requirement already satisfied: duckdb>=0.7.1 in /usr/local/lib/python3.10/dist-packages (from chromadb==0.3.22->-r requirements.txt (line 3)) (0.7.1)
Collecting fastapi>=0.85.1 (from chromadb==0.3.22->-r requirements.txt (line 3))
Downloading fastapi-0.95.1-py3-none-any.whl (56 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 57.0/57.0 kB 5.9 MB/s eta 0:00:00
Collecting uvicorn[standard]>=0.18.3 (from chromadb==0.3.22->-r requirements.txt (line 3))
Downloading uvicorn-0.22.0-py3-none-any.whl (58 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 58.3/58.3 kB 6.4 MB/s eta 0:00:00
Collecting posthog>=2.4.0 (from chromadb==0.3.22->-r requirements.txt (line 3))
Downloading posthog-3.0.1-py2.py3-none-any.whl (37 kB)
Requirement already satisfied: typing-extensions>=4.5.0 in /usr/local/lib/python3.10/dist-packages (from chromadb==0.3.22->-r requirements.txt (line 3)) (4.5.0)
Requirement already satisfied: charset-normalizer>=2.0.0 in /usr/local/lib/python3.10/dist-packages (from pdfminer.six==20221105->-r requirements.txt (line 6)) (2.0.12)
Requirement already satisfied: cryptography>=36.0.0 in /usr/local/lib/python3.10/dist-packages (from pdfminer.six==20221105->-r requirements.txt (line 6)) (40.0.2)
Requirement already satisfied: attrs>=17.3.0 in /usr/local/lib/python3.10/dist-packages (from aiohttp<4.0.0,>=3.8.3->langchain==0.0.166->-r requirements.txt (line 1)) (23.1.0)
Collecting multidict<7.0,>=4.5 (from aiohttp<4.0.0,>=3.8.3->langchain==0.0.166->-r requirements.txt (line 1))
Downloading multidict-6.0.4-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (114 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 114.5/114.5 kB 1.8 MB/s eta 0:00:00
Collecting yarl<2.0,>=1.0 (from aiohttp<4.0.0,>=3.8.3->langchain==0.0.166->-r requirements.txt (line 1))
Downloading yarl-1.9.2-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (268 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 268.8/268.8 kB 24.6 MB/s eta 0:00:00
Collecting frozenlist>=1.1.1 (from aiohttp<4.0.0,>=3.8.3->langchain==0.0.166->-r requirements.txt (line 1))
Downloading frozenlist-1.3.3-cp310-cp310-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (149 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 149.6/149.6 kB 15.1 MB/s eta 0:00:00
Collecting aiosignal>=1.1.2 (from aiohttp<4.0.0,>=3.8.3->langchain==0.0.166->-r requirements.txt (line 1))
Downloading aiosignal-1.3.1-py3-none-any.whl (7.6 kB)
Requirement already satisfied: certifi in /usr/local/lib/python3.10/dist-packages (from clickhouse-connect>=0.5.7->chromadb==0.3.22->-r requirements.txt (line 3)) (2022.12.7)
Requirement already satisfied: pytz in /usr/local/lib/python3.10/dist-packages (from clickhouse-connect>=0.5.7->chromadb==0.3.22->-r requirements.txt (line 3)) (2022.7.1)
Collecting zstandard (from clickhouse-connect>=0.5.7->chromadb==0.3.22->-r requirements.txt (line 3))
Downloading zstandard-0.21.0-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (2.7 MB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 2.7/2.7 MB 62.0 MB/s eta 0:00:00
Collecting lz4 (from clickhouse-connect>=0.5.7->chromadb==0.3.22->-r requirements.txt (line 3))
Downloading lz4-4.3.2-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (1.3 MB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 1.3/1.3 MB 56.4 MB/s eta 0:00:00
Requirement already satisfied: cffi>=1.12 in /usr/local/lib/python3.10/dist-packages (from cryptography>=36.0.0->pdfminer.six==20221105->-r requirements.txt (line 6)) (1.15.1)
Collecting marshmallow<4.0.0,>=3.3.0 (from dataclasses-json<0.6.0,>=0.5.7->langchain==0.0.166->-r requirements.txt (line 1))
Downloading marshmallow-3.19.0-py3-none-any.whl (49 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 49.1/49.1 kB 5.0 MB/s eta 0:00:00
Collecting marshmallow-enum<2.0.0,>=1.5.1 (from dataclasses-json<0.6.0,>=0.5.7->langchain==0.0.166->-r requirements.txt (line 1))
Downloading marshmallow_enum-1.5.1-py2.py3-none-any.whl (4.2 kB)
Collecting typing-inspect>=0.4.0 (from dataclasses-json<0.6.0,>=0.5.7->langchain==0.0.166->-r requirements.txt (line 1))
Downloading typing_inspect-0.8.0-py3-none-any.whl (8.7 kB)
Collecting starlette<0.27.0,>=0.26.1 (from fastapi>=0.85.1->chromadb==0.3.22->-r requirements.txt (line 3))
Downloading starlette-0.26.1-py3-none-any.whl (66 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 66.9/66.9 kB 6.1 MB/s eta 0:00:00
Requirement already satisfied: python-dateutil>=2.8.1 in /usr/local/lib/python3.10/dist-packages (from pandas>=1.3->chromadb==0.3.22->-r requirements.txt (line 3)) (2.8.2)
Requirement already satisfied: six>=1.5 in /usr/local/lib/python3.10/dist-packages (from posthog>=2.4.0->chromadb==0.3.22->-r requirements.txt (line 3)) (1.16.0)
Collecting monotonic>=1.5 (from posthog>=2.4.0->chromadb==0.3.22->-r requirements.txt (line 3))
Downloading monotonic-1.6-py2.py3-none-any.whl (8.2 kB)
Collecting backoff>=1.10.0 (from posthog>=2.4.0->chromadb==0.3.22->-r requirements.txt (line 3))
Downloading backoff-2.2.1-py3-none-any.whl (15 kB)
Requirement already satisfied: idna<4,>=2.5 in /usr/local/lib/python3.10/dist-packages (from requests<3,>=2->langchain==0.0.166->-r requirements.txt (line 1)) (3.4)
Collecting transformers<5.0.0,>=4.6.0 (from sentence-transformers>=2.2.2->chromadb==0.3.22->-r requirements.txt (line 3))
Downloading transformers-4.29.1-py3-none-any.whl (7.1 MB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 7.1/7.1 MB 67.7 MB/s eta 0:00:00
Requirement already satisfied: torch>=1.6.0 in /usr/local/lib/python3.10/dist-packages (from sentence-transformers>=2.2.2->chromadb==0.3.22->-r requirements.txt (line 3)) (2.0.0+cu118)
Requirement already satisfied: torchvision in /usr/local/lib/python3.10/dist-packages (from sentence-transformers>=2.2.2->chromadb==0.3.22->-r requirements.txt (line 3)) (0.15.1+cu118)
Requirement already satisfied: scikit-learn in /usr/local/lib/python3.10/dist-packages (from sentence-transformers>=2.2.2->chromadb==0.3.22->-r requirements.txt (line 3)) (1.2.2)
Requirement already satisfied: scipy in /usr/local/lib/python3.10/dist-packages (from sentence-transformers>=2.2.2->chromadb==0.3.22->-r requirements.txt (line 3)) (1.10.1)
Requirement already satisfied: nltk in /usr/local/lib/python3.10/dist-packages (from sentence-transformers>=2.2.2->chromadb==0.3.22->-r requirements.txt (line 3)) (3.8.1)
Collecting sentencepiece (from sentence-transformers>=2.2.2->chromadb==0.3.22->-r requirements.txt (line 3))
Downloading sentencepiece-0.1.99-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (1.3 MB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 1.3/1.3 MB 64.7 MB/s eta 0:00:00
Collecting huggingface-hub>=0.4.0 (from sentence-transformers>=2.2.2->chromadb==0.3.22->-r requirements.txt (line 3))
Downloading huggingface_hub-0.14.1-py3-none-any.whl (224 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 224.5/224.5 kB 16.5 MB/s eta 0:00:00
Requirement already satisfied: greenlet!=0.4.17 in /usr/local/lib/python3.10/dist-packages (from SQLAlchemy<3,>=1.4->langchain==0.0.166->-r requirements.txt (line 1)) (2.0.2)
Requirement already satisfied: click>=7.0 in /usr/local/lib/python3.10/dist-packages (from uvicorn[standard]>=0.18.3->chromadb==0.3.22->-r requirements.txt (line 3)) (8.1.3)
Collecting h11>=0.8 (from uvicorn[standard]>=0.18.3->chromadb==0.3.22->-r requirements.txt (line 3))
Downloading h11-0.14.0-py3-none-any.whl (58 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 58.3/58.3 kB 6.5 MB/s eta 0:00:00
Collecting httptools>=0.5.0 (from uvicorn[standard]>=0.18.3->chromadb==0.3.22->-r requirements.txt (line 3))
Downloading httptools-0.5.0-cp310-cp310-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (414 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 414.1/414.1 kB 37.6 MB/s eta 0:00:00
Collecting python-dotenv>=0.13 (from uvicorn[standard]>=0.18.3->chromadb==0.3.22->-r requirements.txt (line 3))
Downloading python_dotenv-1.0.0-py3-none-any.whl (19 kB)
Collecting uvloop!=0.15.0,!=0.15.1,>=0.14.0 (from uvicorn[standard]>=0.18.3->chromadb==0.3.22->-r requirements.txt (line 3))
Downloading uvloop-0.17.0-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (4.1 MB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 4.1/4.1 MB 97.8 MB/s eta 0:00:00
Collecting watchfiles>=0.13 (from uvicorn[standard]>=0.18.3->chromadb==0.3.22->-r requirements.txt (line 3))
Downloading watchfiles-0.19.0-cp37-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (1.3 MB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 1.3/1.3 MB 67.5 MB/s eta 0:00:00
Collecting websockets>=10.4 (from uvicorn[standard]>=0.18.3->chromadb==0.3.22->-r requirements.txt (line 3))
Downloading websockets-11.0.3-cp310-cp310-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (129 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 129.9/129.9 kB 14.2 MB/s eta 0:00:00
Requirement already satisfied: pycparser in /usr/local/lib/python3.10/dist-packages (from cffi>=1.12->cryptography>=36.0.0->pdfminer.six==20221105->-r requirements.txt (line 6)) (2.21)
Requirement already satisfied: filelock in /usr/local/lib/python3.10/dist-packages (from huggingface-hub>=0.4.0->sentence-transformers>=2.2.2->chromadb==0.3.22->-r requirements.txt (line 3)) (3.12.0)
Requirement already satisfied: fsspec in /usr/local/lib/python3.10/dist-packages (from huggingface-hub>=0.4.0->sentence-transformers>=2.2.2->chromadb==0.3.22->-r requirements.txt (line 3)) (2023.4.0)
Requirement already satisfied: packaging>=20.9 in /usr/local/lib/python3.10/dist-packages (from huggingface-hub>=0.4.0->sentence-transformers>=2.2.2->chromadb==0.3.22->-r requirements.txt (line 3)) (23.1)
Requirement already satisfied: anyio<5,>=3.4.0 in /usr/local/lib/python3.10/dist-packages (from starlette<0.27.0,>=0.26.1->fastapi>=0.85.1->chromadb==0.3.22->-r requirements.txt (line 3)) (3.6.2)
Requirement already satisfied: sympy in /usr/local/lib/python3.10/dist-packages (from torch>=1.6.0->sentence-transformers>=2.2.2->chromadb==0.3.22->-r requirements.txt (line 3)) (1.11.1)
Requirement already satisfied: networkx in /usr/local/lib/python3.10/dist-packages (from torch>=1.6.0->sentence-transformers>=2.2.2->chromadb==0.3.22->-r requirements.txt (line 3)) (3.1)
Requirement already satisfied: jinja2 in /usr/local/lib/python3.10/dist-packages (from torch>=1.6.0->sentence-transformers>=2.2.2->chromadb==0.3.22->-r requirements.txt (line 3)) (3.1.2)
Requirement already satisfied: triton==2.0.0 in /usr/local/lib/python3.10/dist-packages (from torch>=1.6.0->sentence-transformers>=2.2.2->chromadb==0.3.22->-r requirements.txt (line 3)) (2.0.0)
Requirement already satisfied: cmake in /usr/local/lib/python3.10/dist-packages (from triton==2.0.0->torch>=1.6.0->sentence-transformers>=2.2.2->chromadb==0.3.22->-r requirements.txt (line 3)) (3.25.2)
Requirement already satisfied: lit in /usr/local/lib/python3.10/dist-packages (from triton==2.0.0->torch>=1.6.0->sentence-transformers>=2.2.2->chromadb==0.3.22->-r requirements.txt (line 3)) (16.0.3)
Requirement already satisfied: regex!=2019.12.17 in /usr/local/lib/python3.10/dist-packages (from transformers<5.0.0,>=4.6.0->sentence-transformers>=2.2.2->chromadb==0.3.22->-r requirements.txt (line 3)) (2022.10.31)
Collecting tokenizers!=0.11.3,<0.14,>=0.11.1 (from transformers<5.0.0,>=4.6.0->sentence-transformers>=2.2.2->chromadb==0.3.22->-r requirements.txt (line 3))
Downloading tokenizers-0.13.3-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (7.8 MB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 7.8/7.8 MB 91.0 MB/s eta 0:00:00
Collecting mypy-extensions>=0.3.0 (from typing-inspect>=0.4.0->dataclasses-json<0.6.0,>=0.5.7->langchain==0.0.166->-r requirements.txt (line 1))
Downloading mypy_extensions-1.0.0-py3-none-any.whl (4.7 kB)
Requirement already satisfied: joblib in /usr/local/lib/python3.10/dist-packages (from nltk->sentence-transformers>=2.2.2->chromadb==0.3.22->-r requirements.txt (line 3)) (1.2.0)
Requirement already satisfied: threadpoolctl>=2.0.0 in /usr/local/lib/python3.10/dist-packages (from scikit-learn->sentence-transformers>=2.2.2->chromadb==0.3.22->-r requirements.txt (line 3)) (3.1.0)
Requirement already satisfied: pillow!=8.3.*,>=5.3.0 in /usr/local/lib/python3.10/dist-packages (from torchvision->sentence-transformers>=2.2.2->chromadb==0.3.22->-r requirements.txt (line 3)) (8.4.0)
Requirement already satisfied: sniffio>=1.1 in /usr/local/lib/python3.10/dist-packages (from anyio<5,>=3.4.0->starlette<0.27.0,>=0.26.1->fastapi>=0.85.1->chromadb==0.3.22->-r requirements.txt (line 3)) (1.3.0)
Requirement already satisfied: MarkupSafe>=2.0 in /usr/local/lib/python3.10/dist-packages (from jinja2->torch>=1.6.0->sentence-transformers>=2.2.2->chromadb==0.3.22->-r requirements.txt (line 3)) (2.1.2)
Requirement already satisfied: mpmath>=0.19 in /usr/local/lib/python3.10/dist-packages (from sympy->torch>=1.6.0->sentence-transformers>=2.2.2->chromadb==0.3.22->-r requirements.txt (line 3)) (1.3.0)
Building wheels for collected packages: pygpt4all, llama-cpp-python, hnswlib, sentence-transformers
Building wheel for pygpt4all (setup.py) ... done
Created wheel for pygpt4all: filename=pygpt4all-1.1.0-py3-none-any.whl size=5842 sha256=0dbaeca084ffe2b619700beb3b135b1c8bdee10b2b3886016eb90eb5bce7e2e4
Stored in directory: /root/.cache/pip/wheels/48/e9/c6/7b2548560f0eb92b4c1e4159d8aaeab499240a715d7481e975
Building wheel for llama-cpp-python (pyproject.toml) ... done
Created wheel for llama-cpp-python: filename=llama_cpp_python-0.1.48-cp310-cp310-linux_x86_64.whl size=184481 sha256=7fce06c2c4894481524b18757bd690772014a6e495ab2fda14bfa5b35851999f
Stored in directory: /root/.cache/pip/wheels/eb/06/0e/6ae7b299ed252075128644d31384cac683e9fd768a8538c6be
Building wheel for hnswlib (pyproject.toml) ... done
Created wheel for hnswlib: filename=hnswlib-0.7.0-cp310-cp310-linux_x86_64.whl size=2119825 sha256=e3b2ce92417a5b21d14d5674ba7effff9cc1946df5a48126f4beb5914ccc78eb
Stored in directory: /root/.cache/pip/wheels/8a/ae/ec/235a682e0041fbaeee389843670581ec6c66872db856dfa9a4
Building wheel for sentence-transformers (setup.py) ... done
Created wheel for sentence-transformers: filename=sentence_transformers-2.2.2-py3-none-any.whl size=125926 sha256=bd01429dd41fe0662c4fe33842ddb769a0ea5d00c52ec4dd1bb20efd302ffbfb
Stored in directory: /root/.cache/pip/wheels/62/f2/10/1e606fd5f02395388f74e7462910fe851042f97238cbbd902f
Successfully built pygpt4all llama-cpp-python hnswlib sentence-transformers
Installing collected packages: tokenizers, sentencepiece, monotonic, zstandard, websockets, uvloop, urllib3, python-dotenv, pyllamacpp, pygptj, mypy-extensions, multidict, marshmallow, lz4, llama-cpp-python, httptools, hnswlib, h11, frozenlist, backoff, async-timeout, yarl, watchfiles, uvicorn, typing-inspect, starlette, requests, pygpt4all, openapi-schema-pydantic, marshmallow-enum, clickhouse-connect, aiosignal, posthog, pdfminer.six, huggingface-hub, fastapi, dataclasses-json, aiohttp, transformers, langchain, sentence-transformers, chromadb
Attempting uninstall: urllib3
Found existing installation: urllib3 1.26.15
Uninstalling urllib3-1.26.15:
Successfully uninstalled urllib3-1.26.15
Attempting uninstall: requests
Found existing installation: requests 2.27.1
Uninstalling requests-2.27.1:
Successfully uninstalled requests-2.27.1
Successfully installed aiohttp-3.8.4 aiosignal-1.3.1 async-timeout-4.0.2 backoff-2.2.1 chromadb-0.3.22 clickhouse-connect-0.5.24 dataclasses-json-0.5.7 fastapi-0.95.1 frozenlist-1.3.3 h11-0.14.0 hnswlib-0.7.0 httptools-0.5.0 huggingface-hub-0.14.1 langchain-0.0.166 llama-cpp-python-0.1.48 lz4-4.3.2 marshmallow-3.19.0 marshmallow-enum-1.5.1 monotonic-1.6 multidict-6.0.4 mypy-extensions-1.0.0 openapi-schema-pydantic-1.2.4 pdfminer.six-20221105 posthog-3.0.1 pygpt4all-1.1.0 pygptj-2.0.3 pyllamacpp-2.1.3 python-dotenv-1.0.0 requests-2.30.0 sentence-transformers-2.2.2 sentencepiece-0.1.99 starlette-0.26.1 tokenizers-0.13.3 transformers-4.29.1 typing-inspect-0.8.0 urllib3-1.26.6 uvicorn-0.22.0 uvloop-0.17.0 watchfiles-0.19.0 websockets-11.0.3 yarl-1.9.2 zstandard-0.21.0
[ ]
# make a directory called models
!mkdir models
[ ]
# Download the models. You should replace the URLs with the ones for the models you're going to use.
!wget -P models/ https://gpt4all.io/models/ggml-gpt4all-j-v1.3-groovy.bin
!wget -P models/ https://huggingface.co/Pi3141/alpaca-native-7B-ggml/resolve/397e872bf4c83f4c642317a5bf65ce84a105786e/ggml-model-q4_0.bin
--2023-05-14 17:46:30-- https://gpt4all.io/models/ggml-gpt4all-j-v1.3-groovy.bin
Resolving gpt4all.io (gpt4all.io)... 104.26.1.159, 172.67.71.169, 104.26.0.159, ...
Connecting to gpt4all.io (gpt4all.io)|104.26.1.159|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 3785248281 (3.5G)
Saving to: ‘models/ggml-gpt4all-j-v1.3-groovy.bin’
ggml-gpt4all-j-v1.3 100%[===================>] 3.52G 40.5MB/s in 97s
2023-05-14 17:48:07 (37.1 MB/s) - ‘models/ggml-gpt4all-j-v1.3-groovy.bin’ saved [3785248281/3785248281]
--2023-05-14 17:48:07-- https://huggingface.co/Pi3141/alpaca-native-7B-ggml/resolve/397e872bf4c83f4c642317a5bf65ce84a105786e/ggml-model-q4_0.bin
Resolving huggingface.co (huggingface.co)... 18.67.0.34, 18.67.0.55, 18.67.0.90, ...
Connecting to huggingface.co (huggingface.co)|18.67.0.34|:443... connected.
HTTP request sent, awaiting response... 302 Found
Resolving cdn-lfs.huggingface.co (cdn-lfs.huggingface.co)... 18.64.174.106, 18.64.174.110, 18.64.174.43, ...
Connecting to cdn-lfs.huggingface.co (cdn-lfs.huggingface.co)|18.64.174.106|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 4212727017 (3.9G) [application/octet-stream]
Saving to: ‘models/ggml-model-q4_0.bin’
ggml-model-q4_0.bin 100%[===================>] 3.92G 94.6MB/s in 40s
2023-05-14 17:48:48 (100 MB/s) - ‘models/ggml-model-q4_0.bin’ saved [4212727017/4212727017]
[ ]
# rename example.env to .env (if you want to change the settings, you can do so here)
!mv example.env .env
[ ]
# download any files (.txt, .pdf, or .csv) here that you want to chat with
# project comes with US constitution already in the source_documents/ folder
# !wget -P source-documents/ {URL OF DOCUMENT}
constants.py LICENSE privateGPT.py requirements.txt
ingest.py models/ README.md source_documents/
[ ]
# ingest the documents
!python ingest.py
Loading documents from source_documents
Loaded 1 documents from source_documents
Split into 90 chunks of text (max. 500 tokens each)
llama.cpp: loading model from models/ggml-model-q4_0.bin
llama.cpp: can't use mmap because tensors are not aligned; convert to new format to avoid this
llama_model_load_internal: format = 'ggml' (old version with low tokenizer quality and no mmap support)
llama_model_load_internal: n_vocab = 32000
llama_model_load_internal: n_ctx = 1000
llama_model_load_internal: n_embd = 4096
llama_model_load_internal: n_mult = 256
llama_model_load_internal: n_head = 32
llama_model_load_internal: n_layer = 32
llama_model_load_internal: n_rot = 128
llama_model_load_internal: ftype = 2 (mostly Q4_0)
llama_model_load_internal: n_ff = 11008
llama_model_load_internal: n_parts = 1
llama_model_load_internal: model size = 7B
llama_model_load_internal: ggml ctx size = 4113748.20 KB
llama_model_load_internal: mem required = 5809.33 MB (+ 2052.00 MB per state)
...................................................................................................
.
llama_init_from_file: kv self size = 1000.00 MB
AVX = 1 | AVX2 = 1 | AVX512 = 0 | AVX512_VBMI = 0 | AVX512_VNNI = 0 | FMA = 1 | NEON = 0 | ARM_FMA = 0 | F16C = 1 | FP16_VA = 0 | WASM_SIMD = 0 | BLAS = 0 | SSE3 = 1 | VSX = 0 |
Using embedded DuckDB with persistence: data will be stored in: db
llama_print_timings: load time = 7955.21 ms
llama_print_timings: sample time = 0.00 ms / 1 runs ( 0.00 ms per run)
llama_print_timings: prompt eval time = 93177.06 ms / 122 tokens ( 763.75 ms per token)
llama_print_timings: eval time = 0.00 ms / 1 runs ( 0.00 ms per run)
llama_print_timings: total time = 93267.25 ms
llama_print_timings: load time = 7955.21 ms
llama_print_timings: sample time = 0.00 ms / 1 runs ( 0.00 ms per run)
llama_print_timings: prompt eval time = 87678.71 ms / 116 tokens ( 755.85 ms per token)
llama_print_timings: eval time = 0.00 ms / 1 runs ( 0.00 ms per run)
llama_print_timings: total time = 87759.62 ms
llama_print_timings: load time = 7955.21 ms
llama_print_timings: sample time = 0.00 ms / 1 runs ( 0.00 ms per run)
llama_print_timings: prompt eval time = 70232.13 ms / 96 tokens ( 731.58 ms per token)
llama_print_timings: eval time = 1227.81 ms / 1 runs ( 1227.81 ms per run)
llama_print_timings: total time = 71530.95 ms
llama_print_timings: load time = 7955.21 ms
llama_print_timings: sample time = 0.00 ms / 1 runs ( 0.00 ms per run)
llama_print_timings: prompt eval time = 84546.91 ms / 112 tokens ( 754.88 ms per token)
llama_print_timings: eval time = 0.00 ms / 1 runs ( 0.00 ms per run)
llama_print_timings: total time = 84623.31 ms
llama_print_timings: load time = 7955.21 ms
llama_print_timings: sample time = 0.00 ms / 1 runs ( 0.00 ms per run)
llama_print_timings: prompt eval time = 85620.12 ms / 116 tokens ( 738.10 ms per token)
llama_print_timings: eval time = 0.00 ms / 1 runs ( 0.00 ms per run)
llama_print_timings: total time = 85696.59 ms
llama_print_timings: load time = 7955.21 ms
llama_print_timings: sample time = 0.00 ms / 1 runs ( 0.00 ms per run)
llama_print_timings: prompt eval time = 86097.65 ms / 114 tokens ( 755.24 ms per token)
llama_print_timings: eval time = 0.00 ms / 1 runs ( 0.00 ms per run)
llama_print_timings: total time = 86188.44 ms
llama_print_timings: load time = 7955.21 ms
llama_print_timings: sample time = 0.00 ms / 1 runs ( 0.00 ms per run)
llama_print_timings: prompt eval time = 74171.62 ms / 101 tokens ( 734.37 ms per token)
llama_print_timings: eval time = 0.00 ms / 1 runs ( 0.00 ms per run)
llama_print_timings: total time = 74244.07 ms
llama_print_timings: load time = 7955.21 ms
llama_print_timings: sample time = 0.00 ms / 1 runs ( 0.00 ms per run)
llama_print_timings: prompt eval time = 91600.91 ms / 117 tokens ( 782.91 ms per token)
llama_print_timings: eval time = 0.00 ms / 1 runs ( 0.00 ms per run)
llama_print_timings: total time = 91679.37 ms
llama_print_timings: load time = 7955.21 ms
llama_print_timings: sample time = 0.00 ms / 1 runs ( 0.00 ms per run)
llama_print_timings: prompt eval time = 52194.15 ms / 71 tokens ( 735.13 ms per token)
llama_print_timings: eval time = 0.00 ms / 1 runs ( 0.00 ms per run)
llama_print_timings: total time = 52240.89 ms
llama_print_timings: load time = 7955.21 ms
llama_print_timings: sample time = 0.00 ms / 1 runs ( 0.00 ms per run)
llama_print_timings: prompt eval time = 72686.63 ms / 96 tokens ( 757.15 ms per token)
llama_print_timings: eval time = 0.00 ms / 1 runs ( 0.00 ms per run)
llama_print_timings: total time = 72755.42 ms
llama_print_timings: load time = 7955.21 ms
llama_print_timings: sample time = 0.00 ms / 1 runs ( 0.00 ms per run)
llama_print_timings: prompt eval time = 77032.96 ms / 103 tokens ( 747.89 ms per token)
llama_print_timings: eval time = 0.00 ms / 1 runs ( 0.00 ms per run)
llama_print_timings: total time = 77106.54 ms
llama_print_timings: load time = 7955.21 ms
llama_print_timings: sample time = 0.00 ms / 1 runs ( 0.00 ms per run)
llama_print_timings: prompt eval time = 87515.78 ms / 119 tokens ( 735.43 ms per token)
llama_print_timings: eval time = 0.00 ms / 1 runs ( 0.00 ms per run)
llama_print_timings: total time = 87595.07 ms
llama_print_timings: load time = 7955.21 ms
llama_print_timings: sample time = 0.00 ms / 1 runs ( 0.00 ms per run)
llama_print_timings: prompt eval time = 86388.54 ms / 114 tokens ( 757.79 ms per token)
llama_print_timings: eval time = 0.00 ms / 1 runs ( 0.00 ms per run)
llama_print_timings: total time = 86464.31 ms
llama_print_timings: load time = 7955.21 ms
llama_print_timings: sample time = 0.00 ms / 1 runs ( 0.00 ms per run)
llama_print_timings: prompt eval time = 94732.57 ms / 127 tokens ( 745.93 ms per token)
llama_print_timings: eval time = 0.00 ms / 1 runs ( 0.00 ms per run)
llama_print_timings: total time = 94816.88 ms
llama_print_timings: load time = 7955.21 ms
llama_print_timings: sample time = 0.00 ms / 1 runs ( 0.00 ms per run)
llama_print_timings: prompt eval time = 95393.12 ms / 127 tokens ( 751.13 ms per token)
llama_print_timings: eval time = 0.00 ms / 1 runs ( 0.00 ms per run)
llama_print_timings: total time = 95484.94 ms
llama_print_timings: load time = 7955.21 ms
llama_print_timings: sample time = 0.00 ms / 1 runs ( 0.00 ms per run)
llama_print_timings: prompt eval time = 62926.63 ms / 84 tokens ( 749.13 ms per token)
llama_print_timings: eval time = 0.00 ms / 1 runs ( 0.00 ms per run)
llama_print_timings: total time = 62985.75 ms
llama_print_timings: load time = 7955.21 ms
llama_print_timings: sample time = 0.00 ms / 1 runs ( 0.00 ms per run)
llama_print_timings: prompt eval time = 93757.75 ms / 125 tokens ( 750.06 ms per token)
llama_print_timings: eval time = 0.00 ms / 1 runs ( 0.00 ms per run)
llama_print_timings: total time = 93848.74 ms
llama_print_timings: load time = 7955.21 ms
llama_print_timings: sample time = 0.00 ms / 1 runs ( 0.00 ms per run)
llama_print_timings: prompt eval time = 77850.29 ms / 104 tokens ( 748.56 ms per token)
llama_print_timings: eval time = 0.00 ms / 1 runs ( 0.00 ms per run)
llama_print_timings: total time = 77918.91 ms
llama_print_timings: load time = 7955.21 ms
llama_print_timings: sample time = 0.00 ms / 1 runs ( 0.00 ms per run)
llama_print_timings: prompt eval time = 103428.14 ms / 123 tokens ( 840.88 ms per token)
llama_print_timings: eval time = 0.00 ms / 1 runs ( 0.00 ms per run)
llama_print_timings: total time = 103529.21 ms
llama_print_timings: load time = 7955.21 ms
llama_print_timings: sample time = 0.00 ms / 1 runs ( 0.00 ms per run)
llama_print_timings: prompt eval time = 94194.69 ms / 125 tokens ( 753.56 ms per token)
llama_print_timings: eval time = 0.00 ms / 1 runs ( 0.00 ms per run)
llama_print_timings: total time = 94281.37 ms
llama_print_timings: load time = 7955.21 ms
llama_print_timings: sample time = 0.00 ms / 1 runs ( 0.00 ms per run)
llama_print_timings: prompt eval time = 81215.02 ms / 109 tokens ( 745.09 ms per token)
llama_print_timings: eval time = 0.00 ms / 1 runs ( 0.00 ms per run)
llama_print_timings: total time = 81290.37 ms
llama_print_timings: load time = 7955.21 ms
llama_print_timings: sample time = 0.00 ms / 1 runs ( 0.00 ms per run)
llama_print_timings: prompt eval time = 74170.18 ms / 98 tokens ( 756.84 ms per token)
llama_print_timings: eval time = 0.00 ms / 1 runs ( 0.00 ms per run)
llama_print_timings: total time = 74238.55 ms
llama_print_timings: load time = 7955.21 ms
llama_print_timings: sample time = 0.00 ms / 1 runs ( 0.00 ms per run)
llama_print_timings: prompt eval time = 104273.52 ms / 140 tokens ( 744.81 ms per token)
llama_print_timings: eval time = 0.00 ms / 1 runs ( 0.00 ms per run)
llama_print_timings: total time = 104377.89 ms
llama_print_timings: load time = 7955.21 ms
llama_print_timings: sample time = 0.00 ms / 1 runs ( 0.00 ms per run)
llama_print_timings: prompt eval time = 70830.69 ms / 95 tokens ( 745.59 ms per token)
llama_print_timings: eval time = 0.00 ms / 1 runs ( 0.00 ms per run)
llama_print_timings: total time = 70895.66 ms
llama_print_timings: load time = 7955.21 ms
llama_print_timings: sample time = 0.00 ms / 1 runs ( 0.00 ms per run)
llama_print_timings: prompt eval time = 84829.73 ms / 114 tokens ( 744.12 ms per token)
llama_print_timings: eval time = 0.00 ms / 1 runs ( 0.00 ms per run)
llama_print_timings: total time = 84912.33 ms
llama_print_timings: load time = 7955.21 ms
llama_print_timings: sample time = 0.00 ms / 1 runs ( 0.00 ms per run)
llama_print_timings: prompt eval time = 91037.11 ms / 119 tokens ( 765.02 ms per token)
llama_print_timings: eval time = 0.00 ms / 1 runs ( 0.00 ms per run)
llama_print_timings: total time = 91121.40 ms
llama_print_timings: load time = 7955.21 ms
llama_print_timings: sample time = 0.00 ms / 1 runs ( 0.00 ms per run)
llama_print_timings: prompt eval time = 83682.96 ms / 112 tokens ( 747.17 ms per token)
llama_print_timings: eval time = 1237.62 ms / 1 runs ( 1237.62 ms per run)
llama_print_timings: total time = 85006.08 ms
llama_print_timings: load time = 7955.21 ms
llama_print_timings: sample time = 0.00 ms / 1 runs ( 0.00 ms per run)
llama_print_timings: prompt eval time = 83240.91 ms / 111 tokens ( 749.92 ms per token)
llama_print_timings: eval time = 0.00 ms / 1 runs ( 0.00 ms per run)
llama_print_timings: total time = 83316.03 ms
llama_print_timings: load time = 7955.21 ms
llama_print_timings: sample time = 0.00 ms / 1 runs ( 0.00 ms per run)
llama_print_timings: prompt eval time = 98531.89 ms / 132 tokens ( 746.45 ms per token)
llama_print_timings: eval time = 0.00 ms / 1 runs ( 0.00 ms per run)
llama_print_timings: total time = 98626.29 ms
llama_print_timings: load time = 7955.21 ms
llama_print_timings: sample time = 0.00 ms / 1 runs ( 0.00 ms per run)
llama_print_timings: prompt eval time = 105580.93 ms / 134 tokens ( 787.92 ms per token)
llama_print_timings: eval time = 0.00 ms / 1 runs ( 0.00 ms per run)
llama_print_timings: total time = 105683.98 ms
llama_print_timings: load time = 7955.21 ms
llama_print_timings: sample time = 0.00 ms / 1 runs ( 0.00 ms per run)
llama_print_timings: prompt eval time = 108695.33 ms / 146 tokens ( 744.49 ms per token)
llama_print_timings: eval time = 0.00 ms / 1 runs ( 0.00 ms per run)
llama_print_timings: total time = 108804.10 ms
llama_print_timings: load time = 7955.21 ms
llama_print_timings: sample time = 0.00 ms / 1 runs ( 0.00 ms per run)
llama_print_timings: prompt eval time = 90086.93 ms / 122 tokens ( 738.42 ms per token)
llama_print_timings: eval time = 0.00 ms / 1 runs ( 0.00 ms per run)
llama_print_timings: total time = 90172.37 ms
llama_print_timings: load time = 7955.21 ms
llama_print_timings: sample time = 0.00 ms / 1 runs ( 0.00 ms per run)
llama_print_timings: prompt eval time = 98030.14 ms / 130 tokens ( 754.08 ms per token)
llama_print_timings: eval time = 0.00 ms / 1 runs ( 0.00 ms per run)
llama_print_timings: total time = 98131.77 ms
llama_print_timings: load time = 7955.21 ms
llama_print_timings: sample time = 0.00 ms / 1 runs ( 0.00 ms per run)
llama_print_timings: prompt eval time = 88746.09 ms / 119 tokens ( 745.77 ms per token)
llama_print_timings: eval time = 0.00 ms / 1 runs ( 0.00 ms per run)
llama_print_timings: total time = 88829.57 ms
llama_print_timings: load time = 7955.21 ms
llama_print_timings: sample time = 0.00 ms / 1 runs ( 0.00 ms per run)
llama_print_timings: prompt eval time = 102941.21 ms / 136 tokens ( 756.92 ms per token)
llama_print_timings: eval time = 0.00 ms / 1 runs ( 0.00 ms per run)
llama_print_timings: total time = 103032.19 ms
llama_print_timings: load time = 7955.21 ms
llama_print_timings: sample time = 0.00 ms / 1 runs ( 0.00 ms per run)
llama_print_timings: prompt eval time = 100003.36 ms / 133 tokens ( 751.90 ms per token)
llama_print_timings: eval time = 0.00 ms / 1 runs ( 0.00 ms per run)
llama_print_timings: total time = 100102.35 ms
llama_print_timings: load time = 7955.21 ms
llama_print_timings: sample time = 0.00 ms / 1 runs ( 0.00 ms per run)
llama_print_timings: prompt eval time = 92375.97 ms / 123 tokens ( 751.02 ms per token)
llama_print_timings: eval time = 0.00 ms / 1 runs ( 0.00 ms per run)
llama_print_timings: total time = 92462.44 ms
llama_print_timings: load time = 7955.21 ms
llama_print_timings: sample time = 0.00 ms / 1 runs ( 0.00 ms per run)
llama_print_timings: prompt eval time = 100229.25 ms / 135 tokens ( 742.44 ms per token)
llama_print_timings: eval time = 0.00 ms / 1 runs ( 0.00 ms per run)
llama_print_timings: total time = 100318.54 ms
llama_print_timings: load time = 7955.21 ms
llama_print_timings: sample time = 0.00 ms / 1 runs ( 0.00 ms per run)
llama_print_timings: prompt eval time = 87971.32 ms / 116 tokens ( 758.37 ms per token)
llama_print_timings: eval time = 0.00 ms / 1 runs ( 0.00 ms per run)
llama_print_timings: total time = 88048.33 ms
llama_print_timings: load time = 7955.21 ms
llama_print_timings: sample time = 0.00 ms / 1 runs ( 0.00 ms per run)
llama_print_timings: prompt eval time = 56644.69 ms / 75 tokens ( 755.26 ms per token)
llama_print_timings: eval time = 0.00 ms / 1 runs ( 0.00 ms per run)
llama_print_timings: total time = 56708.32 ms
llama_print_timings: load time = 7955.21 ms
llama_print_timings: sample time = 0.00 ms / 1 runs ( 0.00 ms per run)
llama_print_timings: prompt eval time = 83849.58 ms / 111 tokens ( 755.40 ms per token)
llama_print_timings: eval time = 0.00 ms / 1 runs ( 0.00 ms per run)
llama_print_timings: total time = 83926.28 ms
llama_print_timings: load time = 7955.21 ms
llama_print_timings: sample time = 0.00 ms / 1 runs ( 0.00 ms per run)
llama_print_timings: prompt eval time = 88732.58 ms / 120 tokens ( 739.44 ms per token)
llama_print_timings: eval time = 0.00 ms / 1 runs ( 0.00 ms per run)
llama_print_timings: total time = 88817.69 ms
llama_print_timings: load time = 7955.21 ms
llama_print_timings: sample time = 0.00 ms / 1 runs ( 0.00 ms per run)
llama_print_timings: prompt eval time = 103715.10 ms / 138 tokens ( 751.56 ms per token)
llama_print_timings: eval time = 0.00 ms / 1 runs ( 0.00 ms per run)
llama_print_timings: total time = 103810.76 ms
llama_print_timings: load time = 7955.21 ms
llama_print_timings: sample time = 0.00 ms / 1 runs ( 0.00 ms per run)
llama_print_timings: prompt eval time = 98821.70 ms / 133 tokens ( 743.02 ms per token)
llama_print_timings: eval time = 0.00 ms / 1 runs ( 0.00 ms per run)
llama_print_timings: total time = 98911.89 ms
llama_print_timings: load time = 7955.21 ms
llama_print_timings: sample time = 0.00 ms / 1 runs ( 0.00 ms per run)
llama_print_timings: prompt eval time = 88309.90 ms / 117 tokens ( 754.79 ms per token)
llama_print_timings: eval time = 0.00 ms / 1 runs ( 0.00 ms per run)
llama_print_timings: total time = 88393.30 ms
llama_print_timings: load time = 7955.21 ms
llama_print_timings: sample time = 0.00 ms / 1 runs ( 0.00 ms per run)
llama_print_timings: prompt eval time = 95379.70 ms / 128 tokens ( 745.15 ms per token)
llama_print_timings: eval time = 886.33 ms / 1 runs ( 886.33 ms per run)
llama_print_timings: total time = 96363.96 ms
llama_print_timings: load time = 7955.21 ms
llama_print_timings: sample time = 0.00 ms / 1 runs ( 0.00 ms per run)
llama_print_timings: prompt eval time = 81185.53 ms / 109 tokens ( 744.82 ms per token)
llama_print_timings: eval time = 0.00 ms / 1 runs ( 0.00 ms per run)
llama_print_timings: total time = 81267.66 ms
llama_print_timings: load time = 7955.21 ms
llama_print_timings: sample time = 0.00 ms / 1 runs ( 0.00 ms per run)
llama_print_timings: prompt eval time = 94887.65 ms / 127 tokens ( 747.15 ms per token)
llama_print_timings: eval time = 0.00 ms / 1 runs ( 0.00 ms per run)
llama_print_timings: total time = 94972.22 ms
llama_print_timings: load time = 7955.21 ms
llama_print_timings: sample time = 0.00 ms / 1 runs ( 0.00 ms per run)
llama_print_timings: prompt eval time = 69623.88 ms / 92 tokens ( 756.78 ms per token)
llama_print_timings: eval time = 0.00 ms / 1 runs ( 0.00 ms per run)
llama_print_timings: total time = 69697.63 ms
llama_print_timings: load time = 7955.21 ms
llama_print_timings: sample time = 0.00 ms / 1 runs ( 0.00 ms per run)
llama_print_timings: prompt eval time = 95115.12 ms / 126 tokens ( 754.88 ms per token)
llama_print_timings: eval time = 0.00 ms / 1 runs ( 0.00 ms per run)
llama_print_timings: total time = 95210.15 ms
llama_print_timings: load time = 7955.21 ms
llama_print_timings: sample time = 0.00 ms / 1 runs ( 0.00 ms per run)
llama_print_timings: prompt eval time = 98997.76 ms / 132 tokens ( 749.98 ms per token)
llama_print_timings: eval time = 0.00 ms / 1 runs ( 0.00 ms per run)
llama_print_timings: total time = 99084.24 ms
llama_print_timings: load time = 7955.21 ms
llama_print_timings: sample time = 0.00 ms / 1 runs ( 0.00 ms per run)
llama_print_timings: prompt eval time = 76346.15 ms / 102 tokens ( 748.49 ms per token)
llama_print_timings: eval time = 0.00 ms / 1 runs ( 0.00 ms per run)
llama_print_timings: total time = 76418.22 ms
llama_print_timings: load time = 7955.21 ms
llama_print_timings: sample time = 0.00 ms / 1 runs ( 0.00 ms per run)
llama_print_timings: prompt eval time = 64769.55 ms / 87 tokens ( 744.48 ms per token)
llama_print_timings: eval time = 0.00 ms / 1 runs ( 0.00 ms per run)
llama_print_timings: total time = 64831.54 ms
llama_print_timings: load time = 7955.21 ms
llama_print_timings: sample time = 0.00 ms / 1 runs ( 0.00 ms per run)
llama_print_timings: prompt eval time = 85020.06 ms / 115 tokens ( 739.30 ms per token)
llama_print_timings: eval time = 0.00 ms / 1 runs ( 0.00 ms per run)
llama_print_timings: total time = 85097.15 ms
llama_print_timings: load time = 7955.21 ms
llama_print_timings: sample time = 0.00 ms / 1 runs ( 0.00 ms per run)
llama_print_timings: prompt eval time = 101920.62 ms / 135 tokens ( 754.97 ms per token)
llama_print_timings: eval time = 0.00 ms / 1 runs ( 0.00 ms per run)
llama_print_timings: total time = 102013.26 ms
llama_print_timings: load time = 7955.21 ms
llama_print_timings: sample time = 0.00 ms / 1 runs ( 0.00 ms per run)
llama_print_timings: prompt eval time = 80895.01 ms / 109 tokens ( 742.16 ms per token)
llama_print_timings: eval time = 0.00 ms / 1 runs ( 0.00 ms per run)
llama_print_timings: total time = 80975.25 ms
llama_print_timings: load time = 7955.21 ms
llama_print_timings: sample time = 0.00 ms / 1 runs ( 0.00 ms per run)
llama_print_timings: prompt eval time = 94843.68 ms / 127 tokens ( 746.80 ms per token)
llama_print_timings: eval time = 0.00 ms / 1 runs ( 0.00 ms per run)
llama_print_timings: total time = 94933.48 ms
llama_print_timings: load time = 7955.21 ms
llama_print_timings: sample time = 0.00 ms / 1 runs ( 0.00 ms per run)
llama_print_timings: prompt eval time = 85782.89 ms / 116 tokens ( 739.51 ms per token)
llama_print_timings: eval time = 0.00 ms / 1 runs ( 0.00 ms per run)
llama_print_timings: total time = 85868.64 ms
llama_print_timings: load time = 7955.21 ms
llama_print_timings: sample time = 0.00 ms / 1 runs ( 0.00 ms per run)
llama_print_timings: prompt eval time = 89193.90 ms / 119 tokens ( 749.53 ms per token)
llama_print_timings: eval time = 0.00 ms / 1 runs ( 0.00 ms per run)
llama_print_timings: total time = 89273.36 ms
llama_print_timings: load time = 7955.21 ms
llama_print_timings: sample time = 0.00 ms / 1 runs ( 0.00 ms per run)
llama_print_timings: prompt eval time = 95355.51 ms / 128 tokens ( 744.96 ms per token)
llama_print_timings: eval time = 885.96 ms / 1 runs ( 885.96 ms per run)
llama_print_timings: total time = 96331.75 ms
llama_print_timings: load time = 7955.21 ms
llama_print_timings: sample time = 0.00 ms / 1 runs ( 0.00 ms per run)
llama_print_timings: prompt eval time = 93205.92 ms / 124 tokens ( 751.66 ms per token)
llama_print_timings: eval time = 0.00 ms / 1 runs ( 0.00 ms per run)
llama_print_timings: total time = 93295.21 ms
llama_print_timings: load time = 7955.21 ms
llama_print_timings: sample time = 0.00 ms / 1 runs ( 0.00 ms per run)
llama_print_timings: prompt eval time = 77694.56 ms / 104 tokens ( 747.06 ms per token)
llama_print_timings: eval time = 888.11 ms / 1 runs ( 888.11 ms per run)
llama_print_timings: total time = 78666.38 ms
llama_print_timings: load time = 7955.21 ms
llama_print_timings: sample time = 0.00 ms / 1 runs ( 0.00 ms per run)
llama_print_timings: prompt eval time = 71563.52 ms / 96 tokens ( 745.45 ms per token)
llama_print_timings: eval time = 1245.39 ms / 1 runs ( 1245.39 ms per run)
llama_print_timings: total time = 72889.03 ms
llama_print_timings: load time = 7955.21 ms
llama_print_timings: sample time = 0.00 ms / 1 runs ( 0.00 ms per run)
llama_print_timings: prompt eval time = 70585.34 ms / 96 tokens ( 735.26 ms per token)
llama_print_timings: eval time = 0.00 ms / 1 runs ( 0.00 ms per run)
llama_print_timings: total time = 70646.63 ms
llama_print_timings: load time = 7955.21 ms
llama_print_timings: sample time = 0.00 ms / 1 runs ( 0.00 ms per run)
llama_print_timings: prompt eval time = 77488.75 ms / 104 tokens ( 745.08 ms per token)
llama_print_timings: eval time = 0.00 ms / 1 runs ( 0.00 ms per run)
llama_print_timings: total time = 77561.32 ms
llama_print_timings: load time = 7955.21 ms
llama_print_timings: sample time = 0.00 ms / 1 runs ( 0.00 ms per run)
llama_print_timings: prompt eval time = 75736.15 ms / 101 tokens ( 749.86 ms per token)
llama_print_timings: eval time = 0.00 ms / 1 runs ( 0.00 ms per run)
llama_print_timings: total time = 75809.02 ms
llama_print_timings: load time = 7955.21 ms
llama_print_timings: sample time = 0.00 ms / 1 runs ( 0.00 ms per run)
llama_print_timings: prompt eval time = 91473.18 ms / 120 tokens ( 762.28 ms per token)
llama_print_timings: eval time = 0.00 ms / 1 runs ( 0.00 ms per run)
llama_print_timings: total time = 91553.39 ms
llama_print_timings: load time = 7955.21 ms
llama_print_timings: sample time = 0.00 ms / 1 runs ( 0.00 ms per run)
llama_print_timings: prompt eval time = 63092.67 ms / 85 tokens ( 742.27 ms per token)
llama_print_timings: eval time = 0.00 ms / 1 runs ( 0.00 ms per run)
llama_print_timings: total time = 63156.67 ms
llama_print_timings: load time = 7955.21 ms
llama_print_timings: sample time = 0.00 ms / 1 runs ( 0.00 ms per run)
llama_print_timings: prompt eval time = 36715.59 ms / 50 tokens ( 734.31 ms per token)
llama_print_timings: eval time = 0.00 ms / 1 runs ( 0.00 ms per run)
llama_print_timings: total time = 36754.79 ms
llama_print_timings: load time = 7955.21 ms
llama_print_timings: sample time = 0.00 ms / 1 runs ( 0.00 ms per run)
llama_print_timings: prompt eval time = 74234.89 ms / 99 tokens ( 749.85 ms per token)
llama_print_timings: eval time = 0.00 ms / 1 runs ( 0.00 ms per run)
llama_print_timings: total time = 74307.45 ms
llama_print_timings: load time = 7955.21 ms
llama_print_timings: sample time = 0.00 ms / 1 runs ( 0.00 ms per run)
llama_print_timings: prompt eval time = 84345.38 ms / 114 tokens ( 739.87 ms per token)
llama_print_timings: eval time = 0.00 ms / 1 runs ( 0.00 ms per run)
llama_print_timings: total time = 84422.59 ms
llama_print_timings: load time = 7955.21 ms
llama_print_timings: sample time = 0.00 ms / 1 runs ( 0.00 ms per run)
llama_print_timings: prompt eval time = 89102.74 ms / 118 tokens ( 755.11 ms per token)
llama_print_timings: eval time = 0.00 ms / 1 runs ( 0.00 ms per run)
llama_print_timings: total time = 89184.62 ms
llama_print_timings: load time = 7955.21 ms
llama_print_timings: sample time = 0.00 ms / 1 runs ( 0.00 ms per run)
llama_print_timings: prompt eval time = 70385.23 ms / 96 tokens ( 733.18 ms per token)
llama_print_timings: eval time = 0.00 ms / 1 runs ( 0.00 ms per run)
llama_print_timings: total time = 70455.14 ms
llama_print_timings: load time = 7955.21 ms
llama_print_timings: sample time = 0.00 ms / 1 runs ( 0.00 ms per run)
llama_print_timings: prompt eval time = 70180.50 ms / 92 tokens ( 762.83 ms per token)
llama_print_timings: eval time = 0.00 ms / 1 runs ( 0.00 ms per run)
llama_print_timings: total time = 70245.72 ms
llama_print_timings: load time = 7955.21 ms
llama_print_timings: sample time = 0.00 ms / 1 runs ( 0.00 ms per run)
llama_print_timings: prompt eval time = 76064.79 ms / 103 tokens ( 738.49 ms per token)
llama_print_timings: eval time = 0.00 ms / 1 runs ( 0.00 ms per run)
llama_print_timings: total time = 76139.23 ms
llama_print_timings: load time = 7955.21 ms
llama_print_timings: sample time = 0.00 ms / 1 runs ( 0.00 ms per run)
llama_print_timings: prompt eval time = 87004.75 ms / 117 tokens ( 743.63 ms per token)
llama_print_timings: eval time = 0.00 ms / 1 runs ( 0.00 ms per run)
llama_print_timings: total time = 87082.48 ms
llama_print_timings: load time = 7955.21 ms
llama_print_timings: sample time = 0.00 ms / 1 runs ( 0.00 ms per run)
llama_print_timings: prompt eval time = 82832.10 ms / 112 tokens ( 739.57 ms per token)
llama_print_timings: eval time = 0.00 ms / 1 runs ( 0.00 ms per run)
llama_print_timings: total time = 82910.55 ms
llama_print_timings: load time = 7955.21 ms
llama_print_timings: sample time = 0.00 ms / 1 runs ( 0.00 ms per run)
llama_print_timings: prompt eval time = 65208.59 ms / 87 tokens ( 749.52 ms per token)
llama_print_timings: eval time = 0.00 ms / 1 runs ( 0.00 ms per run)
llama_print_timings: total time = 65272.17 ms
llama_print_timings: load time = 7955.21 ms
llama_print_timings: sample time = 0.00 ms / 1 runs ( 0.00 ms per run)
llama_print_timings: prompt eval time = 85663.75 ms / 114 tokens ( 751.44 ms per token)
llama_print_timings: eval time = 0.00 ms / 1 runs ( 0.00 ms per run)
llama_print_timings: total time = 85744.20 ms
llama_print_timings: load time = 7955.21 ms
llama_print_timings: sample time = 0.00 ms / 1 runs ( 0.00 ms per run)
llama_print_timings: prompt eval time = 77883.46 ms / 104 tokens ( 748.88 ms per token)
llama_print_timings: eval time = 883.96 ms / 1 runs ( 883.96 ms per run)
llama_print_timings: total time = 78840.82 ms
llama_print_timings: load time = 7955.21 ms
llama_print_timings: sample time = 0.00 ms / 1 runs ( 0.00 ms per run)
llama_print_timings: prompt eval time = 91748.26 ms / 122 tokens ( 752.03 ms per token)
llama_print_timings: eval time = 0.00 ms / 1 runs ( 0.00 ms per run)
llama_print_timings: total time = 91839.55 ms
llama_print_timings: load time = 7955.21 ms
llama_print_timings: sample time = 0.00 ms / 1 runs ( 0.00 ms per run)
llama_print_timings: prompt eval time = 87908.63 ms / 120 tokens ( 732.57 ms per token)
llama_print_timings: eval time = 996.64 ms / 1 runs ( 996.64 ms per run)
llama_print_timings: total time = 88996.38 ms
llama_print_timings: load time = 7955.21 ms
llama_print_timings: sample time = 0.00 ms / 1 runs ( 0.00 ms per run)
llama_print_timings: prompt eval time = 91530.13 ms / 122 tokens ( 750.25 ms per token)
llama_print_timings: eval time = 0.00 ms / 1 runs ( 0.00 ms per run)
llama_print_timings: total time = 91610.97 ms
llama_print_timings: load time = 7955.21 ms
llama_print_timings: sample time = 0.00 ms / 1 runs ( 0.00 ms per run)
llama_print_timings: prompt eval time = 100850.35 ms / 134 tokens ( 752.61 ms per token)
llama_print_timings: eval time = 0.00 ms / 1 runs ( 0.00 ms per run)
llama_print_timings: total time = 100946.82 ms
llama_print_timings: load time = 7955.21 ms
llama_print_timings: sample time = 0.00 ms / 1 runs ( 0.00 ms per run)
llama_print_timings: prompt eval time = 98305.58 ms / 131 tokens ( 750.42 ms per token)
llama_print_timings: eval time = 0.00 ms / 1 runs ( 0.00 ms per run)
llama_print_timings: total time = 98396.26 ms
llama_print_timings: load time = 7955.21 ms
llama_print_timings: sample time = 0.00 ms / 1 runs ( 0.00 ms per run)
llama_print_timings: prompt eval time = 111868.42 ms / 150 tokens ( 745.79 ms per token)
llama_print_timings: eval time = 0.00 ms / 1 runs ( 0.00 ms per run)
llama_print_timings: total time = 111970.95 ms
llama_print_timings: load time = 7955.21 ms
llama_print_timings: sample time = 0.00 ms / 1 runs ( 0.00 ms per run)
llama_print_timings: prompt eval time = 79706.58 ms / 107 tokens ( 744.92 ms per token)
llama_print_timings: eval time = 0.00 ms / 1 runs ( 0.00 ms per run)
llama_print_timings: total time = 79784.64 ms
llama_print_timings: load time = 7955.21 ms
llama_print_timings: sample time = 0.00 ms / 1 runs ( 0.00 ms per run)
llama_print_timings: prompt eval time = 94534.19 ms / 126 tokens ( 750.27 ms per token)
llama_print_timings: eval time = 0.00 ms / 1 runs ( 0.00 ms per run)
llama_print_timings: total time = 94617.14 ms
llama_print_timings: load time = 7955.21 ms
llama_print_timings: sample time = 0.00 ms / 1 runs ( 0.00 ms per run)
llama_print_timings: prompt eval time = 98144.63 ms / 132 tokens ( 743.52 ms per token)
llama_print_timings: eval time = 0.00 ms / 1 runs ( 0.00 ms per run)
llama_print_timings: total time = 98234.94 ms
llama_print_timings: load time = 7955.21 ms
llama_print_timings: sample time = 0.00 ms / 1 runs ( 0.00 ms per run)
llama_print_timings: prompt eval time = 44417.95 ms / 59 tokens ( 752.85 ms per token)
llama_print_timings: eval time = 0.00 ms / 1 runs ( 0.00 ms per run)
llama_print_timings: total time = 44461.19 ms
[ ]
!python privateGPT.py
llama.cpp: loading model from models/ggml-model-q4_0.bin
llama.cpp: can't use mmap because tensors are not aligned; convert to new format to avoid this
llama_model_load_internal: format = 'ggml' (old version with low tokenizer quality and no mmap support)
llama_model_load_internal: n_vocab = 32000
llama_model_load_internal: n_ctx = 1000
llama_model_load_internal: n_embd = 4096
llama_model_load_internal: n_mult = 256
llama_model_load_internal: n_head = 32
llama_model_load_internal: n_layer = 32
llama_model_load_internal: n_rot = 128
llama_model_load_internal: ftype = 2 (mostly Q4_0)
llama_model_load_internal: n_ff = 11008
llama_model_load_internal: n_parts = 1
llama_model_load_internal: model size = 7B
llama_model_load_internal: ggml ctx size = 4113748.20 KB
llama_model_load_internal: mem required = 5809.33 MB (+ 2052.00 MB per state)
...................................................................................................
.
llama_init_from_file: kv self size = 1000.00 MB
AVX = 1 | AVX2 = 1 | AVX512 = 0 | AVX512_VBMI = 0 | AVX512_VNNI = 0 | FMA = 1 | NEON = 0 | ARM_FMA = 0 | F16C = 1 | FP16_VA = 0 | WASM_SIMD = 0 | BLAS = 0 | SSE3 = 1 | VSX = 0 |
Using embedded DuckDB with persistence: data will be stored in: db
gptj_model_load: loading model from 'models/ggml-gpt4all-j-v1.3-groovy.bin' - please wait ...
gptj_model_load: n_vocab = 50400
gptj_model_load: n_ctx = 2048
gptj_model_load: n_embd = 4096
gptj_model_load: n_head = 16
gptj_model_load: n_layer = 28
gptj_model_load: n_rot = 64
gptj_model_load: f16 = 2
gptj_model_load: ggml ctx size = 4505.45 MB
gptj_model_load: memory_size = 896.00 MB, n_mem = 57344
gptj_model_load: ................................... done
gptj_model_load: model size = 3609.38 MB / num tensors = 285
Enter a query: how many words in this document?
llama_print_timings: load time = 6182.69 ms
llama_print_timings: sample time = 0.00 ms / 1 runs ( 0.00 ms per run)
llama_print_timings: prompt eval time = 6180.19 ms / 8 tokens ( 772.52 ms per token)
llama_print_timings: eval time = 0.00 ms / 1 runs ( 0.00 ms per run)
llama_print_timings: total time = 6187.11 ms
Traceback (most recent call last):
File "/content/privateGPT/privateGPT.py", line 57, in <module>
main()
File "/content/privateGPT/privateGPT.py", line 42, in main
res = qa(query)
File "/usr/local/lib/python3.10/dist-packages/langchain/chains/base.py", line 140, in __call__
raise e
File "/usr/local/lib/python3.10/dist-packages/langchain/chains/base.py", line 134, in __call__
self._call(inputs, run_manager=run_manager)
File "/usr/local/lib/python3.10/dist-packages/langchain/chains/retrieval_qa/base.py", line 119, in _call
docs = self._get_docs(question)
File "/usr/local/lib/python3.10/dist-packages/langchain/chains/retrieval_qa/base.py", line 181, in _get_docs
return self.retriever.get_relevant_documents(question)
File "/usr/local/lib/python3.10/dist-packages/langchain/vectorstores/base.py", line 366, in get_relevant_documents
docs = self.vectorstore.similarity_search(query, **self.search_kwargs)
File "/usr/local/lib/python3.10/dist-packages/langchain/vectorstores/chroma.py", line 181, in similarity_search
docs_and_scores = self.similarity_search_with_score(query, k, filter=filter)
File "/usr/local/lib/python3.10/dist-packages/langchain/vectorstores/chroma.py", line 228, in similarity_search_with_score
results = self.__query_collection(
File "/usr/local/lib/python3.10/dist-packages/langchain/utils.py", line 50, in wrapper
return func(*args, **kwargs)
File "/usr/local/lib/python3.10/dist-packages/langchain/vectorstores/chroma.py", line 120, in __query_collection
return self._collection.query(
File "/usr/local/lib/python3.10/dist-packages/chromadb/api/models/Collection.py", line 219, in query
return self._client._query(
File "/usr/local/lib/python3.10/dist-packages/chromadb/api/local.py", line 408, in _query
uuids, distances = self._db.get_nearest_neighbors(
File "/usr/local/lib/python3.10/dist-packages/chromadb/db/clickhouse.py", line 583, in get_nearest_neighbors
uuids, distances = index.get_nearest_neighbors(embeddings, n_results, ids)
File "/usr/local/lib/python3.10/dist-packages/chromadb/db/index/hnswlib.py", line 230, in get_nearest_neighbors
raise NoIndexException(
chromadb.errors.NoIndexException: Index not found, please create an instance before querying
PrivateGPT
ALL 5 STAR AI.IO PAGE STUDY
How AI and IoT are Creating An Impact On Industries Today
HELLO AND WELCOME TO THE
5 STAR AI.IOT TOOLS FOR YOUR BUSINESS
ARE NEW WEBSITE IS ABOUT 5 STAR AI and io’t TOOLS on the net.
We prevaid you the best
Artificial Intelligence tools and services that can be used to create and improve BUSINESS websites AND CHANNELS .
This site is includes tools for creating interactive visuals, animations, and videos.
as well as tools for SEO, marketing, and web development.
It also includes tools for creating and editing text, images, and audio. The website is intended to provide users with a comprehensive list of AI-based tools to help them create and improve their business.
https://studio.d-id.com/share?id=078f9242d5185a9494e00852e89e17f7&utm_source=copy
This website is a collection of Artificial Intelligence (AI) tools and services that can be used to create and improve websites. It includes tools for creating interactive visuals, animations, and videos, as well as tools for SEO, marketing, and web development. It also includes tools for creating and editing text, images, and audio. The website is intended to provide users with a comprehensive list of AI-based tools to help them create and improve their websites.
אתר זה הוא אוסף של כלים ושירותים של בינה מלאכותית (AI) שניתן להשתמש בהם כדי ליצור ולשפר אתרים. הוא כולל כלים ליצירת ויזואליה אינטראקטיבית, אנימציות וסרטונים, כמו גם כלים לקידום אתרים, שיווק ופיתוח אתרים. הוא כולל גם כלים ליצירה ועריכה של טקסט, תמונות ואודיו. האתר נועד לספק למשתמשים רשימה מקיפה של כלים מבוססי AI שיסייעו להם ליצור ולשפר את אתרי האינטרנט שלהם.
Hello and welcome to our new site that shares with you the most powerful web platforms and tools available on the web today
All platforms, websites and tools have artificial intelligence AI and have a 5-star rating
All platforms, websites and tools are free and Pro paid
The platforms, websites and the tool's are the best for growing your business in 2022/3
שלום וברוכים הבאים לאתר החדש שלנו המשתף אתכם בפלטפורמות האינטרנט והכלים החזקים ביותר הקיימים היום ברשת. כל הפלטפורמות, האתרים והכלים הם בעלי בינה מלאכותית AI ובעלי דירוג של 5 כוכבים. כל הפלטפורמות, האתרים והכלים חינמיים ומקצועיים בתשלום הפלטפורמות, האתרים והכלים באתר זה הם הטובים ביותר והמועילים ביותר להצמחת ולהגדלת העסק שלך ב-2022/3
A Guide for AI-Enhancing Your Existing Business Application
A guide to improving your existing business application of artificial intelligence
מדריך לשיפור היישום העסקי הקיים שלך בינה מלאכותית
What is Artificial Intelligence and how does it work? What are the 3 types of AI?
What is Artificial Intelligence and how does it work? What are the 3 types of AI? The 3 types of AI are: General AI: AI that can perform all of the intellectual tasks a human can. Currently, no form of AI can think abstractly or develop creative ideas in the same ways as humans. Narrow AI: Narrow AI commonly includes visual recognition and natural language processing (NLP) technologies. It is a powerful tool for completing routine jobs based on common knowledge, such as playing music on demand via a voice-enabled device. Broad AI: Broad AI typically relies on exclusive data sets associated with the business in question. It is generally considered the most useful AI category for a business. Business leaders will integrate a broad AI solution with a specific business process where enterprise-specific knowledge is required. How can artificial intelligence be used in business? AI is providing new ways for humans to engage with machines, transitioning personnel from pure digital experiences to human-like natural interactions. This is called cognitive engagement. AI is augmenting and improving how humans absorb and process information, often in real-time. This is called cognitive insights and knowledge management. Beyond process automation, AI is facilitating knowledge-intensive business decisions, mimicking complex human intelligence. This is called cognitive automation. What are the different artificial intelligence technologies in business? Machine learning, deep learning, robotics, computer vision, cognitive computing, artificial general intelligence, natural language processing, and knowledge reasoning are some of the most common business applications of AI. What is the difference between artificial intelligence and machine learning and deep learning? Artificial intelligence (AI) applies advanced analysis and logic-based techniques, including machine learning, to interpret events, support and automate decisions, and take actions. Machine learning is an application of artificial intelligence (AI) that provides systems the ability to automatically learn and improve from experience without being explicitly programmed. Deep learning is a subset of machine learning in artificial intelligence (AI) that has networks capable of learning unsupervised from data that is unstructured or unlabeled. What are the current and future capabilities of artificial intelligence? Current capabilities of AI include examples such as personal assistants (Siri, Alexa, Google Home), smart cars (Tesla), behavioral adaptation to improve the emotional intelligence of customer support representatives, using machine learning and predictive algorithms to improve the customer’s experience, transactional AI like that of Amazon, personalized content recommendations (Netflix), voice control, and learning thermostats. Future capabilities of AI might probably include fully autonomous cars, precision farming, future air traffic controllers, future classrooms with ambient informatics, urban systems, smart cities and so on. To know more about the scope of artificial intelligence in your business, please connect with our expert.
מהי בינה מלאכותית וכיצד היא פועלת? מהם 3 סוגי הבינה המלאכותית?
מהי בינה מלאכותית וכיצד היא פועלת? מהם 3 סוגי הבינה המלאכותית? שלושת סוגי הבינה המלאכותית הם: בינה מלאכותית כללית: בינה מלאכותית שיכולה לבצע את כל המשימות האינטלקטואליות שאדם יכול. נכון לעכשיו, שום צורה של AI לא יכולה לחשוב בצורה מופשטת או לפתח רעיונות יצירתיים באותן דרכים כמו בני אדם. בינה מלאכותית צרה: בינה מלאכותית צרה כוללת בדרך כלל טכנולוגיות זיהוי חזותי ועיבוד שפה טבעית (NLP). זהו כלי רב עוצמה להשלמת עבודות שגרתיות המבוססות על ידע נפוץ, כגון השמעת מוזיקה לפי דרישה באמצעות מכשיר התומך בקול. בינה מלאכותית רחבה: בינה מלאכותית רחבה מסתמכת בדרך כלל על מערכי נתונים בלעדיים הקשורים לעסק המדובר. זה נחשב בדרך כלל לקטגוריית הבינה המלאכותית השימושית ביותר עבור עסק. מנהיגים עסקיים ישלבו פתרון AI רחב עם תהליך עסקי ספציפי שבו נדרש ידע ספציפי לארגון. כיצד ניתן להשתמש בבינה מלאכותית בעסק? AI מספקת דרכים חדשות לבני אדם לעסוק במכונות, ומעבירה את הצוות מחוויות דיגיטליות טהורות לאינטראקציות טבעיות דמויות אדם. זה נקרא מעורבות קוגניטיבית. בינה מלאכותית מגדילה ומשפרת את האופן שבו בני אדם קולטים ומעבדים מידע, לעתים קרובות בזמן אמת. זה נקרא תובנות קוגניטיביות וניהול ידע. מעבר לאוטומציה של תהליכים, AI מאפשר החלטות עסקיות עתירות ידע, תוך חיקוי אינטליגנציה אנושית מורכבת. זה נקרא אוטומציה קוגניטיבית. מהן טכנולוגיות הבינה המלאכותית השונות בעסק? למידת מכונה, למידה עמוקה, רובוטיקה, ראייה ממוחשבת, מחשוב קוגניטיבי, בינה כללית מלאכותית, עיבוד שפה טבעית וחשיבת ידע הם חלק מהיישומים העסקיים הנפוצים ביותר של AI. מה ההבדל בין בינה מלאכותית ולמידת מכונה ולמידה עמוקה? בינה מלאכותית (AI) מיישמת ניתוח מתקדמות וטכניקות מבוססות לוגיקה, כולל למידת מכונה, כדי לפרש אירועים, לתמוך ולהפוך החלטות לאוטומטיות ולנקוט פעולות. למידת מכונה היא יישום של בינה מלאכותית (AI) המספק למערכות את היכולת ללמוד ולהשתפר מניסיון באופן אוטומטי מבלי להיות מתוכנתים במפורש. למידה עמוקה היא תת-קבוצה של למידת מכונה בבינה מלאכותית (AI) שיש לה רשתות המסוגלות ללמוד ללא פיקוח מנתונים שאינם מובנים או ללא תווית. מהן היכולות הנוכחיות והעתידיות של בינה מלאכותית? היכולות הנוכחיות של AI כוללות דוגמאות כמו עוזרים אישיים (Siri, Alexa, Google Home), מכוניות חכמות (Tesla), התאמה התנהגותית לשיפור האינטליגנציה הרגשית של נציגי תמיכת לקוחות, שימוש בלמידת מכונה ואלגוריתמים חזויים כדי לשפר את חווית הלקוח, עסקאות בינה מלאכותית כמו זו של אמזון, המלצות תוכן מותאמות אישית (Netflix), שליטה קולית ותרמוסטטים ללמידה. יכולות עתידיות של AI עשויות לכלול כנראה מכוניות אוטונומיות מלאות, חקלאות מדויקת, בקרי תעבורה אוויריים עתידיים, כיתות עתידיות עם אינפורמטיקה סביבתית, מערכות עירוניות, ערים חכמות וכן הלאה. כדי לדעת יותר על היקף הבינה המלאכותית בעסק שלך, אנא צור קשר עם המומחה שלנו.
Glossary of Terms
Application Programming Interface(API):
An API, or application programming interface, is a set of rules and protocols that allows different software programs to communicate and exchange information with each other. It acts as a kind of intermediary, enabling different programs to interact and work together, even if they are not built using the same programming languages or technologies. API's provide a way for different software programs to talk to each other and share data, helping to create a more interconnected and seamless user experience.
Artificial Intelligence(AI):
the intelligence displayed by machines in performing tasks that typically require human intelligence, such as learning, problem-solving, decision-making, and language understanding. AI is achieved by developing algorithms and systems that can process, analyze, and understand large amounts of data and make decisions based on that data.
Compute Unified Device Architecture(CUDA):
CUDA is a way that computers can work on really hard and big problems by breaking them down into smaller pieces and solving them all at the same time. It helps the computer work faster and better by using special parts inside it called GPUs. It's like when you have lots of friends help you do a puzzle - it goes much faster than if you try to do it all by yourself.
The term "CUDA" is a trademark of NVIDIA Corporation, which developed and popularized the technology.
Data Processing:
The process of preparing raw data for use in a machine learning model, including tasks such as cleaning, transforming, and normalizing the data.
Deep Learning(DL):
A subfield of machine learning that uses deep neural networks with many layers to learn complex patterns from data.
Feature Engineering:
The process of selecting and creating new features from the raw data that can be used to improve the performance of a machine learning model.
Freemium:
You might see the term "Freemium" used often on this site. It simply means that the specific tool that you're looking at has both free and paid options. Typically there is very minimal, but unlimited, usage of the tool at a free tier with more access and features introduced in paid tiers.
Generative Art:
Generative art is a form of art that is created using a computer program or algorithm to generate visual or audio output. It often involves the use of randomness or mathematical rules to create unique, unpredictable, and sometimes chaotic results.
Generative Pre-trained Transformer(GPT):
GPT stands for Generative Pretrained Transformer. It is a type of large language model developed by OpenAI.
GitHub:
GitHub is a platform for hosting and collaborating on software projects
Google Colab:
Google Colab is an online platform that allows users to share and run Python scripts in the cloud
Graphics Processing Unit(GPU):
A GPU, or graphics processing unit, is a special type of computer chip that is designed to handle the complex calculations needed to display images and video on a computer or other device. It's like the brain of your computer's graphics system, and it's really good at doing lots of math really fast. GPUs are used in many different types of devices, including computers, phones, and gaming consoles. They are especially useful for tasks that require a lot of processing power, like playing video games, rendering 3D graphics, or running machine learning algorithms.
Large Language Model(LLM):
A type of machine learning model that is trained on a very large amount of text data and is able to generate natural-sounding text.
Machine Learning(ML):
A method of teaching computers to learn from data, without being explicitly programmed.
Natural Language Processing(NLP):
A subfield of AI that focuses on teaching machines to understand, process, and generate human language
Neural Networks:
A type of machine learning algorithm modeled on the structure and function of the brain.
Neural Radiance Fields(NeRF):
Neural Radiance Fields are a type of deep learning model that can be used for a variety of tasks, including image generation, object detection, and segmentation. NeRFs are inspired by the idea of using a neural network to model the radiance of an image, which is a measure of the amount of light that is emitted or reflected by an object.
OpenAI:
OpenAI is a research institute focused on developing and promoting artificial intelligence technologies that are safe, transparent, and beneficial to society
Overfitting:
A common problem in machine learning, in which the model performs well on the training data but poorly on new, unseen data. It occurs when the model is too complex and has learned too many details from the training data, so it doesn't generalize well.
Prompt:
A prompt is a piece of text that is used to prime a large language model and guide its generation
Python:
Python is a popular, high-level programming language known for its simplicity, readability, and flexibility (many AI tools use it)
Reinforcement Learning:
A type of machine learning in which the model learns by trial and error, receiving rewards or punishments for its actions and adjusting its behavior accordingly.
Spatial Computing:
Spatial computing is the use of technology to add digital information and experiences to the physical world. This can include things like augmented reality, where digital information is added to what you see in the real world, or virtual reality, where you can fully immerse yourself in a digital environment. It has many different uses, such as in education, entertainment, and design, and can change how we interact with the world and with each other.
Stable Diffusion:
Stable Diffusion generates complex artistic images based on text prompts. It’s an open source image synthesis AI model available to everyone. Stable Diffusion can be installed locally using code found on GitHub or there are several online user interfaces that also leverage Stable Diffusion models.
Supervised Learning:
A type of machine learning in which the training data is labeled and the model is trained to make predictions based on the relationships between the input data and the corresponding labels.
Unsupervised Learning:
A type of machine learning in which the training data is not labeled, and the model is trained to find patterns and relationships in the data on its own.
Webhook:
A webhook is a way for one computer program to send a message or data to another program over the internet in real-time. It works by sending the message or data to a specific URL, which belongs to the other program. Webhooks are often used to automate processes and make it easier for different programs to communicate and work together. They are a useful tool for developers who want to build custom applications or create integrations between different software systems.
מילון מונחים
ממשק תכנות יישומים (API): API, או ממשק תכנות יישומים, הוא קבוצה של כללים ופרוטוקולים המאפשרים לתוכנות שונות לתקשר ולהחליף מידע ביניהן. הוא פועל כמעין מתווך, המאפשר לתוכניות שונות לקיים אינטראקציה ולעבוד יחד, גם אם הן אינן בנויות באמצעות אותן שפות תכנות או טכנולוגיות. ממשקי API מספקים דרך לתוכנות שונות לדבר ביניהן ולשתף נתונים, ועוזרות ליצור חווית משתמש מקושרת יותר וחלקה יותר. בינה מלאכותית (AI): האינטליגנציה שמוצגת על ידי מכונות בביצוע משימות הדורשות בדרך כלל אינטליגנציה אנושית, כגון למידה, פתרון בעיות, קבלת החלטות והבנת שפה. AI מושגת על ידי פיתוח אלגוריתמים ומערכות שיכולים לעבד, לנתח ולהבין כמויות גדולות של נתונים ולקבל החלטות על סמך הנתונים הללו. Compute Unified Device Architecture (CUDA): CUDA היא דרך שבה מחשבים יכולים לעבוד על בעיות קשות וגדולות באמת על ידי פירוקן לחתיכות קטנות יותר ופתרון כולן בו זמנית. זה עוזר למחשב לעבוד מהר יותר וטוב יותר על ידי שימוש בחלקים מיוחדים בתוכו הנקראים GPUs. זה כמו כשיש לך הרבה חברים שעוזרים לך לעשות פאזל - זה הולך הרבה יותר מהר מאשר אם אתה מנסה לעשות את זה לבד. המונח "CUDA" הוא סימן מסחרי של NVIDIA Corporation, אשר פיתחה והפכה את הטכנולוגיה לפופולרית. עיבוד נתונים: תהליך הכנת נתונים גולמיים לשימוש במודל למידת מכונה, כולל משימות כמו ניקוי, שינוי ונימול של הנתונים. למידה עמוקה (DL): תת-תחום של למידת מכונה המשתמש ברשתות עצביות עמוקות עם רבדים רבים כדי ללמוד דפוסים מורכבים מנתונים. הנדסת תכונות: תהליך הבחירה והיצירה של תכונות חדשות מהנתונים הגולמיים שניתן להשתמש בהם כדי לשפר את הביצועים של מודל למידת מכונה. Freemium: ייתכן שתראה את המונח "Freemium" בשימוש לעתים קרובות באתר זה. זה פשוט אומר שלכלי הספציפי שאתה מסתכל עליו יש אפשרויות חינמיות וגם בתשלום. בדרך כלל יש שימוש מינימלי מאוד, אך בלתי מוגבל, בכלי בשכבה חינמית עם יותר גישה ותכונות שהוצגו בשכבות בתשלום. אמנות גנרטיבית: אמנות גנרטיבית היא צורה של אמנות שנוצרת באמצעות תוכנת מחשב או אלגוריתם ליצירת פלט חזותי או אודיו. לרוב זה כרוך בשימוש באקראיות או בכללים מתמטיים כדי ליצור תוצאות ייחודיות, בלתי צפויות ולעיתים כאוטיות. Generative Pre-trained Transformer(GPT): GPT ראשי תיבות של Generative Pre-trained Transformer. זהו סוג של מודל שפה גדול שפותח על ידי OpenAI. GitHub: GitHub היא פלטפורמה לאירוח ושיתוף פעולה בפרויקטי תוכנה
Google Colab: Google Colab היא פלטפורמה מקוונת המאפשרת למשתמשים לשתף ולהריץ סקריפטים של Python בענן Graphics Processing Unit(GPU): GPU, או יחידת עיבוד גרפית, הוא סוג מיוחד של שבב מחשב שנועד להתמודד עם המורכבות חישובים הדרושים להצגת תמונות ווידאו במחשב או במכשיר אחר. זה כמו המוח של המערכת הגרפית של המחשב שלך, והוא ממש טוב לעשות הרבה מתמטיקה ממש מהר. GPUs משמשים סוגים רבים ושונים של מכשירים, כולל מחשבים, טלפונים וקונסולות משחקים. הם שימושיים במיוחד למשימות הדורשות כוח עיבוד רב, כמו משחקי וידאו, עיבוד גרפיקה תלת-ממדית או הפעלת אלגוריתמים של למידת מכונה. מודל שפה גדול (LLM): סוג של מודל למידת מכונה שאומן על כמות גדולה מאוד של נתוני טקסט ומסוגל ליצור טקסט בעל צליל טבעי. Machine Learning (ML): שיטה ללמד מחשבים ללמוד מנתונים, מבלי להיות מתוכנתים במפורש. עיבוד שפה טבעית (NLP): תת-תחום של AI המתמקד בהוראת מכונות להבין, לעבד וליצור שפה אנושית רשתות עצביות: סוג של אלגוריתם למידת מכונה המבוססת על המבנה והתפקוד של המוח. שדות קרינה עצביים (NeRF): שדות קרינה עצביים הם סוג של מודל למידה עמוקה שיכול לשמש למגוון משימות, כולל יצירת תמונה, זיהוי אובייקטים ופילוח. NeRFs שואבים השראה מהרעיון של שימוש ברשת עצבית למודל של זוהר תמונה, שהוא מדד לכמות האור שנפלט או מוחזר על ידי אובייקט. OpenAI: OpenAI הוא מכון מחקר המתמקד בפיתוח וקידום טכנולוגיות בינה מלאכותית שהן בטוחות, שקופות ומועילות לחברה. Overfitting: בעיה נפוצה בלמידת מכונה, שבה המודל מתפקד היטב בנתוני האימון אך גרועים בחדשים, בלתי נראים. נתונים. זה מתרחש כאשר המודל מורכב מדי ולמד יותר מדי פרטים מנתוני האימון, כך שהוא לא מכליל היטב. הנחיה: הנחיה היא פיסת טקסט המשמשת לתכנון מודל שפה גדול ולהנחות את הדור שלו Python: Python היא שפת תכנות פופולרית ברמה גבוהה הידועה בפשטות, בקריאות ובגמישות שלה (כלי AI רבים משתמשים בה) למידת חיזוק: סוג של למידת מכונה שבה המודל לומד על ידי ניסוי וטעייה, מקבל תגמולים או עונשים על מעשיו ומתאים את התנהגותו בהתאם. מחשוב מרחבי: מחשוב מרחבי הוא השימוש בטכנולוגיה כדי להוסיף מידע וחוויות דיגיטליות לעולם הפיזי. זה יכול לכלול דברים כמו מציאות רבודה, שבה מידע דיגיטלי מתווסף למה שאתה רואה בעולם האמיתי, או מציאות מדומה, שבה אתה יכול לשקוע במלואו בסביבה דיגיטלית. יש לו שימושים רבים ושונים, כמו בחינוך, בידור ועיצוב, והוא יכול לשנות את האופן שבו אנו מתקשרים עם העולם ואחד עם השני. דיפוזיה יציבה: דיפוזיה יציבה מייצרת תמונות אמנותיות מורכבות המבוססות על הנחיות טקסט. זהו מודל AI של סינתזת תמונות בקוד פתוח הזמין לכולם. ניתן להתקין את ה-Stable Diffusion באופן מקומי באמצעות קוד שנמצא ב-GitHub או שישנם מספר ממשקי משתמש מקוונים הממנפים גם מודלים של Stable Diffusion. למידה מפוקחת: סוג של למידת מכונה שבה נתוני האימון מסומנים והמודל מאומן לבצע תחזיות על סמך היחסים בין נתוני הקלט והתוויות המתאימות. למידה ללא פיקוח: סוג של למידת מכונה שבה נתוני האימון אינם מסומנים, והמודל מאומן למצוא דפוסים ויחסים בנתונים בעצמו. Webhook: Webhook הוא דרך של תוכנת מחשב אחת לשלוח הודעה או נתונים לתוכנית אחרת דרך האינטרנט בזמן אמת. זה עובד על ידי שליחת ההודעה או הנתונים לכתובת URL ספציפית, השייכת לתוכנית האחרת. Webhooks משמשים לעתים קרובות כדי להפוך תהליכים לאוטומטיים ולהקל על תוכניות שונות לתקשר ולעבוד יחד. הם כלי שימושי למפתחים שרוצים לבנות יישומים מותאמים אישית או ליצור אינטגרציות בין מערכות תוכנה שונות.
WELCOME TO THE
5 STAR AI.IO
TOOLS
FOR YOUR BUSINESS
