タイトル : Llama2を動かしてみよう(CPU)
更新日 : 2023-09-18
カテゴリ : AI、IA、数理
Llama2はCPUで動くのね
GPU持ってないです。😢。CPUも貧弱です。でも、llama.cppは動くらしい。
MetaのLLM【Llama2】の使い方（CPUで7Bを動かす簡単手順） https://ai-wonderland.com/entry/llama2
【Llama2】をローカル+GPUなしで簡単に動かす方法 https://aru-log.com/start-llama2-no-gpu/
ggerganov/llama.cpp https://github.com/ggerganov/llama.cpp
モデルの形式が変更になっているのね
$ python generate.py
gguf_init_from_file: invalid magic number 67676a74
error loading model: llama_model_loader: failed to load model from ./llama-2-7b-chat.ggmlv3.q2_K.bin
llama_load_model_from_file: failed to load model
Traceback (most recent call last):
...
AssertionErrorあれ?動かない。
GGMLからGGUFへ：llama.cppのファイルフォーマット変更 https://note.com/bakushu/n/nbe8d2813c76b
GGML/GGUF/GPTQの違い https://zenn.dev/kun432/scraps/6fc012752afa62
なるほど。ggmlじゃなくて、ggufのモデルファイルを持ってこないとダメなのね。
動かす
from llama_cpp import Llama
# プロンプトを記入
prompt = """
What is the difference between Prince and David Bowie?
"""
# ダウンロードしたModelをセット
llm = Llama(model_path="./XXXX.gguf")
# 生成実行
output = llm(
prompt,max_tokens=500,stop=["System:", "User:", "Assistant:"],echo=True,
)
print(output)結果の抜粋。すげー文章だな。
David Bowie is more famous for his music than he ever was in life.
Prince, on the other hand, lived like a star, not just as an image
but as a real thing that people could watch and experience every day.
He wrote all of his own songs. He played every instrument and sang lead vocals on each song.
The Beatles may have done this too, but for Prince,
it was an obsession. He worked tirelessly at his craft to the point where he couldn\'t help himself
以下省略