[Return]

Report a post

Preview
>>25040
I doubt if such a small model will be able to write anything decent without your help and editing.

but you can try like I described above

download this model for example
https://huggingface.co/TheBloke/Kuchiki-L2-7B-GGUF/resolve/main/kuchiki-l2-7b.Q5_K_M.gguf
drop it into models folder
and try it with llama.cpp
You can split it between cpu and gpu

you can also use model downloader where you
chose model tab download model
put
TheBloke/Kuchiki-L2-7B-GGUF/
into first line press get file list then
copy
kuchiki-l2-7b.Q5_K_M.gguf
into the second line for filename
Post number No.25042
Board Literature
Optional. Describe what's wrong with it.