Hacker News
new
|
past
|
comments
|
ask
|
show
|
jobs
|
submit
login
terafo
on March 31, 2023
|
parent
|
context
|
favorite
| on:
Llama.cpp 30B runs with only 6GB of RAM now
I'd
assume
that 33B model should fit with this(only repo that I know of that implements SparseGPT and GPTQ for LLaMa), I, personally, haven't tried though. But you can try your luck
https://github.com/lachlansneff/sparsellama
Guidelines
|
FAQ
|
Lists
|
API
|
Security
|
Legal
|
Apply to YC
|
Contact
Search: