FilterHN

Ask HN: Best local LLM under 2B paramater and consuming RAM less than 3gb

2 points

1 day ago

| 2 comments

For my local project, I have been finding a good LLM or a SLM that consumes less than 2gb ram, can you get me any models that would aid me?

▲

big-chungus4

1 day ago

[-]

LFM 2.5 1.2B instruct, it's good at instruction following as long as the instruction is short and simple.

Gemma 4 E2B is even better than LFM but I don't know if you can fit it in 3GB.

Qwen 3.5 0.8B and 2B are smart but very inconsistent for instruction following.

If you need absolutely tiny models, check Gemma 3 270m and FunctionGemma, LFM 2.5 230m and Granite 4.0 350m. However those are very stupid models and whether they are feasible for you will depend on your use case.

▲

adithyaharish

16 hours ago

[-]

Thanks for taking out your time and listing out the models, just tried qwen 3.5 2B works well for my use case

▲

macwhisperer

1 day ago

[-]

qwen3 1.7b- q4_k_m is your best best for that size

▲

adithyaharish

16 hours ago

[-]

just checked it out, has mixture of experts, mit fit my use case!