Efficient use of Large system RAM
For example, if I have 128 GB of system RAM but only 16 GB of VRAM, am I still limited to models that fit within GPU memory (aside from CPU offloading techniques like MoE)? Are there ways to increase context size using s
π
This source provides headlines only. Use the button below to read the complete article on the original site.
π° Read the original article on r/LocalLLaMA
Originally published by r/LocalLLaMA. Aggregated on AIWithGhost for educational purposes β full credit and traffic to the original publisher.