I presently use a 3060 in an egpu enclosure connected to a nuc passed through to a vm via proxmox
It works well for some things. MOEs do not seem to. Pass through seems maybe to be the culprit to why MOEs crawl in this config. It’s not enough to run 3.6 27b. My b60 idea seems untenable based on the responses to that
📄
This source provides headlines only. Use the button below to read the complete article on the original site.
📰 Read the original article on r/LocalLLaMA
Originally published by r/LocalLLaMA. Aggregated on AIWithGhost for educational purposes — full credit and traffic to the original publisher.