This guide outlines the setup and optimization of a Proxmox-based VM using an NVIDIA RTX 3060 (12GB) to test and benchmark local LLMs such as DeepSeek, LLaMA, Manus, and Phidata. The system is configured for high-speed inference, allowing real-time interaction with models via Open WebUI or APIs.

Introduction

The NVIDIA Jetson Orin Nano Super is a compact and powerful AI development platform designed for robotics, computer vision, and generative AI applications. Unlike traditional computers, this developer kit is optimized for edge computing, making it ideal for real-time AI inference and deep learning.