1. NousTheta ◦ Multiple Steps: ▪ CPU Usage: 304.4% (3.8x capacity) ▪ Memory Usage: 59.7% (9.55 GB) ◦ Long Context: ▪ CPU Usage: 350.5% (4.4x capacity) ▪ Memory Usage: 59.7% (9.55 GB)
2. Phi3Medium ◦ Multiple Steps: ▪ CPU Usage: 370.9% (4.6x capacity) ▪ Memory Usage: 64.8% (10.37 GB) ◦ Long Context: ▪ CPU Usage: 377.9% (4.7x capacity) ▪ Memory Usage: 64.8% (10.37 GB)
3. Phi3Medium128k ◦ Multiple Steps: ▪ CPU Usage: 374.8% (4.7x capacity) ▪ Memory Usage: 64.7% (10.35 GB) ◦ Long Context: ▪ CPU Usage: 377.0% (4.7x capacity) ▪ Memory Usage: 64.7% (10.35 GB)
4. Phi3Mini ◦ Multiple Steps: ▪ CPU Usage: 324.1% (4.1x capacity) ▪ Memory Usage: 35.3% (5.64 GB) ◦ Long Context: ▪ CPU Usage: 612.2% (7.7x capacity) ▪ Memory Usage: 71.1% (11.38 GB)
5. Llama3_1_8B ◦ Multiple Steps: ▪ CPU Usage: 336.3% (4.2x capacity) ▪ Memory Usage: 38.0% (6.08 GB) ◦ Long Context: ▪ CPU Usage: 369.1% (4.6x capacity) ▪ Memory Usage: 38.0% (6.08 GB)