Llama3_1_8B

• Multiple Steps:
  • GPU Memory Usage: 6113 MiB
  • Total CPU Usage: 12.8 vCPUs (80%)
  • Total Memory Usage: 8.48 GiB
• Long Context:
  • GPU Memory Usage: 6147 MiB
  • Total CPU Usage: 8.64 vCPUs (54%)
  • Total Memory Usage: 8.48 GiB

Phi3Mini

• Multiple Steps:
  • GPU Memory Usage: 3337 MiB
  • Total CPU Usage: 22.496 vCPUs (140.6%)
  • Total Memory Usage: 7.68 GiB
• Long Context:
  • GPU Memory Usage: 3363 MiB
  • Total CPU Usage: 14.4 vCPUs (90%)
  • Total Memory Usage: 7.68 GiB

Phi3Medium128k

• Multiple Steps:
  • GPU Memory Usage: 10931 MiB
  • Total CPU Usage: 11.392 vCPUs (71.2%)
  • Total Memory Usage: 5.28 GiB
• Long Context:
  • GPU Memory Usage: 10973 MiB
  • Total CPU Usage: 7.904 vCPUs (49.4%)
  • Total Memory Usage: 5.28 GiB

Phi3Medium

• Multiple Steps:
  • GPU Memory Usage: 10929 MiB
  • Total CPU Usage: 5.68 vCPUs (35.5%)
  • Total Memory Usage: 5.28 GiB
• Long Context:
  • GPU Memory Usage: 10973 MiB
  • Total CPU Usage: 4.304 vCPUs (26.9%)
  • Total Memory Usage: 5.28 GiB

NousTheta

• Multiple Steps:
  • GPU Memory Usage: 9597 MiB
  • Total CPU Usage: 4.128 vCPUs (25.8%)
  • Total Memory Usage: 6.56 GiB
• Long Context:
  • GPU Memory Usage: 9597 MiB
  • Total CPU Usage: 4.8 vCPUs (30%)
  • Total Memory Usage: 6.4 GiB