Question:(I asked Gemini)
What is the answer generation speed when running the 1.8 tera Kim3 model with ddr5?
Answer:
To put it simply, it is practically impossible to directly run a model of 1.8 trillion (1.8T) parameter scale in a general PC environment (DDR5 system memory). Even if you technically gather the memory to run it, the speed would drop to below 0.0x~0.1 tokens per second, making it unusable.
