Hello! I'd like to learn how to run an inference with a baseline model of Vicuna without Medusa support. Additionally, I’m curious if there has been any analysis done on the memory footprint, and whether Medusa provides any notable improvements compared to the baseline.
Hello! I'd like to learn how to run an inference with a baseline model of Vicuna without Medusa support. Additionally, I’m curious if there has been any analysis done on the memory footprint, and whether Medusa provides any notable improvements compared to the baseline.