04Medusa heads or a separate draft model for a 70B target? Compare acceptance, cost per step and the training each one needs.▼mediumNewOpenAIMetaTogether AIunlockedOne option drafts four tokens for six percent of a target step but guesses each position blind; the other conditions every guess and costs nearly half a step. The answer prices both on Llama 3.3 70B and finds the batch size where the winner flips.Open full answer →
06EAGLE drafts from the target model's hidden states. Why does that raise acceptance, and what did EAGLE-3 change?▼mediumNewOpenAIAnthropicMeta○ sign inA one-layer drafter that beats an 8B model on acceptance sounds wrong until you ask what it is reading. The answer prices the EAGLE head on Llama 3.3 70B, explains the feature trick, and names the three changes in EAGLE-3.Open full answer →