To add on, when using GitHub Copilot you can see a distinction between AI “Credits” and Tokens (in the context window for that conversation). One reason is that each Model can architecturally have different tokenizers, so one what one Model considers as one Token is different from another. To normalize the overall compute costs across different Models, the AI Credit unit is used as an abstraction.

deleted by creator

Sure, see you there! :)

If I may ask, which Quantisation are you running it on? I’m on an Apple setup with oMLX and am still experimenting with finding the right balance