Helpful rules of thumb for understanding tokens:
1 token ~= 4 chars in English
1 token ~= ¾ words
100 tokens ~= 75 words
1-2 sentence ~= 30 tokens
1 paragraph ~= 100 tokens
1,500 words ~= 2048 tokens
Helpful rules of thumb for understanding tokens:
1 token ~= 4 chars in English
1 token ~= ¾ words
100 tokens ~= 75 words
1-2 sentence ~= 30 tokens
1 paragraph ~= 100 tokens
1,500 words ~= 2048 tokens