AI's unbelievable 437× memory shrinkage
In 2023, the word "Unbelievable" took over a million bytes of AI memory. DeepSeek's V4.1 Flash needs 2,670. What changed, and why it makes AI cheaper to run.
Read article
Insights / Notes from the work
Models and Tokenisation
Model behaviour, context windows, tokens and the mechanics behind language models.
Articles
/ Archive
How many words are 128k tokens?
128k tokens are 96k words in English for ChatGPT 3.5 and 4. The ratio is estimated to be 0.75 words per token. However, the answer is not...
Read article
/ Archive
How to Build Training Datasets and Fine-tune ChatGPT
Large-language models (LLMs) are great generalists, but modifications are required for optimisation or specialist tasks. The easiest choice is...
Read article
/ Archive
OpenAI GPT-4 Turbo's 128k token context has a 4k completion limit
Recently, OpenAI released GPT4 turbo preview with 128k at its DevDay. That addresses a serious limitation for Retrieval Augmented Generation (RAG)...
Read article
/ Archive
Bing Chat argues and lies when it gets code wrong
Microsoft could follow Google's $100bn loss. I tried the new Bing Chat (ChatGPT) feature, which was great until it went disastrously wrong. It...
Read article
/ Archive
Google Unveils "Google Bard", its Answer to ChatGPT
The Battle of the AI Chatbots Begins: Google's Bard Takes on ChatGPT.
Read article
/ Archive
Understanding the Power of ChatGPT
ChatGPT is a state-of-the-art language model developed by OpenAI, utilising the Transformer model and fine-tuned through reinforcement learning to...
Read article