24 LLMs

LLMs are everywhere

today we are going to dig into how they work under the hood, high level agenda, future engineering, how you go from a sentence or a text to a model

  • Feature engineering
    • text tokenization
    • word embeddings
  • deep neural networks
    • autoregressive mofdels
    • selfe attention mechanisms
    • transformer architecture
  • Multi-class classification
  • Supervised Learning
    • self-supervised learning
    • insturuciton tuning
  • reinforcemnt learning
    • ...from human dfeedback (RLHF)

want to ground us and give an overview of what is happening

the main idea is that all of the parts are simple, but how people end up using it by stacking things on top of each other is what makes up the models we see today.

DEEP NERUAL NETWORKS

LLMS are pretty much DNNs they take in some text and they output some text

input: the dog chased the output: ball

visualize this better with ascii

how do we even feed the text into these models?

implementation: Linear Algebra how??

TEXT TOKENIZATION

models operate on numbers and vectors and doesnt actually take the text directly tokenization takes the words or sets of words to turn it into numbers?

Use the openai tokenizer to test it out. Some words take up 1 token, some take up more and more. Different tokenizers do better or worse, so it is a deliberate design choice that the model will use. As a designer, you identify which outputs are produced by different tokenizers.

Once you've assigned each substring a token, each token now gets a number. This number is an index in a really, really long vocabulary dictionary. For example

word: many - 8607

now we have gone from a string of characters to a list of numbers. Now we need to do some sort of feature engineering. We can now feed this into our model. It's not just text anymore.

lets go back to the smaller input

the dog chased the:

"The" = 791 -> embed "Dog" = 5679 -> embed "Chased" = 62920 -> embed "The" = 279 -> embed

We embed them individually

then place into the model, which spits out an output prediction

un-embed it

then the output comes as an embedding

5041 = "Ball"

which we turn back into a word

w

what do embeddings look like?

similar words are closer together. So apple will be closer to banana than airplane.

again this is just a sd reperesentation

autoregressive models

The {pad} {pad}

this outputs dog

The dog {pad}

this outputs chased

The dog chased {pad} this outputs the The dog chased the this outputs ball

  • predict one piece at a time,
  • then contactonace with the old outpout to the new input and
  • feed it back into the model to get the next input
  • repeat.

feed that back in for n timesteps to get the n th token x_n

SELF ATTENTION MECHANISMS

maybe its the case that not all tokens matter equally, some may tell us what we need while others dont. Instead of paying attention to some of the input tookens equally, we want to pay attention to these relative tokens with more weight.

in this diagram below (create with askii) x2 is more importatnt to xn than x1 or x3

example, take the sentence:

The dog chewed the bone because it was hungry

What does "it" refer to here? There are two nouns in the sentence, "dog" and "bone"

if we want to measure the weighting of "it", we see that "dog" has the greatest weight

If we change the sentence to:

the dog chewed the bone because it was delicious

Suddenly, now, "it" means the bone, and we can see the weighting is more towards the word bone.

implementing self-attention

we have a key, querey and value we have multiple MLPs, one that provides each one, from the input x1

so it kinda looks like 

x1
V
MLP1 MLP2 MLP3
 v    V    V
 k1   q1   v1
 v    v    v
 > s1 <    ...
   v
   normalize & softmax for each
   v       ...
   a1      v1

so the input inputs the perceptrons to kive these three vectors for key querey and value

the k1 and q1 provide scores s1 ...

those all get normalized and softmaxed, to create a1...an

then we use the dot product across a1 and v1 ... an-1 * v_n-1. to generate xn