Skip to main content

Questions tagged [llm]

Large Language Models (LLMs) are pretrained models that will probabilistically generate natural language texts. The underlying model is typically a Deep Learning one. Examples include GPT models.

Filter by
Sorted by
Tagged with
0 votes
0 answers
7 views

Choosing an evaluation model for LLM for Question and Answering

I was learning the basic of LLM evaluation and the framework introduced in one of the short courses in Deep Learning AI was to generate samples of question and answer which act as the ground truth. ...
ShengXue's user avatar
0 votes
0 answers
8 views

How to construct class proportion confidence interval for an LLM classifier with known bias and precision and recall?

Let's say I have a dataset, $D$, with known ground truth labels. I nonetheless use a few-shot LLM classifier on this dataset to predict $k$ classes for each label. From the LLM results, I get ...
Estimate the estimators's user avatar
1 vote
0 answers
39 views

How to Estimate GPU Memory for training and inference, Data Requirements, and Training Time for Large Language Models?

Today, I faced this question during an interview for an ML Engineer position. I didn't answer it perfectly at the time. How should I answer it ideally? Assume we have models like Transformer, BERT, ...
maplemaple's user avatar
0 votes
0 answers
16 views

How to report few-shot accuracy for LLMs?

I am comparing three prompting techniques in LLMs to check which one is best. All prompting strategies include three examples for in-context learning (few-shot only, no fine-tuning). If I do greedy ...
Jader Martins's user avatar
4 votes
0 answers
80 views

What advantage do sinusoidal positional encodings have over binary positional encodings in transformer LLMs?

I've recently come across an article that discusses the reasons why large language models use sinusoidal functions to generate positional encodings — as per the famous paper Attention Is All You Need (...
Philip Voinea's user avatar
0 votes
0 answers
20 views

Where does the equation $ C = 6 \times N \times T $ come from for Large Language Models, especially with a simple explanation for both passes?

Why $ C = 6 \times N \times T $? I'm trying to understand the computational steps specifically during the backward pass of neural networks in relation to the widely cited formula ( C = 6 \times N \...
Charlie Parker's user avatar
0 votes
0 answers
11 views

Gradient flow through sampled tokens when training RNNs (but without teacher forcing)

Suppose we want to train an autoregressive generative language model based on a recurrent neural network (RNN) architecture without teacher forcing: At each timestep, the RNN takes an input token $x_t$...
Ben JW's user avatar
  • 101
0 votes
0 answers
69 views

Are LLMs stateful or stateless during the generation process?

Are LLMs like OpenAI's gpt-* stateful or stateless during the generation of the response? I've read a couple of articles like this but am still not quite sure about that. I understand they are ...
Dr. Hans-Peter Störr's user avatar
0 votes
0 answers
27 views

Why is my (Mistral) LLM (almost completely) stopping to learn on my synthetic data after the first epoch, yet not overfitting?

I am creating synthetic task-oriented dialogs that are rather complex. Training and validation losses suggest that the model (almost completely) stops learning, but does not start overfitting: How ...
DaveFar's user avatar
  • 93
0 votes
0 answers
70 views

Why did the OpenAI's scaling law paper underestimate the importance of data in model scaling?

The Chinchilla paper (Hoffmann, Jordan, et al. "Training compute-optimal large language models." arXiv preprint arXiv:2203.15556 (2022).) famously found that when scaling a model, you should ...
user35734's user avatar
  • 406
0 votes
0 answers
205 views

block_size in transformers: does it dictate effective context length in LLMs?

I would like to understand how the block_size parameter in the huggingface transformers library works, particularly in comparison with model_max_length. I am interested in models being able to attend ...
Nucular's user avatar
  • 453
0 votes
0 answers
22 views

Does Positional Interpolation Change Llama's Architecture?

I'm currently exploring Meta's positional interpolation method, which aims to increase the context size in their large language model. This method extends the context length from n x n into n′ x n′. ...
user219313's user avatar
0 votes
0 answers
24 views

Evaluation metrics for chunking and synthesis steps for Q&A system

I am doing research and interested in the following question: What are the evaluation metrics for chunking, retrieval and synthesis steps for Q&A, when I do NLQ with LLMs? I am looking for ...
Anakin Skywalker's user avatar
0 votes
0 answers
34 views

LLMs' latency and their usability for inference

I am trying to use a transformer decoder (LLM, for simplicity) to label a collection of texts, later to be used for training a classifier. I tried multiple 7B models, which I can save on my local ...
David Harar's user avatar
0 votes
1 answer
638 views

Are embedding in GPT models trainable model parameters? [closed]

I have tried to search from a few sources, but I did not see any one of them specifically talking about this issue. For example This blog post seems to imply that the embedding used in transformer is ...
Sam's user avatar
  • 403
1 vote
0 answers
47 views

Why are LLMs generative models [duplicate]

According to Wikipedia: A generative model is a statistical model of the joint probability distribution $P ( X , Y )$ on given observable variable $X$ and target variable $Y$; A discriminative model ...
Sam's user avatar
  • 403
0 votes
0 answers
82 views

How can BERT/Transformer models accept input batches of different sizes?

I understand that all inputs in a batch need to be of the same size. However, it seems BERT/Transformers models can accept batches with different sizes as input. How is that possible? I thought we ...
The Wanderer's user avatar
0 votes
1 answer
828 views

What is the Llama2 number of steps? [closed]

Llama2 is pretrained with 2 trillion of tokens: $2\times10^9$, and its batch size is of $4\times 10^6$. We can calculate the number of steps (times we upgrade the parameters) per epoch as follows: $$\...
Noether's user avatar
0 votes
1 answer
332 views

Measuring perplexity over a limited domain in an LLM

Are there papers/a literature on measuring perplexity in using a Large Language Model such as ChatGPT/Flan over a limited domain? I want to prompt an LLM to do movie recommendations/next job ...
piedpiper's user avatar
  • 113
1 vote
0 answers
196 views

How to determine EC2 instance type and memory for LLM inference endpoint [closed]

I am trying to estimate the costs required for hosting a fine tuned large language model for real time inference. There will be 100s of users querying the endpoint concurrently for multiple use cases ...
user3711946's user avatar
0 votes
1 answer
61 views

Using embeddings to anonymize information

This might be a stupid question, so bear with me. I was wondering if embeddings can be used to anonymize input text. I couldn't find any information online that says that embeddings can be 1:1 decoded ...
funerr's user avatar
  • 103
5 votes
1 answer
1k views

How to count the number of neurons in GPT-2?

In a recent update from OpenAI, they mentioned the discovery of N neurons in their GPT-2 model. This finding raises the question: how did they arrive at this calculation? In their publication, they ...
ofou's user avatar
  • 61
3 votes
0 answers
217 views

Attention is All You Need: How to calculate params number of the models?

I want to re-calculate the last column of Table 3 of Attention is All You Need, i.e. number of params in the models. But numbers from my calculation do not match. Model Params from Table 3 ($\times ...
Judd's user avatar
  • 31
2 votes
1 answer
39 views

Accuracy of probability estimate from generative autoregressive language model

My understanding is that a discriminative classifier such as a CNN that takes an input $x$ and produces a discrete output label $y$ is typically trained to predict the best value of $y$, and would not ...
sunfishstanford's user avatar
1 vote
0 answers
81 views

Why does the best performing adapter-based parameter-efficient fine-tuning depend on the language model being fine-tuned?

https://arxiv.org/abs/2304.01933 shows that the best performing adapter-based parameter-efficient fine-tuning depends on the language model being fine-tuned: E.g., LORA is the best adapter for LlaMa-...
Franck Dernoncourt's user avatar
2 votes
1 answer
641 views

How do LLMs transform tokens into vectors?

I know about tokenization algorithms like BPE and some other basics of tokenization from the Hugging Face course. I've also heard about word2vec and other algorithms for assigning words to vectors. I'...
jskattt797's user avatar
14 votes
4 answers
1k views

Why do language models like InstructGPT and LLM utilize reinforcement learning instead of supervised learning to learn based on user-ranked examples?

Why do language models like InstructGPT and LLM utilize reinforcement learning instead of supervised learning to learn based on user-ranked examples? Language models like InstructGPT and ChatGPT are ...
resnet's user avatar
  • 1,270
1 vote
1 answer
145 views

How to evaluate Natural Question-Answer Generation pairs?

I am trying to generate Natural Question-Answer for a specific domain. I am using a Large Language Model (LLM). I have only context to generate question-answers but don't have any ground truth. How to ...
Aaditya Ura's user avatar