Questions tagged [llm]
Large Language Models (LLMs) are pretrained models that will probabilistically generate natural language texts. The underlying model is typically a Deep Learning one. Examples include GPT models.
17
questions
0
votes
0
answers
19
views
Does Positional Interpolation Change Llama's Architecture?
I'm currently exploring Meta's positional interpolation method, which aims to increase the context size in their large language model. This method extends the context length from n x n into n′ x n′. ...
0
votes
0
answers
7
views
Evaluation metrics for chunking and synthesis steps for Q&A system
I am doing research and interested in the following question:
What are the evaluation metrics for chunking, retrieval and synthesis steps for Q&A, when I do NLQ with LLMs? I am looking for ...
- 229
0
votes
0
answers
31
views
LLMs' latency and their usability for inference
I am trying to use a transformer decoder (LLM, for simplicity) to label a collection of texts, later to be used for training a classifier.
I tried multiple 7B models, which I can save on my local ...
- 326
0
votes
1
answer
136
views
Are embedding in GPT models trainable model parameters? [closed]
I have tried to search from a few sources, but I did not see any one of them specifically talking about this issue. For example This blog post seems to imply that the embedding used in transformer is ...
- 363
1
vote
0
answers
44
views
Why are LLMs generative models [duplicate]
According to Wikipedia:
A generative model is a statistical model of the joint probability distribution $P ( X , Y )$ on given observable variable $X$ and target variable $Y$;
A discriminative model ...
- 363
0
votes
0
answers
42
views
How can BERT/Transformer models accept input batches of different sizes?
I understand that all inputs in a batch need to be of the same size. However, it seems BERT/Transformers models can accept batches with different sizes as input.
How is that possible? I thought we ...
- 657
0
votes
1
answer
276
views
What is the Llama2 number of steps? [closed]
Llama2 is pretrained with 2 trillion of tokens: $2\times10^9$, and its batch size is of $4\times 10^6$.
We can calculate the number of steps (times we upgrade the parameters) per epoch as follows:
$$\...
- 3
0
votes
1
answer
213
views
Measuring perplexity over a limited domain in an LLM
Are there papers/a literature on measuring perplexity in using a Large Language Model such as ChatGPT/Flan over a limited domain?
I want to prompt an LLM to do movie recommendations/next job ...
- 113
0
votes
0
answers
164
views
Can you fine tune Falcon instruct models
Can you fine tune instruction fine tuned LLM like (Falcon 7B instruct and Falcon 40B instruct)? I have seen tutorials where the base models are fine tuned but can't find any which fine tune the ...
- 111
1
vote
0
answers
167
views
How to determine EC2 instance type and memory for LLM inference endpoint [closed]
I am trying to estimate the costs required for hosting a fine tuned large language model for real time inference. There will be 100s of users querying the endpoint concurrently for multiple use cases ...
- 111
0
votes
1
answer
49
views
Using embeddings to anonymize information
This might be a stupid question, so bear with me.
I was wondering if embeddings can be used to anonymize input text.
I couldn't find any information online that says that embeddings can be 1:1 decoded ...
- 103
2
votes
0
answers
100
views
Attention is All You Need: How to calculate params number of the models?
I want to re-calculate the last column of Table 3 of Attention is All You Need,
i.e. number of params in the models.
But numbers from my calculation do not match.
Model
Params from Table 3 ($\times ...
- 21
2
votes
1
answer
31
views
Accuracy of probability estimate from generative autoregressive language model
My understanding is that a discriminative classifier such as a CNN that takes an input $x$ and produces a discrete output label $y$ is typically trained to predict the best value of $y$, and would not ...
1
vote
0
answers
68
views
Why does the best performing adapter-based parameter-efficient fine-tuning depend on the language model being fine-tuned?
https://arxiv.org/abs/2304.01933 shows that the best performing adapter-based parameter-efficient fine-tuning depends on the language model being fine-tuned:
E.g., LORA is the best adapter for LlaMa-...
- 45.9k
2
votes
1
answer
409
views
How do LLMs transform tokens into vectors?
I know about tokenization algorithms like BPE and some other basics of tokenization from the Hugging Face course. I've also heard about word2vec and other algorithms for assigning words to vectors. I'...
- 133
10
votes
4
answers
874
views
Why do language models like InstructGPT and LLM utilize reinforcement learning instead of supervised learning to learn based on user-ranked examples?
Why do language models like InstructGPT and LLM utilize reinforcement learning instead of supervised learning to learn based on user-ranked examples?
Language models like InstructGPT and ChatGPT are ...
- 1,200
1
vote
1
answer
118
views
How to evaluate Natural Question-Answer Generation pairs?
I am trying to generate Natural Question-Answer for a specific domain. I am using a Large Language Model (LLM). I have only context to generate question-answers but don't have any ground truth. How to ...
- 313