So I’m trying to expand the Roberta Pretrained Model and I was doing a basic model for testing but I’m getting this error from TensorFlow: ValueError: Output tensors of a Functional model must be the output of a TensorFlow Layer. which is from the Model api of Keras but I don’t exactly know what’s causing it. Code: Full error traceback:
Tag: huggingface-transformers
How to replace PyTorch model layer’s tensor with another layer of same shape in Huggingface model?
Given a Huggingface model, e.g. I can access a layer’s tensor as such: [out]: Given the another tensor of the same shape that I’ve pre-defined from somewhere else, in this case, for illustration, I’m creating a random tensor but this can be any tensor that is pre-defined. Note: I’m not trying to replace a layer with a random tensor but
How to input embeddings directly to a huggingface model instead of tokens?
I’m going over the huggingface tutorial where they showed how tokens can be fed into a model to generate hidden representations: But how can I input word embeddings directly instead of tokens? That is, I have another model that generates word embeddings and I need to feed those into the model Answer Most (every?) huggingface encoder model supports that with
Pytorch model object has no attribute ‘predict’ BERT
I had train a BertClassifier model using pytorch. After creating my best.pt I would like to make in production my model and using it to predict and classifier starting from a sample, so I resume them from the checkpoint. Otherwise after put it in evaluation and freeze model, I use .predict to make in work on my sample but I’m
transformers longformer classification problem with f1, precision and recall classification
I am replicating code from this page and I am getting F1, precision and recall to be 0. I got accuracy as shown by the author. What could be reason? I looked into compute_metrics function and it seems to be correct. I tried some toy data as below and precision_recall_fscore_support seems to be giving a correct answer as I am
transformers AutoTokenizer.tokenize introducing extra characters
I am using HuggingFace transformers AutoTokenizer to tokenize small segments of text. However this tokenization is splitting incorrectly in the middle of words and introducing # characters to the tokens. I have tried several different models with the same results. Here is an example of a piece of text and the tokens that were created from it. And here is
BERT get sentence embedding
I am replicating code from this page. I have downloaded the BERT model to my local system and getting sentence embedding. I have around 500,000 sentences for which I need sentence embedding and it is taking a lot of time. Is there a way to expedite the process? Would sending batches of sentences rather than one sentence at a time
Load a model as DPRQuestionEncoder in HuggingFace
I would like to load the BERT’s weights (or whatever transformer) into a DPRQuestionEncoder architecture, such that I can use the HuggingFace save_pretrained method and plug the saved model into the RAG architecture to do end-to-end fine-tuning. But I got the following error I am using the last version of Transformers. Answer As already mentioned in the comments, DPRQuestionEncoder does
Can BERT output be fixed in shape, irrespective of string size?
I am confused about using huggingface BERT models and about how to make them yield a prediction at a fixed shape, regardless of input size (i.e., input string length). I tried to call the tokenizer with the parameters padding=True, truncation=True, max_length = 15, but the prediction output dimensions for inputs = [“a”, “a”*20, “a”*100, “abcede”*20000] are not fixed. What am
Hugging Face: NameError: name ‘sentences’ is not defined
I am following this tutorial here: https://huggingface.co/transformers/training.html – though, I am coming across an error, and I think the tutorial is missing an import, but i do not know which. These are my current imports: Current code: The error: Answer The error states that you do not have a variable called sentences in the scope. I believe the tutorial presumes