UnparsedFlagAccessError: Trying to access flag

Question

I'm a beginner with BERT and i'm trying to run the code that the developers put as example. Unfortunately, when i'm in the cell that use the bert tokenizer i had this error

---------------------------------------------------------------------------
UnparsedFlagAccessError                   Traceback (most recent call last)
/home/guidotto/BERT_model.ipynb Cella 13 in ()
----> 1 tokenizer.tokenize("This here's an example of using the BERT tokenizer")

File ~/lucenv/lib/python3.10/site-packages/bert/tokenization.py:192, in FullTokenizer.tokenize(self, text)
    190 def tokenize(self, text):
    191   split_tokens = []
--> 192   for token in self.basic_tokenizer.tokenize(text):
    193     if preserve_token(token, self.vocab):
    194       split_tokens.append(token)

File ~/lucenv/lib/python3.10/site-packages/bert/tokenization.py:237, in BasicTokenizer.tokenize(self, text)
    235 split_tokens = []
    236 for token in orig_tokens:
--> 237   if preserve_token(token, self.vocab):
    238     split_tokens.append(token)
    239     continue

File ~/lucenv/lib/python3.10/site-packages/bert/tokenization.py:41, in preserve_token(token, vocab)
     39 def preserve_token(token, vocab):
     40   """Returns True if the token should forgo tokenization and be preserved."""
---> 41   if not FLAGS.preserve_unused_tokens:
     42     return False
     43   if token not in vocab:
...
    477 else:
--> 478   raise _exceptions.UnparsedFlagAccessError(
    479       'Trying to access flag --%s before flags were parsed.' % name)

UnparsedFlagAccessError: Trying to access flag --preserve_unused_tokens before flags were parsed.

I'm working with the 2.9.1 tensorflow version, anyone knows how to fix this? I tried to downgrade tensorflow but the system doesn't let me install a version before that the 2.8. Here the code :

# This is a path to an uncased (all lowercase) version of BERT
BERT_MODEL_HUB = "https://tfhub.dev/google/bert_uncased_L-12_H-768_A-12/1"

def create_tokenizer_from_hub_module():
  """Get the vocab file and casing info from the Hub module."""
  with tf.Graph().as_default():
    bert_module = hub.Module(BERT_MODEL_HUB)
    tokenization_info = bert_module(signature="tokenization_info", as_dict=True)
    with tf.compat.v1.Session() as sess:
      vocab_file, do_lower_case = sess.run([tokenization_info["vocab_file"],
                                            tokenization_info["do_lower_case"]])
      
  return bert.tokenization.FullTokenizer(
      vocab_file=vocab_file, do_lower_case=do_lower_case)

tokenizer = create_tokenizer_from_hub_module()

tokenizer.tokenize("This here's an example of using the BERT tokenizer")

user_14774959 · Answer

This code looks a bit old to be used with Tensorflow 2.x. See here, the solution would be to downgrade to tensorflow 1.x.

However you can obtain the same results using Tensorflow 2 (recommended). See for example the following code:

import bert
import tensorflow_hub as hub

BertTokenizer = bert.bert_tokenization.FullTokenizer

bert_layer = hub.KerasLayer("https://tfhub.dev/tensorflow/bert_en_uncased_L-12_H-768_A-12/2",
                            trainable=False)
vocabulary_file = bert_layer.resolved_object.vocab_file.asset_path.numpy()
to_lower_case = bert_layer.resolved_object.do_lower_case.numpy()
tokenizer = BertTokenizer(vocabulary_file, to_lower_case)

# Test tokenizer
tokenizer.tokenize("This here's an example of using the BERT tokenizer")

The output is:

['this',
 'here',
 "'",
 's',
 'an',
 'example',
 'of',
 'using',
 'the',
 'bert',
 'token',
 '##izer']

Note: the following dependency is required pip install bert-for-tf2

UnparsedFlagAccessError: Trying to access flag

1 Answers