Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

TypeError while running DocProduct/docproduct/prediction.py #8

Closed
WrathofBhuvan11 opened this issue Jul 12, 2019 · 19 comments
Closed

TypeError while running DocProduct/docproduct/prediction.py #8

WrathofBhuvan11 opened this issue Jul 12, 2019 · 19 comments

Comments

@WrathofBhuvan11
Copy link

File "predictor.py", line 314, in
gen = GenerateQADoc()
File "predictor.py", line 273, in init
load_pretrain=False
File "predictor.py", line 85, in init
self.predict(questions=question, answers=answer, dataset=False)
File "predictor.py", line 154, in predict
model_outputs = self.model(model_inputs)
File "/home/bhuvanesh/anaconda3/envs/stark/lib/python3.6/site-packages/tensorflow/python/keras/engine/base_layer.py", line 703, in call
outputs = self.call(inputs, *args, **kwargs)
File "/home/bhuvanesh/Documents/pystuff/Medical_dr_source/DocProduct/docproduct/models.py", line 114, in call
(inputs['q_input_ids'], inputs['q_segment_ids'], inputs['q_input_masks']))[self.layer_ind]
File "/home/bhuvanesh/anaconda3/envs/stark/lib/python3.6/site-packages/tensorflow/python/keras/engine/base_layer.py", line 703, in call
outputs = self.call(inputs, *args, **kwargs)
File "/home/bhuvanesh/Documents/pystuff/Medical_dr_source/DocProduct/docproduct/bert.py", line 169, in call
trainable=self.trainable)
File "/home/bhuvanesh/Documents/pystuff/Medical_dr_source/DocProduct/docproduct/bert.py", line 141, in _wrap_layer
build_output = build_func(input_layer)
File "/home/bhuvanesh/anaconda3/envs/stark/lib/python3.6/site-packages/tensorflow/python/keras/engine/base_layer.py", line 696, in call
self.build(input_shapes)
File "/home/bhuvanesh/Documents/pystuff/Medical_dr_source/DocProduct/keras_bert/keras_multi_head/multi_head_attention.py", line 94, in build
name='%s_Wq' % self.name,
File "/home/bhuvanesh/anaconda3/envs/stark/lib/python3.6/site-packages/tensorflow/python/keras/engine/base_layer.py", line 534, in add_weight
use_resource=use_resource)
File "/home/bhuvanesh/anaconda3/envs/stark/lib/python3.6/site-packages/tensorflow/python/training/checkpointable/base.py", line 497, in _add_variable_with_custom_getter
**kwargs_for_getter)
File "/home/bhuvanesh/anaconda3/envs/stark/lib/python3.6/site-packages/tensorflow/python/keras/engine/base_layer.py", line 1873, in make_variable
use_resource=use_resource)
File "/home/bhuvanesh/anaconda3/envs/stark/lib/python3.6/site-packages/tensorflow/python/ops/variable_scope.py", line 2234, in variable
use_resource=use_resource)
File "/home/bhuvanesh/anaconda3/envs/stark/lib/python3.6/site-packages/tensorflow/python/ops/variable_scope.py", line 2224, in
previous_getter = lambda **kwargs: default_variable_creator(None, **kwargs)
File "/home/bhuvanesh/anaconda3/envs/stark/lib/python3.6/site-packages/tensorflow/python/ops/variable_scope.py", line 2196, in default_variable_creator
constraint=constraint)
File "/home/bhuvanesh/anaconda3/envs/stark/lib/python3.6/site-packages/tensorflow/python/ops/resource_variable_ops.py", line 312, in init
constraint=constraint)
File "/home/bhuvanesh/anaconda3/envs/stark/lib/python3.6/site-packages/tensorflow/python/ops/resource_variable_ops.py", line 417, in _init_from_args
initial_value(), name="initial_value", dtype=dtype)
File "/home/bhuvanesh/anaconda3/envs/stark/lib/python3.6/site-packages/tensorflow/python/keras/engine/base_layer.py", line 1860, in
shape, dtype=dtype, partition_info=partition_info)
File "/home/bhuvanesh/anaconda3/envs/stark/lib/python3.6/site-packages/tensorflow/python/ops/init_ops.py", line 468, in call
scale /= max(1., (fan_in + fan_out) / 2.)
TypeError: unsupported operand type(s) for /: 'Dimension' and 'float'
@Santosh-Gupta
Copy link
Member

Can you sent a link to the colab notebook you're using?

@WrathofBhuvan11
Copy link
Author

I'm running on my system... i have all pretrained models ...
except ....
loader('bert/embeddings/word_embeddings'), {here}
it's from keras_bert directory
actually i think i knw where's the mistake.. i think this word emebeddings is missing can i get the link of it

File "", line 1, in
runfile('/home/bhuvanesh/Documents/pystuff/Medical_dr_source/DocProduct/docproduct/predictor.py', wdir='/home/bhuvanesh/Documents/pystuff/Medical_dr_source/DocProduct/docproduct')

File "/home/bhuvanesh/anaconda3/envs/stark/lib/python3.6/site-packages/spyder_kernels/customize/spydercustomize.py", line 668, in runfile
execfile(filename, namespace)

File "/home/bhuvanesh/anaconda3/envs/stark/lib/python3.6/site-packages/spyder_kernels/customize/spydercustomize.py", line 108, in execfile
exec(compile(f.read(), filename, 'exec'), namespace)

File "/home/bhuvanesh/Documents/pystuff/Medical_dr_source/DocProduct/docproduct/predictor.py", line 314, in
gen = GenerateQADoc()

File "/home/bhuvanesh/Documents/pystuff/Medical_dr_source/DocProduct/docproduct/predictor.py", line 273, in init
load_pretrain=True#False # <TypeError: unsupported operand type(s) for /: 'Dimension' and 'float'>

File "/home/bhuvanesh/Documents/pystuff/Medical_dr_source/DocProduct/docproduct/predictor.py", line 76, in init
layer_ind=layer_ind)

File "/home/bhuvanesh/Documents/pystuff/Medical_dr_source/DocProduct/docproduct/models.py", line 87, in init
model=self.biobert, config=config, checkpoint_file=checkpoint_file, training=False)

File "/home/bhuvanesh/Documents/pystuff/Medical_dr_source/DocProduct/keras_bert/loader.py", line 72, in load_model_weights_from_checkpoint
loader('bert/embeddings/word_embeddings'),

File "/home/bhuvanesh/anaconda3/envs/stark/lib/python3.6/site-packages/tensorflow/python/keras/engine/base_layer.py", line 1462, in set_weights
str(weights)[:50] + '...')

ValueError: You called set_weights(weights) on layer "Embedding-Token" with a weight list of length 1, but the layer was expecting 0 weights. Provided weights: [array([[-0.03812443, -0.08015507, -0.01168962, .....

@Santosh-Gupta
Copy link
Member

"i think this word emebeddings is missing can i get the link of it"

It's a little tricky to diagnose the exact issue. Could you post the code you are using?

@WrathofBhuvan11
Copy link
Author

can I use keras_bert from "https://github.com/CyberZHG/keras-bert" ?
the code i'm using is...
https://github.com/re-search/DocProduct/blob/master/keras_bert/loader.py
I'm using keras_bert folder from your git rep. I'm not sure from where to download bert word embeddings
line.no.57
def load_model_weights_from_checkpoint(model,
config,
checkpoint_file,
training=False):
"""Load trained official model from checkpoint.
:param model: Built keras model.
:param config: Loaded configuration file.
:param checkpoint_file: The path to the checkpoint files, should end with '.ckpt'.
:param training: If training, the whole model will be returned.
Otherwise, the MLM and NSP parts will be ignored.
"""
the path you gave in this function defination...

@WrathofBhuvan11
Copy link
Author

https://github.com/CyberZHG/keras-bert#download-pretrained-checkpoints
does your keras_bert has these functions to download Bert word-embeddings?

@WrathofBhuvan11
Copy link
Author

WrathofBhuvan11 commented Jul 12, 2019 •
edited

Problem is all rep. on gpt estimator or keras_bert doesn't support tf 1.9 it's tf 2.0 and i'm using anaconda and they didn't update to tf 2.0 it's still 1.9. Yesterday i had to write gpt2 estimator for tf 1.9.
Now prblm same for keras_bert....
Successfully built keras-bert keras-transformer keras-pos-embd keras-multi-head keras-layer-normalization keras-position-wise-feed-forward keras-embed-sim keras-self-attention
ERROR: docproduct 0.0.0 requires pyarrow, which is not installed.
ERROR: docproduct 0.0.0 requires tensorflow-gpu==2.0.0-alpha0, which is not installed.
ERROR: docproduct 0.0.0 has requirement keras-pos-embd==0.9.0, but you'll have keras-pos-embd 0.11.0 which is incompatible.
ERROR: docproduct 0.0.0 has requirement keras-transformer==0.21.0, but you'll have keras-transformer 0.28.0 which is incompatible.
ERROR: docproduct 0.0.0 has requirement tensorflow==2.0.0-alpha0, but you'll have tensorflow 1.9.0 which is incompatible.
Installing collected packages: keras-pos-embd, keras-self-attention, keras-multi-head, keras-layer-normalization, keras-position-wise-feed-forward, keras-embed-sim, keras-transformer, keras-bert
Found existing installation: keras-pos-embd 0.9.0
Uninstalling keras-pos-embd-0.9.0:
Successfully uninstalled keras-pos-embd-0.9.0
Found existing installation: keras-transformer 0.21.0
Uninstalling keras-transformer-0.21.0:
Successfully uninstalled keras-transformer-0.21.0
Successfully installed keras-bert-0.68.1 keras-embed-sim-0.7.0 keras-layer-normalization-0.12.0 keras-multi-head-0.20.0 keras-pos-embd-0.11.0 keras-position-wise-feed-forward-0.6.0 keras-self-attention-0.41.0 keras-transformer-0.28.0

@Santosh-Gupta
Copy link
Member

Ah, interesting. I believe TF 2.0 is still in beta, so it may be a while before Anaconda is updated to 2.0. This code is in TF 2.0 because we submitted it to a TF 2.0 hackathon.

@WrathofBhuvan11
Copy link
Author

In [48]: loader
Out[48]: <function main.checkpoint_loader.._loader(name)>

In [49]: loader('bert/embeddings/word_embeddings')
Out[49]:
array([[-0.03812443, -0.08015507, -0.01168962, ..., -0.03758528,
-0.04412125, 0.00953547],
[ 0.01136853, -0.02929683, -0.03054058, ..., -0.03894736,
-0.06458649, 0.00042958],
[-0.03499779, -0.01893356, 0.00504081, ..., -0.03684894,
-0.03942461, -0.03027621],
...,
[ 0.00039876, -0.0074348 , -0.04399593, ..., -0.03482178,
-0.0749266 , -0.01116488],
[-0.0215879 , 0.00865485, -0.02265086, ..., -0.08096945,
0.00631786, -0.03537379],
[ 0.05276028, -0.02393056, -0.02634131, ..., -0.04740848,
-0.04738597, -0.00887593]], dtype=float32)

In [50]: model.get_layer(name='Embedding-Token').set_weights([
...: loader('bert/embeddings/word_embeddings'),
...: ])
Traceback (most recent call last):

File "", line 2, in
loader('bert/embeddings/word_embeddings'),

File "/home/bhuvanesh/anaconda3/envs/stark/lib/python3.6/site-packages/tensorflow/python/keras/engine/base_layer.py", line 1462, in set_weights
str(weights)[:50] + '...')

ValueError: You called set_weights(weights) on layer "Embedding-Token" with a weight list of length 1, but the layer was expecting 0 weights. Provided weights: [array([[-0.03812443, -0.08015507, -0.01168962, .....

@Santosh-Gupta
Copy link
Member

Could you include the code you used? It's difficult to diagnose from just the error message.

@WrathofBhuvan11
Copy link
Author

WrathofBhuvan11 commented Jul 17, 2019 •
edited

pretrained_path = "...../DocProduct/BioBertFolder/biobert_v1.0_pubmed_pmc/"
code="/DocProduct/keras_bert/loader.py"
line 58
def load_model_weights_from_checkpoint(model,
config,
checkpoint_file,
#model.layers[1].trainable=False
training=False):
"""Load trained official model from checkpoint.
:param model: Built keras model.
:param config: Loaded configuration file.
:param checkpoint_file: The path to the checkpoint files, should end with '.ckpt'.
:param training: If training, the whole model will be returned.
Otherwise, the MLM and NSP parts will be ignored.

here i'm getting error
in previous comment i uploaded u the single line of tht function load_model_weights_from_checkpoint

@WrathofBhuvan11
Copy link
Author

WrathofBhuvan11 commented Jul 17, 2019 •
edited

Could you include the code you used? It's difficult to diagnose from just the error message.

in this code https://github.com/re-search/DocProduct/blob/master/keras_bert/loader.py
pretrained_path = "...../DocProduct/BioBertFolder/biobert_v1.0_pubmed_pmc/"(this model of bioBert)
function "load_model_weights_from_checkpoint" has a problem even when i run single code snippets say
loader = checkpoint_loader(checkpoint_file)

model.get_layer(name='Embedding-Token').set_weights([
    loader('bert/embeddings/word_embeddings'),
])# "this line shows the same Value Error"

@Santosh-Gupta
Copy link
Member

I'm still not sure how to exactly reproduce the error from this info. Can you give a link to the colab notebook which reproduces this error? If the error you are receiving is fully reproduced, then I can figure out where the error is coming from and possibly offer a fix.

@WrathofBhuvan11
Copy link
Author

WrathofBhuvan11 commented Jul 17, 2019 •
edited

colab link :
"https://colab.research.google.com/drive/11hAr1qo7VCSmIjWREFwyTFblU2LVeh1R#scrollTo=ztuTHeW4jFiH&forceEdit=true&offline=true&sandboxMode=true"
#@title Load model weights and Q&A data. Double click to see code

from docproduct.predictor import RetreiveQADoc

pretrained_path = 'BioBertFolder/biobert_v1.0_pubmed_pmc/'
#ffn_weight_file = None
bert_ffn_weight_file = 'newFolder/models/bertffn_crossentropy/bertffn'
embedding_file = 'Float16EmbeddingsExpanded5-27-19.pkl'

doc = RetreiveQADoc(pretrained_path=pretrained_path,
ffn_weight_file=None,
bert_ffn_weight_file=bert_ffn_weight_file,
embedding_file=embedding_file)

@Santosh-Gupta
Copy link
Member

That notebook doesn't seem to have an error in it. Did you mean to send a different notebook?

@WrathofBhuvan11
Copy link
Author

WrathofBhuvan11 commented Jul 17, 2019 via email

@Santosh-Gupta
Copy link
Member

Ah so the issue is not with the code, but with your system?

You tried running the identical code both in colab and on your system, but the code works in colab, and not with your system?

@WrathofBhuvan11
Copy link
Author

WrathofBhuvan11 commented Jul 17, 2019 via email

@Santosh-Gupta
Copy link
Member

I think I figured out your issue. You're using Tensorflow 1.9. We didn't develop this in Tensorflow 1.x, we did it in Tensorflow 2.0 alpha.

@WrathofBhuvan11
Copy link
Author

WrathofBhuvan11 commented Jul 17, 2019 via email

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@Santosh-Gupta @WrathofBhuvan11