Skip to content

AttributeError: 'POS' object has no attribute 'model' numpy.core._exceptions._ArrayMemoryError: #48

@henrywang0314

Description

@henrywang0314

Code
from ckiptagger import data_utils, construct_dictionary, WS, POS, NER
data_utils.download_data_gdown("./")
ws = WS("./data")
pos = POS("./data")
ner = NER("./data")
sentence_list = ["""傅達仁今將執行安樂死,卻突然爆出自己20年前遭緯來體育台封殺,
他不懂自己哪裡得罪到電視台。",
"美國參議院針對今天總統布什所提名的勞工部長趙小蘭展開認可聽
證會,預料她將會很順利通過參議院支持,成為該國有史以來第一
位的華裔女性內閣成員。",
"",
"土地公有政策??還是土地婆有政策。.",
"… 你確定嗎… 不要再騙了……",
"最多容納59,000個人,或5.9萬人,再多就不行了.這是環評的結
論.",
"科長說:1,坪數對人數為1:3。2,可以再增加。"""]

word_s = ws(sentence_list,
sentence_segmentation=True,
segment_delimiter_set={'?', '?', '!', '!', '。', ',',
',', ';', ':', '、'})

word_p = pos(word_s)

print(word_p)

Getting this Error

Traceback (most recent call last):
File "c:\Users\Administrator\Downloads...\python_file.py", line 4, in
pos = POS("./data")
^^^^^^^^^^^^^
File "C:\ProgramData\anaconda3\envs\myenv\Lib\site-packages\ckiptagger\api.py", line 160, in init
config.w_token_to_vector, config.w_embedding_d = _load_embedding(os.path.join(data_dir, "embedding_word"))
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\ProgramData\anaconda3\envs\myenv\Lib\site-packages\ckiptagger\api.py", line 357, in _load_embedding
vector_list = np.load(vector_file)
^^^^^^^^^^^^^^^^^^^^
File "C:\ProgramData\anaconda3\envs\myenv\Lib\site-packages\numpy\lib\npyio.py", line 432, in load
return format.read_array(fid, allow_pickle=allow_pickle,
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\ProgramData\anaconda3\envs\myenv\Lib\site-packages\numpy\lib\format.py", line 790, in
read_array
array = numpy.fromfile(fp, dtype=dtype, count=count)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
numpy.core._exceptions._ArrayMemoryError: Unable to allocate 1.52 GiB for an array with shape (406737300,) and data type float32
Exception ignored in: <function POS.del at 0x0000020709DE6340>
Traceback (most recent call last):
File "C:\ProgramData\anaconda3\envs\myenv\Lib\site-packages\ckiptagger\api.py", line 185, in del
self.model.sess.close()
^^^^^^^^^^
AttributeError: 'POS' object has no attribute 'model'

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions