Line limit: 1000 Total bytes processed: "1.20 MB" 2023-11-07T11:03:24+02:00 annif.exception.OperationFailedException: Error in tokenization of text 'jenni toivanen 2023-11-07T11:03:24+02:00 ValueError: character U+3e3ffe4e is not in range [U+0000; U+10ffff] 2023-11-07T11:03:24+02:00 ERROR:annif:Exception on /v1/projects/yso-fi/suggest [POST] 2023-11-07T11:03:24+02:00 tukea, turvaa vai suoritus muiden 2023-11-07T11:03:24+02:00 joukossa?' 2023-11-07T11:03:24+02:00 annif.exception.OperationFailedException: Voikko error in analysis of word 'tukea' 2023-11-07T11:03:24+02:00 Traceback (most recent call last): 2023-11-07T11:03:24+02:00 Traceback (most recent call last): 2023-11-07T11:03:24+02:00 Traceback (most recent call last): 2023-11-07T11:03:24+02:00 The above exception was the direct cause of the following exception: 2023-11-07T11:03:24+02:00 The above exception was the direct cause of the following exception: 2023-11-07T11:03:24+02:00 2023-11-07T11:03:24+02:00 2023-11-07T11:03:24+02:00 2023-11-07T11:03:24+02:00 2023-11-07T11:03:24+02:00 2023-11-07T11:03:24+02:00 File "/usr/local/lib/python3.10/site-packages/voikko/libvoikko.py", line 682, in analyze 2023-11-07T11:03:24+02:00 File "/usr/local/lib/python3.10/site-packages/sklearn/feature_extraction/text.py", line 1434, in transform 2023-11-07T11:03:24+02:00 File "/usr/local/lib/python3.10/site-packages/sklearn/feature_extraction/text.py", line 1276, in _count_vocab 2023-11-07T11:03:24+02:00 File "/usr/local/lib/python3.10/site-packages/sklearn/feature_extraction/text.py", line 112, in _analyze 2023-11-07T11:03:24+02:00 File "/usr/local/lib/python3.10/site-packages/flask_cors/extension.py", line 176, in wrapped_function 2023-11-07T11:03:24+02:00 File "/usr/local/lib/python3.10/site-packages/flask/app.py", line 2529, in wsgi_app 2023-11-07T11:03:24+02:00 File "/usr/local/lib/python3.10/site-packages/flask/app.py", line 1825, in full_dispatch_request 2023-11-07T11:03:24+02:00 File "/usr/local/lib/python3.10/site-packages/flask/app.py", line 1823, in full_dispatch_request 2023-11-07T11:03:24+02:00 File "/usr/local/lib/python3.10/site-packages/flask/app.py", line 1799, in dispatch_request 2023-11-07T11:03:24+02:00 File "/usr/local/lib/python3.10/site-packages/connexion/decorators/validation.py", line 399, in wrapper 2023-11-07T11:03:24+02:00 File "/usr/local/lib/python3.10/site-packages/connexion/decorators/validation.py", line 196, in wrapper 2023-11-07T11:03:24+02:00 File "/usr/local/lib/python3.10/site-packages/connexion/decorators/uri_parsing.py", line 149, in wrapper 2023-11-07T11:03:24+02:00 File "/usr/local/lib/python3.10/site-packages/connexion/decorators/parameter.py", line 120, in wrapper 2023-11-07T11:03:24+02:00 File "/usr/local/lib/python3.10/site-packages/connexion/decorators/decorator.py", line 68, in wrapper 2023-11-07T11:03:24+02:00 File "/Annif/annif/suggestion.py", line 159, in 2023-11-07T11:03:24+02:00 File "/Annif/annif/rest.py", line 99, in _hit_sets_to_list 2023-11-07T11:03:24+02:00 File "/Annif/annif/rest.py", line 99, in 2023-11-07T11:03:24+02:00 File "/Annif/annif/rest.py", line 174, in _suggest 2023-11-07T11:03:24+02:00 File "/Annif/annif/rest.py", line 122, in suggest 2023-11-07T11:03:24+02:00 File "/Annif/annif/project.py", line 253, in suggest 2023-11-07T11:03:24+02:00 File "/Annif/annif/project.py", line 253, in suggest 2023-11-07T11:03:24+02:00 File "/Annif/annif/project.py", line 234, in 2023-11-07T11:03:24+02:00 File "/Annif/annif/project.py", line 143, in _suggest_with_backend 2023-11-07T11:03:24+02:00 File "/Annif/annif/project.py", line 143, in _suggest_with_backend 2023-11-07T11:03:24+02:00 File "/Annif/annif/lexical/mllm.py", line 87, in generate_candidates 2023-11-07T11:03:24+02:00 File "/Annif/annif/lexical/mllm.py", line 156, in generate_candidates 2023-11-07T11:03:24+02:00 File "/Annif/annif/backend/mllm.py", line 166, in _suggest 2023-11-07T11:03:24+02:00 File "/Annif/annif/backend/mllm.py", line 153, in _generate_candidates 2023-11-07T11:03:24+02:00 File "/Annif/annif/backend/ensemble.py", line 71, in _suggest_batch 2023-11-07T11:03:24+02:00 File "/Annif/annif/backend/ensemble.py", line 46, in 2023-11-07T11:03:24+02:00 File "/Annif/annif/backend/ensemble.py", line 45, in _suggest_with_sources 2023-11-07T11:03:24+02:00 File "/Annif/annif/backend/backend.py", line 143, in suggest 2023-11-07T11:03:24+02:00 File "/Annif/annif/backend/backend.py", line 143, in suggest 2023-11-07T11:03:24+02:00 File "/Annif/annif/backend/backend.py", line 129, in _suggest_batch 2023-11-07T11:03:24+02:00 File "/Annif/annif/backend/backend.py", line 129, in 2023-11-07T11:03:24+02:00 File "/Annif/annif/analyzer/voikko.py", line 35, in _normalize_word 2023-11-07T11:03:24+02:00 File "/Annif/annif/analyzer/voikko.py", line 33, in _normalize_word 2023-11-07T11:03:24+02:00 File "/Annif/annif/analyzer/analyzer.py", line 56, in tokenize_words 2023-11-07T11:03:24+02:00 File "/Annif/annif/analyzer/analyzer.py", line 51, in 2023-11-07T11:03:24+02:00 File "/Annif/annif/analyzer/analyzer.py", line 50, in tokenize_words 2023-11-07T11:03:24+02:00 value = self.__lib.voikko_mor_analysis_value_ucs4(cAnalysis, key) 2023-11-07T11:03:24+02:00 sent_tokens = vectorizer.transform(sentences) 2023-11-07T11:03:24+02:00 self.suggest([doc.text for doc in doc_batch], backend_params) 2023-11-07T11:03:24+02:00 self._normalize_word(word) 2023-11-07T11:03:24+02:00 rv = self.handle_user_exception(e) 2023-11-07T11:03:24+02:00 rv = self.dispatch_request() 2023-11-07T11:03:24+02:00 return { 2023-11-07T11:03:24+02:00 return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) 2023-11-07T11:03:24+02:00 return self.backend.suggest(texts, beparams) 2023-11-07T11:03:24+02:00 return self.backend.suggest(texts, beparams) 2023-11-07T11:03:24+02:00 return self._suggest_with_backend(texts, backend_params) 2023-11-07T11:03:24+02:00 return self._suggest_with_backend(texts, backend_params) 2023-11-07T11:03:24+02:00 return self._suggest_batch(texts, params=beparams) 2023-11-07T11:03:24+02:00 return self._suggest_batch(texts, params=beparams) 2023-11-07T11:03:24+02:00 return self._model.generate_candidates(text, self.project.analyzer) 2023-11-07T11:03:24+02:00 return generate_candidates(text, analyzer, self._vectorizer, self._index) 2023-11-07T11:03:24+02:00 return function(request) 2023-11-07T11:03:24+02:00 return function(**kwargs) 2023-11-07T11:03:24+02:00 return cors_after_request(app.make_response(f(*args, **kwargs))) 2023-11-07T11:03:24+02:00 return _hit_sets_to_list(hit_sets, project.subjects, lang) 2023-11-07T11:03:24+02:00 return [ 2023-11-07T11:03:24+02:00 return [ 2023-11-07T11:03:24+02:00 return [ 2023-11-07T11:03:24+02:00 result = self.voikko.analyze(word) 2023-11-07T11:03:24+02:00 result = _suggest(project_id, documents, parameters) 2023-11-07T11:03:24+02:00 response = self.full_dispatch_request() 2023-11-07T11:03:24+02:00 response = function(request) 2023-11-07T11:03:24+02:00 response = function(request) 2023-11-07T11:03:24+02:00 response = function(request) 2023-11-07T11:03:24+02:00 raise OperationFailedException( 2023-11-07T11:03:24+02:00 raise OperationFailedException( 2023-11-07T11:03:24+02:00 project_id: self.project.registry.get_project(project_id).suggest(texts) 2023-11-07T11:03:24+02:00 for feature in analyze(doc): 2023-11-07T11:03:24+02:00 doc = tokenizer(doc) 2023-11-07T11:03:24+02:00 candidates = self._generate_candidates(text) 2023-11-07T11:03:24+02:00 batch_by_source = self._suggest_with_sources(texts, sources) 2023-11-07T11:03:24+02:00 _, X = self._count_vocab(raw_documents, fixed_vocab=True) 2023-11-07T11:03:24+02:00 [self._suggest(text, params) for text in texts], 2023-11-07T11:03:24+02:00 [self._suggest(text, params) for text in texts], 2023-11-07T11:03:24+02:00 (batch.filter(limit, threshold) for batch in self.batches) 2023-11-07T11:03:24+02:00 annif.exception.OperationFailedException: Error in tokenization of text 'liikunnallisesti aktiivinen vanhempi näkee yleensä vaivaa lapsensa liikkumisen eteen sekä 2023-11-07T11:03:24+02:00 ValueError: character U+3e3ffe4e is not in range [U+0000; U+10ffff] 2023-11-07T11:03:24+02:00 ERROR:annif:Exception on /v1/projects/yso-fi/suggest [POST] 2023-11-07T11:03:24+02:00 rajoittaa lapsen viettämää aikaa passiivisten ajanviettotapojen parissa.' 2023-11-07T11:03:24+02:00 annif.exception.OperationFailedException: Voikko error in analysis of word 'vaivaa' 2023-11-07T11:03:24+02:00 Traceback (most recent call last): 2023-11-07T11:03:24+02:00 Traceback (most recent call last): 2023-11-07T11:03:24+02:00 Traceback (most recent call last): 2023-11-07T11:03:24+02:00 The above exception was the direct cause of the following exception: 2023-11-07T11:03:24+02:00 The above exception was the direct cause of the following exception: 2023-11-07T11:03:24+02:00 2023-11-07T11:03:24+02:00 2023-11-07T11:03:24+02:00 2023-11-07T11:03:24+02:00 2023-11-07T11:03:24+02:00 File "/usr/local/lib/python3.10/site-packages/voikko/libvoikko.py", line 682, in analyze 2023-11-07T11:03:24+02:00 File "/usr/local/lib/python3.10/site-packages/sklearn/feature_extraction/text.py", line 1434, in transform 2023-11-07T11:03:24+02:00 File "/usr/local/lib/python3.10/site-packages/sklearn/feature_extraction/text.py", line 1276, in _count_vocab 2023-11-07T11:03:24+02:00 File "/usr/local/lib/python3.10/site-packages/sklearn/feature_extraction/text.py", line 112, in _analyze 2023-11-07T11:03:24+02:00 File "/usr/local/lib/python3.10/site-packages/flask_cors/extension.py", line 176, in wrapped_function 2023-11-07T11:03:24+02:00 File "/usr/local/lib/python3.10/site-packages/flask/app.py", line 2529, in wsgi_app 2023-11-07T11:03:24+02:00 File "/usr/local/lib/python3.10/site-packages/flask/app.py", line 1825, in full_dispatch_request 2023-11-07T11:03:24+02:00 File "/usr/local/lib/python3.10/site-packages/flask/app.py", line 1823, in full_dispatch_request 2023-11-07T11:03:24+02:00 File "/usr/local/lib/python3.10/site-packages/flask/app.py", line 1799, in dispatch_request 2023-11-07T11:03:24+02:00 File "/usr/local/lib/python3.10/site-packages/connexion/decorators/validation.py", line 399, in wrapper 2023-11-07T11:03:24+02:00 File "/usr/local/lib/python3.10/site-packages/connexion/decorators/validation.py", line 196, in wrapper 2023-11-07T11:03:24+02:00 File "/usr/local/lib/python3.10/site-packages/connexion/decorators/uri_parsing.py", line 149, in wrapper 2023-11-07T11:03:24+02:00 File "/usr/local/lib/python3.10/site-packages/connexion/decorators/parameter.py", line 120, in wrapper 2023-11-07T11:03:24+02:00 File "/usr/local/lib/python3.10/site-packages/connexion/decorators/decorator.py", line 68, in wrapper 2023-11-07T11:03:24+02:00 File "/Annif/annif/suggestion.py", line 159, in 2023-11-07T11:03:24+02:00 File "/Annif/annif/rest.py", line 99, in _hit_sets_to_list 2023-11-07T11:03:24+02:00 File "/Annif/annif/rest.py", line 99, in 2023-11-07T11:03:24+02:00 File "/Annif/annif/rest.py", line 174, in _suggest 2023-11-07T11:03:24+02:00 File "/Annif/annif/rest.py", line 122, in suggest 2023-11-07T11:03:24+02:00 File "/Annif/annif/project.py", line 253, in suggest 2023-11-07T11:03:24+02:00 File "/Annif/annif/project.py", line 253, in suggest 2023-11-07T11:03:24+02:00 File "/Annif/annif/project.py", line 234, in 2023-11-07T11:03:24+02:00 File "/Annif/annif/project.py", line 143, in _suggest_with_backend 2023-11-07T11:03:24+02:00 File "/Annif/annif/project.py", line 143, in _suggest_with_backend 2023-11-07T11:03:24+02:00 File "/Annif/annif/lexical/mllm.py", line 87, in generate_candidates 2023-11-07T11:03:24+02:00 File "/Annif/annif/lexical/mllm.py", line 156, in generate_candidates 2023-11-07T11:03:24+02:00 File "/Annif/annif/backend/mllm.py", line 166, in _suggest 2023-11-07T11:03:24+02:00 File "/Annif/annif/backend/mllm.py", line 153, in _generate_candidates 2023-11-07T11:03:24+02:00 File "/Annif/annif/backend/ensemble.py", line 71, in _suggest_batch 2023-11-07T11:03:24+02:00 File "/Annif/annif/backend/ensemble.py", line 46, in 2023-11-07T11:03:24+02:00 File "/Annif/annif/backend/ensemble.py", line 45, in _suggest_with_sources 2023-11-07T11:03:24+02:00 File "/Annif/annif/backend/backend.py", line 143, in suggest 2023-11-07T11:03:24+02:00 File "/Annif/annif/backend/backend.py", line 143, in suggest 2023-11-07T11:03:24+02:00 File "/Annif/annif/backend/backend.py", line 129, in _suggest_batch 2023-11-07T11:03:24+02:00 File "/Annif/annif/backend/backend.py", line 129, in 2023-11-07T11:03:24+02:00 File "/Annif/annif/analyzer/voikko.py", line 35, in _normalize_word 2023-11-07T11:03:24+02:00 File "/Annif/annif/analyzer/voikko.py", line 33, in _normalize_word 2023-11-07T11:03:24+02:00 File "/Annif/annif/analyzer/analyzer.py", line 56, in tokenize_words 2023-11-07T11:03:24+02:00 File "/Annif/annif/analyzer/analyzer.py", line 51, in 2023-11-07T11:03:24+02:00 File "/Annif/annif/analyzer/analyzer.py", line 50, in tokenize_words 2023-11-07T11:03:24+02:00 value = self.__lib.voikko_mor_analysis_value_ucs4(cAnalysis, key) 2023-11-07T11:03:24+02:00 sent_tokens = vectorizer.transform(sentences) 2023-11-07T11:03:24+02:00 self.suggest([doc.text for doc in doc_batch], backend_params) 2023-11-07T11:03:24+02:00 self._normalize_word(word) 2023-11-07T11:03:24+02:00 rv = self.handle_user_exception(e) 2023-11-07T11:03:24+02:00 rv = self.dispatch_request() 2023-11-07T11:03:24+02:00 return { 2023-11-07T11:03:24+02:00 return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) 2023-11-07T11:03:24+02:00 return self.backend.suggest(texts, beparams) 2023-11-07T11:03:24+02:00 return self.backend.suggest(texts, beparams) 2023-11-07T11:03:24+02:00 return self._suggest_with_backend(texts, backend_params) 2023-11-07T11:03:24+02:00 return self._suggest_with_backend(texts, backend_params) 2023-11-07T11:03:24+02:00 return self._suggest_batch(texts, params=beparams) 2023-11-07T11:03:24+02:00 return self._suggest_batch(texts, params=beparams) 2023-11-07T11:03:24+02:00 return self._model.generate_candidates(text, self.project.analyzer) 2023-11-07T11:03:24+02:00 return generate_candidates(text, analyzer, self._vectorizer, self._index) 2023-11-07T11:03:24+02:00 return function(request) 2023-11-07T11:03:24+02:00 return function(**kwargs) 2023-11-07T11:03:24+02:00 return cors_after_request(app.make_response(f(*args, **kwargs))) 2023-11-07T11:03:24+02:00 return _hit_sets_to_list(hit_sets, project.subjects, lang) 2023-11-07T11:03:24+02:00 return [ 2023-11-07T11:03:24+02:00 return [ 2023-11-07T11:03:24+02:00 return [ 2023-11-07T11:03:24+02:00 result = self.voikko.analyze(word) 2023-11-07T11:03:24+02:00 result = _suggest(project_id, documents, parameters) 2023-11-07T11:03:24+02:00 response = self.full_dispatch_request() 2023-11-07T11:03:24+02:00 response = function(request) 2023-11-07T11:03:24+02:00 response = function(request) 2023-11-07T11:03:24+02:00 response = function(request) 2023-11-07T11:03:24+02:00 raise OperationFailedException( 2023-11-07T11:03:24+02:00 raise OperationFailedException( 2023-11-07T11:03:24+02:00 project_id: self.project.registry.get_project(project_id).suggest(texts) 2023-11-07T11:03:24+02:00 for feature in analyze(doc): 2023-11-07T11:03:24+02:00 doc = tokenizer(doc) 2023-11-07T11:03:24+02:00 candidates = self._generate_candidates(text) 2023-11-07T11:03:24+02:00 batch_by_source = self._suggest_with_sources(texts, sources) 2023-11-07T11:03:24+02:00 _, X = self._count_vocab(raw_documents, fixed_vocab=True) 2023-11-07T11:03:24+02:00 [self._suggest(text, params) for text in texts], 2023-11-07T11:03:24+02:00 [self._suggest(text, params) for text in texts], 2023-11-07T11:03:24+02:00 (batch.filter(limit, threshold) for batch in self.batches) 2023-11-07T11:03:24+02:00 annif.exception.OperationFailedException: Error in tokenization of text 'niiden on 􋍵nsinnäkin nähty haittaavan tavoitteiden saa󲿾uttamista (emmons & king, 1988), toisaalt򪛿 2023-11-07T11:03:24+02:00 ValueError: character U+3e3ffe4e is not in range [U+0000; U+10ffff] 2023-11-07T11:03:24+02:00 ERROR:annif:Exception on /v1/projects/yso-fi/suggest [POST] 2023-11-07T11:03:24+02:00 niitä on pidetty jo itsessään haitallisia hyvinvoinnille (riediger, 2007).' 2023-11-07T11:03:24+02:00 annif.exception.OperationFailedException: Voikko error in analysis of word 'niitä' 2023-11-07T11:03:24+02:00 Traceback (most recent call last): 2023-11-07T11:03:24+02:00 Traceback (most recent call last): 2023-11-07T11:03:24+02:00 Traceback (most recent call last): 2023-11-07T11:03:24+02:00 The above exception was the direct cause of the following exception: 2023-11-07T11:03:24+02:00 The above exception was the direct cause of the following exception: 2023-11-07T11:03:24+02:00 2023-11-07T11:03:24+02:00 2023-11-07T11:03:24+02:00 2023-11-07T11:03:24+02:00 2023-11-07T11:03:24+02:00 File "/usr/local/lib/python3.10/site-packages/voikko/libvoikko.py", line 682, in analyze 2023-11-07T11:03:24+02:00 File "/usr/local/lib/python3.10/site-packages/sklearn/feature_extraction/text.py", line 1434, in transform 2023-11-07T11:03:24+02:00 File "/usr/local/lib/python3.10/site-packages/sklearn/feature_extraction/text.py", line 1276, in _count_vocab 2023-11-07T11:03:24+02:00 File "/usr/local/lib/python3.10/site-packages/sklearn/feature_extraction/text.py", line 112, in _analyze 2023-11-07T11:03:24+02:00 File "/usr/local/lib/python3.10/site-packages/flask_cors/extension.py", line 176, in wrapped_function 2023-11-07T11:03:24+02:00 File "/usr/local/lib/python3.10/site-packages/flask/app.py", line 2529, in wsgi_app 2023-11-07T11:03:24+02:00 File "/usr/local/lib/python3.10/site-packages/flask/app.py", line 1825, in full_dispatch_request 2023-11-07T11:03:24+02:00 File "/usr/local/lib/python3.10/site-packages/flask/app.py", line 1823, in full_dispatch_request 2023-11-07T11:03:24+02:00 File "/usr/local/lib/python3.10/site-packages/flask/app.py", line 1799, in dispatch_request 2023-11-07T11:03:24+02:00 File "/usr/local/lib/python3.10/site-packages/connexion/decorators/validation.py", line 399, in wrapper 2023-11-07T11:03:24+02:00 File "/usr/local/lib/python3.10/site-packages/connexion/decorators/validation.py", line 196, in wrapper 2023-11-07T11:03:24+02:00 File "/usr/local/lib/python3.10/site-packages/connexion/decorators/uri_parsing.py", line 149, in wrapper 2023-11-07T11:03:24+02:00 File "/usr/local/lib/python3.10/site-packages/connexion/decorators/parameter.py", line 120, in wrapper 2023-11-07T11:03:24+02:00 File "/usr/local/lib/python3.10/site-packages/connexion/decorators/decorator.py", line 68, in wrapper 2023-11-07T11:03:24+02:00 File "/Annif/annif/suggestion.py", line 159, in 2023-11-07T11:03:24+02:00 File "/Annif/annif/rest.py", line 99, in _hit_sets_to_list 2023-11-07T11:03:24+02:00 File "/Annif/annif/rest.py", line 99, in 2023-11-07T11:03:24+02:00 File "/Annif/annif/rest.py", line 174, in _suggest 2023-11-07T11:03:24+02:00 File "/Annif/annif/rest.py", line 122, in suggest 2023-11-07T11:03:24+02:00 File "/Annif/annif/project.py", line 253, in suggest 2023-11-07T11:03:24+02:00 File "/Annif/annif/project.py", line 253, in suggest 2023-11-07T11:03:24+02:00 File "/Annif/annif/project.py", line 234, in 2023-11-07T11:03:24+02:00 File "/Annif/annif/project.py", line 143, in _suggest_with_backend 2023-11-07T11:03:24+02:00 File "/Annif/annif/project.py", line 143, in _suggest_with_backend 2023-11-07T11:03:24+02:00 File "/Annif/annif/lexical/mllm.py", line 87, in generate_candidates 2023-11-07T11:03:24+02:00 File "/Annif/annif/lexical/mllm.py", line 156, in generate_candidates 2023-11-07T11:03:24+02:00 File "/Annif/annif/backend/mllm.py", line 166, in _suggest 2023-11-07T11:03:24+02:00 File "/Annif/annif/backend/mllm.py", line 153, in _generate_candidates 2023-11-07T11:03:24+02:00 File "/Annif/annif/backend/ensemble.py", line 71, in _suggest_batch 2023-11-07T11:03:24+02:00 File "/Annif/annif/backend/ensemble.py", line 46, in 2023-11-07T11:03:24+02:00 File "/Annif/annif/backend/ensemble.py", line 45, in _suggest_with_sources 2023-11-07T11:03:24+02:00 File "/Annif/annif/backend/backend.py", line 143, in suggest 2023-11-07T11:03:24+02:00 File "/Annif/annif/backend/backend.py", line 143, in suggest 2023-11-07T11:03:24+02:00 File "/Annif/annif/backend/backend.py", line 129, in _suggest_batch 2023-11-07T11:03:24+02:00 File "/Annif/annif/backend/backend.py", line 129, in 2023-11-07T11:03:24+02:00 File "/Annif/annif/analyzer/voikko.py", line 35, in _normalize_word 2023-11-07T11:03:24+02:00 File "/Annif/annif/analyzer/voikko.py", line 33, in _normalize_word 2023-11-07T11:03:24+02:00 File "/Annif/annif/analyzer/analyzer.py", line 56, in tokenize_words 2023-11-07T11:03:24+02:00 File "/Annif/annif/analyzer/analyzer.py", line 51, in 2023-11-07T11:03:24+02:00 File "/Annif/annif/analyzer/analyzer.py", line 50, in tokenize_words 2023-11-07T11:03:24+02:00 value = self.__lib.voikko_mor_analysis_value_ucs4(cAnalysis, key) 2023-11-07T11:03:24+02:00 sent_tokens = vectorizer.transform(sentences) 2023-11-07T11:03:24+02:00 self.suggest([doc.text for doc in doc_batch], backend_params) 2023-11-07T11:03:24+02:00 self._normalize_word(word) 2023-11-07T11:03:24+02:00 rv = self.handle_user_exception(e) 2023-11-07T11:03:24+02:00 rv = self.dispatch_request() 2023-11-07T11:03:24+02:00 return { 2023-11-07T11:03:24+02:00 return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) 2023-11-07T11:03:24+02:00 return self.backend.suggest(texts, beparams) 2023-11-07T11:03:24+02:00 return self.backend.suggest(texts, beparams) 2023-11-07T11:03:24+02:00 return self._suggest_with_backend(texts, backend_params) 2023-11-07T11:03:24+02:00 return self._suggest_with_backend(texts, backend_params) 2023-11-07T11:03:24+02:00 return self._suggest_batch(texts, params=beparams) 2023-11-07T11:03:24+02:00 return self._suggest_batch(texts, params=beparams) 2023-11-07T11:03:24+02:00 return self._model.generate_candidates(text, self.project.analyzer) 2023-11-07T11:03:24+02:00 return generate_candidates(text, analyzer, self._vectorizer, self._index) 2023-11-07T11:03:24+02:00 return function(request) 2023-11-07T11:03:24+02:00 return function(**kwargs) 2023-11-07T11:03:24+02:00 return cors_after_request(app.make_response(f(*args, **kwargs))) 2023-11-07T11:03:24+02:00 return _hit_sets_to_list(hit_sets, project.subjects, lang) 2023-11-07T11:03:24+02:00 return [ 2023-11-07T11:03:24+02:00 return [ 2023-11-07T11:03:24+02:00 return [ 2023-11-07T11:03:24+02:00 result = self.voikko.analyze(word) 2023-11-07T11:03:24+02:00 result = _suggest(project_id, documents, parameters) 2023-11-07T11:03:24+02:00 response = self.full_dispatch_request() 2023-11-07T11:03:24+02:00 response = function(request) 2023-11-07T11:03:24+02:00 response = function(request) 2023-11-07T11:03:24+02:00 response = function(request) 2023-11-07T11:03:24+02:00 raise OperationFailedException( 2023-11-07T11:03:24+02:00 raise OperationFailedException( 2023-11-07T11:03:24+02:00 project_id: self.project.registry.get_project(project_id).suggest(texts) 2023-11-07T11:03:24+02:00 for feature in analyze(doc): 2023-11-07T11:03:24+02:00 doc = tokenizer(doc) 2023-11-07T11:03:24+02:00 candidates = self._generate_candidates(text) 2023-11-07T11:03:24+02:00 batch_by_source = self._suggest_with_sources(texts, sources) 2023-11-07T11:03:24+02:00 _, X = self._count_vocab(raw_documents, fixed_vocab=True) 2023-11-07T11:03:24+02:00 [self._suggest(text, params) for text in texts], 2023-11-07T11:03:24+02:00 [self._suggest(text, params) for text in texts], 2023-11-07T11:03:24+02:00 (batch.filter(limit, threshold) for batch in self.batches) 2023-11-07T11:03:24+02:00 annif.exception.OperationFailedException: Error in tokenization of text 'sivuaineopintojen suhteen eroa havaittiin vain kahden sivuaineryhmän välil􊚿ä.' 2023-11-07T11:03:24+02:00 ValueError: character U+3e3ffe4e is not in range [U+0000; U+10ffff] 2023-11-07T11:03:24+02:00 ERROR:annif:Exception on /v1/projects/yso-fi/suggest [POST] 2023-11-07T11:03:24+02:00 annif.exception.OperationFailedException: Voikko error in analysis of word 'eroa' 2023-11-07T11:03:24+02:00 Traceback (most recent call last): 2023-11-07T11:03:24+02:00 Traceback (most recent call last): 2023-11-07T11:03:24+02:00 Traceback (most recent call last): 2023-11-07T11:03:24+02:00 The above exception was the direct cause of the following exception: 2023-11-07T11:03:24+02:00 The above exception was the direct cause of the following exception: 2023-11-07T11:03:24+02:00 2023-11-07T11:03:24+02:00 2023-11-07T11:03:24+02:00 2023-11-07T11:03:24+02:00 2023-11-07T11:03:24+02:00 File "/usr/local/lib/python3.10/site-packages/voikko/libvoikko.py", line 682, in analyze 2023-11-07T11:03:24+02:00 File "/usr/local/lib/python3.10/site-packages/sklearn/feature_extraction/text.py", line 1434, in transform 2023-11-07T11:03:24+02:00 File "/usr/local/lib/python3.10/site-packages/sklearn/feature_extraction/text.py", line 1276, in _count_vocab 2023-11-07T11:03:24+02:00 File "/usr/local/lib/python3.10/site-packages/sklearn/feature_extraction/text.py", line 112, in _analyze 2023-11-07T11:03:24+02:00 File "/usr/local/lib/python3.10/site-packages/flask_cors/extension.py", line 176, in wrapped_function 2023-11-07T11:03:24+02:00 File "/usr/local/lib/python3.10/site-packages/flask/app.py", line 2529, in wsgi_app 2023-11-07T11:03:24+02:00 File "/usr/local/lib/python3.10/site-packages/flask/app.py", line 1825, in full_dispatch_request 2023-11-07T11:03:24+02:00 File "/usr/local/lib/python3.10/site-packages/flask/app.py", line 1823, in full_dispatch_request 2023-11-07T11:03:24+02:00 File "/usr/local/lib/python3.10/site-packages/flask/app.py", line 1799, in dispatch_request 2023-11-07T11:03:24+02:00 File "/usr/local/lib/python3.10/site-packages/connexion/decorators/validation.py", line 399, in wrapper 2023-11-07T11:03:24+02:00 File "/usr/local/lib/python3.10/site-packages/connexion/decorators/validation.py", line 196, in wrapper 2023-11-07T11:03:24+02:00 File "/usr/local/lib/python3.10/site-packages/connexion/decorators/uri_parsing.py", line 149, in wrapper 2023-11-07T11:03:24+02:00 File "/usr/local/lib/python3.10/site-packages/connexion/decorators/parameter.py", line 120, in wrapper 2023-11-07T11:03:24+02:00 File "/usr/local/lib/python3.10/site-packages/connexion/decorators/decorator.py", line 68, in wrapper 2023-11-07T11:03:24+02:00 File "/Annif/annif/suggestion.py", line 159, in 2023-11-07T11:03:24+02:00 File "/Annif/annif/rest.py", line 99, in _hit_sets_to_list 2023-11-07T11:03:24+02:00 File "/Annif/annif/rest.py", line 99, in 2023-11-07T11:03:24+02:00 File "/Annif/annif/rest.py", line 174, in _suggest 2023-11-07T11:03:24+02:00 File "/Annif/annif/rest.py", line 122, in suggest 2023-11-07T11:03:24+02:00 File "/Annif/annif/project.py", line 253, in suggest 2023-11-07T11:03:24+02:00 File "/Annif/annif/project.py", line 253, in suggest 2023-11-07T11:03:24+02:00 File "/Annif/annif/project.py", line 234, in 2023-11-07T11:03:24+02:00 File "/Annif/annif/project.py", line 143, in _suggest_with_backend 2023-11-07T11:03:24+02:00 File "/Annif/annif/project.py", line 143, in _suggest_with_backend 2023-11-07T11:03:24+02:00 File "/Annif/annif/lexical/mllm.py", line 87, in generate_candidates 2023-11-07T11:03:24+02:00 File "/Annif/annif/lexical/mllm.py", line 156, in generate_candidates 2023-11-07T11:03:24+02:00 File "/Annif/annif/backend/mllm.py", line 166, in _suggest 2023-11-07T11:03:24+02:00 File "/Annif/annif/backend/mllm.py", line 153, in _generate_candidates 2023-11-07T11:03:24+02:00 File "/Annif/annif/backend/ensemble.py", line 71, in _suggest_batch 2023-11-07T11:03:24+02:00 File "/Annif/annif/backend/ensemble.py", line 46, in 2023-11-07T11:03:24+02:00 File "/Annif/annif/backend/ensemble.py", line 45, in _suggest_with_sources 2023-11-07T11:03:24+02:00 File "/Annif/annif/backend/backend.py", line 143, in suggest 2023-11-07T11:03:24+02:00 File "/Annif/annif/backend/backend.py", line 143, in suggest 2023-11-07T11:03:24+02:00 File "/Annif/annif/backend/backend.py", line 129, in _suggest_batch 2023-11-07T11:03:24+02:00 File "/Annif/annif/backend/backend.py", line 129, in 2023-11-07T11:03:24+02:00 File "/Annif/annif/analyzer/voikko.py", line 35, in _normalize_word 2023-11-07T11:03:24+02:00 File "/Annif/annif/analyzer/voikko.py", line 33, in _normalize_word 2023-11-07T11:03:24+02:00 File "/Annif/annif/analyzer/analyzer.py", line 56, in tokenize_words 2023-11-07T11:03:24+02:00 File "/Annif/annif/analyzer/analyzer.py", line 51, in 2023-11-07T11:03:24+02:00 File "/Annif/annif/analyzer/analyzer.py", line 50, in tokenize_words 2023-11-07T11:03:24+02:00 value = self.__lib.voikko_mor_analysis_value_ucs4(cAnalysis, key) 2023-11-07T11:03:24+02:00 sent_tokens = vectorizer.transform(sentences) 2023-11-07T11:03:24+02:00 self.suggest([doc.text for doc in doc_batch], backend_params) 2023-11-07T11:03:24+02:00 self._normalize_word(word) 2023-11-07T11:03:24+02:00 rv = self.handle_user_exception(e) 2023-11-07T11:03:24+02:00 rv = self.dispatch_request() 2023-11-07T11:03:24+02:00 return { 2023-11-07T11:03:24+02:00 return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) 2023-11-07T11:03:24+02:00 return self.backend.suggest(texts, beparams) 2023-11-07T11:03:24+02:00 return self.backend.suggest(texts, beparams) 2023-11-07T11:03:24+02:00 return self._suggest_with_backend(texts, backend_params) 2023-11-07T11:03:24+02:00 return self._suggest_with_backend(texts, backend_params) 2023-11-07T11:03:24+02:00 return self._suggest_batch(texts, params=beparams) 2023-11-07T11:03:24+02:00 return self._suggest_batch(texts, params=beparams) 2023-11-07T11:03:24+02:00 return self._model.generate_candidates(text, self.project.analyzer) 2023-11-07T11:03:24+02:00 return generate_candidates(text, analyzer, self._vectorizer, self._index) 2023-11-07T11:03:24+02:00 return function(request) 2023-11-07T11:03:24+02:00 return function(**kwargs) 2023-11-07T11:03:24+02:00 return cors_after_request(app.make_response(f(*args, **kwargs))) 2023-11-07T11:03:24+02:00 return _hit_sets_to_list(hit_sets, project.subjects, lang) 2023-11-07T11:03:24+02:00 return [ 2023-11-07T11:03:24+02:00 return [ 2023-11-07T11:03:24+02:00 return [ 2023-11-07T11:03:24+02:00 result = self.voikko.analyze(word) 2023-11-07T11:03:24+02:00 result = _suggest(project_id, documents, parameters) 2023-11-07T11:03:24+02:00 response = self.full_dispatch_request() 2023-11-07T11:03:24+02:00 response = function(request) 2023-11-07T11:03:24+02:00 response = function(request) 2023-11-07T11:03:24+02:00 response = function(request) 2023-11-07T11:03:24+02:00 raise OperationFailedException( 2023-11-07T11:03:24+02:00 raise OperationFailedException( 2023-11-07T11:03:24+02:00 project_id: self.project.registry.get_project(project_id).suggest(texts) 2023-11-07T11:03:24+02:00 for feature in analyze(doc): 2023-11-07T11:03:24+02:00 doc = tokenizer(doc) 2023-11-07T11:03:24+02:00 candidates = self._generate_candidates(text) 2023-11-07T11:03:24+02:00 batch_by_source = self._suggest_with_sources(texts, sources) 2023-11-07T11:03:24+02:00 _, X = self._count_vocab(raw_documents, fixed_vocab=True) 2023-11-07T11:03:24+02:00 [self._suggest(text, params) for text in texts], 2023-11-07T11:03:24+02:00 [self._suggest(text, params) for text in texts], 2023-11-07T11:03:24+02:00 (batch.filter(limit, threshold) for batch in self.batches) 2023-11-07T11:03:24+02:00 INFO:annif:finished initializing projects 2023-11-07T11:03:24+02:00 2023-11-07T09:03:16.876Z INFO [omikuji::model] Loaded model with 3 trees; it took 0.09s 2023-11-07T11:03:24+02:00 2023-11-07T09:03:16.851Z INFO [omikuji::model] Loading tree from data/projects/thema-school-bonsai-fi/omikuji-model/tree2.cbor... 2023-11-07T11:03:24+02:00 2023-11-07T09:03:16.822Z INFO [omikuji::model] Loading tree from data/projects/thema-school-bonsai-fi/omikuji-model/tree1.cbor... 2023-11-07T11:03:24+02:00 2023-11-07T09:03:16.785Z INFO [omikuji::model] Loading tree from data/projects/thema-school-bonsai-fi/omikuji-model/tree0.cbor... 2023-11-07T11:03:24+02:00 2023-11-07T09:03:16.785Z INFO [omikuji::model] Loaded model settings Settings { n_features: 61903, classifier_loss_type: Hinge }... 2023-11-07T11:03:24+02:00 2023-11-07T09:03:16.784Z INFO [omikuji::model] Loading model settings from data/projects/thema-school-bonsai-fi/omikuji-model/settings.json... 2023-11-07T11:03:24+02:00 2023-11-07T09:03:16.784Z INFO [omikuji::model] Loading model from data/projects/thema-school-bonsai-fi/omikuji-model... 2023-11-07T11:03:24+02:00 2023-11-07T09:03:15.646Z INFO [omikuji::model] Loaded model with 3 trees; it took 0.07s 2023-11-07T11:03:24+02:00 2023-11-07T09:03:15.622Z INFO [omikuji::model] Loading tree from data/projects/thema-school-parabel-fi/omikuji-model/tree2.cbor... 2023-11-07T11:03:24+02:00 2023-11-07T09:03:15.606Z INFO [omikuji::model] Loading tree from data/projects/thema-school-parabel-fi/omikuji-model/tree1.cbor... 2023-11-07T11:03:24+02:00 2023-11-07T09:03:15.573Z INFO [omikuji::model] Loading tree from data/projects/thema-school-parabel-fi/omikuji-model/tree0.cbor... 2023-11-07T11:03:24+02:00 2023-11-07T09:03:15.573Z INFO [omikuji::model] Loaded model settings Settings { n_features: 27588, classifier_loss_type: Hinge }... 2023-11-07T11:03:24+02:00 2023-11-07T09:03:15.572Z INFO [omikuji::model] Loading model settings from data/projects/thema-school-parabel-fi/omikuji-model/settings.json... 2023-11-07T11:03:24+02:00 2023-11-07T09:03:15.572Z INFO [omikuji::model] Loading model from data/projects/thema-school-parabel-fi/omikuji-model... 2023-11-07T11:02:55+02:00 2023-11-07T09:02:54.609Z INFO [omikuji::model] Loaded model with 3 trees; it took 0.49s 2023-11-07T11:02:55+02:00 2023-11-07T09:02:54.511Z INFO [omikuji::model] Loading tree from data/projects/thema-nonfict-bonsai-fi/omikuji-model/tree2.cbor... 2023-11-07T11:02:55+02:00 2023-11-07T09:02:54.347Z INFO [omikuji::model] Loading tree from data/projects/thema-nonfict-bonsai-fi/omikuji-model/tree1.cbor... 2023-11-07T11:02:55+02:00 2023-11-07T09:02:54.122Z INFO [omikuji::model] Loading tree from data/projects/thema-nonfict-bonsai-fi/omikuji-model/tree0.cbor... 2023-11-07T11:02:55+02:00 2023-11-07T09:02:54.122Z INFO [omikuji::model] Loaded model settings Settings { n_features: 144386, classifier_loss_type: Hinge }... 2023-11-07T11:02:55+02:00 2023-11-07T09:02:54.122Z INFO [omikuji::model] Loading model settings from data/projects/thema-nonfict-bonsai-fi/omikuji-model/settings.json... 2023-11-07T11:02:55+02:00 2023-11-07T09:02:54.122Z INFO [omikuji::model] Loading model from data/projects/thema-nonfict-bonsai-fi/omikuji-model... 2023-11-07T11:02:51+02:00 2023-11-07T09:02:50.959Z INFO [omikuji::model] Loaded model with 3 trees; it took 0.95s 2023-11-07T11:02:50+02:00 2023-11-07T09:02:50.736Z INFO [omikuji::model] Loading tree from data/projects/thema-nonfict-parabel-fi/omikuji-model/tree2.cbor... 2023-11-07T11:02:50+02:00 2023-11-07T09:02:50.467Z INFO [omikuji::model] Loading tree from data/projects/thema-nonfict-parabel-fi/omikuji-model/tree1.cbor... 2023-11-07T11:02:50+02:00 2023-11-07T09:02:50.012Z INFO [omikuji::model] Loading tree from data/projects/thema-nonfict-parabel-fi/omikuji-model/tree0.cbor... 2023-11-07T11:02:50+02:00 2023-11-07T09:02:50.012Z INFO [omikuji::model] Loaded model settings Settings { n_features: 68014, classifier_loss_type: Hinge }... 2023-11-07T11:02:50+02:00 2023-11-07T09:02:50.011Z INFO [omikuji::model] Loading model settings from data/projects/thema-nonfict-parabel-fi/omikuji-model/settings.json... 2023-11-07T11:02:50+02:00 2023-11-07T09:02:50.011Z INFO [omikuji::model] Loading model from data/projects/thema-nonfict-parabel-fi/omikuji-model... 2023-11-07T11:02:46+02:00 /usr/local/lib/python3.10/site-packages/sklearn/base.py:348: InconsistentVersionWarning: Trying to unpickle estimator LinearSVC from version 1.3.1 when using version 1.3.2. This might lead to breaking code or invalid results. Use at your own risk. For more info please refer to: 2023-11-07T11:02:46+02:00 https://scikit-learn.org/stable/model_persistence.html#security-maintainability-limitations 2023-11-07T11:02:46+02:00 warnings.warn( 2023-11-07T11:02:42+02:00 2023-11-07T09:02:40.651Z INFO [omikuji::model] Loaded model with 3 trees; it took 0.22s 2023-11-07T11:02:42+02:00 2023-11-07T09:02:40.591Z INFO [omikuji::model] Loading tree from data/projects/thema-fict-bonsai-fi/omikuji-model/tree2.cbor... 2023-11-07T11:02:42+02:00 2023-11-07T09:02:40.492Z INFO [omikuji::model] Loading tree from data/projects/thema-fict-bonsai-fi/omikuji-model/tree1.cbor... 2023-11-07T11:02:42+02:00 2023-11-07T09:02:40.433Z INFO [omikuji::model] Loading tree from data/projects/thema-fict-bonsai-fi/omikuji-model/tree0.cbor... 2023-11-07T11:02:42+02:00 2023-11-07T09:02:40.433Z INFO [omikuji::model] Loaded model settings Settings { n_features: 115953, classifier_loss_type: Hinge }... 2023-11-07T11:02:42+02:00 2023-11-07T09:02:40.433Z INFO [omikuji::model] Loading model settings from data/projects/thema-fict-bonsai-fi/omikuji-model/settings.json... 2023-11-07T11:02:42+02:00 2023-11-07T09:02:40.433Z INFO [omikuji::model] Loading model from data/projects/thema-fict-bonsai-fi/omikuji-model... 2023-11-07T11:02:38+02:00 2023-11-07T09:02:38.050Z INFO [omikuji::model] Loaded model with 3 trees; it took 0.20s 2023-11-07T11:02:38+02:00 2023-11-07T09:02:37.975Z INFO [omikuji::model] Loading tree from data/projects/thema-fict-parabel-fi/omikuji-model/tree2.cbor... 2023-11-07T11:02:38+02:00 2023-11-07T09:02:37.913Z INFO [omikuji::model] Loading tree from data/projects/thema-fict-parabel-fi/omikuji-model/tree1.cbor... 2023-11-07T11:02:38+02:00 2023-11-07T09:02:37.849Z INFO [omikuji::model] Loading tree from data/projects/thema-fict-parabel-fi/omikuji-model/tree0.cbor... 2023-11-07T11:02:38+02:00 2023-11-07T09:02:37.849Z INFO [omikuji::model] Loaded model settings Settings { n_features: 52287, classifier_loss_type: Hinge }... 2023-11-07T11:02:38+02:00 2023-11-07T09:02:37.848Z INFO [omikuji::model] Loading model settings from data/projects/thema-fict-parabel-fi/omikuji-model/settings.json... 2023-11-07T11:02:38+02:00 2023-11-07T09:02:37.848Z INFO [omikuji::model] Loading model from data/projects/thema-fict-parabel-fi/omikuji-model... 2023-11-07T11:02:32+02:00 2023-11-07T09:02:31.794Z INFO [omikuji::model] Loaded model with 3 trees; it took 5.18s 2023-11-07T11:02:31+02:00 2023-11-07T09:02:30.537Z INFO [omikuji::model] Loading tree from data/projects/kauno-finna-bonsai-fi/omikuji-model/tree2.cbor... 2023-11-07T11:02:31+02:00 2023-11-07T09:02:29.049Z INFO [omikuji::model] Loading tree from data/projects/kauno-finna-bonsai-fi/omikuji-model/tree1.cbor... 2023-11-07T11:02:26+02:00 2023-11-07T09:02:26.613Z INFO [omikuji::model] Loading tree from data/projects/kauno-finna-bonsai-fi/omikuji-model/tree0.cbor... 2023-11-07T11:02:26+02:00 2023-11-07T09:02:26.613Z INFO [omikuji::model] Loaded model settings Settings { n_features: 189653, classifier_loss_type: Hinge }... 2023-11-07T11:02:26+02:00 2023-11-07T09:02:26.612Z INFO [omikuji::model] Loading model settings from data/projects/kauno-finna-bonsai-fi/omikuji-model/settings.json... 2023-11-07T11:02:26+02:00 2023-11-07T09:02:26.612Z INFO [omikuji::model] Loading model from data/projects/kauno-finna-bonsai-fi/omikuji-model... 2023-11-07T11:02:25+02:00 2023-11-07T09:02:25.721Z INFO [omikuji::model] Loaded model with 3 trees; it took 9.94s 2023-11-07T11:02:25+02:00 2023-11-07T09:02:22.286Z INFO [omikuji::model] Loading tree from data/projects/kauno-ks-bonsai-fi/omikuji-model/tree2.cbor... 2023-11-07T11:02:25+02:00 2023-11-07T09:02:18.798Z INFO [omikuji::model] Loading tree from data/projects/kauno-ks-bonsai-fi/omikuji-model/tree1.cbor... 2023-11-07T11:02:25+02:00 2023-11-07T09:02:15.785Z INFO [omikuji::model] Loading tree from data/projects/kauno-ks-bonsai-fi/omikuji-model/tree0.cbor... 2023-11-07T11:02:25+02:00 2023-11-07T09:02:15.785Z INFO [omikuji::model] Loaded model settings Settings { n_features: 559304, classifier_loss_type: Hinge }... 2023-11-07T11:02:25+02:00 2023-11-07T09:02:15.784Z INFO [omikuji::model] Loading model settings from data/projects/kauno-ks-bonsai-fi/omikuji-model/settings.json... 2023-11-07T11:02:25+02:00 2023-11-07T09:02:15.784Z INFO [omikuji::model] Loading model from data/projects/kauno-ks-bonsai-fi/omikuji-model... 2023-11-07T11:02:05+02:00 2023-11-07T09:02:05.757Z INFO [omikuji::model] Loaded model with 3 trees; it took 0.05s 2023-11-07T11:02:05+02:00 2023-11-07T09:02:05.743Z INFO [omikuji::model] Loading tree from data/projects/ykl-omikuji-bonsai-en/omikuji-model/tree2.cbor... 2023-11-07T11:02:05+02:00 2023-11-07T09:02:05.730Z INFO [omikuji::model] Loading tree from data/projects/ykl-omikuji-bonsai-en/omikuji-model/tree1.cbor... 2023-11-07T11:02:05+02:00 2023-11-07T09:02:05.713Z INFO [omikuji::model] Loading tree from data/projects/ykl-omikuji-bonsai-en/omikuji-model/tree0.cbor... 2023-11-07T11:02:05+02:00 2023-11-07T09:02:05.713Z INFO [omikuji::model] Loaded model settings Settings { n_features: 12977, classifier_loss_type: Hinge }... 2023-11-07T11:02:05+02:00 2023-11-07T09:02:05.704Z INFO [omikuji::model] Loading model settings from data/projects/ykl-omikuji-bonsai-en/omikuji-model/settings.json... 2023-11-07T11:02:05+02:00 2023-11-07T09:02:05.704Z INFO [omikuji::model] Loading model from data/projects/ykl-omikuji-bonsai-en/omikuji-model... 2023-11-07T11:02:05+02:00 2023-11-07T09:02:05.648Z INFO [omikuji::model] Loaded model with 3 trees; it took 0.03s 2023-11-07T11:02:05+02:00 2023-11-07T09:02:05.632Z INFO [omikuji::model] Loading tree from data/projects/ykl-omikuji-parabel-en/omikuji-model/tree2.cbor... 2023-11-07T11:02:05+02:00 2023-11-07T09:02:05.627Z INFO [omikuji::model] Loading tree from data/projects/ykl-omikuji-parabel-en/omikuji-model/tree1.cbor... 2023-11-07T11:02:05+02:00 2023-11-07T09:02:05.623Z INFO [omikuji::model] Loading tree from data/projects/ykl-omikuji-parabel-en/omikuji-model/tree0.cbor... 2023-11-07T11:02:05+02:00 2023-11-07T09:02:05.623Z INFO [omikuji::model] Loaded model settings Settings { n_features: 12977, classifier_loss_type: Hinge }... 2023-11-07T11:02:05+02:00 2023-11-07T09:02:05.623Z INFO [omikuji::model] Loading model settings from data/projects/ykl-omikuji-parabel-en/omikuji-model/settings.json... 2023-11-07T11:02:05+02:00 2023-11-07T09:02:05.623Z INFO [omikuji::model] Loading model from data/projects/ykl-omikuji-parabel-en/omikuji-model... 2023-11-07T11:02:05+02:00 2023-11-07T09:02:05.531Z INFO [omikuji::model] Loaded model with 3 trees; it took 0.03s 2023-11-07T11:02:05+02:00 2023-11-07T09:02:05.516Z INFO [omikuji::model] Loading tree from data/projects/ykl-omikuji-bonsai-sv/omikuji-model/tree2.cbor... 2023-11-07T11:02:05+02:00 2023-11-07T09:02:05.508Z INFO [omikuji::model] Loading tree from data/projects/ykl-omikuji-bonsai-sv/omikuji-model/tree1.cbor... 2023-11-07T11:02:05+02:00 2023-11-07T09:02:05.502Z INFO [omikuji::model] Loading tree from data/projects/ykl-omikuji-bonsai-sv/omikuji-model/tree0.cbor... 2023-11-07T11:02:05+02:00 2023-11-07T09:02:05.502Z INFO [omikuji::model] Loaded model settings Settings { n_features: 25365, classifier_loss_type: Hinge }... 2023-11-07T11:02:05+02:00 2023-11-07T09:02:05.502Z INFO [omikuji::model] Loading model settings from data/projects/ykl-omikuji-bonsai-sv/omikuji-model/settings.json... 2023-11-07T11:02:05+02:00 2023-11-07T09:02:05.502Z INFO [omikuji::model] Loading model from data/projects/ykl-omikuji-bonsai-sv/omikuji-model... 2023-11-07T11:02:05+02:00 2023-11-07T09:02:05.422Z INFO [omikuji::model] Loaded model with 3 trees; it took 0.03s 2023-11-07T11:02:05+02:00 2023-11-07T09:02:05.408Z INFO [omikuji::model] Loading tree from data/projects/ykl-omikuji-parabel-sv/omikuji-model/tree2.cbor... 2023-11-07T11:02:05+02:00 2023-11-07T09:02:05.401Z INFO [omikuji::model] Loading tree from data/projects/ykl-omikuji-parabel-sv/omikuji-model/tree1.cbor... 2023-11-07T11:02:05+02:00 2023-11-07T09:02:05.394Z INFO [omikuji::model] Loading tree from data/projects/ykl-omikuji-parabel-sv/omikuji-model/tree0.cbor... 2023-11-07T11:02:05+02:00 2023-11-07T09:02:05.394Z INFO [omikuji::model] Loading model settings from data/projects/ykl-omikuji-parabel-sv/omikuji-model/settings.json... 2023-11-07T11:02:05+02:00 2023-11-07T09:02:05.394Z INFO [omikuji::model] Loading model from data/projects/ykl-omikuji-parabel-sv/omikuji-model... 2023-11-07T11:02:05+02:00 2023-11-07T09:02:05.394Z INFO [omikuji::model] Loaded model settings Settings { n_features: 25365, classifier_loss_type: Hinge }... 2023-11-07T11:02:05+02:00 2023-11-07T09:02:05.314Z INFO [omikuji::model] Loaded model with 3 trees; it took 0.55s 2023-11-07T11:02:05+02:00 2023-11-07T09:02:05.216Z INFO [omikuji::model] Loading tree from data/projects/ykl-omikuji-bonsai-fi/omikuji-model/tree2.cbor... 2023-11-07T11:02:05+02:00 2023-11-07T09:02:05.019Z INFO [omikuji::model] Loading tree from data/projects/ykl-omikuji-bonsai-fi/omikuji-model/tree1.cbor... 2023-11-07T11:02:04+02:00 2023-11-07T09:02:04.767Z INFO [omikuji::model] Loading tree from data/projects/ykl-omikuji-bonsai-fi/omikuji-model/tree0.cbor... 2023-11-07T11:02:04+02:00 2023-11-07T09:02:04.767Z INFO [omikuji::model] Loaded model settings Settings { n_features: 183181, classifier_loss_type: Hinge }... 2023-11-07T11:02:04+02:00 2023-11-07T09:02:04.766Z INFO [omikuji::model] Loading model settings from data/projects/ykl-omikuji-bonsai-fi/omikuji-model/settings.json... 2023-11-07T11:02:04+02:00 2023-11-07T09:02:04.766Z INFO [omikuji::model] Loading model from data/projects/ykl-omikuji-bonsai-fi/omikuji-model... 2023-11-07T11:02:04+02:00 2023-11-07T09:02:04.140Z INFO [omikuji::model] Loaded model with 3 trees; it took 0.85s 2023-11-07T11:02:03+02:00 2023-11-07T09:02:03.946Z INFO [omikuji::model] Loading tree from data/projects/ykl-omikuji-parabel-fi/omikuji-model/tree2.cbor... 2023-11-07T11:02:03+02:00 2023-11-07T09:02:03.728Z INFO [omikuji::model] Loading tree from data/projects/ykl-omikuji-parabel-fi/omikuji-model/tree1.cbor... 2023-11-07T11:02:03+02:00 2023-11-07T09:02:03.301Z INFO [omikuji::model] Loading tree from data/projects/ykl-omikuji-parabel-fi/omikuji-model/tree0.cbor... 2023-11-07T11:02:03+02:00 2023-11-07T09:02:03.301Z INFO [omikuji::model] Loaded model settings Settings { n_features: 183181, classifier_loss_type: Hinge }... 2023-11-07T11:02:03+02:00 2023-11-07T09:02:03.292Z INFO [omikuji::model] Loading model settings from data/projects/ykl-omikuji-parabel-fi/omikuji-model/settings.json... 2023-11-07T11:02:03+02:00 2023-11-07T09:02:03.292Z INFO [omikuji::model] Loading model from data/projects/ykl-omikuji-parabel-fi/omikuji-model... 2023-11-07T11:02:03+02:00 /usr/local/lib/python3.10/site-packages/sklearn/base.py:348: InconsistentVersionWarning: Trying to unpickle estimator TfidfVectorizer from version 1.3.1 when using version 1.3.2. This might lead to breaking code or invalid results. Use at your own risk. For more info please refer to: 2023-11-07T11:02:03+02:00 https://scikit-learn.org/stable/model_persistence.html#security-maintainability-limitations 2023-11-07T11:02:03+02:00 warnings.warn( 2023-11-07T11:02:03+02:00 /usr/local/lib/python3.10/site-packages/sklearn/base.py:348: InconsistentVersionWarning: Trying to unpickle estimator TfidfTransformer from version 1.3.1 when using version 1.3.2. This might lead to breaking code or invalid results. Use at your own risk. For more info please refer to: 2023-11-07T11:02:03+02:00 https://scikit-learn.org/stable/model_persistence.html#security-maintainability-limitations 2023-11-07T11:02:03+02:00 warnings.warn( 2023-11-07T11:01:59+02:00 2023-11-07T09:01:59.168Z INFO [omikuji::model] Loaded model with 3 trees; it took 27.66s 2023-11-07T11:01:58+02:00 2023-11-07T09:01:49.492Z INFO [omikuji::model] Loading tree from data/projects/yso-bonsai-en/omikuji-model/tree2.cbor... 2023-11-07T11:01:39+02:00 2023-11-07T09:01:39.427Z INFO [omikuji::model] Loading tree from data/projects/yso-bonsai-en/omikuji-model/tree1.cbor... 2023-11-07T11:01:35+02:00 2023-11-07T09:01:31.512Z INFO [omikuji::model] Loading tree from data/projects/yso-bonsai-en/omikuji-model/tree0.cbor... 2023-11-07T11:01:35+02:00 2023-11-07T09:01:31.512Z INFO [omikuji::model] Loaded model settings Settings { n_features: 554139, classifier_loss_type: Hinge }... 2023-11-07T11:01:35+02:00 2023-11-07T09:01:31.512Z INFO [omikuji::model] Loading model settings from data/projects/yso-bonsai-en/omikuji-model/settings.json... 2023-11-07T11:01:35+02:00 2023-11-07T09:01:31.512Z INFO [omikuji::model] Loading model from data/projects/yso-bonsai-en/omikuji-model... 2023-11-07T11:00:46+02:00 2023-11-07T09:00:45.669Z INFO [omikuji::model] Loaded model with 3 trees; it took 15.82s 2023-11-07T11:00:40+02:00 2023-11-07T09:00:39.667Z INFO [omikuji::model] Loading tree from data/projects/yso-bonsai-sv/omikuji-model/tree2.cbor... 2023-11-07T11:00:36+02:00 2023-11-07T09:00:34.710Z INFO [omikuji::model] Loading tree from data/projects/yso-bonsai-sv/omikuji-model/tree1.cbor... 2023-11-07T11:00:30+02:00 2023-11-07T09:00:29.848Z INFO [omikuji::model] Loading tree from data/projects/yso-bonsai-sv/omikuji-model/tree0.cbor... 2023-11-07T11:00:30+02:00 2023-11-07T09:00:29.848Z INFO [omikuji::model] Loading model settings from data/projects/yso-bonsai-sv/omikuji-model/settings.json... 2023-11-07T11:00:30+02:00 2023-11-07T09:00:29.848Z INFO [omikuji::model] Loading model from data/projects/yso-bonsai-sv/omikuji-model... 2023-11-07T11:00:30+02:00 2023-11-07T09:00:29.848Z INFO [omikuji::model] Loaded model settings Settings { n_features: 574561, classifier_loss_type: Hinge }... 2023-11-07T10:59:30+02:00 2023-11-07T08:59:23.725Z INFO [omikuji::model] Loaded model with 3 trees; it took 56.34s 2023-11-07T10:59:11+02:00 2023-11-07T08:59:04.603Z INFO [omikuji::model] Loading tree from data/projects/yso-bonsai-fi/omikuji-model/tree2.cbor... 2023-11-07T10:58:53+02:00 2023-11-07T08:58:46.840Z INFO [omikuji::model] Loading tree from data/projects/yso-bonsai-fi/omikuji-model/tree1.cbor... 2023-11-07T10:58:33+02:00 2023-11-07T08:58:27.404Z INFO [omikuji::model] Loading tree from data/projects/yso-bonsai-fi/omikuji-model/tree0.cbor... 2023-11-07T10:58:33+02:00 2023-11-07T08:58:27.404Z INFO [omikuji::model] Loaded model settings Settings { n_features: 1173853, classifier_loss_type: Hinge }... 2023-11-07T10:58:33+02:00 2023-11-07T08:58:27.389Z INFO [omikuji::model] Loading model settings from data/projects/yso-bonsai-fi/omikuji-model/settings.json... 2023-11-07T10:58:33+02:00 2023-11-07T08:58:27.389Z INFO [omikuji::model] Loading model from data/projects/yso-bonsai-fi/omikuji-model... 2023-11-07T10:58:33+02:00 /usr/local/lib/python3.10/site-packages/sklearn/base.py:348: InconsistentVersionWarning: Trying to unpickle estimator TfidfVectorizer from version 1.3.0 when using version 1.3.2. This might lead to breaking code or invalid results. Use at your own risk. For more info please refer to: 2023-11-07T10:58:33+02:00 https://scikit-learn.org/stable/model_persistence.html#security-maintainability-limitations 2023-11-07T10:58:33+02:00 warnings.warn( 2023-11-07T10:58:14+02:00 /usr/local/lib/python3.10/site-packages/sklearn/base.py:348: InconsistentVersionWarning: Trying to unpickle estimator TfidfTransformer from version 1.3.0 when using version 1.3.2. This might lead to breaking code or invalid results. Use at your own risk. For more info please refer to: 2023-11-07T10:58:14+02:00 https://scikit-learn.org/stable/model_persistence.html#security-maintainability-limitations 2023-11-07T10:58:14+02:00 warnings.warn( 2023-11-07T10:57:13+02:00 /usr/local/lib/python3.10/site-packages/sklearn/base.py:348: InconsistentVersionWarning: Trying to unpickle estimator BaggingClassifier from version 1.3.0 when using version 1.3.2. This might lead to breaking code or invalid results. Use at your own risk. For more info please refer to: 2023-11-07T10:57:13+02:00 https://scikit-learn.org/stable/model_persistence.html#security-maintainability-limitations 2023-11-07T10:57:13+02:00 warnings.warn( 2023-11-07T10:57:13+02:00 /usr/local/lib/python3.10/site-packages/sklearn/base.py:348: InconsistentVersionWarning: Trying to unpickle estimator DecisionTreeClassifier from version 1.3.0 when using version 1.3.2. This might lead to breaking code or invalid results. Use at your own risk. For more info please refer to: 2023-11-07T10:57:13+02:00 https://scikit-learn.org/stable/model_persistence.html#security-maintainability-limitations 2023-11-07T10:57:13+02:00 warnings.warn( 2023-11-07T10:57:11+02:00 /usr/local/lib/python3.10/site-packages/sklearn/base.py:348: InconsistentVersionWarning: Trying to unpickle estimator CountVectorizer from version 1.3.0 when using version 1.3.2. This might lead to breaking code or invalid results. Use at your own risk. For more info please refer to: 2023-11-07T10:57:11+02:00 https://scikit-learn.org/stable/model_persistence.html#security-maintainability-limitations 2023-11-07T10:57:11+02:00 warnings.warn( 2023-11-07T10:57:05+02:00 To enable the following instructions: AVX2 FMA, in other operations, rebuild TensorFlow with the appropriate compiler flags. 2023-11-07T10:57:05+02:00 2023-11-07 10:57:04.100300: I tensorflow/core/platform/cpu_feature_guard.cc:182] This TensorFlow binary is optimized to use available CPU instructions in performance-critical operations. 2023-11-07T10:57:00+02:00 [2023-11-07 10:56:59 +0200] [47] [INFO] Booting worker with pid: 47 2023-11-07T10:57:00+02:00 [2023-11-07 10:56:59 +0200] [1] [ERROR] Worker (pid:7) was sent code 134! 2023-11-07T10:57:00+02:00 corrupted size vs. prev_size