Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

при сборе kommersant возникает исключение #10

Open
stroykova opened this issue Mar 21, 2021 · 1 comment
Open

Comments

@stroykova
Copy link
Contributor

при сборе kommersant возникает исключение:

Traceback (most recent call last):
  File "/usr/local/lib/python3.7/site-packages/twisted/internet/defer.py", line 654, in _runCallbacks
    current.result = callback(current.result, *args, **kw)
  File "/usr/local/lib/python3.7/site-packages/scrapy/utils/defer.py", line 150, in f
    return deferred_from_coro(coro_f(*coro_args, **coro_kwargs))
  File "/code/newsbot/pipelines.py", line 46, in process_item
    item = self._process_metrics(spider, item, self._metrics)
  File "/code/newsbot/pipelines.py", line 79, in _process_metrics
    item[m] = spider.process_metric(item.get(m, ["0"]))
  File "/code/newsbot/spiders/news.py", line 115, in process_metric
    return metrics[0]
IndexError: list index out of range

@Avenon
Copy link
Contributor

Avenon commented Mar 24, 2021

Ошибка возникает из-за некорректоного xpath для поля comm_count (судя по всем количество комментариев). Это единственный паук у которого он задан, в других пауках просто стоит прочерк '-'.
Я сейчас проверил, просто убрав путь и сделав прочерк как у других, ошибки нет.
@iggisv9t Если такой вариант решения проблемы подойдет, то закомичу.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants