Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Weibo export does not include data file #982

Open
kerchner opened this issue May 6, 2019 · 2 comments
Open

Weibo export does not include data file #982

kerchner opened this issue May 6, 2019 · 2 comments
Assignees
@lwrubel
Labels
bug
Milestone
Backlog

Comments

@kerchner
Copy link
Member

kerchner commented May 6, 2019

Running an export from a weibo collection seems to only result in the README file. As an example, try exporting either json or xlsx from collection 349 on production.
@kerchner kerchner added the bug label May 6, 2019
@kerchner kerchner added this to the Backlog milestone May 6, 2019
@lwrubel
Copy link
Collaborator

lwrubel commented May 8, 2019 •
edited
Loading

No data file is provided because sfmutils.warc_iter reports in the sfm_weiboexporter_1 container log: "Processed 0 records. Yielded 0 items" and "Bad json in record" for each WARC in the collection.

The JSON doesn't appear to have errors (used a JSON linter and pretty-printed) upon inspection of what is in the log.

@lwrubel
Copy link
Collaborator

lwrubel commented May 21, 2019

JSON is also fine when reviewed in the source warcs. The issue seems to be with warc_iter.py or the warcio library, but needs more research.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@lwrubel lwrubel

Labels
bug
Projects
None yet
Milestone
Backlog
Development

No branches or pull requests
2 participants
@lwrubel @kerchner