Expose shared document library state

[suguanYang]

Summary

• Adds document_metadata to canonical documents and job creation metadata, then copies it through publication and exposes it in document responses.
• Keeps GET /api/v1/documents document-only; parsing state remains on job creation/read responses rather than being projected into document listing.
• Returns the planned document_id from job creation and job read responses before publication, so clients can correlate active parsing state immediately.
• Removes the namespace migration/backfill helper in favor of client-side compatibility between default and legacy Notebook namespaces.

Part of #176.

Testing

• uv run --all-packages pytest apps/api/tests/contract/test_documents_contract.py apps/api/tests/contract/test_job_creation_contract.py apps/api/tests/contract/test_job_read_contract.py -q
• uv run pytest apps/api/tests apps/worker/tests/contract -q
• make check
• git diff --check