Skip to content

Enhanced TvDatafeed library for asynchronous operations and flexible data output #61

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Open
KoushikEng wants to merge 5 commits into
base: main
Choose a base branch
from
Open

Enhanced TvDatafeed library for asynchronous operations and flexible data output #61

KoushikEng wants to merge 5 commits into from

Conversation

@KoushikEng
Copy link

@KoushikEng KoushikEng commented Nov 25, 2024

Implemented significant updates to the TvDatafeed library, focusing on improving its performance and usability:

  • Migrated to websockets from websocket for efficient asynchronous operations.
  • Introduced asynchronous methods for fetching multiple symbol's historical data simultaneously, improving speed and scalability.
  • Enabled flexible data output, allowing users to choose between raw lists and pandas DataFrames.
  • Refactored core functionalities for better maintainability and extensibility.
  • Modified usage guide in Readme
  • Modified requirments.txt

These changes maintain backward compatibility while offering modernized functionality for TradingView data retrieval.
Looking forward to your feedback!

@NumairSayed
Copy link

NumairSayed commented Dec 11, 2024

Thank you for the contribution @koushikghosh11, Did this patch have any changes in execution time? Also, can you suggest how can we make it real time? through persistent connections maybe. I am new to this and looking forward to your suggestion.

@KoushikEng
Copy link
Author

KoushikEng commented Apr 10, 2025

Hey @NumairSayed ,
Apologies for the delayed response—just catching up on old threads, and I appreciate your patience.

Regarding the execution time: I haven’t benchmarked it precisely, but I do believe there’s a noticeable improvement due to the use of asynchronous fetching for multiple symbols. It helps reduce the overall wait time compared to sequential requests, especially when working with a larger set of tickers.

As for real-time data, the TvDatafeedLive class already supports that functionality. I’d suggest taking a look at the usage guide for it—it’s designed exactly for that purpose, possibly using persistent connections depending on your setup.

If you run into anything unclear or need help around the updates I made to the TvDatafeed class, feel free to tag me. Happy to help out anytime.

semoi pushed a commit to semoi/tvdatafeed that referenced this pull request Nov 20, 2025
@claude
docs: add comprehensive CLAUDE.md project documentation
0ead26c 
Create detailed project documentation for AI-assisted development:
- Project overview and fork objectives
- Complete architecture breakdown (TvDatafeed, TvDatafeedLive, Seis, Consumer)
- Critical issues analysis (rongardF#62 2FA auth, rongardF#63 get_hist broken, etc.)
- Pull requests integration roadmap (rongardF#73, rongardF#69, rongardF#61, rongardF#37, rongardF#30)
- Development workflow and technical guidelines
- WebSocket protocol documentation
- Debugging tips and best practices
- Suggested development phases and roadmap

This documentation will serve as reference for both Claude CLI and Claude Web
to maintain consistency when working on this tvdatafeed fork project.
semoi pushed a commit to semoi/tvdatafeed that referenced this pull request Nov 21, 2025
@claude
docs: Add comprehensive pull requests analysis and integration plan
c76fd88 
Analyzed all 5 open pull requests from original repository with detailed
technical evaluation, integration plans, and recommendations.

**Documentation Created (85 KB, 5 files):**

1. **PR_ANALYSIS_REPORT.md** (45 KB, ~1200 lines)
   - Complete technical analysis of all 5 PRs
   - Code samples and implementation details
   - Pros/cons for each PR
   - Conflict analysis with current codebase
   - Detailed action plans

2. **PR_EXECUTIVE_SUMMARY.md** (7.5 KB)
   - 1-page summary for decision makers
   - Priority recommendations
   - Resource requirements (28 agent-days over 4-6 weeks)
   - Immediate decisions required
   - Reading time: 5 minutes

3. **PR_INTEGRATION_PLAN.md** (16 KB)
   - Detailed sprint-by-sprint plan (Weeks 1-6)
   - Complete checklists for each PR
   - Timeline with milestones
   - Agent assignments by task
   - Reading time: 15-20 minutes

4. **PR_QUICK_REFERENCE.md** (7.2 KB)
   - Cheat sheet and quick reference
   - Visual decision matrix
   - Copy-paste commands
   - Immediate next steps

5. **README.md** (6 KB)
   - Index and navigation guide
   - Reading workflow by role
   - Documentation structure

**Pull Requests Analysis:**

✅ **PR rongardF#37** - Verbose Logging (INTEGRATE - P1)
- Effort: 15 minutes
- Impact: 100% users (cleaner logs)
- Risk: None
- Ready to implement immediately

✅ **PR rongardF#69** - Date Range Search (INTEGRATE - P2)
- Effort: 1 week
- Impact: 80% users (backtesting use case)
- Risk: Low
- Critical feature for quants/traders

⏸️ **PR rongardF#30** - 2FA + Pro Data (INVESTIGATE - P3)
- 2FA: Priority P1 in roadmap - needs investigation
- Pro Data: >10K bars for paid accounts
- Action: Clone fork and analyze 26 commits
- Decision: GO/NO-GO after investigation

⏸️ **PR rongardF#73** - Rate Limiting + Features (INVESTIGATE - P4)
- Rate limiting: Priority P2 in roadmap
- Fundamental data: Additional financial data
- Bulk/stream features: Evaluate vs TvDatafeedLive
- Action: Access code and analyze 9 commits

❌ **PR rongardF#61** - Async Operations (REJECT)
- Reason: Incompatible with threading architecture
- Would require complete rewrite of datafeed.py
- Alternative: Implement get_hist_multi() with ThreadPoolExecutor

**Recommendations:**

Week 1: Integrate PR rongardF#37 (15 min)
Week 2: Integrate PR rongardF#69 (4-5 days)
Week 3: Investigate PR rongardF#30 and rongardF#73 (2 days)
Week 4: GO/NO-GO decisions
Weeks 5-6: Integrate validated features

**Resources Required:**
- 28 agent-days over 4-6 weeks
- Optional: TradingView Pro account for testing (~$50/month)

**Benefits:**
- Verbose logging: Clean production logs (100% users)
- Date range: Precise backtesting (80% users)
- 2FA support: Secure access (30% users)
- Rate limiting: API stability (100% users)

All documentation in /home/user/tvdatafeed/docs/
Start with PR_EXECUTIVE_SUMMARY.md (5 min read)
semoi pushed a commit to semoi/tvdatafeed that referenced this pull request Nov 22, 2025
@claude
docs: Update PR analysis documents after PR30 integration
0ae50f3 
- PR_ANALYSIS_REPORT.md: PR rongardF#30 status changed to INTÉGRÉ (2025-11-22)
- PR_EXECUTIVE_SUMMARY.md: Phase 1 marked as COMPLETE
- PR_INTEGRATION_PLAN.md: Sprint 3A and 4 marked as complete
- PR_QUICK_REFERENCE.md: Decision matrix updated with integration dates
- docs/README.md: Phase 1 completion status updated

Summary of Phase 1 completion:
- PR rongardF#37 (Verbose Logging): ✅ INTÉGRÉ
- PR rongardF#69 (Date Range Search): ✅ INTÉGRÉ
- PR rongardF#30 (2FA/TOTP): ✅ INTÉGRÉ - Security review 8.5/10 APPROVED
- PR rongardF#61 (Async): ❌ REJECTED - Incompatible with threading
- PR rongardF#73 (Rate Limiting): ⏸️ Deferred to Phase 2

Reviewed by Agent Architecte Lead
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

1 more reviewer

@joesixpack joesixpack joesixpack approved these changes

Reviewers whose approvals may not affect merge requirements

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@KoushikEng @NumairSayed @joesixpack