Our team has developed a documentation aggregator for popular languages and frameworks which runs python scripts in the backend and scraps data from the respective webpages and displays the output on terminal similar to the working of manpages and further saves and retrieves the ouput displayed in text files and csv files (for scrapped tables)
- Scrapes documentations React Docs,Flutter Docs,Expressjs docs, Python docs, Django docs and learning platform like Geeks for Geeks
- Displays results on CLI as text similar to the working of manpages
- Falls back to Google Generative AI-generated responses if no result is found and also allows user to even generate responses from Google Generative AI directly
- Allows storage and retrieval of results for further use in the form of text and csv files
- CLI-based, accessible from the terminal in CodeEditor, easing the search process while coding, by just running the main.py script
- Ensure you have python3 installed in your system as some of the dependencies may depend upon it
- Run pip install -r requirements.txt (ensure your terminal indicates that you are in the root folder before running this command)
- in case of google gen ai requirement, you may need to run pip install -U google-generativeai (in case of module not found error)
- Requires env file for api key for Google GenAI
After setting up the files, open the terminal and run main.py cli to run interactive cli and see the help page
Once the script starts running, user can type search , for examples:
User can use : doc show to show previous saved results and select the file based on index
User can use : doc remove to show previous saved results and select the file based on index to delete it
User can list down the content of webman directory to show the previous saved outputs
- Further testing in order to refine the scrappers
- Building a package out of it to make it scalable
- Add more scrappers
- Prompting user to add his/her API_KEY for using the Google GenAI