Generate 3D renderings of an existing 2D image by modifying the subject based on a text prompt
Report Bug
·
Request Feature
- Table of Contents
- About The Project
- Proposed Architecture
- Preliminary Results
- Hardware Requirements
- Setup and Usage
- License
- Contributors
- Base Repositories
- Acknowledgements
This project is a part of the Boston University Course: GRS CS640 - Artificial Intelligence and involves the merger of two base papers:
A wide range of editing effects are now available to content creators thanks to extensive research into changing the appearance and style of objects in photographs. However, majority of the research in this field focuses on global editing rather than localized editing. To address this Text2LIVE developed an algorithm with localized editing of images using only text prompt. Given the substantial work being done on 3D objects and the widespread usage of 3D models in CAD-modeling and video games, the same flexibility and range of editing effects ought to be available in 3D. Due to this, we propose Text2LIVE-3D, which gives the same degree of creative control over the appearance and style of 3D models as can be done with 2D photographs.
We recommend an Nvidia GPU for Training the models. As per our experimentation the following specifications are recommended:
- Text2LIVE: Nvidia A100 (or any GPU with VRAM greater than 18 GB)
- DreamFusion3D: Nvidia Tesla V100 (or any GPU with VRAM greater than 16 GB)
For setup and usage, please follow the instructions in the readme:
Distributed under the GNU AGPL V3 License. See LICENSE for more information.
- LinkedIn: animikh-aich
- Email: [email protected]
- GitHub: animikhaich
- Twitter: @AichAnimikh
- LinkedIn: hipatil
- Email: [email protected]
- GitHub: HiPatil
- LinkedIn: vedika-srivastava
- Email: [email protected]
- GitHub: VedikaSrivastava
Our work aims to derive and build on top of the two projects:
- Text2LIVE: https://text2live.github.io/ (Paper)
- DreamFusion3D: https://dreamfusion3d.github.io/ (Paper)
Hence, this directory contains a copy of the base repositories for the above.
- Text2LIVE has an open source official implementation: https://github.com/omerbt/Text2LIVE
- DreamFusion3D has an open source unofficial implementation with modifications including replacement of Imagen by Stable Diffusion: https://github.com/ashawkey/stable-dreamfusion
- Omer Bar-Tal (Text2LIVE)
- Kiui - Jiaxiang Tang (Stable Dreamfusion)