Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Review/Increase the parameterization of the notebooks #877

Open
LiliamLeme opened this issue Nov 22, 2024 · 3 comments
Open

Review/Increase the parameterization of the notebooks #877

LiliamLeme opened this issue Nov 22, 2024 · 3 comments

Comments

@LiliamLeme
Copy link
Contributor

LiliamLeme commented Nov 22, 2024

Review notebooks and increase the parameterization. Follow overview of the NB made by Sreedhar ( #862 ).

This task most likely will be break in more tasks:

  1. investigate if there a point where it could use better the parametrization of the NBs
    Example: DDL table definition could be done in separate file.

2)Add the log library to the notebooks.
2.1 Investigate and define how the logs will be stored for this repo and which libraries we should use for this purpose. LIke will be using python telemetry using Open Telemetry or something different/
2.2 Implement the changes in the NBs.

  1. Evaluate if any performance improvements/techniques can be applied.

  2. If necessary, alter the documentation accordingly.

@LiliamLeme LiliamLeme self-assigned this Nov 22, 2024
@sreedhar-guda
Copy link
Contributor

sreedhar-guda commented Nov 22, 2024

@LiliamLeme
For your review/refrence, here is current implementation #862 . These use multiple ways of parameterization: %%configure, notebook parameters tag and an external config file application.cfg which uses Python's configparser module to read it.

@LiliamLeme
Copy link
Contributor Author

LiliamLeme commented Nov 27, 2024

thanks @sreedhar-guda I was thinking in your notebook implementation when I created. So I appreciate

@LiliamLeme
Copy link
Contributor Author

FYI @ydaponte I changed the description.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

3 participants