Skip to content

Commit f14f13c

Browse files
avara1986avara1986
authored
Added retries to requests wrapper (#56)
* Added retry system to requests wrapper * Fix pylint errors * Updated docs
1 parent 477ea42 commit f14f13c

File tree

14 files changed

+314
-116
lines changed

14 files changed

+314
-116
lines changed

‎CONTRIBUTING.md

Lines changed: 65 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,7 @@
11
# Contributing
22

3+
## Installation
4+
35
After cloning this repo, create a [virtualenv](https://virtualenv.pypa.io/en/stable/) and ensure dependencies are installed by running:
46

57
```sh
@@ -30,4 +32,66 @@ If you wish to run against a specific version defined in the `tox.ini` file:
3032
tox -e py36
3133
```
3234

33-
Tox can only use whatever versions of Python are installed on your system. When you create a pull request, Travis will also be running the same tests and report the results, so there is no need for potential contributors to try to install every single version of Python on their own system ahead of time. We appreciate opening issues and pull requests to make PyMS even more stable & useful!
35+
Tox can only use whatever versions of Python are installed on your system. When you create a pull request, Travis will also be running the same tests and report the results, so there is no need for potential contributors to try to install every single version of Python on their own system ahead of time.
36+
37+
## Pipenv
38+
39+
### Advantages over plain pip and requirements.txt
40+
[Pipenv](https://pipenv.readthedocs.io/en/latest/) generates two files: a `Pipfile`and a `Pipfile.lock`.
41+
* `Pipfile`: Is a high level declaration of the dependencies of your project. It can contain "dev" dependencies (usually test related stuff) and "standard" dependencies which are the ones you'll need for your project to function
42+
* `Pipfile.lock`: Is the "list" of all the dependencies your Pipfile has installed, along with their version and their hashes. This prevents two things: Conflicts between dependencies and installing a malicious module.
43+
44+
### How to...
45+
46+
Here the most 'common' `pipenv` commands, for a more in-depth explanation please refer to the [official documentation](https://pipenv.readthedocs.io/en/latest/).
47+
48+
#### Install pipenv
49+
```bash
50+
pip install pipenv
51+
```
52+
53+
#### Install dependencies defined in a Pipfile
54+
```bash
55+
pipenv install
56+
```
57+
58+
#### Install both dev and "standard" dependencies defined in a Pipfile
59+
```bash
60+
pipenv install --dev
61+
```
62+
63+
#### Install a new module
64+
```bash
65+
pipenv install django
66+
```
67+
68+
#### Install a new dev module (usually test related stuff)
69+
```bash
70+
pipenv install nose --dev
71+
```
72+
73+
#### Install dependencies in production
74+
```bash
75+
pipenv install --deploy
76+
```
77+
78+
#### Start a shell
79+
```bash
80+
pipenv shell
81+
```
82+
83+
## Documentation
84+
85+
This project use MkDocs
86+
87+
* `mkdocs new [dir-name]` - Create a new project.
88+
* `mkdocs serve` - Start the live-reloading docs server.
89+
* `mkdocs build` - Build the documentation site.
90+
* `mkdocs help` - Print this help message.
91+
92+
### Project layout
93+
94+
mkdocs.yml # The configuration file.
95+
docs/
96+
index.md # The documentation homepage.
97+
... # Other markdown pages, images and other files.

‎README.md

Lines changed: 32 additions & 69 deletions
Original file line numberDiff line numberDiff line change
@@ -10,8 +10,30 @@
1010
PyMS, Python MicroService, is a collections of libraries, best practices and recommended ways to build
1111
microservices with Python.
1212

13+
## Documentation
14+
1315
To know how use, install or build a project see the docs: https://py-ms.readthedocs.io/en/latest/
1416

17+
## Motivation
18+
19+
When we started to create microservice with no idea, we were looking for tutorials, guides, best practices, but we found
20+
nothing to create professional projects. Most articles say:
21+
- "Install flask"
22+
- "Create routes"
23+
- (Sometimes) "Create a swagger specs"
24+
- "TA-DA! you have a microservice"
25+
26+
But... what happens with our configuration out of code like Kubernetes configmap? what happens with transactionality?
27+
If we have many microservices, what happens with traces?.
28+
29+
There are many problems around Python and microservices and we can`t find anyone to give a solution.
30+
31+
We start creating these projects to try to solve all the problems we have found in our professional lives about
32+
microservices architecture.
33+
34+
Nowadays, is not perfect and we have a looong roadmap, but we hope this library could help other felas and friends ;)
35+
36+
1537
## Installation
1638
```bash
1739
pip install py-ms
@@ -23,80 +45,21 @@ pip install py-ms
2345
Module to read yaml or json configuration from a dictionary or a path.
2446

2547
### pyms/flask/app
26-
With the funcion `create_app` initialize the Flask app, register [blueprints](http://flask.pocoo.org/docs/0.12/blueprints/)
27-
and intialize all libraries like Swagger, database, trace system, custom logger format, etc.
48+
With the function `create_app` initialize the Flask app, register [blueprints](http://flask.pocoo.org/docs/0.12/blueprints/)
49+
and initialize all libraries such as Swagger, database, trace system, custom logger format, etc.
50+
51+
### pyms/flask/services
52+
Integrations and wrappers over common libs like request, swagger, connexion
2853

2954
### pyms/flask/healthcheck
30-
This views is usually used by Kubernetes, Eureka and other systems to check if our application is up and running.
55+
This view is usually used by Kubernetes, Eureka and other systems to check if our application is running.
3156

3257
### pyms/logger
3358
Print logger in JSON format to send to server like Elasticsearch. Inject span traces in logger.
3459

35-
### pyms/rest_template
36-
Encapsulate common rest operations between business services propagating trace headers if configured.
37-
3860
### pyms/tracer
39-
Create an injector `flask_opentracing.FlaskTracer` to use in our projects
40-
41-
## Pipenv
42-
43-
### Advantages over plain pip and requirements.txt
44-
[Pipenv](https://pipenv.readthedocs.io/en/latest/) generates two files: a `Pipfile`and a `Pipfile.lock`.
45-
* `Pipfile`: Is a high level declaration of the dependencies of your project. It can contain "dev" dependencies (usually test related stuff) and "standard" dependencies which are the ones you'll need for your project to function
46-
* `Pipfile.lock`: Is the "list" of all the dependencies your Pipfile has installed, along with their version and their hashes. This prevents two things: Conflicts between dependencies and installing a malicious module.
47-
48-
### How to...
49-
50-
Here the most 'common' `pipenv` commands, for a more in-depth explanation please refer to the [official documentation](https://pipenv.readthedocs.io/en/latest/).
51-
52-
#### Install pipenv
53-
```bash
54-
pip install pipenv
55-
```
56-
57-
#### Install dependencies defined in a Pipfile
58-
```bash
59-
pipenv install
60-
```
61-
62-
#### Install both dev and "standard" dependencies defined in a Pipfile
63-
```bash
64-
pipenv install --dev
65-
```
66-
67-
#### Install a new module
68-
```bash
69-
pipenv install django
70-
```
71-
72-
#### Install a new dev module (usually test related stuff)
73-
```bash
74-
pipenv install nose --dev
75-
```
76-
77-
#### Install dependencies in production
78-
```bash
79-
pipenv install --deploy
80-
```
81-
82-
#### Start a shell
83-
```bash
84-
pipenv shell
85-
```
86-
87-
## Documentation
88-
89-
This project use MkDocs
90-
91-
* `mkdocs new [dir-name]` - Create a new project.
92-
* `mkdocs serve` - Start the live-reloading docs server.
93-
* `mkdocs build` - Build the documentation site.
94-
* `mkdocs help` - Print this help message.
95-
96-
### Project layout
97-
98-
mkdocs.yml # The configuration file.
99-
docs/
100-
index.md # The documentation homepage.
101-
... # Other markdown pages, images and other files.
61+
Create an injector `flask_opentracing.FlaskTracer` to use in our projects.
10262

63+
## How To Contrib
64+
We appreciate opening issues and pull requests to make PyMS even more stable & useful! See [This doc](COONTRIBUTING.md)
65+
for more details

‎docs/index.md

Lines changed: 20 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -3,6 +3,26 @@
33
PyMS, Python MicroService, is a collection of libraries, best practices and recommended ways to build
44
microservices with Python.
55

6+
## Motivation
7+
8+
When we started to create microservice with no idea, we were looking for tutorials, guides, best practices, but we found
9+
nothing to create professional projects. Most articles say:
10+
- "Install flask"
11+
- "Create routes"
12+
- (Sometimes) "Create a swagger specs"
13+
- "TA-DA! you have a microservice"
14+
15+
But... what happens with our configuration out of code like Kubernetes configmap? what happens with transactionality?
16+
If we have many microservices, what happens with traces?.
17+
18+
There are many problems around Python and microservices and we can`t find anyone to give a solution.
19+
20+
We start creating these projects to try to solve all the problems we have found in our professional lives about
21+
microservices architecture.
22+
23+
Nowadays, is not perfect and we have a looong roadmap, but we hope this library could help other felas and friends ;)
24+
25+
626
## Index:
727
* [PyMS structure](structure.md)
828
* [Configuration](configuration.md)

‎docs/structure.md

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -7,6 +7,9 @@ Module to read yaml or json configuration from a dictionary or a path.
77
With the function `create_app` initialize the Flask app, register [blueprints](http://flask.pocoo.org/docs/0.12/blueprints/)
88
and initialize all libraries such as Swagger, database, trace system, custom logger format, etc.
99

10+
### pyms/flask/services
11+
Integrations and wrappers over common libs like request, swagger, connexion
12+
1013
### pyms/flask/healthcheck
1114
This view is usually used by Kubernetes, Eureka and other systems to check if our application is running.
1215

‎pylintrc

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -54,7 +54,7 @@ confidence=
5454
# --enable=similarities". If you want to run only the classes checker, but have
5555
# no Warning level messages displayed, use"--disable=all --enable=classes
5656
# --disable=W"
57-
disable=C0301
57+
disable=C0301,C0111,C0103,logging-format-interpolation
5858

5959
# Enable the message, report, category or checker with the given id(s). You can
6060
# either give multiple identifier separated by comma (,) or put this option

‎pyms/__init__.py

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -2,4 +2,4 @@
22

33
__email__ = "[email protected]"
44

5-
__version__ = "1.1.0"
5+
__version__ = "1.2.0"

‎pyms/flask/app/create_app.py

Lines changed: 10 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -8,7 +8,7 @@
88
from pyms.config.conf import get_conf
99
from pyms.constants import LOGGER_NAME, SERVICE_ENVIRONMENT
1010
from pyms.flask.healthcheck import healthcheck_blueprint
11-
from pyms.flask.services.driver import ServicesManager
11+
from pyms.flask.services.driver import ServicesManager, DriverService
1212
from pyms.logger import CustomJsonFormatter
1313
from pyms.tracer.main import init_lightstep_tracer
1414

@@ -35,6 +35,8 @@ def __call__(cls, *args, **kwargs):
3535
class Microservice(metaclass=SingletonMeta):
3636
service = None
3737
application = None
38+
swagger = DriverService
39+
requests = DriverService
3840

3941
def __init__(self, *args, **kwargs):
4042
self.service = kwargs.get("service", os.environ.get(SERVICE_ENVIRONMENT, "ms"))
@@ -70,14 +72,15 @@ def init_logger(self):
7072
def init_app(self) -> Flask:
7173
if getattr(self, "swagger", False):
7274
app = connexion.App(__name__, specification_dir=os.path.join(self.path, self.swagger.path))
73-
app.add_api(self.swagger.file,
74-
arguments={'title': self.config.APP_NAME},
75-
base_path=self.config.APPLICATION_ROOT
76-
)
75+
app.add_api(
76+
self.swagger.file,
77+
arguments={'title': self.config.APP_NAME},
78+
base_path=self.config.APPLICATION_ROOT
79+
)
7780

7881
# Invert the objects, instead connexion with a Flask object, a Flask object with
7982
application = app.app
80-
application._connexion_app = app
83+
application.connexion_app = app
8184
else:
8285
application = Flask(__name__, static_folder=os.path.join(self.path, 'static'),
8386
template_folder=os.path.join(self.path, 'templates'))
@@ -121,4 +124,4 @@ def add_error_handler(self, code_or_exception, handler):
121124
:param code_or_exception: HTTP error code or exception
122125
:param handler: callback for error handler
123126
"""
124-
self.application._connexion_app.add_error_handler(code_or_exception, handler)
127+
self.application.connexion_app.add_error_handler(code_or_exception, handler)

‎pyms/flask/app/create_config.py

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -3,4 +3,4 @@
33

44
def config():
55
ms = Microservice()
6-
return ms.config
6+
return ms.config

‎pyms/flask/healthcheck/__init__.py

Lines changed: 2 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -1,9 +1,3 @@
1-
"""Init file
2-
"""
3-
from __future__ import unicode_literals, print_function, absolute_import, division
1+
from pyms.flask.healthcheck.healthcheck import healthcheck_blueprint
42

5-
from flask import Blueprint
6-
7-
healthcheck_blueprint = Blueprint('healthcheck', __name__, static_url_path='/static')
8-
9-
from pyms.flask.healthcheck import healthcheck
3+
__all__ = ['healthcheck_blueprint']

‎pyms/flask/healthcheck/healthcheck.py

Lines changed: 5 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,8 @@
1-
"""Healthcheck
2-
"""
3-
from pyms.flask.healthcheck import healthcheck_blueprint
1+
from __future__ import unicode_literals, print_function, absolute_import, division
2+
3+
from flask import Blueprint
4+
5+
healthcheck_blueprint = Blueprint('healthcheck', __name__, static_url_path='/static')
46

57

68
@healthcheck_blueprint.route('/healthcheck', methods=['GET'])

0 commit comments

Comments
 (0)