Skip to content

Commit 92310d7

Browse files
committed
Merge pull request #2 from cedadev/py3-devel
Py3 devel
2 parents f1203e7 + dd83542 commit 92310d7

20 files changed

+397
-142
lines changed

.pydevproject

Lines changed: 3 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -1,9 +1,7 @@
11
<?xml version="1.0" encoding="UTF-8" standalone="no"?>
2-
<?eclipse-pydev version="1.0"?>
3-
4-
<pydev_project>
5-
<pydev_property name="org.python.pydev.PYTHON_PROJECT_INTERPRETER">ndg-httpsclient-py2.7</pydev_property>
6-
<pydev_property name="org.python.pydev.PYTHON_PROJECT_VERSION">python 2.7</pydev_property>
2+
<?eclipse-pydev version="1.0"?><pydev_project>
3+
<pydev_property name="org.python.pydev.PYTHON_PROJECT_INTERPRETER">ndg-https-client-py3.4</pydev_property>
4+
<pydev_property name="org.python.pydev.PYTHON_PROJECT_VERSION">python 3.0</pydev_property>
75
<pydev_pathproperty name="org.python.pydev.PROJECT_SOURCE_PATH">
86
<path>/ndg_httpsclient</path>
97
</pydev_pathproperty>

INSTALL

Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,8 @@
1+
Installation notes
2+
==================
3+
4+
For install on Mac Yosemite, special header file include paths are needed for
5+
PyOpenSSL dependency source builds. Install XCode and add explicit include path
6+
for ffi package and generic include:
7+
8+
export CPATH=/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.10.sdk/usr/include/ffi:/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.10.sdk/usr/include/

README.md

Lines changed: 89 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -2,3 +2,92 @@ A HTTPS client implementation for httplib and urllib2 based on
22
PyOpenSSL. PyOpenSSL provides a more fully featured SSL implementation over the
33
default provided with Python and importantly enables full verification of the
44
SSL peer.
5+
6+
Releases
7+
========
8+
0.4.0
9+
-----
10+
* Made dual compatible with Python 2 / 3.
11+
12+
0.3.3
13+
-----
14+
* Fix to add in AnotherName for ``subjectAltNames`` field - added for support for CACert issued
15+
certs (thanks to Gu1).
16+
* Fix to HTTP Basic Auth option for ``ndg.httpsclient.utils.main``
17+
* Fix to ``ServerSSLCertVerification`` so that it can pass a function-based callback instead of using ``__call__``. In newer versions of OpenSSL (>= 0.14) the latter failed because of a request for ``__name__`` attribute.
18+
19+
0.3.2
20+
-----
21+
* Fix to SubjectAltNames support check - should only be enabled if pyasn1 is
22+
installed.
23+
* Fix to open_url: HTTP Request object was being created inside if headers is
24+
None block - now corrected to create regardless.
25+
* Added http basic auth support to script. (Thanks to Willem van Engen)
26+
27+
0.3.1
28+
-----
29+
* extended utils functions to support keyword for passing additional urllib2
30+
handlers.
31+
32+
0.3.0
33+
-----
34+
* Added ndg.httpsclient.utils.fetch_stream_from_url function and added
35+
parameter for data to post in open_url and fetch_* methods.
36+
* fix to ndg.httpsclient.utils module _should_use_proxy and open_url functions
37+
38+
0.2.0
39+
-----
40+
* added support for SSL verification with subjectAltNames using pyasn1
41+
* fixed minor bug - SSL cert DN prefix matching
42+
43+
0.1.0
44+
-----
45+
Initial release
46+
47+
Prerequisites
48+
=============
49+
This has been developed and tested for Python 2.6 and 2.7 with pyOpenSSL 0.13 and 0.14.
50+
Version 0.4.0 tested with pyOpenSSL 0.15.1 and Python 2.7 and 3.4. Note that proxy support
51+
is only available from Python 2.6.2 onwards. pyasn1 is required for correct SSL
52+
verification with subjectAltNames.
53+
54+
Installation
55+
============
56+
Installation can be performed using easy_install or pip.
57+
58+
Running ndg_httpclient
59+
======================
60+
A simple script for fetching data using HTTP or HTTPS GET from a specified URL.
61+
62+
Parameter:
63+
64+
``url``
65+
The URL of the resource to be fetched
66+
67+
Options:
68+
69+
``-h, --help``
70+
Show help message and exit.
71+
72+
``-c FILE, --certificate=FILE``
73+
Certificate file - defaults to ``$HOME/credentials.pem``
74+
75+
``-k FILE, --private-key=FILE``
76+
Private key file - defaults to the certificate file
77+
78+
``-t DIR, --ca-certificate-dir=DIR``
79+
Trusted CA certificate file directory.
80+
81+
``-d, --debug``
82+
Print debug information - this may be useful in solving problems with HTTP or
83+
HTTPS access to a server.
84+
85+
``-p FILE, --post-data-file=FILE``
86+
POST data file
87+
88+
``-f FILE, --fetch=FILE``
89+
Output file
90+
91+
``-n, --no-verify-peer``
92+
Skip verification of peer certificate.
93+

ndg/httpsclient/https.py

Lines changed: 15 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -12,9 +12,18 @@
1212
__revision__ = '$Id$'
1313
import logging
1414
import socket
15-
from httplib import HTTPS_PORT
16-
from httplib import HTTPConnection
17-
from urllib2 import AbstractHTTPHandler
15+
import sys
16+
17+
if sys.version_info[0] > 2:
18+
from http.client import HTTPS_PORT
19+
from http.client import HTTPConnection
20+
21+
from urllib.request import AbstractHTTPHandler
22+
else:
23+
from httplib import HTTPS_PORT
24+
from httplib import HTTPConnection
25+
26+
from urllib2 import AbstractHTTPHandler
1827

1928

2029
from OpenSSL import SSL
@@ -81,7 +90,8 @@ def connect(self):
8190

8291
def close(self):
8392
"""Close socket and shut down SSL connection"""
84-
self.sock.close()
93+
if hasattr(self.sock, "close"):
94+
self.sock.close()
8595

8696

8797
class HTTPSContextHandler(AbstractHTTPHandler):
@@ -106,7 +116,7 @@ def __init__(self, ssl_context, debuglevel=0):
106116
ssl_context)
107117
self.ssl_context = ssl_context
108118
else:
109-
self.ssl_context = SSL.Context(SSL.SSLv23_METHOD)
119+
self.ssl_context = SSL.Context(SSL.TLSv1_METHOD)
110120

111121
def https_open(self, req):
112122
"""Opens HTTPS request

ndg/httpsclient/ssl_context_util.py

Lines changed: 7 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -8,7 +8,12 @@
88
__license__ = "BSD - see LICENSE file in top-level directory"
99
__contact__ = "[email protected]"
1010
__revision__ = '$Id$'
11-
import urlparse
11+
import sys
12+
13+
if sys.version_info[0] > 2:
14+
import urllib.parse as urlparse_
15+
else:
16+
import urlparse as urlparse_
1217

1318
from OpenSSL import SSL
1419

@@ -85,7 +90,7 @@ def set_peer_verification_for_url_hostname(ssl_context, url,
8590
'''Convenience routine to set peer verification callback based on
8691
ServerSSLCertVerification class'''
8792
if not if_verify_enabled or (ssl_context.get_verify_mode() & SSL.VERIFY_PEER):
88-
urlObj = urlparse.urlparse(url)
93+
urlObj = urlparse_.urlparse(url)
8994
hostname = urlObj.hostname
9095
server_ssl_cert_verif = ServerSSLCertVerification(hostname=hostname)
9196
verify_callback_ = server_ssl_cert_verif.get_verify_server_cert_func()

ndg/httpsclient/ssl_peer_verification.py

Lines changed: 12 additions & 11 deletions
Original file line numberDiff line numberDiff line change
@@ -14,7 +14,8 @@
1414
from ndg.httpsclient.subj_alt_name import SubjectAltName
1515
from pyasn1.codec.der import decoder as der_decoder
1616
SUBJ_ALT_NAME_SUPPORT = True
17-
except ImportError, e:
17+
18+
except ImportError as e:
1819
SUBJ_ALT_NAME_SUPPORT = False
1920
SUBJ_ALT_NAME_SUPPORT_MSG = (
2021
'SubjectAltName support is disabled - check pyasn1 package '
@@ -40,7 +41,7 @@ class ServerSSLCertVerification(object):
4041
'userid': 'UID'
4142
}
4243
SUBJ_ALT_NAME_EXT_NAME = 'subjectAltName'
43-
PARSER_RE_STR = '/(%s)=' % '|'.join(DN_LUT.keys() + DN_LUT.values())
44+
PARSER_RE_STR = '/(%s)=' % '|'.join(list(DN_LUT.keys()) + list(DN_LUT.values()))
4445
PARSER_RE = re.compile(PARSER_RE_STR)
4546

4647
__slots__ = ('__hostname', '__certDN', '__subj_alt_name_match')
@@ -156,12 +157,12 @@ def __call__(self, connection, peerCert, errorStatus, errorDepth,
156157
return preverifyOK
157158

158159
def get_verify_server_cert_func(self):
159-
def verify_server_cert(connection, peerCert, errorStatus, errorDepth,
160-
preverifyOK):
161-
return self.__call__(connection, peerCert, errorStatus,
162-
errorDepth, preverifyOK)
163-
164-
return verify_server_cert
160+
def verify_server_cert(connection, peerCert, errorStatus, errorDepth,
161+
preverifyOK):
162+
return self.__call__(connection, peerCert, errorStatus,
163+
errorDepth, preverifyOK)
164+
165+
return verify_server_cert
165166

166167
@classmethod
167168
def _get_subj_alt_name(cls, peer_cert):
@@ -195,15 +196,15 @@ def _getCertDN(self):
195196
return self.__certDN
196197

197198
def _setCertDN(self, val):
198-
if isinstance(val, basestring):
199+
if isinstance(val, str):
199200
# Allow for quoted DN
200201
certDN = val.strip('"')
201202

202203
dnFields = self.__class__.PARSER_RE.split(certDN)
203204
if len(dnFields) < 2:
204205
raise TypeError('Error parsing DN string: "%s"' % certDN)
205206

206-
self.__certDN = zip(dnFields[1::2], dnFields[2::2])
207+
self.__certDN = list(zip(dnFields[1::2], dnFields[2::2]))
207208
self.__certDN.sort()
208209

209210
elif not isinstance(val, list):
@@ -226,7 +227,7 @@ def _getHostname(self):
226227
return self.__hostname
227228

228229
def _setHostname(self, val):
229-
if not isinstance(val, basestring):
230+
if not isinstance(val, str):
230231
raise TypeError("Expecting string type for hostname "
231232
"attribute")
232233
self.__hostname = val

ndg/httpsclient/ssl_socket.py

Lines changed: 9 additions & 22 deletions
Original file line numberDiff line numberDiff line change
@@ -12,7 +12,7 @@
1212
from datetime import datetime
1313
import logging
1414
import socket
15-
from cStringIO import StringIO
15+
from io import BytesIO
1616

1717
from OpenSSL import SSL
1818

@@ -61,22 +61,20 @@ def buf_size(self):
6161
@buf_size.setter
6262
def buf_size(self, value):
6363
"""Buffer size for makefile method recv() operations"""
64-
if not isinstance(value, (int, long)):
65-
raise TypeError('Expecting int or long type for "buf_size"; '
64+
if not isinstance(value, int):
65+
raise TypeError('Expecting int type for "buf_size"; '
6666
'got %r instead' % type(value))
6767
self.__buf_size = value
6868

6969
def close(self):
7070
"""Shutdown the SSL connection and call the close method of the
7171
underlying socket"""
72-
# try:
73-
# self.__ssl_conn.shutdown()
74-
# except SSL.Error:
75-
# # Make errors on shutdown non-fatal
76-
# pass
77-
7872
if self._makefile_refs < 1:
79-
self.__ssl_conn.shutdown()
73+
try:
74+
self.__ssl_conn.shutdown()
75+
except (SSL.Error, SSL.SysCallError):
76+
# Make errors on shutdown non-fatal
77+
pass
8078
else:
8179
self._makefile_refs -= 1
8280

@@ -236,7 +234,7 @@ def makefile(self, *args):
236234
_buf_size = self.buf_size
237235

238236
i=0
239-
stream = StringIO()
237+
stream = BytesIO()
240238
startTime = datetime.utcnow()
241239
try:
242240
dat = self.__ssl_conn.recv(_buf_size)
@@ -261,17 +259,6 @@ def makefile(self, *args):
261259
stream.seek(0)
262260

263261
return stream
264-
265-
# def makefile(self, mode='r', bufsize=-1):
266-
#
267-
# """Make and return a file-like object that
268-
# works with the SSL connection. Just use the code
269-
# from the socket module."""
270-
#
271-
# self._makefile_refs += 1
272-
# # close=True so as to decrement the reference count when done with
273-
# # the file-like object.
274-
# return socket._fileobject(self.socket, mode, bufsize, close=True)
275262

276263
def getsockname(self):
277264
"""

ndg/httpsclient/subj_alt_name.py

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -14,7 +14,7 @@
1414
try:
1515
from pyasn1.type import univ, constraint, char, namedtype, tag
1616

17-
except ImportError, e:
17+
except ImportError as e:
1818
import_error_msg = ('Error importing pyasn1, subjectAltName check for SSL '
1919
'peer verification will be disabled. Import error '
2020
'is: %s' % e)
Lines changed: 20 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,20 @@
1+
-----BEGIN CERTIFICATE-----
2+
MIIDLjCCAhagAwIBAgIBATANBgkqhkiG9w0BAQUFADA3MREwDwYDVQQLDAhTZWN1
3+
cml0eTEUMBIGA1UEAwwLTkRHIFRlc3QgQ0ExDDAKBgNVBAoMA05ERzAeFw0xNTAx
4+
MjExNDMzMThaFw0yMDAxMjAxNDMzMThaMDcxETAPBgNVBAsMCFNlY3VyaXR5MRQw
5+
EgYDVQQDDAtOREcgVGVzdCBDQTEMMAoGA1UECgwDTkRHMIIBIjANBgkqhkiG9w0B
6+
AQEFAAOCAQ8AMIIBCgKCAQEArq4QKUTRq45nCDR/p+OlHIIN8+ugUbiCfteazbTG
7+
rX8vIQ9HxSuz/xvxTw+E0KgA4YSK2SJJP4QiCjlMKYS3Rt8o361GNtnRmeo5qyBu
8+
GMSv73XL1uuqumggUZyrhhksckR7gyNFnKVXzZjAQPepsT0xBjs5uEAEqXJzAf+r
9+
24AnT3MZRh7gsyEe3sZjd75kZVwcrWhrocyKlMCR77yEr+uP4pg+dEMhDMKKxlaF
10+
C5RPMotOpWm/7AToHrGia34WSmcxvuOwxOkI4xEW6mxWMaVTBCXUh6Wb/0m/x8Nv
11+
9VvS2UBC4sCp4MqlDpySxQpT1RgrhMTEmtUOh50l4eEhdwIDAQABo0UwQzASBgNV
12+
HRMBAf8ECDAGAQH/AgEAMA4GA1UdDwEB/wQEAwIBBjAdBgNVHQ4EFgQUkEvQjGOP
13+
Oj5DZEvsm96AdiiFXWgwDQYJKoZIhvcNAQEFBQADggEBAGD0kQASmNzvtYL+JUGf
14+
gTPyJhADl9Ai9GvZJsY/wX0IRTxRl5y08Dqlg3qyGG3GzL918cr1sVCYnLepNQES
15+
T0MIz50DCKGryNSc74JHPDxpYaSV6whmNH5iwh8fy6tmJwF3FWbGXD2ddc+ofJqP
16+
WPPJtzqxuuJ6iXQIFqD9mEn3iXVcvFuSzpdpH9paORTKB0j4gya9zctB8LP0ZXIE
17+
//wREc+4msnmoTn+qkFAOPBg9WnvoipfyCXPgbTagxlofVjZ7gAgYIefqhXBTQdd
18+
5tnYdyQQBRcUXQS2bBX03q8ftcxOjc3SvXI4MvrqofuFPwu4GnrspnC0KQYlXwEI
19+
7ds=
20+
-----END CERTIFICATE-----
Lines changed: 20 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,20 @@
1+
-----BEGIN CERTIFICATE-----
2+
MIIDLjCCAhagAwIBAgIBATANBgkqhkiG9w0BAQUFADA3MREwDwYDVQQLDAhTZWN1
3+
cml0eTEUMBIGA1UEAwwLTkRHIFRlc3QgQ0ExDDAKBgNVBAoMA05ERzAeFw0xNTAx
4+
MjExNDMzMThaFw0yMDAxMjAxNDMzMThaMDcxETAPBgNVBAsMCFNlY3VyaXR5MRQw
5+
EgYDVQQDDAtOREcgVGVzdCBDQTEMMAoGA1UECgwDTkRHMIIBIjANBgkqhkiG9w0B
6+
AQEFAAOCAQ8AMIIBCgKCAQEArq4QKUTRq45nCDR/p+OlHIIN8+ugUbiCfteazbTG
7+
rX8vIQ9HxSuz/xvxTw+E0KgA4YSK2SJJP4QiCjlMKYS3Rt8o361GNtnRmeo5qyBu
8+
GMSv73XL1uuqumggUZyrhhksckR7gyNFnKVXzZjAQPepsT0xBjs5uEAEqXJzAf+r
9+
24AnT3MZRh7gsyEe3sZjd75kZVwcrWhrocyKlMCR77yEr+uP4pg+dEMhDMKKxlaF
10+
C5RPMotOpWm/7AToHrGia34WSmcxvuOwxOkI4xEW6mxWMaVTBCXUh6Wb/0m/x8Nv
11+
9VvS2UBC4sCp4MqlDpySxQpT1RgrhMTEmtUOh50l4eEhdwIDAQABo0UwQzASBgNV
12+
HRMBAf8ECDAGAQH/AgEAMA4GA1UdDwEB/wQEAwIBBjAdBgNVHQ4EFgQUkEvQjGOP
13+
Oj5DZEvsm96AdiiFXWgwDQYJKoZIhvcNAQEFBQADggEBAGD0kQASmNzvtYL+JUGf
14+
gTPyJhADl9Ai9GvZJsY/wX0IRTxRl5y08Dqlg3qyGG3GzL918cr1sVCYnLepNQES
15+
T0MIz50DCKGryNSc74JHPDxpYaSV6whmNH5iwh8fy6tmJwF3FWbGXD2ddc+ofJqP
16+
WPPJtzqxuuJ6iXQIFqD9mEn3iXVcvFuSzpdpH9paORTKB0j4gya9zctB8LP0ZXIE
17+
//wREc+4msnmoTn+qkFAOPBg9WnvoipfyCXPgbTagxlofVjZ7gAgYIefqhXBTQdd
18+
5tnYdyQQBRcUXQS2bBX03q8ftcxOjc3SvXI4MvrqofuFPwu4GnrspnC0KQYlXwEI
19+
7ds=
20+
-----END CERTIFICATE-----

0 commit comments

Comments
 (0)