Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add FileDownloader class with file:// protocol #430

Open
wants to merge 1 commit into
base: main
Choose a base branch
from
Open
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
62 changes: 62 additions & 0 deletions pooch/downloaders.py
Original file line number Diff line number Diff line change
Expand Up @@ -75,6 +75,7 @@
"http": HTTPDownloader,
"sftp": SFTPDownloader,
"doi": DOIDownloader,
"file": FileDownloader,
}

parsed_url = parse_url(url)
Expand Down Expand Up @@ -502,6 +503,67 @@
if sftp is not None:
sftp.close()

class FileDownloader: # pylint: disable=too-few-public-methods
"""
Download manager for fetching files over a file system mounted in the operating system.

When called, downloads the given file path into the specified local file.
Uses :mod:`shutil` to copy files from paths.

Note: Does not support a progressbar.
"""

def __init__(self, progressbar=False, chunk_size=1024, **kwargs):
self.kwargs = kwargs
self.progressbar = progressbar
self.chunk_size = chunk_size

Check warning on line 519 in pooch/downloaders.py

View check run for this annotation

Codecov / codecov/patch

pooch/downloaders.py#L517-L519

Added lines #L517 - L519 were not covered by tests

def __call__(
self, url, output_file, pooch, check_only=False
): # pylint: disable=R0914
"""
Download the given file in the filesystem to the given output file.

Uses :func:`shutil.copyfile` or :func:`shutil.copyfileobj`.

Parameters
----------
url : str
The url path to the file you want to download.
output_file : str or file-like object
Path (and file name) to which the file will be downloaded.
pooch : :class:`~pooch.Pooch`
The instance of :class:`~pooch.Pooch` that is calling this method.
check_only : bool
If True, will only check if a file exists in the directory and
**without downloading the file**. Will return ``True`` if the file
exists and ``False`` otherwise.

Returns
-------
availability : bool or None
If ``check_only==True``, returns a boolean indicating if the file
is available on the server. Otherwise, returns ``None``.

"""
import pathlib # pylint: disable=C0415

Check warning on line 549 in pooch/downloaders.py

View check run for this annotation

Codecov / codecov/patch

pooch/downloaders.py#L549

Added line #L549 was not covered by tests

parsed_url = parse_url(url)
source_path = pathlib.Path(parsed_url['netloc'] + parsed_url['path'])

Check warning on line 552 in pooch/downloaders.py

View check run for this annotation

Codecov / codecov/patch

pooch/downloaders.py#L551-L552

Added lines #L551 - L552 were not covered by tests

if check_only:
return source_path.exists()

Check warning on line 555 in pooch/downloaders.py

View check run for this annotation

Codecov / codecov/patch

pooch/downloaders.py#L554-L555

Added lines #L554 - L555 were not covered by tests

import shutil # pylint: disable=C0415

Check warning on line 557 in pooch/downloaders.py

View check run for this annotation

Codecov / codecov/patch

pooch/downloaders.py#L557

Added line #L557 was not covered by tests

ispath = not hasattr(output_file, "write")
if ispath:
shutil.copyfile(source_path, output_file)

Check warning on line 561 in pooch/downloaders.py

View check run for this annotation

Codecov / codecov/patch

pooch/downloaders.py#L559-L561

Added lines #L559 - L561 were not covered by tests
else:
with source_path.open('rb') as fsrc:
shutil.copyfileobj(fsrc, output_file)

Check warning on line 564 in pooch/downloaders.py

View check run for this annotation

Codecov / codecov/patch

pooch/downloaders.py#L563-L564

Added lines #L563 - L564 were not covered by tests

return None

Check warning on line 566 in pooch/downloaders.py

View check run for this annotation

Codecov / codecov/patch

pooch/downloaders.py#L566

Added line #L566 was not covered by tests

class DOIDownloader: # pylint: disable=too-few-public-methods
"""
Expand Down
Loading