Downloader

Build Status Coverage Status Maintainability

Automates file downloads. Supports batch downloading via newline-delimited link dump in a text file. Uses http.rb and Addressable under the hood and Thor for the CLI.

Installation

Add this line to your application's Gemfile:

gem "downloader", git: "https://github.com/strawberryjello/downloader.git"

This pulls from the master branch by default. You can also specify a version, eg:

gem "downloader", "0.3.1", git: "https://github.com/strawberryjello/downloader.git"

And then execute:

$ bundle

Or install it yourself by cloning the repo and running the ff. in the root directory:

$ rake install

Usage

Download a single file:

Downloader.download(<URL>)
  • Note: This will also return the original filename of the file you downloaded

Download multiple files whose URLs are saved to a text file:

Downloader.batch("/path/to/file", "/path/to/destination/directory")

Command-line options for Downloader.batch can be passed in via a hash, eg:

options = {:numbered_filenames => true}
Downloader.batch("/path/to/file", "/path/to/destination/directory", options)

CLI

Downloader.download and Downloader.batch are available via CLI. You can also run the commands during development using bundle exec exe/downloader <command> <options>.

List the available commands:

$ downloader

Download a single file:

$ downloader download <url>

Download multiple files:

$ downloader batch /path/to/file /path/to/destination/directory

CLI Options

Downloader.batch accepts the ff. options:

  • --numbered-filenames - Rename files to be downloaded with numbers according to their order in the input file; file extensions, if any, will be retained

  • --scheme-host - The scheme and host in one string, for files containing relative URLs

  • --scheme - The scheme, for files containing scheme-less URLs

A help command is also available (Thor built-in) that will display the options for each command, eg:

$ downloader help batch

Tests and Documentation Coverage

Tests can be run locally using rspec or rake in the top-level directory.

To check documentation coverage (provided by Inch), run inch in the top-level directory.

License

This project is licensed under the MIT license.