New upstream version 2021.01.24.1

ChangeLog

@@ -1,3 +1,27 @@
+version 2021.01.24
+
+Core
+* Introduce --output-na-placeholder (#27896)
+
+Extractors
+* [franceculture] Make thumbnail optional (#18807)
+* [franceculture] Fix extraction (#27891, #27903)
+* [njpwworld] Fix extraction (#27890)
+* [comedycentral] Fix extraction (#27905)
+* [wat] Fix format extraction (#27901)
++ [americastestkitchen:season] Add support for seasons (#27861)
++ [trovo] Add support for trovo.live (#26125)
++ [aol] Add support for yahoo videos (#26650)
+* [yahoo] Fix single video extraction
+* [lbry] Unescape lbry URI (#27872)
+* [9gag] Fix and improve extraction (#23022)
+* [americastestkitchen] Improve metadata extraction for ATK episodes (#27860)
+* [aljazeera] Fix extraction (#20911, #27779)
++ [minds] Add support for minds.com (#17934)
+* [ard] Fix title and description extraction (#27761)
++ [spotify] Add support for Spotify Podcasts (#27443)
+
+
 version 2021.01.16
 
 Core

docs/supportedsites.md

@@ -46,10 +46,11 @@
  - **Amara**
  - **AMCNetworks**
  - **AmericasTestKitchen**
+ - **AmericasTestKitchenSeason**
  - **anderetijden**: npo.nl, ntr.nl, omroepwnl.nl, zapp.nl and npo3.nl
  - **AnimeOnDemand**
  - **Anvato**
- - **aol.com**
+ - **aol.com**: Yahoo screen and movies
  - **APA**
  - **Aparat**
  - **AppleConnect**
@@ -192,8 +193,6 @@
  - **CNNArticle**
  - **CNNBlogs**
  - **ComedyCentral**
- - **ComedyCentralFullEpisodes**
- - **ComedyCentralShortname**
  - **ComedyCentralTV**
  - **CondeNast**: Condé Nast media group: Allure, Architectural Digest, Ars Technica, Bon Appétit, Brides, Condé Nast, Condé Nast Traveler, Details, Epicurious, GQ, Glamour, Golf Digest, SELF, Teen Vogue, The New Yorker, Vanity Fair, Vogue, W Magazine, WIRED
  - **CONtv**
@@ -506,6 +505,9 @@
  - **Mgoon**
  - **MGTV**: 芒果TV
  - **MiaoPai**
+ - **minds**
+ - **minds:channel**
+ - **minds:group**
  - **MinistryGrid**
  - **Minoto**
  - **miomio.tv**
@@ -859,6 +861,8 @@
  - **Sport5**
  - **SportBox**
  - **SportDeutschland**
+ - **spotify**
+ - **spotify:show**
  - **Spreaker**
  - **SpreakerPage**
  - **SpreakerShow**
@@ -940,12 +944,13 @@
  - **TNAFlixNetworkEmbed**
  - **toggle**
  - **ToonGoggles**
- - **Tosh**: Tosh.0
  - **tou.tv**
  - **Toypics**: Toypics video
  - **ToypicsUser**: Toypics user profile
  - **TrailerAddict** (Currently broken)
  - **Trilulilu**
+ - **Trovo**
+ - **TrovoVod**
  - **TruNews**
  - **TruTV**
  - **Tube8**

test/test_YoutubeDL.py

@@ -633,13 +633,20 @@ def test_prepare_filename(self):
             'title2': '%PATH%',
         }
 
-        def fname(templ):
-            ydl = YoutubeDL({'outtmpl': templ})
+        def fname(templ, na_placeholder='NA'):
+            params = {'outtmpl': templ}
+            if na_placeholder != 'NA':
+                params['outtmpl_na_placeholder'] = na_placeholder
+            ydl = YoutubeDL(params)
             return ydl.prepare_filename(info)
         self.assertEqual(fname('%(id)s.%(ext)s'), '1234.mp4')
         self.assertEqual(fname('%(id)s-%(width)s.%(ext)s'), '1234-NA.mp4')
-        # Replace missing fields with 'NA'
-        self.assertEqual(fname('%(uploader_date)s-%(id)s.%(ext)s'), 'NA-1234.mp4')
+        NA_TEST_OUTTMPL = '%(uploader_date)s-%(width)d-%(id)s.%(ext)s'
+        # Replace missing fields with 'NA' by default
+        self.assertEqual(fname(NA_TEST_OUTTMPL), 'NA-NA-1234.mp4')
+        # Or by provided placeholder
+        self.assertEqual(fname(NA_TEST_OUTTMPL, na_placeholder='none'), 'none-none-1234.mp4')
+        self.assertEqual(fname(NA_TEST_OUTTMPL, na_placeholder=''), '--1234.mp4')
         self.assertEqual(fname('%(height)d.%(ext)s'), '1080.mp4')
         self.assertEqual(fname('%(height)6d.%(ext)s'), '  1080.mp4')
         self.assertEqual(fname('%(height)-6d.%(ext)s'), '1080  .mp4')

youtube-dl.1

@@ -70,7 +70,7 @@ Force extraction to use the generic extractor
 .B \-\-default\-search \f[I]PREFIX\f[]
 Use this prefix for unqualified URLs.
 For example "gvsearch2:" downloads two videos from google videos for
-youtube\-dl "large apple".
+youtube\- dl "large apple".
 Use the value "auto" to let youtube\-dl guess ("auto_warning" to emit a
 warning when guessing).
 "error" just throws an error.
@@ -82,7 +82,7 @@ if this is not possible instead of searching.
 .B \-\-ignore\-config
 Do not read configuration files.
 When given in the global configuration file /etc/youtube\-dl.conf: Do
-not read the user configuration in ~/.config/youtube\- dl/config
+not read the user configuration in ~/.config/youtube\-dl/config
 (%APPDATA%/youtube\-dl/config.txt on Windows)
 .RS
 .RE
@@ -187,9 +187,9 @@ Playlist video to end at (default is last)
 .B \-\-playlist\-items \f[I]ITEM_SPEC\f[]
 Playlist video items to download.
 Specify indices of the videos in the playlist separated by commas like:
-"\-\-playlist\-items 1,2,5,8" if you want to download videos indexed 1,
+"\-\- playlist\-items 1,2,5,8" if you want to download videos indexed 1,
 2, 5, 8 in the playlist.
-You can specify range: "\-\-playlist\-items 1\-3,7,10\-13", it will
+You can specify range: " \-\-playlist\-items 1\-3,7,10\-13", it will
 download the videos at index 1, 2, 3, 7, 10, 11, 12 and 13.
 .RS
 .RE
@@ -380,7 +380,7 @@ while downloading (some players may not be able to play it)
 .TP
 .B \-\-external\-downloader \f[I]COMMAND\f[]
 Use the specified external downloader.
-Currently supports aria2c,avconv,axel,curl,ffmpeg,httpie,wget
+Currently supports aria2c,avconv,axel,c url,ffmpeg,httpie,wget
 .RS
 .RE
 .TP
@@ -408,6 +408,12 @@ Output filename template, see the "OUTPUT TEMPLATE" for all the info
 .RS
 .RE
 .TP
+.B \-\-output\-na\-placeholder \f[I]PLACEHOLDER\f[]
+Placeholder value for unavailable meta fields in output filename
+template (default is "NA")
+.RS
+.RE
+.TP
 .B \-\-autonumber\-start \f[I]NUMBER\f[]
 Specify the start value for %(autonumber)s (default is 1)
 .RS
@@ -462,7 +468,7 @@ Write video annotations to a .annotations.xml file
 .TP
 .B \-\-load\-info\-json \f[I]FILE\f[]
 JSON file containing the video information (created with the
-"\-\-write\-info\-json" option)
+"\-\-write\- info\-json" option)
 .RS
 .RE
 .TP
@@ -686,7 +692,7 @@ lower bound of a range for randomized sleep before each download
 .B \-\-max\-sleep\-interval \f[I]SECONDS\f[]
 Upper bound of a range for randomized sleep before each download
 (maximum possible number of seconds to sleep).
-Must only be used along with \-\-min\-sleep\-interval.
+Must only be used along with \-\-min\- sleep\-interval.
 .RS
 .RE
 .SS Video Format Options:
@@ -753,7 +759,7 @@ Subtitle format, accepts formats preference, for example: "srt" or
 .TP
 .B \-\-sub\-lang \f[I]LANGS\f[]
 Languages of the subtitles to download (optional) separated by commas,
-use \-\-list\- subs for available language tags
+use \-\-list\-subs for available language tags
 .RS
 .RE
 .SS Authentication Options:
@@ -809,8 +815,8 @@ List all supported multiple\-system operators
 .SS Post\-processing Options:
 .TP
 .B \-x, \-\-extract\-audio
-Convert video files to audio\-only files (requires ffmpeg or avconv and
-ffprobe or avprobe)
+Convert video files to audio\-only files (requires ffmpeg/avconv and
+ffprobe/avprobe)
 .RS
 .RE
 .TP
@@ -838,7 +844,7 @@ Give these arguments to the postprocessor
 .RE
 .TP
 .B \-k, \-\-keep\-video
-Keep the video file on disk after the post\- processing; the video is
+Keep the video file on disk after the post\-processing; the video is
 erased by default
 .RS
 .RE
@@ -869,7 +875,7 @@ Parse additional metadata like song title / artist from the video title.
 The format syntax is the same as \-\-output.
 Regular expression with named capture groups may also be used.
 The parsed parameters replace existing values.
-Example: \-\-metadata\-from\- title "%(artist)s \- %(title)s" matches a
+Example: \-\-metadata\-from\-title "%(artist)s \- %(title)s" matches a
 title like "Coldplay \- Paradise".
 Example (regex): \-\-metadata\-from\-title "(?P.+?) \- (?P
 .RS
@@ -1219,7 +1225,8 @@ Each aforementioned sequence when referenced in an output template will
 be replaced by the actual value corresponding to the sequence name.
 Note that some of the sequences are not guaranteed to be present since
 they depend on the metadata obtained by a particular extractor.
-Such sequences will be replaced with \f[C]NA\f[].
+Such sequences will be replaced with placeholder value provided with
+\f[C]\-\-output\-na\-placeholder\f[] (\f[C]NA\f[] by default).
 .PP
 For example for \f[C]\-o\ %(title)s\-%(id)s.%(ext)s\f[] and an mp4 video
 with title \f[C]youtube\-dl\ test\ video\f[] and id

youtube-dl.bash-completion

@@ -4,7 +4,7 @@ __youtube_dl()
     COMPREPLY=()
     cur="${COMP_WORDS[COMP_CWORD]}"
     prev="${COMP_WORDS[COMP_CWORD-1]}"
-    opts="--help --version --update --ignore-errors --abort-on-error --dump-user-agent --list-extractors --extractor-descriptions --force-generic-extractor --default-search --ignore-config --config-location --flat-playlist --mark-watched --no-mark-watched --no-color --proxy --socket-timeout --source-address --force-ipv4 --force-ipv6 --geo-verification-proxy --cn-verification-proxy --geo-bypass --no-geo-bypass --geo-bypass-country --geo-bypass-ip-block --playlist-start --playlist-end --playlist-items --match-title --reject-title --max-downloads --min-filesize --max-filesize --date --datebefore --dateafter --min-views --max-views --match-filter --no-playlist --yes-playlist --age-limit --download-archive --include-ads --limit-rate --retries --fragment-retries --skip-unavailable-fragments --abort-on-unavailable-fragment --keep-fragments --buffer-size --no-resize-buffer --http-chunk-size --test --playlist-reverse --playlist-random --xattr-set-filesize --hls-prefer-native --hls-prefer-ffmpeg --hls-use-mpegts --external-downloader --external-downloader-args --batch-file --id --output --autonumber-size --autonumber-start --restrict-filenames --auto-number --title --literal --no-overwrites --continue --no-continue --no-part --no-mtime --write-description --write-info-json --write-annotations --load-info-json --cookies --cache-dir --no-cache-dir --rm-cache-dir --write-thumbnail --write-all-thumbnails --list-thumbnails --quiet --no-warnings --simulate --skip-download --get-url --get-title --get-id --get-thumbnail --get-description --get-duration --get-filename --get-format --dump-json --dump-single-json --print-json --newline --no-progress --console-title --verbose --dump-pages --write-pages --youtube-print-sig-code --print-traffic --call-home --no-call-home --encoding --no-check-certificate --prefer-insecure --user-agent --referer --add-header --bidi-workaround --sleep-interval --max-sleep-interval --format --all-formats --prefer-free-formats --list-formats --youtube-include-dash-manifest --youtube-skip-dash-manifest --merge-output-format --write-sub --write-auto-sub --all-subs --list-subs --sub-format --sub-lang --username --password --twofactor --netrc --video-password --ap-mso --ap-username --ap-password --ap-list-mso --extract-audio --audio-format --audio-quality --recode-video --postprocessor-args --keep-video --no-post-overwrites --embed-subs --embed-thumbnail --add-metadata --metadata-from-title --xattrs --fixup --prefer-avconv --prefer-ffmpeg --ffmpeg-location --exec --convert-subs"
+    opts="--help --version --update --ignore-errors --abort-on-error --dump-user-agent --list-extractors --extractor-descriptions --force-generic-extractor --default-search --ignore-config --config-location --flat-playlist --mark-watched --no-mark-watched --no-color --proxy --socket-timeout --source-address --force-ipv4 --force-ipv6 --geo-verification-proxy --cn-verification-proxy --geo-bypass --no-geo-bypass --geo-bypass-country --geo-bypass-ip-block --playlist-start --playlist-end --playlist-items --match-title --reject-title --max-downloads --min-filesize --max-filesize --date --datebefore --dateafter --min-views --max-views --match-filter --no-playlist --yes-playlist --age-limit --download-archive --include-ads --limit-rate --retries --fragment-retries --skip-unavailable-fragments --abort-on-unavailable-fragment --keep-fragments --buffer-size --no-resize-buffer --http-chunk-size --test --playlist-reverse --playlist-random --xattr-set-filesize --hls-prefer-native --hls-prefer-ffmpeg --hls-use-mpegts --external-downloader --external-downloader-args --batch-file --id --output --output-na-placeholder --autonumber-size --autonumber-start --restrict-filenames --auto-number --title --literal --no-overwrites --continue --no-continue --no-part --no-mtime --write-description --write-info-json --write-annotations --load-info-json --cookies --cache-dir --no-cache-dir --rm-cache-dir --write-thumbnail --write-all-thumbnails --list-thumbnails --quiet --no-warnings --simulate --skip-download --get-url --get-title --get-id --get-thumbnail --get-description --get-duration --get-filename --get-format --dump-json --dump-single-json --print-json --newline --no-progress --console-title --verbose --dump-pages --write-pages --youtube-print-sig-code --print-traffic --call-home --no-call-home --encoding --no-check-certificate --prefer-insecure --user-agent --referer --add-header --bidi-workaround --sleep-interval --max-sleep-interval --format --all-formats --prefer-free-formats --list-formats --youtube-include-dash-manifest --youtube-skip-dash-manifest --merge-output-format --write-sub --write-auto-sub --all-subs --list-subs --sub-format --sub-lang --username --password --twofactor --netrc --video-password --ap-mso --ap-username --ap-password --ap-list-mso --extract-audio --audio-format --audio-quality --recode-video --postprocessor-args --keep-video --no-post-overwrites --embed-subs --embed-thumbnail --add-metadata --metadata-from-title --xattrs --fixup --prefer-avconv --prefer-ffmpeg --ffmpeg-location --exec --convert-subs"
     keywords=":ytfavorites :ytrecommended :ytsubscriptions :ytwatchlater :ythistory"
     fileopts="-a|--batch-file|--download-archive|--cookies|--load-info"
     diropts="--cache-dir"

youtube-dl.fish

@@ -66,6 +66,7 @@ complete --command youtube-dl --long-option external-downloader-args --descripti
 complete --command youtube-dl --long-option batch-file --short-option a --description 'File containing URLs to download ('"'"'-'"'"' for stdin), one URL per line. Lines starting with '"'"'#'"'"', '"'"';'"'"' or '"'"']'"'"' are considered as comments and ignored.' --require-parameter
 complete --command youtube-dl --long-option id --description 'Use only video ID in file name'
 complete --command youtube-dl --long-option output --short-option o --description 'Output filename template, see the "OUTPUT TEMPLATE" for all the info'
+complete --command youtube-dl --long-option output-na-placeholder --description 'Placeholder value for unavailable meta fields in output filename template (default is "%default")'
 complete --command youtube-dl --long-option autonumber-size
 complete --command youtube-dl --long-option autonumber-start --description 'Specify the start value for %(autonumber)s (default is %default)'
 complete --command youtube-dl --long-option restrict-filenames --description 'Restrict filenames to only ASCII characters, and avoid "&" and spaces in filenames'
@@ -144,7 +145,7 @@ complete --command youtube-dl --long-option ap-mso --description 'Adobe Pass mul
 complete --command youtube-dl --long-option ap-username --description 'Multiple-system operator account login'
 complete --command youtube-dl --long-option ap-password --description 'Multiple-system operator account password. If this option is left out, youtube-dl will ask interactively.'
 complete --command youtube-dl --long-option ap-list-mso --description 'List all supported multiple-system operators'
-complete --command youtube-dl --long-option extract-audio --short-option x --description 'Convert video files to audio-only files (requires ffmpeg or avconv and ffprobe or avprobe)'
+complete --command youtube-dl --long-option extract-audio --short-option x --description 'Convert video files to audio-only files (requires ffmpeg/avconv and ffprobe/avprobe)'
 complete --command youtube-dl --long-option audio-format --description 'Specify audio format: "best", "aac", "flac", "mp3", "m4a", "opus", "vorbis", or "wav"; "%default" by default; No effect without -x'
 complete --command youtube-dl --long-option audio-quality --description 'Specify ffmpeg/avconv audio quality, insert a value between 0 (better) and 9 (worse) for VBR or a specific bitrate like 128K (default %default)'
 complete --command youtube-dl --long-option recode-video --description 'Encode the video to another format if necessary (currently supported: mp4|flv|ogg|webm|mkv|avi)' --arguments 'mp4 flv ogg webm mkv' --exclusive