Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[youtube] Add options to skip webpage and js player for video extraction #860

Merged
merged 12 commits into from
Sep 6, 2021

Conversation

coletdjnz
Copy link
Member

@coletdjnz coletdjnz commented Sep 2, 2021

Please follow the guide below

  • You will be asked some questions, please read them carefully and answer honestly
  • Put an x into all the boxes [ ] relevant to your pull request (like that [x])
  • Use Preview tab to see how your pull request will actually look like

Before submitting a pull request make sure you have:

In order to be accepted and merged into yt-dlp each piece of code must be in public domain or released under Unlicense. Check one of the following options:

  • I am the original author of this code and I am willing to release it under Unlicense
  • I am not the original author of this code but it is in public domain or released under Unlicense (provide reliable evidence)

What is the purpose of your pull request?

  • (Future?) Bug fix
  • Improvement
  • New extractor
  • New feature

Description of your pull request and other information

Not sure what to call this.

This code was originally from #682.

This PR adds:

  • fallback player url extraction when it fails to be extracted from the webpage
  • don't download js player unnecessarily for clients that don't require it
  • try extract js player url from any additional client configs
  • ability to skip the js player usage/download (inc iframe fallback).
    • This overrides the clients' config setting.
    • This can be enabled with --extractor-args youtube:player_skip=js
    • Formats that require signature decryption will be skipped automatically (since no player url)
  • ability to skip the webpage download
    • This can be enabled with --extractor-args youtube:player_skip=webpage

KNOWN ISSUES:

  • authentication for multi-channel accounts and multi-account cookies may not work correctly if the webpage (and any additional client configs) are not downloaded.

TODO:

  • documentation
  • player_skip=js or something else?

@coletdjnz
Copy link
Member Author

coletdjnz commented Sep 2, 2021

Some example usage

  1. --extractor-args youtube:player_skip=webpage,configs,js;player_client=android,web
    • This makes extraction solely rely on the API - so no webpage downloads. Might help with avoiding 429s in some situations.
    • Since the web client usually needs the player js, formats from the web client requiring signature decryption will be skipped.
    • 3 initial API requests (android, web, initial data)

2. --extractor-args youtube:player_skip=webpage,configs,js;player_client=android
- Same as above, but without the web client (some minor metadata may be missed - be sure to check)
- 2 initial API requests (android, initial data)
Do not use - missing metadata

3. --extractor-args youtube:player_skip=webpage;player_client=android
- since the android client doesn't require the player js, this is the same as 2.
Do not use - missing metadata

  1. --extractor-args youtube:player_skip=webpage

    • basically forces fallbacks as if the webpage failed to download.
    • player url should be downloaded using the iframe API fallback (once, only if required)
  2. --extractor-args youtube:player_skip=webpage;player_client=web_music

    • config webpage for web_music is downloaded (thus player js details), so web formats should work correctly.

Not recommended:

  1. --extractor-args youtube:player_skip=webpage,configs,js;player_client=web
    • In some cases you'll get a no formats error (formats from the web client requiring signature decryption will be skipped).
  2. --extractor-args youtube:player_skip=js;player_client=web

yt_dlp/extractor/youtube.py Outdated Show resolved Hide resolved
yt_dlp/extractor/youtube.py Outdated Show resolved Hide resolved
yt_dlp/extractor/youtube.py Outdated Show resolved Hide resolved
@coletdjnz coletdjnz added the pending-fixes PR has had changes requested label Sep 3, 2021
coletdjnz and others added 5 commits September 5, 2021 06:32
Co-authored-by: pukkandan <pukkandan.ytdlp@gmail.com>
Co-authored-by: pukkandan <pukkandan.ytdlp@gmail.com>
Co-authored-by: pukkandan <pukkandan.ytdlp@gmail.com>
@coletdjnz coletdjnz removed the pending-fixes PR has had changes requested label Sep 5, 2021
@coletdjnz coletdjnz changed the title [youtube] Formalise API fallbacks for YouTubeIE [youtube] Add options to skip webpage and js player for video extraction Sep 5, 2021
yt_dlp/extractor/youtube.py Outdated Show resolved Hide resolved
yt_dlp/extractor/youtube.py Show resolved Hide resolved
yt_dlp/extractor/youtube.py Outdated Show resolved Hide resolved
README.md Outdated Show resolved Hide resolved
@pukkandan pukkandan merged commit b6de707 into yt-dlp:master Sep 6, 2021
nixxo pushed a commit to nixxo/yt-dlp that referenced this pull request Nov 22, 2021
* fallback player url extraction when it fails to be extracted from the webpage
* don't download js player unnecessarily for clients that don't require it
* try to extract js player url from any additional client configs
* ability to skip the js player usage/download using `player_skip=js`
* ability to skip the initial webpage download using `player_skip=webpage`

known issue:
* authentication for multi-channel accounts and multi-account cookies may not work correctly if the webpage or client configs are skipped
*  formats from the web client requiring signature decryption will be skipped if player js extraction is skipped

Authored by: coletdjnz
@coletdjnz coletdjnz deleted the player_js_skip branch March 4, 2022 06:04
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants