Skip to content

Fix ScreenScraper credential URL-encoding and colon parsing in userCreds - #270

Closed
saitamasahil wants to merge 2 commits into
Gemba:masterfrom
saitamasahil:fix-screenscraper-credential-encoding
Closed

saitamasahil wants to merge 2 commits into
Gemba:masterfrom
saitamasahil:fix-screenscraper-credential-encoding

Conversation

@saitamasahil

Copy link
Copy Markdown

Description

While investigating a thread-limiting issue reported in Scrappy (saitamasahil/scrappy#26), I analyzed ScreenScraper authentication errors and identified two issues in Skyscraper's credential handling:

  1. Missing URL-encoding for ScreenScraper HTTP GET requests:
    In src/skyscraper.cpp (prepareScreenscraper) and src/screenscraper.cpp (getSearchResults), user and password are appended as raw strings into the request URL (&ssid=...&sspassword=...).

    • If a password contains reserved URL characters like &, +, #, or %, the HTTP query string gets corrupted (e.g., + becomes a space, & splits the query, # cuts off the URL), causing ScreenScraper to return Erreur de login and drop the scraper to 1-thread guest limits.
    • This has been a recurring point of confusion across the retro emulation scene (previously noted by Lars Muldjord in muldjord/skyscraper#297 and documented on the Batocera Wiki).
  2. Colons in password break userCreds parsing:
    In src/skyscraper.cpp (loadConfig), config.userCreds.split(":") checks for length() == 2. If a password contains a colon (:), length() is > 2, causing both config.user and config.password to be skipped and left empty.

Changes Made

  • Encoded user and password using QString::fromUtf8(QUrl::toPercentEncoding(...)) in src/skyscraper.cpp and src/screenscraper.cpp.
  • Updated userCreds parsing in src/skyscraper.cpp to use indexOf(':') so credentials are split only at the first colon, preserving any colons within the password.

@saitamasahil

Copy link
Copy Markdown
Author

For additional context and references across the retro emulation scene regarding ScreenScraper credentials and special characters:

  1. Official Batocera Wiki - Scrape From Guide
    Documents the exact parsing/escaping issue with ScreenScraper credentials:

    "If your username/password contains any special characters (like ?, Ó, ±, or №) it won't be parsed correctly by ES and you'll fail to login. Should have chosen a simpler password! ... You must escape special characters with a preceding backslash (\), eg. specialcharacter?inmypassword would become specialcharacter\?inmypassword. If the \ itself is in your password, just use \\."

  2. Batocera Issue #8807 - ScreenScraper Login issue
    Batocera maintainer lbrpdx referencing ScreenScraper login failures caused by unescaped password symbols:

    "Do you use any special character by any chance? If so, you might need to escape them with \ (or temporarily change your password and remove these characters for testing). See https://wiki.batocera.org/scrape_from"

  3. Skyscraper Issue #297 - No login with screenscraper module
    Lars Muldjord identifying special characters as the primary suspect when ScreenScraper rejects credentials:

    "Maybe it's some special characters it doesn't like, that's my best guess."

  4. RetroPie / EmulationStation PR #743
    When RetroPie added user credentials for ScreenScraper, they explicitly applied URL-encoding to both parameters to avoid GET parameter corruption:

    path = path + "&ssid=" + HttpReq::urlEncode(user) + "&sspassword=" + HttpReq::urlEncode(pass);

  5. Skyscraper Issue #316 - Quotes in config.ini
    Demonstrates how delicate credential parsing has historically been in config.ini, causing Skyscraper to silently drop to anonymous guest mode.

With QUrl::toPercentEncoding(...) and the indexOf(':') first-colon split in this PR, users don't need workarounds, backslash escaping, or simple alphanumeric passwords—credentials with &, +, #, %, or : in config.ini are safely encoded and accepted directly by ScreenScraper.

@Gemba Gemba mentioned this pull request Sep 20, 2026
Gemba added a commit that referenced this pull request Sep 20, 2026
* Fix for #270

- Screenscraper.fr user password may contiain a colon
- Apply percent encoding of credentials in URL because Screenscraper.fr only allows HTTP GET
- Doc update and testcases
@Gemba

Gemba commented Sep 20, 2026

Copy link
Copy Markdown
Owner

I have fixed it with 050eea4.

It boils down to the possible use of colon and the percent encoding.

The others are false flags (produced by the stochastical mix-n-match machine): Either not related to Skyscraper or can be reduced to user error.

Thanks for tabling this and your efforts on this, but no thanks for the AI efforts. I am closing this unmerged, as I was not comfortable with the suggested implementation.

@Gemba Gemba closed this Sep 20, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants