• Has this change been discussed on the forum or in an issue before?
• Does the code follow the SFML Code Style Guide?
• Have you provided some example/test code for your changes?
• If you have additional steps which need to be performed, please list them as tasks!

Description

Here's a fix issue #647
All the tests have passed on https://github.com/FRex/SFML/actions/runs/13015976076

It has 3 commits (for explanation of each see below):

1. adding tests for saving, loading and failing to load a file with Unicode filename.
2. fix for sf::Image saving/loading to/from Unicode filenames.
   1. fix for printing std::filesystem::path to std::string to then print error to sf::err().

EDIT: The last commit from above was put in PR #3407 and merged, now this PR has jus sf::Image tests and fixes for Unicode load/save filenames.

The tests check for 4 different interesting Unicode filenames: one with a Spanish letter, one with a Polish letter, one with a CJK character, and one with an emoji.

The Spanish letter is a Latin 1 character, and often works on Windows due to codepage 850 aka Latin 1 (which has values that overlap with Unicode, by design) being set. The Polish character will fail with that codepage since it's above 0xff. The CJK character will not work in any of these extended ASCII/codepage settings, it requires real Unicode support to work.

The emoji makes sure characters above 0xffff (outside of Unicode's first plane, the Basic Multilingual Plane) are handled properly. These characters take 3 UTF-8 bytes/codeunits or 2 wchars/UTF-16 codeunits (using surrogate pairs). Some older software (especially made before emojis became so popular online) sometimes fails on them, if they assume that UTF-16 is fixed width, so this is a good test for robust Unicode handling.

The fix in sf::Image uses std::ifstream/std::ofstream which accept std::filesystem::path so the Unicode stuff never leaves std:: parts, stb_/stbi_ then use callbacks to read to/from these streams, just how SFML's own streams.

EDIT: see above, this commit is no longer in this PR.

In Utils.cpp there is a helper function for making error messages that puts std::filesystem::path into a char stream, which throws an exception on Windows if that path contains Unicode. The fix is to always convert to UTF-8. The reinterpret_cast is because in C++17 u8path is char string, but in C++20 it's char8_t string which cannot be put into a char stream. It's same data and char8_t is same size as char so it works.

Tasks

• Tested on Linux
• Tested on Windows
• Tested on macOS
• Tested on iOS
• Tested on Android

How to test this PR?

Run the below program on Windows and see that files 🐌.png, 日.png, ń.png and ñ.png appear and load, as expected, and error message is printed properly in UTF-8 (if redirected to file) or at least doesn't throw an exception and just has some boxy characters in place of Unicode (if in default terminal).

Before this PR due to using .string() all these operations would throw a "No mapping for the unicode character exists in the target multi-byte code page" exception when path contains Unicode outside of current codepage (so most likely ñ works, but the other three don't, for most Western/American developers on default English language Windows).

#include <SFML/Graphics/Image.hpp>
#include <iostream>

int main()
{
    std::cerr << std::boolalpha; // to print true/false to std::cerr

    const char32_t totry[] = {
        0xf1,    // small n with tilde, outside of ascii but inside common latin1 codepage
        0x144,   // small n with acute accent, outside of latin1 codepage
        0x65E5,  // CJK symbol for sun, outside of any european codepage
        0x1F40C, // snail emoji, outside of Basic Multilingual Plane
    };

    char32_t utf32name[6] = {'X', '.', 'p', 'n', 'g', '\0'};

    for(const auto c : totry)
    {
        utf32name[0] = c;
        utf32name[1] = '.';
        sf::Image img = sf::Image(sf::Vector2u(32, 32), sf::Color::Red);
        std::cerr << img.saveToFile(utf32name) << std::endl; // successful save
        std::cerr << img.loadFromFile(utf32name) << std::endl; // successful load
        utf32name[1] = ',';
        std::cerr << !img.loadFromFile(utf32name) << std::endl; // failed load
    }
}