Finish multiplayer support

CHANGELOG.md

@@ -15,6 +15,9 @@ With thanks to **@parkerlreed** for his donation this month!
 
 ### Added
 
+- Added support for emulated LAN multiplayer.
+  Only works on an actual LAN.
+  [#225](https://github.com/JesseTG/melonds-ds/issues/225)
 - Integrated support for the GBA Memory Expansion Pak.
   [#44](https://github.com/JesseTG/melonds-ds/issues/44)
 - Integrated support for the Rumble Pak.
@@ -475,4 +478,4 @@ This is an overview of this release's changes. Pretty cool, huh?
 
 ### Changed
 
-- Test release to see what happens.
+- Test release to see what happens.

LanMultiplayer.md

@@ -0,0 +1,84 @@
+# What is LAN Multiplayer?
+The Nintendo DS has several forms of multiplayer,
+including local Wi-Fi (also called LAN), Nintendo Wi-Fi Connection (WFC) and infrared.
+You'll know when games use local Wi-Fi if the game mentions "Wi-Fi" and "players nearby",
+while games using WFC mention "players around the world" and use friend codes.
+The Nintendo DS local Wi-Fi does not use friend codes.
+This page only explains local Wi-Fi multiplayer.
+
+Now, the Nintendo DS local Wi-Fi isn't the normal Wi-Fi in your house,
+it is a [mesh network that uses specialized hardware](https://melonds.kuribo64.net/comments.php?id=25).
+This means that games expect extremely low latency,
+which is achievable between two consoles directly connected with special hardware,
+but harder to achieve with two computers with a router in-between,
+and simply **impossible** to achieve through the Internet.
+**LAN multiplayer does not work through the Internet and neither with VPNs or tunnels such as Hamachi**.
+This is not something that can be fixed easily.
+**The only way to use emulated LAN Multiplayer is on an actual, low latency, wired LAN connection**.
+
+The latency requirements are so extreme that even in LAN, you might still have issues.
+That is why using Wi-Fi in your LAN connection is not recommended,
+Wi-Fi simply adds too much latency,
+and the connection will drop frequently.
+The recommended way to use the emulated Wi-Fi LAN connection is with a wired LAN connection between the computers.
+
+## What is a MAC address?
+Every Nintendo DS (and every device capable of Wi-Fi) comes with an identifier built-in in its firmware called a "MAC address".
+For three or more Nintendo DS consoles to connect in a local Wi-Fi multiplayer network,
+all three need to have different MAC addresses.
+You can see the emulated console's MAC address in a game with Nintendo WFC
+by going to "Nintendo WFC Settings", "Options", "System Information".
+
+Some games will refuse to load save files
+that were created on a console with a different MAC address
+than the console loading the file.
+That is why it is important to pay attention to your MAC address when sharing save files across devices.
+
+So how does the emulated console obtain a MAC address?
+It depends on the core option "MAC address mode", in the "Network" category:
+
+* Set from firmware:
+The default setting.
+The emulator will use the firmware file's MAC address for the emulated console.
+If there is no firmware file, then a default MAC address of "00-09-BF-11-22-33" will be used.
+This setting will cause issues on LAN multiplayer with more than 2 players
+if the same firmware (or the default firmware) is used on more than one device.
+
+* Derive from libretro username:
+The emulator will use your username as set on the libretro frontend to automatically generate a MAC address.
+Such generated MAC addresses will start with "00-08-BF".
+Devices with the same username set will always generate the same MAC address.
+Useful when syncing save files across devices
+to guarantee the emulated console's MAC address is the same on all devices.
+This setting will cause issues on LAN multiplayer
+if more than one player has the same username.
+The username can be set on RetroArch by going to "Settings", "User", "Username".
+
+* Read from a file:
+By adding a txt file that contains a formatted MAC address
+in the "system" folder of the libretro frontend or a subfolder of the "system" folder named "melonDS DS",
+an option will appear to set the MAC address from the file.
+The file must be encoded in ASCII,
+and the MAC address must be formatted as "00:00:00:00:00".
+The MAC address must be the first line in the file,
+and there must be no other text before the MAC address.
+The MAC address must only use uppercase letters and digits in the hexadecimal numbers.
+As with all other settings,
+this setting will cause issues if more than one player in a multiplayer session has the same MAC address.
+
+## Starting a multiplayer session (RetroArch)
+Before starting a multiplayer session,
+it is recommended that all set a proper username in "Settings", "User", "Username", in RetroArch,
+and set the MAC address mode to "Derive from libretro username".
+
+In RetroArch, such a multiplayer session can be started through the "Netplay" menu either before starting a game or during a game.
+One player should host and the others should use the "Refresh Netplay LAN List" option to join.
+In game, you'd look for something mentioning local multiplayer and "players nearby".
+In Pokémon games, this can be accessed on the top floor of Pokémon centers,
+in Mario Kart DS, it is the "Multiplayer" option in the menu etc.
+There are a ton of pages online saying that "Netplay is not network emulation".
+For various reasons, these pages are wrong when it comes to melonDS DS.
+
+Again, emulated LAN multiplayer only works if all players are on the same real local network.
+This means the same house, apartment etc.
+For better results, a wired connection is recommended.

src/libretro/config/config.cpp

@@ -26,6 +26,7 @@
 #include <initializer_list>
 #include <memory>
 #include <span>
+#include <random>
 #include <string_view>
 #include <utility>
 #include <vector>
@@ -588,8 +589,6 @@ static void MelonDsDs::config::ParseFirmwareOptions(CoreConfig& config) noexcept
         retro::warn("Failed to get value for {}; defaulting to existing firmware value", firmware::WFC_DNS);
         config.SetDnsServer(nullopt);
     }
-
-    // TODO: Make MAC address configurable with a file at runtime
 }
 
 static void MelonDsDs::config::ParseAudioOptions(CoreConfig& config) noexcept {
@@ -627,10 +626,10 @@ static void MelonDsDs::config::ParseAudioOptions(CoreConfig& config) noexcept {
 }
 
 static void MelonDsDs::config::ParseNetworkOptions(CoreConfig& config) noexcept {
-#ifdef HAVE_NETWORKING
     ZoneScopedN(TracyFunction);
     using retro::get_variable;
 
+#ifdef HAVE_NETWORKING
     if (optional<NetworkMode> value = ParseNetworkMode(get_variable(network::NETWORK_MODE))) {
         config.SetNetworkMode(*value);
     } else {
@@ -647,6 +646,62 @@ static void MelonDsDs::config::ParseNetworkOptions(CoreConfig& config) noexcept
     }
 #endif
 #endif
+
+    if (string_view macAddrModeText = get_variable(network::MAC_ADDRESS_MODE); macAddrModeText == values::FROM_USERNAME) {
+        optional<string_view> retro_username = retro::username();
+        if (retro_username) {
+            // The first 3 bytes of a MAC address are special:
+            // they identify the manufacturer
+            // 00:09:BF is the manufacturer of the default firmware.
+            // We use 00:08:BF to differentiate and because
+            // it is not assigned to anyone else.
+            // (Not that this MAC address will be used outside the emulated network)
+            melonDS::MacAddress addr{0x00, 0x08, 0xBF, 0, 0, 0};
+            std::uint32_t seed = 0;
+            // We use our own hashing algorithm to guarantee same output with different compilers
+            // Our hash does not need to be very good because we already have an rng engine.
+            for (char c : *retro_username) {
+                // Protect against signed char/unsigned char differences
+                if (c < 0) {
+                    c *= -1;
+                }
+                std::uint32_t thisCharSeed = c;
+                // 1419857 = 17^5
+                // Shift left 4 bits (multiply by 16), then add the previous value 5 times,
+                // but done in a quick multiplications instead of a loop.
+                // (a * 17) = (a * 16) + a = (a << 4) + a
+                // Why 5 times? Because UINT32_MAX/(17^5) > CHAR_MAX, assuming char is <= 8 bits,
+                // which means there is no risk of overflow, which is implementation-defined.
+                thisCharSeed *= 1419857;
+                seed ^= thisCharSeed;
+            }
+            // We use a specific engine here to guarantee same output with different devices and compilers
+            // This is the same as std::mt19937 but with the fixed uint32_t
+            // instead of the implementation-defined uint_fast32_t
+            std::mersenne_twister_engine<std::uint32_t,
+                             32, 624, 397, 31,
+                             0x9908b0df, 11,
+                             0xffffffff, 7,
+                             0x9d2c5680, 15,
+                             0xefc60000, 18, 1812433253> rng_engine{seed};
+            for (int i = 3; i <= 5; i++) {
+                // Usually using modulo is a bad way to narrow a random distribution
+                // However because UINT32_MAX + 1 % 256 = 0, it is actually ok.
+                // Another way to think is that we are taking the eight lowest bits
+                addr[i] = rng_engine() % 256;
+            }
+            config.SetMacAddress(addr);
+        } else {
+            config.SetMacAddress(nullopt);
+        }
+    } else if (macAddrModeText == values::FIRMWARE) {
+        config.SetMacAddress(nullopt);
+    } else if (optional<melonDS::MacAddress> fileAddress = ParseMacAddress(macAddrModeText)) {
+        config.SetMacAddress(fileAddress);
+    } else {
+        retro::warn("Failed to get value for {}; defaulting to existing firmware value", network::MAC_ADDRESS_MODE);
+        config.SetMacAddress(nullopt);
+    }
 }
 
 static void MelonDsDs::config::ParseScreenOptions(CoreConfig& config) noexcept {
@@ -767,6 +822,11 @@ struct FirmwareEntry {
     struct stat stat;
 };
 
+struct MacAddressEntry {
+    std::string description;
+    std::string printedAddress;
+};
+
 static time_t NewestTimestamp(const struct stat& statbuf) noexcept {
     return std::max({statbuf.st_atime, statbuf.st_mtime, statbuf.st_ctime});
 }
@@ -832,6 +892,7 @@ bool MelonDsDs::RegisterCoreOptions() noexcept {
 
     vector<string> dsiNandPaths;
     vector<FirmwareEntry> firmware;
+    vector<MacAddressEntry> macAddresses;
     optional<string_view> sysdir = retro::get_system_directory();
 
     if (subdir) {
@@ -845,6 +906,13 @@ bool MelonDsDs::RegisterCoreOptions() noexcept {
             ZoneScopedN("MelonDsDs::config::set_core_options::find_system_files::paths");
             for (const retro::dirent& d : retro::readdir(string(path), true)) {
                 ZoneScopedN("MelonDsDs::config::set_core_options::find_system_files::paths::dirent");
+                if (optional<melonDS::MacAddress> address = ParseMacAddressFile(d)) {
+                    string_view prettyPath{d.path};
+                    prettyPath.remove_prefix(sysdir->size() + 1);
+                    std::string description = fmt::format("Read from file \"{}\" ({})", prettyPath, PrintMacAddress(*address));
+                    macAddresses.emplace_back(MacAddressEntry{std::move(description), PrintMacAddress(*address)});
+                }
+
                 if (IsDsiNandImage(d)) {
                     dsiNandPaths.emplace_back(d.path);
                 } else if (IsFirmwareImage(d, header)) {
@@ -912,7 +980,24 @@ bool MelonDsDs::RegisterCoreOptions() noexcept {
         retro_assert(firmwarePathOption->default_value != nullptr);
         retro_assert(firmwarePathDsiOption->default_value != nullptr);
     }
-
+    if(!macAddresses.empty()) {
+        ZoneScopedN("MelonDsDs::config::set_core_options::init_mac_address_options");
+        retro_core_option_v2_definition* macAddressOption = find_if(definitions.begin(), definitions.end(), [](const auto& def) {
+            return string_is_equal(def.key, MelonDsDs::config::network::MAC_ADDRESS_MODE);
+        });
+        retro_assert(macAddressOption != definitions.end());
+        int existingOptions = 0;
+        while (macAddressOption->values[existingOptions++].value != nullptr && existingOptions != RETRO_NUM_CORE_OPTION_VALUES_MAX) {
+        }
+        existingOptions--;
+        int length = std::min((int)macAddresses.size(), (int)RETRO_NUM_CORE_OPTION_VALUES_MAX - existingOptions);
+        for (int i = 0; i < length; ++i) {
+            macAddressOption->values[i + existingOptions] = { macAddresses[i].printedAddress.data(), macAddresses[i].description.data() };
+        }
+        if(length + existingOptions < RETRO_NUM_CORE_OPTION_VALUES_MAX) {
+            macAddressOption->values[length + existingOptions] = { nullptr, nullptr };
+        }
+    }
 
     // TODO: Pass in the LibPCap instance created by NetState
     // TODO: Create a DynamicOption class, pass in instances of that

src/libretro/config/constants.cpp

@@ -16,7 +16,10 @@
 
 #include "constants.hpp"
 
+#include <SPI_Firmware.h>
 #include <algorithm>
+#include <regex>
+#include <fmt/format.h>
 #include <string>
 #include <fmt/ranges.h>
 #include <net/net_compat.h>
@@ -25,6 +28,7 @@
 #include <file/file_path.h>
 #include <streams/file_stream.h>
 
+#include "retro/file.hpp"
 #include "types.hpp"
 #include "environment.hpp"
 #include "retro/dirent.hpp"
@@ -169,4 +173,97 @@ bool MelonDsDs::config::IsFirmwareImage(const retro::dirent& file, Firmware::Fir
 
     memcpy(&header, &buffer, sizeof(buffer));
     return true;
-}
+}
+
+// A MAC address has 6 bytes, each with two hexadecimal characters,
+// and 5 colons (:) for separators
+constexpr int MacAddressStringSize = 2*6 + 5;
+
+std::optional<melonDS::MacAddress> MelonDsDs::config::ParseMacAddressFile(const retro::dirent &file) noexcept {
+    ZoneScopedN(TracyFunction);
+    ZoneText(file.path, strnlen(file.path, sizeof(file.path)));
+    retro::debug("Reading file {}", file.path);
+
+    if(!file.is_regular_file()) {
+        retro::debug("{} is not a regular file, it's not a mac address file", file.path);
+        return std::nullopt;
+    }
+
+    if(!string_ends_with(file.path, ".txt")) {
+        retro::debug("{} is not a mac address file, it does not end with .txt", file.path);
+        return std::nullopt;
+    }
+    if (file.size < MacAddressStringSize) {
+        retro::debug("{} is not a mac address file, it is too small", file.path);
+        return std::nullopt;
+    }
+
+    char buffer[MacAddressStringSize];
+    retro::rfile_ptr stream = retro::make_rfile(file.path, RETRO_VFS_FILE_ACCESS_READ, RETRO_VFS_FILE_ACCESS_HINT_NONE);
+
+    int64_t bytesRead = filestream_read(stream.get(), &buffer, sizeof(buffer));
+    if (bytesRead < MacAddressStringSize) {
+        if (bytesRead < 0) {
+            retro::warn("Failed to read {}", file.path);
+        } else {
+            retro::warn("Tried to read {} bytes, ended up reading {} bytes instead", MacAddressStringSize, bytesRead);
+        }
+        return std::nullopt;
+    }
+
+    std::optional<melonDS::MacAddress> ret =  MelonDsDs::config::ParseMacAddress(std::string_view{buffer, sizeof(buffer)});
+    if (!ret.has_value()) {
+        retro::debug("Could not read the mac address from \"{}\"", std::string_view{buffer, sizeof(buffer)});
+    }
+    return ret;
+}
+
+std::optional<melonDS::MacAddress> MelonDsDs::config::ParseMacAddress(std::string_view s) noexcept {
+    // This would be 5 lines if scanf worked on string_view
+    melonDS::MacAddress ret;
+    const static std::regex pattern{"^([[:xdigit:]]{2})[:-]([[:xdigit:]]{2})[:-]([[:xdigit:]]{2})[:-]([[:xdigit:]]{2})[:-]([[:xdigit:]]{2})[:-]([[:xdigit:]]{2})$", std::regex_constants::extended};
+    std::match_results<std::string_view::const_iterator> results;
+    if (!std::regex_match(s.cbegin(), s.cend(), results, pattern)) {
+        return std::nullopt;
+    } else {
+        int readOctets = 0;
+        bool firstMatch = true;
+        for (const std::sub_match<std::string_view::const_iterator> &octetMatch : results) {
+            if (firstMatch) {
+                // The first match is always the whole string
+                firstMatch = false;
+                continue;
+            }
+            if (readOctets == 6) {
+                break;
+            }
+            char octetNullString[3];
+            if (octetMatch.length() != 2) {
+                return std::nullopt;
+            }
+            std::string_view::const_iterator matchIter = octetMatch.first;
+            octetNullString[0] = *matchIter;
+            matchIter++;
+            octetNullString[1] = *matchIter;
+            octetNullString[2] = '\0';
+            char *end = nullptr;
+            unsigned long octetValue = std::strtoul(octetNullString, &end, 16);
+            if (end != octetNullString + 2) {
+                return std::nullopt;
+            }
+            if (octetValue > 255) {
+                // Not sure how this would be possible
+                return std::nullopt;
+            }
+            ret[readOctets++] = octetValue;
+        }
+        if (readOctets != 6) {
+            return std::nullopt;
+        }
+        return ret;
+    }
+}
+
+std::string MelonDsDs::config::PrintMacAddress(const melonDS::MacAddress &address) noexcept {
+    return fmt::format("{:02X}:{:02X}:{:02X}:{:02X}:{:02X}:{:02X}", address[0], address[1], address[2], address[3], address[4], address[5]);
+}