[Feature] - make conversion possible with special fiat currency at st…

README.md

@@ -83,7 +83,10 @@ Main features:
       - [Last price](#last-price)
       - [Last trades](#last-trades)
       - [Conversion](#conversion)
+        - [Example: Print the value of 1000 Shiba Inu expressed in Solana on all exchanges where such conversion is possible](#example-print-the-value-of-1000-shiba-inu-expressed-in-solana-on-all-exchanges-where-such-conversion-is-possible)
+        - [Example: Print the value of 1000 EUR expressed in Ethereum on all exchanges where such conversion is possible](#example-print-the-value-of-1000-eur-expressed-in-ethereum-on-all-exchanges-where-such-conversion-is-possible)
       - [Conversion path](#conversion-path)
+        - [Example: Print the fastest conversion path from Shiba Inu to Solana on all exchanges where such conversion is possible](#example-print-the-fastest-conversion-path-from-shiba-inu-to-solana-on-all-exchanges-where-such-conversion-is-possible)
       - [Withdraw fees](#withdraw-fees)
         - [Example 1: query all withdraw fees](#example-1-query-all-withdraw-fees)
         - [Example 2: query ETH withdraw fees on Kraken and Kucoin](#example-2-query-eth-withdraw-fees-on-kraken-and-kucoin)
@@ -384,31 +387,84 @@ coincenter last-trades dot-usdt,binance,huobi --n 15
 
 #### Conversion
 
-From a starting amount to a destination currency, `conversion` examines the shortest conversion path (in terms of number of conversion) possible to reach the destination currency, on optional list of exchanges, and return the converted amount on each exchange where such conversion is possible. The trade fees are not taken into account for this conversion.
+From a starting amount to a destination currency, `conversion` examines the shortest conversion path (in terms of number of conversion) possible to reach the destination currency, on optional list of exchanges, and return the converted amount on each exchange where such conversion is possible. The trade fees **are** taken into account for this conversion.
 
 The conversion path chosen is the fastest (in terms of number of markets, so trades) possible.
 
-Example: Print the fastest conversion from Shiba Inu to Solana on all exchanges where such conversion is possible
+**Important note**: fiat conversions (including fiat-like crypto-currencies like `USDT`) are allowed even if the corresponding market is not an exchange market only as a first or a last conversion. No trade fees are taken into account for fiat conversions.
+
+##### Example: Print the value of 1000 Shiba Inu expressed in Solana on all exchanges where such conversion is possible
 
 ```bash
 coincenter conversion 1000shib-sol
 ```
 
+Possible output:
+
+```bash
++----------+------------------------------+
+| Exchange | 1000 SHIB converted into SOL |
++----------+------------------------------+
+| binance  | 0.00020324242724052 SOL      |
+| bithumb  | 0.00020297245011085 SOL      |
+| huobi    | 0.00020276144764053 SOL      |
+| kraken   | 0.00020323457500308 SOL      |
+| kucoin   | 0.00020338540268269 SOL      |
+| upbit    | 0.00020302426059262 SOL      |
++----------+------------------------------+
+```
+
+##### Example: Print the value of 1000 EUR expressed in Ethereum on all exchanges where such conversion is possible
+
+```bash
+coincenter conversion 1000eur-eth
+```
+
+Possible output:
+
+```bash
++----------+-----------------------------+
+| Exchange | 1000 EUR converted into ETH |
++----------+-----------------------------+
+| binance  | 0.31121398375705967 ETH     |
+| bithumb  | 0.29715335705445442 ETH     |
+| huobi    | 0.30978082796054002 ETH     |
+| kraken   | 0.3110542008206231 ETH      |
+| kucoin   | 0.31194281985067939 ETH     |
+| upbit    | 0.29672491761070147 ETH     |
++----------+-----------------------------+
+```
+
 #### Conversion path
 
-This option is similar to previous one ([conversion](#conversion)) but takes a starting currency code instead of an amount to convert.
+This option is similar to previous one ([conversion](#conversion)) but takes a starting currency code instead of an amount to convert, and only real markets from exchanges are taken into account (in particular, fiat conversions that are not proposed by the exchange as a market are not possible).
 
 It will print the result as an ordered list of markets for each exchange.
 
 Option `conversion-path` is used internally for [multi trade](#multi-trade) but is provided as a stand-alone query for information.
 
-Example: Print the fastest conversion path from Shiba Inu to Solana on all exchanges where such conversion is possible
+##### Example: Print the fastest conversion path from Shiba Inu to Solana on all exchanges where such conversion is possible
 
 ```bash
 coincenter conversion-path shib-sol
 ```
 
-**Note**: when several conversion paths of same length are possible, it will favor the ones not involving fiat currencies.
+Possible output:
+
+```bash
++----------+--------------------------------------+
+| Exchange | Fastest conversion path for SHIB-SOL |
++----------+--------------------------------------+
+| binance  | SHIB-FDUSD,SOL-FDUSD                 |
+| bithumb  | SHIB-KRW,SOL-KRW                     |
+| huobi    | SHIB-USDT,SOL-USDT                   |
+| kraken   | SHIB-USDT,SOL-USDT                   |
+| kucoin   | SHIB-USDC,SOL-USDC                   |
+| upbit    | SHIB-KRW,SOL-KRW                     |
++----------+--------------------------------------+
+```
+
+**Note**: when several conversion paths of same length are possible, it will favor the ones not involving fiat currencies (stable coins are **not** considered as fiat currencies).
 
 #### Withdraw fees
 

src/api/common/include/exchangepublicapi.hpp

@@ -29,7 +29,15 @@ class ExchangePublic : public ExchangeBase {
   static constexpr int kDefaultDepth = MarketOrderBook::kDefaultDepth;
   static constexpr int kNbLastTradesDefault = 100;
 
-  enum class MarketPathMode : int8_t { kStrict, kWithLastFiatConversion };
+  enum class MarketPathMode : int8_t {
+    // Only authorize conversions from real markets of the exchange.
+    // In particular, fiat conversions will be forbidden  if the fiat pair does not exist as a real market on the
+    // exchange.
+    kStrict,
+
+    // Authorize unique fiat conversion at one extremity of the conversion path (beginning or end, but not both).
+    kWithPossibleFiatConversionAtExtremity
+  };
 
   using Fiats = CommonAPI::Fiats;
 
@@ -59,8 +67,8 @@ class ExchangePublic : public ExchangeBase {
     MarketOrderBookMap marketOrderBookMap;
     Fiats fiats = queryFiats();
     MarketSet markets;
-    MarketsPath conversionPath =
-        findMarketsPath(from.currencyCode(), toCurrency, markets, fiats, MarketPathMode::kWithLastFiatConversion);
+    MarketsPath conversionPath = findMarketsPath(from.currencyCode(), toCurrency, markets, fiats,
+                                                 MarketPathMode::kWithPossibleFiatConversionAtExtremity);
     return convert(from, toCurrency, conversionPath, fiats, marketOrderBookMap, priceOptions);
   }
 
@@ -112,7 +120,7 @@ class ExchangePublic : public ExchangeBase {
   ///         or empty array if conversion is not possible
   /// For instance, findMarketsPath("XLM", "XRP") can return:
   ///   - XLM-USDT
-  ///   - XRP-USDT (and not USDT-XRP, as the pair defined on the exchange is XRP-USDT)
+  ///   - XRP-USDT
   MarketsPath findMarketsPath(CurrencyCode fromCurrencyCode, CurrencyCode toCurrencyCode, MarketSet &markets,
                               const Fiats &fiats, MarketPathMode marketsPathMode = MarketPathMode::kStrict);
 

src/api/common/include/fiatconverter.hpp

@@ -2,13 +2,16 @@
 
 #include <mutex>
 #include <optional>
+#include <string_view>
 #include <unordered_map>
 
 #include "cct_string.hpp"
 #include "curlhandle.hpp"
 #include "currencycode.hpp"
+#include "file.hpp"
 #include "market.hpp"
 #include "monetaryamount.hpp"
+#include "reader.hpp"
 #include "timedef.hpp"
 
 namespace cct {
@@ -32,10 +35,17 @@ class CoincenterInfo;
 /// Conversion methods are thread safe.
 class FiatConverter {
  public:
+  static File GetRatesCacheFile(std::string_view dataDir);
+
   /// Creates a FiatConverter able to perform live queries to free converter api.
   /// @param ratesUpdateFrequency the minimum time needed between two currency rates updates
   FiatConverter(const CoincenterInfo &coincenterInfo, Duration ratesUpdateFrequency);
 
+  /// Creates a FiatConverter able to perform live queries to free converter api.
+  /// @param ratesUpdateFrequency the minimum time needed between two currency rates updates
+  /// @param reader the reader from which to load the initial rates conversion cache
+  FiatConverter(const CoincenterInfo &coincenterInfo, Duration ratesUpdateFrequency, const Reader &reader);
+
   std::optional<double> convert(double amount, CurrencyCode from, CurrencyCode to);
 
   std::optional<MonetaryAmount> convert(MonetaryAmount amount, CurrencyCode to) {

src/api/common/src/exchangepublicapi.cpp

@@ -60,27 +60,29 @@ std::optional<MonetaryAmount> ExchangePublic::convert(MonetaryAmount from, Curre
   for (Market mk : conversionPath) {
     switch (mk.type()) {
       case Market::Type::kFiatConversionMarket: {
-        // should be last market
-        const bool isToCurrencyFiatLike = fiats.contains(toCurrency);
-        if (!isToCurrencyFiatLike) {
-          // convert of fiat like crypto-currency (stable coin) to fiat currency is only possible if the destination
-          // currency is a fiat. It cannot be done for an intermediate conversion
-          return std::nullopt;
-        }
         const CurrencyCode mFromCurrencyCode = from.currencyCode();
         const CurrencyCode mToCurrencyCode = mk.opposite(mFromCurrencyCode);
+
         const CurrencyCode fiatLikeFrom = _coincenterInfo.tryConvertStableCoinToFiat(mFromCurrencyCode);
         const CurrencyCode fiatFromLikeCurCode = fiatLikeFrom.isNeutral() ? mFromCurrencyCode : fiatLikeFrom;
+
         const CurrencyCode fiatLikeTo = _coincenterInfo.tryConvertStableCoinToFiat(mToCurrencyCode);
         const CurrencyCode fiatToLikeCurCode = fiatLikeTo.isNeutral() ? mToCurrencyCode : fiatLikeTo;
 
         const bool isFromFiatLike = fiatLikeFrom.isDefined() || fiats.contains(mFromCurrencyCode);
         const bool isToFiatLike = fiatLikeTo.isDefined() || fiats.contains(mToCurrencyCode);
 
-        if (isFromFiatLike && isToFiatLike) {
-          return _fiatConverter.convert(MonetaryAmount(from, fiatFromLikeCurCode), fiatToLikeCurCode);
+        if (!isFromFiatLike || !isToFiatLike) {
+          return std::nullopt;
         }
-        return std::nullopt;
+
+        const auto optConvertedAmount =
+            _fiatConverter.convert(MonetaryAmount(from, fiatFromLikeCurCode), fiatToLikeCurCode);
+        if (!optConvertedAmount) {
+          return std::nullopt;
+        }
+        from = *optConvertedAmount;
+        break;
       }
       case Market::Type::kRegularExchangeMarket: {
         const auto it = marketOrderBookMap.find(mk);
@@ -107,11 +109,13 @@ namespace {
 // Struct containing a currency and additional information to create markets with detailed information (order, market
 // type)
 struct CurrencyDir {
+  enum class Dir : int8_t { kExchangeOrder, kReversed };
+
   constexpr std::strong_ordering operator<=>(const CurrencyDir &) const noexcept = default;
 
   CurrencyCode cur;
-  bool isLastRealMarketReversed = false;
-  bool isRegularExchangeMarket = false;
+  Dir dir = Dir::kExchangeOrder;
+  Market::Type marketType = Market::Type::kRegularExchangeMarket;
 };
 
 using CurrencyDirPath = SmallVector<CurrencyDir, 3>;
@@ -122,7 +126,9 @@ class CurrencyDirFastestPathComparator {
 
   bool operator()(const CurrencyDirPath &lhs, const CurrencyDirPath &rhs) {
     // First, favor paths with the least number of non regular markets
-    const auto hasNonRegularMarket = [](CurrencyDir curDir) { return !curDir.isRegularExchangeMarket; };
+    const auto hasNonRegularMarket = [](CurrencyDir curDir) {
+      return curDir.marketType != Market::Type::kRegularExchangeMarket;
+    };
     const auto lhsNbNonRegularMarkets = std::ranges::count_if(lhs, hasNonRegularMarket);
     const auto rhsNbNonRegularMarkets = std::ranges::count_if(rhs, hasNonRegularMarket);
     if (lhsNbNonRegularMarkets != rhsNbNonRegularMarkets) {
@@ -158,13 +164,12 @@ MarketsPath ExchangePublic::findMarketsPath(CurrencyCode fromCurrency, CurrencyC
     return ret;
   }
 
-  const auto isFiatLike = [this, marketsPathMode, &fiats](CurrencyCode cur) {
-    return (marketsPathMode == MarketPathMode::kWithLastFiatConversion &&
-            _coincenterInfo.tryConvertStableCoinToFiat(cur).isDefined()) ||
-           fiats.contains(cur);
+  const auto isFiatConvertible = [this, marketsPathMode, &fiats](CurrencyCode cur) {
+    return marketsPathMode == MarketPathMode::kWithPossibleFiatConversionAtExtremity &&
+           (_coincenterInfo.tryConvertStableCoinToFiat(cur).isDefined() || fiats.contains(cur));
   };
 
-  const auto isToCurrencyFiatLike = isFiatLike(toCurrency);
+  const auto isToCurrencyFiatConvertible = isFiatConvertible(toCurrency);
 
   CurrencyDirFastestPathComparator comp(_commonApi);
 
@@ -181,22 +186,27 @@ MarketsPath ExchangePublic::findMarketsPath(CurrencyCode fromCurrency, CurrencyC
     if (visitedCurrencies.contains(cur)) {
       continue;
     }
+
     if (cur == toCurrency) {
       // stop criteria
       const int nbCurDir = path.size();
       ret.reserve(nbCurDir - 1);
       for (int curDirPos = 1; curDirPos < nbCurDir; ++curDirPos) {
         const auto curDir = path[curDirPos];
-        const auto marketType =
-            curDir.isRegularExchangeMarket ? Market::Type::kRegularExchangeMarket : Market::Type::kFiatConversionMarket;
-        if (curDir.isLastRealMarketReversed) {
-          ret.emplace_back(curDir.cur, path[curDirPos - 1].cur, marketType);
-        } else {
-          ret.emplace_back(path[curDirPos - 1].cur, curDir.cur, marketType);
+        switch (curDir.dir) {
+          case CurrencyDir::Dir::kExchangeOrder:
+            ret.emplace_back(path[curDirPos - 1].cur, curDir.cur, curDir.marketType);
+            break;
+          case CurrencyDir::Dir::kReversed:
+            ret.emplace_back(curDir.cur, path[curDirPos - 1].cur, curDir.marketType);
+            break;
+          default:
+            unreachable();
         }
       }
       return ret;
     }
+
     // Retrieve markets if not already done
     if (markets.empty()) {
       std::lock_guard<std::mutex> guard(_tradableMarketsMutex);
@@ -206,26 +216,40 @@ MarketsPath ExchangePublic::findMarketsPath(CurrencyCode fromCurrency, CurrencyC
         return ret;
       }
     }
-    bool alreadyInsertedTargetCurrency = false;
+
+    bool reachedTargetCurrency = false;
     for (Market mk : markets | std::views::filter([cur](Market mk) { return mk.canTrade(cur); })) {
-      const bool isLastRealMarketReversed = cur == mk.quote();
-      constexpr bool isRegularExchangeMarket = true;
+      const auto dir = cur == mk.quote() ? CurrencyDir::Dir::kReversed : CurrencyDir::Dir::kExchangeOrder;
       const CurrencyCode newCur = mk.opposite(cur);
-      alreadyInsertedTargetCurrency |= newCur == toCurrency;
+
+      reachedTargetCurrency = reachedTargetCurrency || (newCur == toCurrency);
 
       CurrencyDirPath &newPath = searchPaths.emplace_back(path);
-      newPath.emplace_back(newCur, isLastRealMarketReversed, isRegularExchangeMarket);
+      newPath.emplace_back(newCur, dir, Market::Type::kRegularExchangeMarket);
       std::ranges::push_heap(searchPaths, comp);
     }
-    if (isToCurrencyFiatLike && !alreadyInsertedTargetCurrency && isFiatLike(cur)) {
-      constexpr bool isLastRealMarketReversed = false;
-      constexpr bool isRegularExchangeMarket = false;
-      const CurrencyCode newCur = toCurrency;
 
-      CurrencyDirPath &newPath = searchPaths.emplace_back(std::move(path));
-      newPath.emplace_back(newCur, isLastRealMarketReversed, isRegularExchangeMarket);
-      std::ranges::push_heap(searchPaths, comp);
+    if (isFiatConvertible(cur)) {
+      if (isToCurrencyFiatConvertible && !reachedTargetCurrency) {
+        CurrencyDirPath &newPath = searchPaths.emplace_back(path);
+        newPath.emplace_back(toCurrency, CurrencyDir::Dir::kExchangeOrder, Market::Type::kFiatConversionMarket);
+        std::ranges::push_heap(searchPaths, comp);
+      } else if (path.size() == 1 && searchPaths.empty()) {
+        // A conversion is possible from starting fiat currency
+        for (Market mk : markets) {
+          if (fiats.contains(mk.base())) {
+            CurrencyDirPath &newPath = searchPaths.emplace_back(path);
+            newPath.emplace_back(mk.base(), CurrencyDir::Dir::kExchangeOrder, Market::Type::kFiatConversionMarket);
+            std::ranges::push_heap(searchPaths, comp);
+          } else if (fiats.contains(mk.quote())) {
+            CurrencyDirPath &newPath = searchPaths.emplace_back(path);
+            newPath.emplace_back(mk.quote(), CurrencyDir::Dir::kExchangeOrder, Market::Type::kFiatConversionMarket);
+            std::ranges::push_heap(searchPaths, comp);
+          }
+        }
+      }
     }
+
     visitedCurrencies.insert(std::move(cur));
   }
 

src/api/common/src/fiatconverter.cpp

@@ -45,22 +45,21 @@ string LoadCurrencyConverterAPIKey(std::string_view dataDir) {
   return std::move(freeConverterIt->get_ref<string&>());
 }
 
-File GetRatesCacheFile(std::string_view dataDir) {
-  return {dataDir, File::Type::kCache, kRatesCacheFile, File::IfError::kNoThrow};
-}
-
 }  // namespace
 
 FiatConverter::FiatConverter(const CoincenterInfo& coincenterInfo, Duration ratesUpdateFrequency)
+    : FiatConverter(coincenterInfo, ratesUpdateFrequency, GetRatesCacheFile(coincenterInfo.dataDir())) {}
+
+FiatConverter::FiatConverter(const CoincenterInfo& coincenterInfo, Duration ratesUpdateFrequency,
+                             const Reader& fiatsCacheReader)
     : _curlHandle1(kFiatConverterSource1BaseUrl, coincenterInfo.metricGatewayPtr(), PermanentCurlOptions(),
                    coincenterInfo.getRunMode()),
       _curlHandle2(kFiatConverterSource2BaseUrl, coincenterInfo.metricGatewayPtr(), PermanentCurlOptions(),
                    coincenterInfo.getRunMode()),
       _ratesUpdateFrequency(ratesUpdateFrequency),
       _apiKey(LoadCurrencyConverterAPIKey(coincenterInfo.dataDir())),
       _dataDir(coincenterInfo.dataDir()) {
-  const File ratesCacheFile = GetRatesCacheFile(_dataDir);
-  const json data = ratesCacheFile.readAllJson();
+  const json data = fiatsCacheReader.readAllJson();
 
   _pricesMap.reserve(data.size());
   for (const auto& [marketStr, rateAndTimeData] : data.items()) {
@@ -73,6 +72,10 @@ FiatConverter::FiatConverter(const CoincenterInfo& coincenterInfo, Duration rate
   log::debug("Loaded {} fiat currency rates from {}", _pricesMap.size(), kRatesCacheFile);
 }
 
+File FiatConverter::GetRatesCacheFile(std::string_view dataDir) {
+  return {dataDir, File::Type::kCache, kRatesCacheFile, File::IfError::kNoThrow};
+}
+
 void FiatConverter::updateCacheFile() const {
   json data;
   for (const auto& [market, priceTimeValue] : _pricesMap) {

src/api/common/test/exchangeprivateapi_test.cpp

@@ -88,7 +88,7 @@ class ExchangePrivateTest : public ::testing::Test {
   LoadConfiguration loadConfiguration{kDefaultDataDir, LoadConfiguration::ExchangeConfigFileType::kTest};
   CoincenterInfo coincenterInfo{settings::RunMode::kTestKeys, loadConfiguration};
   CommonAPI commonAPI{coincenterInfo, Duration::max()};
-  FiatConverter fiatConverter{coincenterInfo, Duration::max()};  // max to avoid real Fiat converter queries
+  FiatConverter fiatConverter{coincenterInfo, Duration::max(), Reader()};  // max to avoid real Fiat converter queries
 
   MockExchangePublic exchangePublic{kSupportedExchanges[0], fiatConverter, commonAPI, coincenterInfo};
   APIKey key{"test", "testUser", "", "", ""};