-
Notifications
You must be signed in to change notification settings - Fork 0
Framework introduction – Marketplace
This page will give an in-depth overview over the way different marketplaces are modelled in the recommerce simulation framework.
- Overview
- Action and State
- Step
- Episode
- Customer
- Owner
- Market Types
- Market Environments
- Available marketplace classes
The general marketplace is represented as a series of actions and states. A state describes the current condition of the marketplace, whereas each marketplace participant (namely the vendors performs one action during each step of the simulation, influencing the subsequent state(s). After a configurable amount of steps the marketplace gets reset, meaning most of its properties get randomly shuffled. Such a collection of steps is called an episode, which is the main way of configuring the length of the simulation.
In our framework, an action is the process of a vendor setting a price for one its products. In the most complex of marketplaces, the circular economy with rebuy prices, the action is comprised of three separate prices:
- Price for the new product
- Price for the refurbished product
- Price at which used products are bought back from owners
A state represents the current situation of the marketplace. In every step each vendor performs an action which modifies the state of the next step as well as the remaining step phase.
In order allow our vendors to perform informed actions, they need to know certain details about the current market state, most importantly how many products they sold to customers during the previous step, which allows them to find optimal prices.
For this, certain, configurable parts of the complete state are passed to the vendors before they take an action. In our framework, we call this slice of the current state the vendor's observation of the market state. How much information is provided to each vendor can be configured in the market_config.json file at the start of the simulation.
A step is divided into a number of intervals. Within each interval, one vendor is active. The order at which the different vendors act is given by the order they were inserted into the marketplace. For example, in the training task, the trained agent will be the first to take an action, followed by its competitors. Within its interval, a vendor sets prices based on its policy. Consequently, the state will be updated for the following vendors.
At the end of an interval, first the customers and then the owners arrive and make their respective decisions. The amount of customers arriving during each interval is given by the equation total_number_of_customers_per_step / number_of_vendors. The (re)sales done during this interval are added to the total profit the vendor makes during this step. In the case of a training RL-agent, the profit will then influence the agent's policy for subsequent steps.
An episode is made up of a larger number of steps. This so-called episode length is configured in the market_config.json file, through the episode_length parameter. At the end of an episode, the market state is reset, meaning most of its properties get randomly shuffled. All major metrics, such as profits and sales numbers, are also recorded on a per-episode basis.
Customers are part of the marketplace itself. Unlikely vendors, customers fill a more passive role. At the end of each interval within a step, a configurable amount of customers arrives at the marketplace to make purchasing decisions. Each customer has the option of either buying any one new or refurbished product, or to buy no product at all.
All purchasing decisions are modelled through probability distributions. For this, customers first assign each available product a preference ratio. These ratios are then normalized, and a multinomial distribution is used to draw samples. For specific formulas about the implementation of the customers decision-making process, see this bachelor's thesis.
TODO: More thesis links!
After the customers have made their purchasing decisions, owners of current products (bought during one of the previous steps) are given the option to sell their now used products back to one of the vendors. As of now, owners as well as customers are memoryless, meaning owners do not know the price at which they bought their product.
During each step, only a small percentage of owners are given the option to re-sell their product, as in the real market, many owners will only consider reselling their product after a certain amount of time has passed. If an owner decides to sell their product back to one of the vendors, the vendor will pay the respective price they set during that step and add the item to their inventory. In subsequent steps, the item can be sold again as a refurbished product.
Similar to how the customers work, the owners will first calculate preference ratios for the different rebuy prices offered by the vendors. In addition, all owners have two preference ratios for holding or throwing away their products. The ratios are again normalizd and used as input for a multinomial distribution to draw samples from.
Our simulation framework features three major market types, explained in the sections below. When referencing market types, we refer to the amount of prices the vendors set in that market. On the other hand, the amount of vendors present in the market is given by the respective market environment.
At the bottom of this section, a list of all current (non-abstract) marketplace classes in the framework is given.
In the classic linear economy products are sold once and then discarded by their owners. This market type is the least complex and least refined of the three market types we implemented, and does not see much use. In this market type, vendors set only one price, the new price.
In the circular economy vendors sell new products and refurbished products. In this market type, owners give their product back to the vendors for free, meaning the vendors do not pay for the used products. This means that vendors set two prices: The new price and the refurbished price. This market type is an intermediary and simpler version of the third market type.
As the name says, this third market type is a more complex version of the previous one. Here, vendors need to set three prices at once: The new price, refurbished price and rebuy price, at which used items are bought back from owners. This is the market type we primarily use in the simulation framework, as it is closest to what we would expect from a real market.
All of the above market types can be used within different market environments in our simulation framework. The market environment defines the amount of vendors present in the simulation. There are three different market environments we implemented:
This market environment consists of a single vendor. All customers and owners interact with this vendor, and there is no competition.
In this market environment, there are two vendors directly competing with each other. Customers and owners can choose to buy and sell products from any one of them.
In an oligopoly environment, the amount of vendors present can be freely configured by the user interacting with the framework, simply through the amount defined in the respective environment_config.json file.
This section includes all current (non-abstract) marketplace classes that are available for use in the framework. Please note that marketplaces are only compatible with a subset of vendors. As this list can change over time, we add a timestamp:
This list was last updated on: 24/07/2022.
LinearEconomyMonopolyLinearEconomyDuopolyLinearEconomyOligopoly
CircularEconomyMonopolyCircularEconomyDuopolyCircularEconomyOligopoly
CircularEconomyRebuyPriceMonopolyCircularEconomyRebuyPriceDuopolyCircularEconomyRebuyPriceOligopoly
Online Marketplace Simulation: A Testbed for Self-Learning Agents is the 2021/2022 bachelor's project of the Enterprise Platform and Integration Concepts (@hpi-epic, epic.hpi.de) research group of the Hasso Plattner Institute.