Differences

This shows you the differences between two versions of the page.

--- librematch:backend [2022-10-10 19:30] – [Poller] simonsan
+++ librematch:backend [2022-10-23 21:19] (current) – removed nimda
@@ Line 1: / Line 1: @@
-====== LibreMatch Backend ======
-----
-===== Design =====
-==== Requirements ====
-=== User Stories ===
-Some things may be out of scope, we can still collect them here, to have an overview and include some of them in the design process.
-**User** stands for the user of the //LibreMatch// API.
-^As a/an ^I want to... ^So that.. ^
-| user | have a stable API | I can rely on it |
-| user | request win data of the games | I can easily determine the outcome of a match |
-| statistician | request database dumps for a time period | I can easily analyse a big amount of data without firing thousands of requests |
-| authenticated tournament organizer| delay matches showing up for a limited amount of time | I can take care that people are not being spoiled |
-| user | be able to get live updates about ongoing games | I can update my e.g. overlay immediately |
-| user | be able to log in with my Steam account | I can create lobbies from outside the game |
-| user | be able to query for matches of a player since a timestamp | I can update a list of matches of a certain user more easily |
-| user | be able to quickly start using the API | I am not spending much time reading the documentation or spend time implementing a client to use it |
-| user | be able to get all matches by a player from different games at once | I don't need to make several requests |
-| user | be notified of match state updates of all matches or a subset | I don't have to keep polling for updates and deriving state updates myself |
-| user | get notified when changes are made to a lobby | I can update my lobby tables and reduce the amount of requests |
-| recorded matches analyser | get notified when a new replay becomes available (e.g. in a leaderboard/from a player) | I can download and parse it in a timely fashion |
-| user | embed a player report function in my page | people can report suspicious activity (e.g. spoofing) to the official franchise |
-==== Additional Tooling ====
-== LibreMatch Proxy ==
-We will need a running instance of a Steam Client for each game. To circumvent limitations (as only holding one login to steam per account on a certain moment in time is needed) we use a proxy to forward our requests on behalf of a certain session.
-//Usage//
-Base URL: **https://rlink.api.tools.uber.space/relic**
-You need to get an //API_KEY// for the proxy from the team members.
-Then you can make calls to the proxy e.g. for endpoint `/game/news/getNews` call:
-<code>https://proxy.api.tools.uber.space/relic/game/news/getNews</code>
-You need to supply a header with the //API_KEY// e.g. <code>API_KEY=...</code>
-All query parameters that you add to the request will be forwarded to the Relic Link API on your behalf.
-== Insomnia  ==
-[[ https://insomnia.rest/ | Insomnia ]] is a API design platform.
-<WRAP center round download 60%>
-We have created some {{ :insomnia_librematchexamples.json | LibreMatch examples}} to use with their client.
-</WRAP>
-==== Architecture ====
-=== Introduction ===
-== Example (Bright Sky) ==
-[[ https://brightsky.dev/ | Bright Sky ]] pulls weather data from DWD and implements a JSON-API on top of it for its users.
-Their architecture looks like on the image on the right side. It's some kind of the same what we also want to achieve.
-{{:librematch:architecture_1_.png?400 |}}
-This is taken from the [[ https://github.com/jdemaeyer/brightsky/#architecture | Official repository ]]:
-  * The brightsky worker, which leverages the logic contained in the brightsky Python package to retrieve weather records from the DWD server, parse them, and store them in a database. It will periodically poll the DWD servers for new data.
-  * The brightsky webserver (API), which serves as gate to our database and processes all queries for weather records coming from the outside world.
-  * A PostgreSQL database consisting of two relevant tables:
-    *  sources contains information on the locations for which we hold weather records, and
-    * weather contains the history of actual meteorological measurements (or forecasts) for these locations.
-  * A Redis server, which is used as the backend of the worker's task queue.
-== LibreMatch (Draft) ==
-{{ ::librematch-arch-draft.png?400|}}
-Basically we should poll the Relic Link API endpoints for all data every so many seconds (this time is our minimum latency for our whole system).
-So the time frame in which we see certain information (Match infos, wins of games) and could deliver information to our API.
-Then we parse the results into a data structure of ourselves to make error handling easier and also deal with types of data and not handle everything as a huge JSON-Array that is being exposed by the Relic Link API. This data we send to the database which is our persistence layer. From our persistence layer we are able to request data for our endpoints that we expose to the users.
-Some endpoints may require a Pub/Sub API (e.g. live updates for match information), some are fine with the usual REST-API approach.
-{{:librematch:librematch_2022-10-09_3_.jpg?600|}}
-=== Data sources ===
-{{:librematch:data_sources.png?600 |}}
-There are a few available data sources (resource) we can query our data from.
-== Relic Link API ==
-  * Authenticated endpoints\\ Requests to these endpoints should be rather limited and only for really essential things.\\ Most of our data we should get from the /community-Endpoints.
-  * Community endpoints\\ Data here is relatively scarce, though most of our data we should fetch from here.
-== ageofempires.com API ==
-== Steamworks API ==
-=== Proxy ===
-{{:librematch:proxy.png?600|}}
-see [[librematch:backend#librematch_proxy|LibreMatch Proxy]]
-=== Collector ===
-{{:librematch:collector.png?600| }}
-== Polling ==
-== Parsing ==
-== Cache ==
-** Intermediate Cache **
-== Parser ==
-=== Database ===
-TODO: Create an overview over all data we can get from the RelicLink API.
-== Schema ==
-First step:
-[[ https://github.com/amacneil/dbmate | Version control our database schema with a lightweight, framework-agnostic database migration tool ]]
-== Postgres Statistics Extension ==
-To visualise long-running requests and loads we should use some statistics extension for Postgres.
-== Schema ==
-== Partitioning ==
-== Replication ==
-=== Backend ===
-== Authentication ==
-Sometimes resources will need authorisation to be accessed by privileged community members. Such members could be tournament organizers and admins that want to setup a schedule within the backend to delay the access to certain player information for a limited amount of time. This could be needed during tournaments, so people aren't able to find out results of matches before they are officially concluded (e.g. in case of casting recorded games) and spoil other people with the results.
-The Relic Link ID aka RLINK_ID is a unique identifier for every player on the Relic Link platform, no matter if the person bought a game on Steam or Microsoft Store. Authentication for a player on that platform could be achieved by utilising [[ https://partner.steamgames.com/doc/features/auth#website | web browser based authentication with Steam OpenID ]].
-== REST-API ==
-** OpenAPI **
-To make it easy to start using our API we should provide an [[ https://en.wikipedia.org/wiki/OpenAPI_Specification | OpenAPI file ]] so that the users are able to generate clients for our API for their favourite programming language. An [[ https://swagger.io/ | OpenAPI ]] file also works well as a complete documentation of the API. There are several [[ https://openapi.tools/ | Tools ]] that make dealing with an OpenAPI a breeze.
-TODO: [[ https://github.com/tokio-rs/axum/pull/459 | Generating OpenAPI ]] files from within [[ https://github.com/tokio-rs/axum | axum ]] is in the making.
-== Pub/Sub-API ==
-Some information for players might want to be constantly queried by the users of our API, for example match information. These information we should supply via an endpoint that can be subscribed to and communicates via a Websocket.
-==== Implementation ====
-==== Roadmap ====