feat: simplify special asset pairs calculations

CHANGELOG.md

@@ -46,6 +46,10 @@ Ref: https://keepachangelog.com/en/1.0.0/
 
 ## Unreleased
 
+### Bug Fixes
+
+- [2315](https://github.com/umee-network/umee/pull/2215) Improve reliability of MaxBorrow, MaxWithdraw when special asset pairs present.
+
 ### Improvements
 
 - [2299](https://github.com/umee-network/umee/pull/2299) Upgrade Cosmos SDK to v0.47.

x/leverage/EXAMPLES.md

@@ -38,76 +38,76 @@ Once special asset pairs are taken into account, the position Behaves As:
 | ---------- | ------ | ------------------- |
 | $40 A      | $20 B  | 0.5 (special)       |
 | $50 A      | $20 C  | 0.4 (special)       |
-| $10 A      | $1  D  | 0.1 = min(0.1, 0.4) |
-| $40 D      | $4  D  | 0.1                 |
-| $260 D     | -      | 0.1                 |
+| $10 A      | -      | 0.4                 |
+| $300 D     | -      | 0.1                 |
+| -          | $20 D  | 0.1                 |
 
-Note that the position is arranged above such that an asset prefers to be in the highest row it can occupy (hence the unused collateral at the bottom, as all borrows filled in from the top).
-It also reflects an order of "special pairs then regular assets; both categories sorted by collateral weight from highest to lowest" to maximize efficiency.
+For special pairs, the position is arranged above such that an asset prefers to be in the highest row it can occupy (as long as there is enough of the other asset to complete the pair).
+Unpaired assets are the remainder after special pairs are created.
 
 Suppose I wish to compute `MaxBorrow(B)` on this position.
-Naively, I would simply see how much `B` can be borrowed by the unused collateral `D` at the bottom row.
-This would consume `$260 D` collateral at a weight of `0.1` for a max borrow of `$26`.
+Naively, I would simply see how much `B` can be borrowed in addition to the existing borrowed `D` by the unused collateral `D` and `A` at the bottom row.
+
+This would consume `$140 D` collateral at a weight of `0.1` for a max borrow of `$14`.
 However, this actually underestimates the max borrow amount because asset `B` qualifies for a special asset pair.
 
+(Note that `$10 A * 0.4` and `$160 D * 0.1` can cover a borrow of `$20 D` using their collateral weights, which is why `$140 D` collateral is the maximum we used above.)
+
 What will actually happen, is any newly borrowed `B` will be paired with collateral `A` in the highest priority special asset pair (and also any collateral `A` that is floating around in regular assets) before being matched with leftover collateral.
-First it will take from the `$10 A` sitting in normal assets (and displace the `$1 D` which was being covered by that collateral onto the unused collateral at the bottom).
-If the `$1 D` could only be partially moved due to a limited amount of unused collateral, we would compute the amount of `A` collateral that would be freed up, and the resulting size of the `B` max borrow, and return there.
-(This logic is a recursive `MaxWithdraw(A)`)
 
-Position after first displacement of collateral `A`:
+First it will take from the `$10 A` sitting in normal assets.
+
+If the `$10 A` could only be partially used due to a limited amount of unused collateral, we would compute the amount of `A` collateral that would be freed up, and the resulting size of the `B` max borrow, and return there.
+
+Position after first pairing of new borrow `B` with collateral `A`:
 
 | Collateral | Borrow | Weight            | Change in Position |
 | ---------- | ------ | ----------------- | ------------------ |
 | $50 A      | $25 B  | 0.5 (special)     | +$10 A       +$5 B |
 | $50 A      | $20 C  | 0.4 (special)     |                    |
-| $0 A       | $0 D   | -                 | -$10 A       -$1 D |
-| $50 D      | $5 D   | 0.1               | +$100 D      +$1 D |
-| $250 D     | -      | 0.1               | -$100 D            |
+| $0 A       | -      | -                 | -$10 A             |
+| $300 D     | -      | 0.1               |                    |
+|            | $20 D  | 0.1               |                    |
+
+But there is still unused borrow limit available after the step above, so the `B` looks for any more collateral `A` that can be moved to the topmost special pair.
+
+Since the existing `$20 D` borrowed can be covered by `$200 D` collateral at a weight of `0.1`, an additional `$100 D` can borrow `$10 B`.
+
+The total max borrow returned by the leverage module will be `$5B` (special) `+ $10B` (normal) = `$15`. This is greater than the original `$14`.
+
+However, there is an edge case present here: repeating `MaxBorrow(B)` would displace assets from an existing special pair to one of higher weight, increasing the total borrowed further.
+
+We could take collateral `A` from the special pair `($50 A, $20 C)` since that pair has lower weight than `A <-> B`.
 
-But there is still unused collateral available after the step above, so the `B` looks for any more collateral `A` that can be moved to the topmost special pair.
-This time, it takes collateral `A` from the special pair `($50 A, $20 C)` since that pair has lower weight.
 Again, this displaces borrowed `C` which must find a new row to land in.
 The displaced `C` first looks for lower weight special asset pairs that allow borrowed `C` (and finds none), then attempts to insert itself into the ordinary asset rows.
-Since `C` has a higher collateral weight than `D`, it displaces all borrowed `D` to lower rows.
-If the displacement fills all unused collateral before completing, returns with the amount of newly borrowed `B`.
-This is still part of the recursive `MaxWithdraw(A)` mentioned above, since it is matching existing collateral `A` with new borrowed `B`, effectively withdrawing the `A` from the rest of the position.
 
-Position after second displacement of collateral `A`:
+Position after hypothetical displacement of collateral `A` from special pair:
 
-| Collateral | Borrow | Weight            | Change in Position |
-| ---------- | ------ | ----------------- | ------------------ |
-| $100 A     | $50 B  | 0.5 (special)     | +$50A        +$25B |
-| $0 A       | $0 C   | -   (special)     | -$50A        -$20C |
-| $200 D     | $20 C  | 0.1               | +$200D       +$20C |
-| $50 D      | $5 D   | 0.1               |                    |
-| $50 D      | -      | 0.1               | -$100 D            |
-
-Note that the `(A,C, 0.4)` special pair which was used is now unused, as its collateral was moved to the more efficient pair `(A,B,0.5)`.
-There is still a little left over collateral `D`, so with all the special pairs dealt with, the ordinary assets can be settled.
-Due to the rule "borrowed assets are listed by collateral weight, descending" any remaining borrow `B` will insert itself below rows containing borrowed `A`, but above any rows containing borrowed `C` or `D`.
-This functions as a recursive `MaxWithdraw(D)` from this example position since only `D` collateral is being affected.
-Position after final displacement of collateral `D`:
+| Collateral | Borrow  | Weight            | Change in Position |
+| ---------- | ------- | ----------------- | ------------------ |
+| $75 A      | $37.5 B | 0.5 (special)     | +$25A      +$12.5B |
+| $25 A      | $10 C   | -   (special)     | -$25A        -$10C |
+| $300 D     | -       | 0.1               |                    |
+| -          | $10 C   | 0.3               |              +$10C |
+| -          | $20 D   | 0.1               |                    |
 
-| Collateral | Borrow | Weight            | Change in Position |
-| ---------- | ------ | ----------------- | ------------------ |
-| $100 A     | $50 B  | 0.5 (special)     |                    |
-| $50 D      | $5 B   | 0.1               | +$50D         +$5B |
-| $200 D     | $20 C  | 0.1               |                    |
-| $50 D      | $5 D   | 0.1               |                    |
-| $0 D       | -      | 0.1               | -$50D              |
+Note that the `(A,C, 0.4)` special pair has been cut in half, as some of its collateral was moved to the more efficient pair `(A,B,0.5)`.
 
-The position in the table above can be found at `TestMaxBorrowScenarioA` in the [unit tests](./types/documented_test.go).
+The `C` could not be fully displaced into normal assets because `$300 D` and only support a total of `$30` borrowed value at its weight of `0.1`.
 
-Since this position had only `D` collateral under ordinary assets, the displacement is simple.
-In a mixed position, borrows are actually being bumped down one row at a time until the bottom row (unused collateral) has been filled up.
+Still, the resuling max borrow, if this procedure were to be implemented in the module, would be `$17.5 B`.
+
+The position in the table above can be found at `TestMaxBorrowScenarioA` in the [unit tests](./types/documented_test.go).
 
 Overall Result:
 
-- Initial displacement of collateral A added $5B borrows
-- Second displacement of collateral A added $25B borrows
-- Final displacement of collateral D added $5B borrows
+- Borrow limit without special assets would be `$74`, so max borrow would be (74 - 60) = `$14`
+- By moving normal assets to special pairs, leverage module can increase max borrow to `$15`
+- A more perfect algorithm would rearrange special pairs to give a result of `MaxBorrow(B) = $17.5`.
 
-Therefore `MaxBorrow(B) = $25 + $5 + $5 = $35`.
+Practical notes on the edge case:
 
-This is greater than the naive estimate of `$26` from the start of the example.
+- Max borrow will only be underestimated if an existing special pair can be cannibalized by new borrows into a higher weighted special pair.
+- A user can approach the theoretical limit by executing multiple max borrow transactions in a row without doing any calculations.
+- They can also use a `MsgBorrow($17.5B)` directly if they know the final amount.

x/leverage/README.md

@@ -221,16 +221,15 @@ A user's borrow limit is the sum of the contributions from each collateral they
 
 The full calculation of a user's borrow limit is as follows:
 
-1. Calculate the USD value of the user's collateral assets, using the _lower_ of either spot price or historic price for each asset. Collateral with missing prices is treated as zero-valued.
-2. Calculate the USD value of the user's borrowed assets, using the _higher_ of either spot price or historic price for each asset. Borrowed assets with missing prices cause any transaction which could increased borrowed value or decrease borrow limit to fail.
+1. Calculate the USD value of the user's collateral assets, using the _lower_ of either spot price or historic price for each asset. Collateral with missing prices is treated as zero-valued when attempting to borrow new assets or withdraw collateral, but will block any liquidations until collateral price returns.
+2. Calculate the USD value of the user's borrowed assets, using the _higher_ of either spot price or historic price for each asset. Borrowed assets with missing prices block any new borrowing or withdrawing of collateral, but are trated as zero valued during liquidations.
 3. Sort all `Special Asset Pairs` with assets matching parts of the user's position, starting with the highest `Special Collateral Weight`.
 4. For each special asser pair, match collateral tokens with borrowed tokens until one of the two runs out. The matched amounts satisfy `Collateral Value (A) * Special Collateral Weight (A,B) = Borrowed Value (B)` for each special asset pair `[A,B,CW]`. Subtract the collateral and borrowed tokens from the user's remaining position.
-5. Then sort the user's remaining collateral tokens by `Collateral Weight` and sort their remaining borrowed tokens by  the same.
-6. Starting with the highest collateral weight in each list, match collateral tokens with borrowed tokens until either collateral or borrowed tokens are exhausted. The matched amounts satisfy `Collateral Value (A) * Minimum Collateral Weight (A,B) = Borrowed Value (B)`
-7. If collateral tokens remain after the matching is complete, then the user has borrowed less than their borrow limit. `Borrow Limit = Borrowed Value + sum(collateral value * collateral weight)` summed over all remaining collateral tokens.
-8. If borrowed tokens remain after the matching is complete, then the user has borrowed more than their borrow limit. `Borrow Limit = Borrowed Value - sum(borrowed value)` summed over all remaining borrowed tokens.
+5. Then sum the `CollateralValue * CollateralWeight` for each unpaired collateral token, and subtract the sum of `BorrowedValue` for each unpaired borrow token. This value is the user's unused borrow limit (and is negative if they are over limit.)
+6. Also sum `CollateralValue` for each collateral token, and subtract the sum of `BorrowedValue / max(0.5,CollateralWeight)` for each borrowed token. This value is the user's unused collateral according to borrow factor (and can also be negative, in which case it should be multiplied by the weighted average collateral weight of the collateral to reflect actual usage).
+7. The user's current borrowed value, plus the lower of their unused borrow limit or unused collateral, is their borrow limit.
 
-Note that the borrow limit described in step 7 is the user's ideal borrow limit, their maximum borrowed value if all additional borrowed tokens had collateral weight equal to the weight of the remaining collateral.
+Note that the result of step 7 is the user's ideal borrow limit, their maximum borrowed value if all additional borrowed tokens had collateral weight greater than or equal to the weight of the remaining collateral, so as not to be limited by borrow factor.
 When borrowing tokens with inferior `Borrow Factor`, the user's actual borrow limit will be lower.
 
 #### Example Borrow Limit Calculation
@@ -245,54 +244,18 @@ When borrowing tokens with inferior `Borrow Factor`, the user's actual borrow li
 > Collateral: $20 ATOM (0.6) + $20 UMEE (0.35)
 > Borrowed: $20 ATOM (0.6)
 >
-> Then collateral is matched with borrowed assets starting with the highest weights. First, $20 ATOM collateral is matched with $12 borrowed ATOM. Then, $20 UMEE collateral is matched with $7 borrowed ATOM.
-> Collateral is now exhausted, and $1 borrowed ATOM remains. Thus the user is $1 above their borrow limit. Their borrow limit must be $50 - $1 = $49.
+> Using collateral weights on the unpaired position, `$20 * 0.6 + $20 * 0.35 = $19`, and `$19 - $20 = $-1`, so the user's borrow limit is one dollar less than their current total borrowed value. `BorrowLimit = $49`.
+>
+> Using borrow factor on the unpaired position, `$20 + $20 - ($20 / 0.6) = $6.67`, which is greater than `$-1` so borrow factor does not cause any additional restrictions.
 
 #### Max Borrow
 
 This calculation must sometimes be done in reverse, for example when computing `MaxWithdraw` or `MaxBorrow` based on what change in the user's position would produce a `Borrow Limit` exactly equal to their borrowed value.
-The result of these calculations will vary depending on the asset requested, and where its collateral weight would be sorted in the lists mentioned in step 5, or if it is part of any special pairs.
+The result of these calculations will vary depending on the asset requested, or if it is part of any special pairs.
 
 #### Example Max Borrow Calculation
 
-> Assume the following collateral weights: AKT 0.3, BNB 0.4, CSMT 0.5, DOT 0.6, ETH 0.7, hereby abbreviated as denoms `A,B,C,D,E` and special asset pairs [AKT, BNB, 0.5] and [CSMT, DOT, 0.8] abbreviated as `[A,B,0.5]` and `[C,D,0.8]`. We will calculate the `MaxBorrow(B)` of a borrower with the following existing position:
->
-> Collateral: $20 A, $20 B, $50 C, $20 D, $30 E
-> Borrowed: $5 B, $45 D
->
-> The new borrow of B will appear here on the user's position (ordered by by special pairs first, then collateral weight from highest to lowest, as the algortihm would match assets):
->
-> | Collateral | Borrowed | Effective Collateral Weight |
-> | - | - | - |
-> | $50 C | $40 D | 0.8 |
-> | $10 A | $5 B | 0.5 <--- 1st insertion |
-> | $8.33 E | $5 D | min(0.7,0.6) |
-> | $41.66 E | - | <--- 2nd insertion |
-> | $20 D | - | <--- 3rd insertion |
-> | $50 C | - | <--- 4th insertion |
-> | $20 B | - | <--- 5th insertion |
-> | $10 A | - | <--- 1st deletion |
->
-> A new borrow of B will be matched with some collateral of A (row marked `1st Deletion`) and add an additional $10 A, $5 B to an existing special pair (`1st insertion`). Then, it will match with unused collateral (`2nd - 5th insertion`).
-> The resulting sorted position would be:
->
-> | Collateral | Borrowed | Effective Collateral Weight |
-> | - | - | - |
-> | $50 C | $40 D | 0.8 |
-> | $20 A | $10 B | 0.5 |
-> | $8.33 E | $5 D | min(0.7,0.6) |
-> | $41.66 E | $16.66 B | min(0.7,0.4) |
-> | $20 D | $8 B | min(0.6,0.4) |
-> | $50 C | $20 B | min(0.5,0.4) |
-> | $20 B | $8 B | min(0.4,0.4) |
->
-> Since the borrowed amount of B increased from $5 to ($10 + $16.66 + $8 + $20 + $8) = $62.66, we determine that `MaxBorrow(B) = $57.66` (and then convert from dollars back to tokens in queries.)
-> Note that the calculation first had to locate the collateral A which would be moved from its regular row to a special asset row (and would have done so even if that meant orphaning some collateral that was previousy matched with it or a borrow from a lower priority special pair with collateral A)
-> After such displaced assets are dealt with, including chain reactions, remaining borrowed B is inserted into the regular rows.
-> It cannot bump borrowed assets with a greater or equal collateral weight, but will displace lower-weighted borrows down to the bottom, then fill all emptry rows.
-
-The computation above for max borrow will behave differently for different tokens, given the presence or absence of special asset pairs and the collateral weight of the new borrow and the existing borrows being displaced.
-It will abort and return zero if all collateral is in use.
+See [EXAMPLES.md](./EXAMPLES.md) for an example of a max borrow calculation, including an edge case that would cause the module to return less than the true maximum after special pairs.
 
 #### Liquidation Threshold