Earn = Revenue - Expenses ??

[moresearch]

Hello,

We're working with the SWE-Lancer benchmark and have a couple of questions regarding the "earn" metric and how to calculate costs.

1. "Earn" Definition: The paper mentions mapping model performance to monetary value, but it doesn't explicitly define what "earn" encompasses. Does it solely refer to the potential payout for successfully completed tasks, or does it also factor in aspects like the cost of computation and resources used by the AI model? A clearer definition would be very helpful for comparing different models and understanding their true economic impact.

2. Cost Calculation: How can we determine the number of tokens used and the associated cost for running each agent on a daily basis? Understanding the computational resources required is crucial for practical applications and for assessing the overall cost-effectiveness of different approaches. Could you provide some guidance or examples on how to calculate these costs?

Thanks in advance for clarifying these points! This will greatly improve our understanding and utilization of the SWE-Lancer benchmark.

[James4Ever0]

I would give the AI a crypto wallet, and if the balance in the wallet increases, that is income.

I would set a fixed cost rate, called "tax", which is adjustable by me.

You work with crypto. You know crypto works with AI right? Feel free to check Cybergod.