Work continues on the KoboldAI Horde apace, and in the past week I’ve added oauth auhentication but kept the anonymous access live as well. Let me tell you, figuring out how to use the flask.dance module for discord was a complete PITA.
But I got it done, and even added github authentication (and hopefully google soon, if they stop asking for silly things)
The authentication was added to allow me to build the next step, which is to be able to balance usage from the community.
The thing I expect to happen is that as the service becomes more popular, we might have way more clients trying to use the horde than we have actual contributing servers, and therefore we would run into performance bottlenecks. This could become unworkable especially if things like bots start using this service anonymously etc.
To provide some automatic balance, I have now introduced the concept of Kudos into the mix. A Kudos is a reward gained when someone contributes to the well-being horde in some way. There are two built-in methods to gain Kudos at the moment:
- Any time your KoboldAI server generates text for someone, you receive tokens based on the amount of text generated and the power of your model.
- KoboldAI servers get “uptime” rewards every 10 minutes they stay online and check-in looking for more requests. This incentivizes people leaving their instance running even in non-busy times.
On the flip-side, people using the Horde as clients are likewise consuming Kudos to do so. Every prompt you send, will reduce your balance by the same amount of Kudos rewarded to the server which fulfilled your request. Effectively you’re “thanking” the people running the instance which helped you.
But unlike typical currencies, you can still use this service even if you don’t have any Kudos. You simply go negative, which represents your “debt” to the community. This allows people to use the Horde anonymously as well (through the common Anonymous account) which simply keeps going more and more in negative Kudos.
But to avoid abusing this service by people who want to take and give nothing back, I needed to find a way to disincentivize negative kudos balances. And I just implemented the solution to do that which is a fairly simple one.
You kudos balance corresponds to your priority in the queue of requests
This means that if there’s plenty of servers around and not a lot of requests, nothing much changes, even for anonymous clients and they get to enjoy the full speed of the horde.
However on times of congestion, the requests sorted by kudos-order. The people with the most kudos get to have their requests fulfilled first, while those with the least, such as the Anonymous account, will get their generated last.
With some other tools provided such as bridge-based priorities, and server-owner-preference, this should allow owners of servers to make sure that not only “good” actors are the first to use their resources, but also to find their own ways to provide a service to selected members of the community. For example someone might set their family members to have priority on their server gens, even if those members are deep in negative kudos.
Along with that, I’ve deployed a kudos transfer service where peers can send kudos directly to others. So someone who has accumulated a large amount, might use that to reward good behavior in the community, such as writing documentation, or helping others. The kudos transfer can also happen via REST API, which can allow even automated tools to participate, such as discord bots which might trigger on special smileys assigned to posts and give a small kudos reward to the recipient.
There are of course many other things to consider, such as actions by malicious actors who might try to game the system. I really hope I won’t have to deploy countermeasures for people abusing community goodwill.
But until then, I really enjoy seeing the excitement from the KAI community, especially as heavy models like the fresh Erebus 13B are deployed into the horde. With the kudos infrastructure in place, there is now potential for new dynamics and reward systems to emerge organically from the community which will serve as an excellent guide on how to best proceed with this project.