Self-Governing Hybrid Societies and Deception

Throughout history, deception has played significant roles in shaping societies, and some would even argue that deception influenced the rise and fall of entire civilisations. In political competition, deception plays a crucial role as a strategy for both individuals and groups [21]. If too many individuals select deceptive strategies to communicate preferences, then an ecosystem emerges where preferences of people are distorted on a public level, e.g., an echo chamber. This can lead to undesired group outcomes, inconsistent with individual preferences. Think of when every person declares publicly false preferences. In turn, the incremental declaration of false preferences is treated by the group as true preferences. Hence, the group becomes self-deceived by commonly held beliefs. This feeds back into the individual as peer-pressure, leading to mob behaviour and potentially undesired outcomes. Unfortunately, organised groups can exploit this phenomenon [21].

On a societal level, deception takes the form of disinformation—the deliberate (intentional) propagation of false information in order to mislead others for various motives [6, 7]. It has been argued that no matter whether deception is well intended or not, it might eventually lead to erosion of trust [3]. In the case of self-organising societies, for instance, cooperation is destabilised when deception is present, which then prevents systems from organising themselves against external and internal threats. On the other hand, deception can stimulate self-organisation by building public support towards one’s goals, e.g., by exaggerating threats to democracies posed by other nations or by performing provocative diplomatic actions [38].

Often, deception has also been used as a governmental defensive mechanism, either through strong control and censorship of information flows in totalitarian regimes, or as a strategical offensive mechanism through deceptive military intelligence operations [26].

In the legal domain, deception has been extensively regulated, e.g., laws against perjury or false statements under oath, and, some argue, even used to establish national identities and constitutional rights, by playing with the human tendency of forming groups when presented with a constructed national identity, or by adhering to a fictitious religious belief [47].

We can safely infer from the vast literature that deception is and has been pervasive in human society throughout history. Nowadays, however, societies are confronted with a pervasiveness of deception never seen before [2], due to a combination of factors that influence the dynamics of a very complex system of information dissemination and consumption, which we call the Infosphere [8]. The existence of the Infosphere implies the existence of hybrid societies, where multiple types of agents, both human and artificial, interact. Due to the advancement of AI, the Infosphere is threatened by several deception risks, some of which are deepfakes, ChatGPT, trollbots, and so on [28, 44].

What does this mean for the future of hybrid societies when faced with fully autonomous AI agents which will be able to learn and develop their own methods for deceiving? Will societies break down, or will they somehow manage to govern themselves, promote cooperation, and become resilient to deceptive attacks? In this article, we answer these questions using large-scale evolutionary agent-based simulations, similarly used to analyse agent societies in References [1, 40, 43, 46], and interpret the results from the perspective of Machine Behaviour (MB) [37].

To understand how machines behave in the wild (e.g., in open multi-agent systems, on the Web, on the Infosphere [8]), the MB approach can be used instead of mainstream work on AI [37]. Rather than aiming to maximize or optimize algorithmic performance of AI agents, MB focuses on defining measures of micro and macro outcomes to answer broad questions such as how machines behave in different environments and whether human interactions with agents alter societal outcomes. MB allows researchers to interpret the actions of intelligent agents as part of a wider self-organising ecosystem that ranges from the technical aspects that underlie the design of AI agents to the security constraints that govern interactions between agents (humans and machines alike).

In the context of self-organising systems [22], MB can be used to describe the overall behaviour of machines and humans as an ecosystem. An example of how MB can be applied is given by [43] where it is used to interpret how deception in large complex systems can lead to the Tragedy of the Digital Commons (TDC).¹ In self-organising systems, TDC is reached because self-interested agents adopt the non-cooperative behaviour of free-riding by deciding to defect from contributing to the public good (a resource) while exploiting it. However, as Ostrom argued [31, 32], cooperative behaviour can re-emerge if agents manage to establish institutional rules that govern their interactions.

Assuming that deceptive agents seem cooperative to others, while in reality their strategies are not visible [4], how do we investigate and deal with them? Furthermore, how do we avoid a society where agents adopt a free-riding behaviour, whether that is deception or defection? Will humans and machines eventually manage to establish governing rules in the face of deception?

To answer such questions, we must look into the socio-cognitive factors that influence deceptive interactions of self-organising hybrid societies. Regarding the deception capabilities of AI agents, one must consider multiple levels of inter-related social skills, according to Castelfranchi’s perspective on trust and deception [5]. Moreover, if we are to assume that machines will adopt humans’ strategies of social interaction, we must also take into account agents’ different capacities of learning from each other. This type of learning is equivalent to social learning, where humans imitate the behaviour of other humans to better adapt to social contexts. Indeed, the literature in evolutionary agent-based modelling (ABM) shows how social learning influences the evolution of cooperation [1, 43, 46]. From an evolutionary perspective, it is particularly interesting how stronger levels of social learning can lead self-organising agent societies to select various mechanisms that promote cooperation in the face of free-riding. Such studies have looked at multi-agent interactions which represent some form of public goods games (PGGs). In PGGs, agents can either choose to be cooperative and contribute to maintain the public good, or free-ride and exploit it. Sarkadi et al. adopted the perspective that knowledge itself can be a public good that is maintained and exploited by agents through either the curation and sharing of honest information, or by not contributing to the knowledge (not sharing information) or polluting it with false information [43]. We will use the same principle to model our multi-agent interactions.

Taking socio-cognitive factors into consideration, the authors in Reference [43] showed that for interactions like PGGs, cooperation can be re-established in the face of deception for strong social learning (\(s\rightarrow \infty\)) if the right self-regulatory mechanisms are in place. This means cooperation can be re-established if agents do not make any mistakes when adapting and learning—a very strong assumption. The mechanism that they identified was called Peer-Hybrid Interrogation, which combines the peer-punishment of Defectors with the interrogation and peer-punishment of potential Deceivers.

Following this line of work, we design an agent-based model of socio-cognitive dynamics of deception, and empirically test this model through extensive simulations, in order to answer two main questions:

To answer Q1 and Q2, we start by reproducing the voluntary PGGs in Reference [43] for both strong and weak conditions of social learning as a baseline. Social learning can be weak/intermediate (a stochastic process that takes a fixed value for \(s \in [0,\infty)\)), or strong (a deterministic process where \(s = \infty\)) [46]. In the weak/intermediate stochastic process, the higher the value for \(s\), the stronger the tendency of adopting the better strategy. In the deterministic process, when \(s = \infty\), the agents will always adopt the better strategy and when \(s \rightarrow 0\), a coin toss decides whether the better strategy is adopted.

After that, we extend this evolutionary framework to introduce a new type of PGG, namely PGG*, to better represent and capture the socio-cognitive dynamics of deception and deception detection. We do this by changing the cognitive functions for computing trust and the payoffs for Deception and Peer-Hybrid Interrogation; we also add a reward in the Peer-Hybrid Interrogator’s PGG payoff. Doing so, we (i) redefine the trust model to take into account the existence of Deceivers, not just Defectors; (ii) based on the new trust model, capture the more realistic interactions where Deceivers need to take into account the possibility of deceiving other Deceivers or being deceived themselves; and (ii) design a decentralised reward mechanism for the Peer-Hybrid Interrogators.

After defining PGG*, we run extensive simulations to explore the long-run frequencies and temporal dynamics of a hybrid society where agents interact according to the new game. Differently from Reference [43], we show that in PGG* cooperation can be re-established even for weak/intermediate social learning.

Subsequently, we extend the methodology for studying deception in agent-based simulations, by performing a resilience test where deception attacks on the entire population are performed at fixed intervals. Since we are modelling a hybrid society, we are referring here to the term resilience as the resilience of a society to recognise [36], but also respond, to deception. In our model, the concept of resilience takes the form of organisational and ecological resilience in complex systems [9].

Finally, we discuss our findings considering Q1 and Q2, and we contrast and compare them with existing and future work on deceptive AI from the lens of MB.

The work in Reference [43] introduced deception in the evolutionary PGG literature as a new free-riding strategy and explored six PGG setups to test what kind of punishment strategies are best able to deal with deception. The six PGGs were the following:

The assumptions made in Reference [43] were the following, namely (1) that a population of agents can self-organise according and change types according to the available strategies, (2) that the frequency of adopted strategies by the agents change according to the process of mutation and social learning, (3) that the PGG interactions are voluntary, i.e., there exists the Loner strategy, and (4) that social learning can either be strong or weak/intermediate. By strong social learning, it is understood that an agent compares the payoff given by its current strategy with another agent’s strategy and will copy the other agent strategy only if the other agent’s payoff is higher than its own. By weak social learning, it is understood that agents sometimes make mistakes, and that according to a stochastic process agents compare their strategies with others’ strategies, however they will not be always capable of distinguishing the difference in payoff. In this weak social learning process, a social learning parameter takes a value represented by real number. The higher the value of that number, the stronger the ability of the agent to distinguish between the payoffs of the strategy.

The findings in Reference [43] showed that cooperation can be re-established and deception can be best dealt with only in the sixth PGG (with Peer-Hybrid Interrogators) and only under the assumption of strong social learning. That is, agents were not allowed to make mistakes when choosing the better strategy in the social learning process. According to their findings, the strategy able to deal with deception and defection is the one of Peer-Hybrid Interrogators. This type of strategy combines the peer-to-peer punishment of defectors strategy with a peer-to-peer punishment and interrogation strategy of deceivers. It is different from a pool-punishment strategy, which would imply that agents that punish make an additional contribution to punishment pool in advance, i.e., centralise their resources in order to try and catch the defectors or deceivers before the PGG is played (and before deception and defection actually happen).

Another assumption that was made in Reference [43] was that trust in society was related to the number of all agents that both cooperated and only seemed to cooperate (Deceivers). This meant that the trust model was not considering the presence of potential Deceivers. This made agents more trusting towards each other and easier for Deceivers to exploit the agents that blindly assigned trust in them.

In this article, we relax both (i) the assumption of strong social learning, i.e., we only use the deterministic process to reproduce previous results, and (ii) the assumption that trust is not affected by the presence of Deceivers.

Similarly to References [1, 43, 46], we assume that intelligent and self-interested agents adapt their behaviour in a hybrid society based on the social learning (imitation) model. The strength of social learning \(s\) directly influences agents to copy the strategies of their peers based on their peers strategy payoffs. Additionally, these agents might tend to explore other strategies at random, which is driven by the mutation (exploration) rate \(\mu\). The combination of exploration and social learning is equivalent to a model that balances social (driven by \(s\)) and asocial learning (driven by \(\mu\)) [25]. In this evolutionary setting, agents with greater social learning skills leave very small error margins when imitating others’ payoffs and, thus, manage to imitate the behaviour of their peers more successfully compared with other agents. See Section 5 Methods and Models, where Algorithm 1 describes how the public goods game drives the evolution of the population over time \(t\) and number of iterations \(T\). The inputs of Algorithm 1 are the population distribution at \(t=0\), namely \(k_0\), the social learning strength \(s\), the mutation rate \(\mu\), and the number of iterations \(T\), whereas the output is the population distribution at \(t=T\), namely \(k_T\). Algorithm 2 describes the social learning (imitation) process used in line 7 of Algorithm 1. Algorithm 2 takes as inputs the population distribution \(k\), the payoffs from the PGG \(\Pi\), and the social learning strength \(s\) for the details of the social learning process, then it outputs the population distribution \(k\) after the social learning process has taken place.

We model a voluntary PGG, called PGG*, played between a sample \(M\) of a fixed agent population \(N\), over a period of \(T\) iterations. At every iteration, each participant first decides whether to contribute, or not, to the public pool with an amount \(c\gt 0\), and then receives a payout and a payoff. The payout, equal to \(r\times c \times \frac{M_C}{M}\), where \(r\) is a multiplier (an incentive to participate to the PGG, i.e., an amount which multiplies the amount contributed \(c\)), and \(M_C\) is the number of contributors to the PGG, is the same for all the participants and it increases with the frequency of cooperative behaviour: if \(M_C = M\), then the social good is maximised and each participant receives the maximum amount \(r \times c\). On the other hand, the payoffs of the participants depend on the strategies they use to play the PGG.

Because PGG* is a voluntary one, we also have Non-participants (Loners) that never receive the payout from the PGG, but always receive a fixed amount \(\sigma\) as payoff, no matter which other strategies are used in a PGG. The Loner’s role is to give a chance to other strategies to invade the population, e.g., to secure neutral drift. This strategy is similarly used in References [1, 15, 43, 46].

Previous studies show that in the absence of punishment free-riding (taking the payout without contributing to the public pool) becomes the dominant (evolutionary stable) strategy [46]. Following the state-of-the-art, we include in the PGG the following cooperative and free-riding strategies: Cooperation (\(C\)) and Peer-Hybrid Interrogation (\(H_{PeP}\)) as cooperative strategies, i.e., that contribute to the public pool; Defection (\(D\)) and Deception (\(Dec\)) as free-riding strategies, i.e., that do not contribute to the public pool; and Non-Participation (\(L\)).

Agents that select the classic cooperative strategy, namely \(C\), receive the PGG payout (Equation (2), see Methods 5), pay the PGG contribution \(c\), and pay \(\beta\), which represents a fixed tax for \(H_{PeP}\) to exist. The agents that select the classic free-riding strategy, namely \(D\), receive the PGG payout and do not pay the contribution, but if caught they do pay a punishment fee imposed by \(H_{PeP}\).

The non-classic strategies for cooperation and free-riding, initially introduced in Reference [43], namely \(H_{PeP}\) and \(Dec\), are influenced by a model of trust. In Reference [43], the authors considered that trust increased with the prevalence of \(C\) and only decreased with the prevalence of \(D\) because \(Dec\) were oblivious to the presence of other \(Dec\) in the PGG. Here, we make a different and more realistic assumption, that trust in society is influenced by both the presence of \(D\) and \(Dec\). By treating trust in this way, \(Dec\) need to consider that agents in the game who seem to be \(C\), might in fact be other \(Dec\). In Methods 5, we describe how trust is computed.

By extending the agent-based framework in Reference [43], we modify the two non-classic strategies to better reflect socio-cognitive dynamics and the new trust model. \(Dec\) receives the PGG payout and does not pay the PGG contribution, but it can be interrogated by \(H_{PeP}\) and, if caught, it must pay a tax for deception. Additionally, it must pay a cost of deception that depends on the cognitive load of the \(Dec\), the risk of leakage from \(Dec\), and on \(Dec\)’s communicative skill. In this article, we change the cognitive load function for the \(Dec\) to include other \(Dec\)s as targets to be deceived based on the new trust model. This is closer to what happens in the real-world where, for instance, con-men try to outsmart other con-men. See Methods 5 for a full description of the new \(Dec\).

The Peer-Hybrid Interrogator (\(H_{PeP}\)), while it contributes to the public pool and receives the PGG payout, has two main regulatory roles: to punish \(D\) and to detect and punish \(Dec\). Differently from Reference [43], we do the following: (i) introduce a reward for \(H_{PeP}\) that can be discounted; (ii) introduce the interrogation skill; (iii) introduce the cognitive load for interrogation; and (iv) re-define the entire model of \(H_{PeP}\) to reflect the influence of the new socio-cognitive factors and the new model of trust. In summary, \(H_{PeP}\) are rewarded for finding and punishing \(Dec\) and \(D\), but pay the associated socio-cognitive costs for doing their job. See Methods 5 for a full description of the new \(H_{PeP}\).

We use a mixed methodology that comprises of evolutionary game theory and mechanism design in ABM, together with cognitive modelling and resilience testing of complex systems. The resulting output of the method is an evolutionary PGG where deception and deception detection are present. We validate our model with individual agent-based simulations instead of analytical methods due to the complexity of the model [34, 35], which arises form the integration of cognitive functions responsible for deception and deception detection. Our choice of parameters for evolutionary agent-based models of PGGs is consistent with the previous literature that used these games for studying the emergence of cooperation under social learning [1, 43, 46].

To answer Q1 and Q2 using agent-based simulations, we formulated the hypotheses in Table 2. We reproduced the results for the PGG model in Reference [43] and confirm that, under their assumptions, strong social learning (\(s\rightarrow \infty\)) can indeed re-establish cooperation in hybrid societies, whereas weak social learning cannot. The two social learning conditions from Reference [43] are contrasted in Figures 2(a) and 2(b).

However, in PGG*, where the improved \(H_{PeP}\) mechanism is present, cooperation can be re-established in hybrid societies, even if agents do not have unlimited social learning abilities, i.e., they are now allowed to make mistakes in the social learning process. We show this in Figure 2(c). Additionally, Figure 1 shows that \(H_{PeP}\) in PGG* is actually better at re-establishing cooperation compared with the model in Reference [43]. Another major difference between PGG* and the PGG in Reference [43], that can be observed by comparing Figures 2(c) and 2(b), is that PGG*’s variance is lower, which indicates greater overall stability as a system. This can also be observed in the temporal dynamics in Figure 3(a).

Even if social learning is not infinite (\(s \ne \infty\)), if it is high enough (\(s \ge 100\)) then cooperation can be re-established. Figure 4(a) shows how the frequency of \(H_{PeP}\) increases together with \(s\) up to \(s \approx 100\), and then it stabilises around \(80\%\).

Unfortunately, the larger a hybrid society becomes, the more likely it is for \(D\) to become evolutionary stable. Figures 4(c) and 4(d) show how, by increasing \(N\), the proportion of \(H_{PeP}\) decreases, while the proportion of \(D\) increases. For \(N \gt 100\), the frequency of \(H_{PeP}\) starts dropping, and \(H_{PeP}\) becomes a completely unsustainable strategy for \(N \ge 200\). On the other hand, the number of players \(M\) does not seem to affect the evolutionary stability of \(H_{PeP}\) (see Figure 4(b)).

Regarding resilience, that is the ability of societies to revert back to a cooperative strategy, our model shows that these are resilient in the face of full-fledged deceptive attacks under certain conditions w.r.t. attack intervals.

We show that if given enough time between the attack intervals (20,000 iterations), \(H_{PeP}\) manage to re-invade the population and re-establish cooperation in the long-run (see Figure 5(c)). By looking at Figure 5, we can observe how the overall cooperation is significantly decreased compared to the games where full-fledged deceptive attacks are not artificially caused. The mean of the long-run avg. frequencies of free-riders (\(D + Dec\)) is greater than the ones of cooperators (\(C + H_{Pep}\)) for the attacks that happen at intervals of 5,000 and 10,000 iterations. By looking at Figures 5 and 6, we can also observe how over a long run simulation of \(T=10^5\), it is sufficient to have a full-fledged deceptive attack on all agents at the same time every 10,000 iterations in order to break overall cooperation by having free-riders (\(D\) or \(Dec\)) prevent cooperative strategies from immediately re-invading. This happens despite \(H_{PeP}\) being dominant by having the highest avg. of the long-run avg. frequency because the combined averages of \(Dec\) and \(D\) outweigh the combined averages of \(H_{PeP}\) and \(C\) (see Figure 5(b)). This effect is even stronger for more frequent attacks at intervals of 5,000 iterations, where \(H_{PeP}\) does not even become the dominant strategy in the long-run - \(Dec\) becomes the dominant strategy (see Figure 5(a)).

How can the results be interpreted from the MB perspective to answer Q1 and Q2? We know from Reference [43] that there are indeed risks of machines to adopt deceptive behaviour from social interactions with other agents (humans or machines). This enhances the negative effects that lead to a TDC, which according to Reference [43] can be avoided if social learning is infinite. In other words, agents do not make any mistakes when they adopt other agents’ strategies, which is a very optimistic scenario. It is very difficult, if not impossible, to guarantee that a self-organising and adaptive complex system will always be able to learn perfectly—mistakes and/or accidents happen, especially in complex and dynamic environments. On the other hand, our results indicate that things might not be so bleak when it comes to respond to deception in the future, even if social learning is not infinite.

In fact, regarding Q1, we show that if more realistic assumptions are met regarding the socio-cognitive factors involved in deception and deception detection, then infinite social learning capabilities of agents are not necessary for the TDC to be avoided. Moreover, we have shown that, under the new assumptions, Peer-Hybrid Interrogation is actually better at fighting off free-riders, despite the limits imposed on the social learning skills of agents. What we can infer from both studies (Reference [43] and ours), is that social learning is indeed a crucial factor in avoiding TDC. We have actually showed here that by increasing the social learning skills of agents, we can reduce the likelihood of free-riding, and by extension, of TDC happening. The smarter the agents are at adopting others’ behaviour, the more likely it is for them to learn how to detect and punish deception (Figure 4(a)).

Regarding Q2, we show that self-organising societies, which can select the Peer-Hybrid Interrogation decentralised regulatory mechanism - an equivalent of a deliberative democracy [14]—are resilient to large-scale deceptive attacks under certain conditions. In the real world, deception is sometimes coordinated as an internal or external attack on the population of agents. An example is when for short periods the majority of agents in a system can be externally incentivised to (or tricked into) spread disinformation. Such attacks, if recurring, take their toll on the system’s levels of cooperation. However, we show how the system, if given the opportunity (and enough time to respond), fights back and re-establishes cooperation. So, the effects of social learning and the resilience test demonstrate that self-organising hybrid societies can (i) get better at fighting off free-riders and (ii) fully recover from external attacks that incentivise agents of the hybrid society to spread disinformation.

This self-governing system does have its limits though regarding sustainability. Our results show that while our Peer-Hybrid Interrogation mechanism can deal with free-riders for different sizes of interactions (Figure 4(b)), it cannot handle population sizes greater than 200 agents (Figure 4(c)). This reflects the difficulties of regulating disinformation in large and open multi-agent systems, such as Twitter, Facebook, and so on, where huge amounts of agents access and share information [2]. On the other hand, there are Facebook and WhatsApp groups or moderated Reddit threads where this type of decentralised peer-to-peer mechanism might be implemented successfully. In the real-world, Peer-Hybrid Interrogation might take various forms. One of them is a normative reputation mechanism that promotes trust, an example of which is how the anonymous silk-road users were reviewing each other’s online business behaviours [23].

Previous work has proposed normative approaches for regulating agent behaviour in multi-agent systems. The normative approach might not be entirely distinct from the MB approach, as it aims to give agents the necessary knowledge to reason about the behaviour and the related rewards or sanctions of the possible behaviours available to them in a multi-agent system. The MB approach was proposed as a discipline to study the co-behaviour of humans and AI as part of their shared ecosystem. Another compatible approach with MB that is different from the normative one is that of mechanism design. In contrast to normative approaches, approaches that use mechanism design for regulating agent behaviour focus on the design of the multi-agent system itself by assuming that agents generally operate with self-interest [29]. However, mechanism design has mostly been explored for regulating free-riding behaviour within a purely economical paradigm [16], and except for [43] and [40], it has not been explored for regulating deception in socio-cognitive systems.

What we have done in this article is give a self-organising population of agents the opportunity to self-select a strategy that regulates free-riding behaviour (deception and defection), without having to specify norms that govern their behaviour. Agents choose to select strategies that interrogate and punish/sanction free-riders because they are self-interested to manage information.

Regarding deception, there have not been many approaches developed for multi-agent systems to manage/regulate it explicitly, except for extensive work on trust, reputation and image formation mechanisms for managing deception, such as the one of Schillo et al. that created a trust model for reputation [45], Sabater et al. who introduce the notion of image of other agents in socio-cognitive systems [39] and Yu and Singh who developed a weighted majority belief update mechanisms for detecting deception [49]. All of these decentralised mechanism are, in principle, compatible with our Peer-Hybrid Interrogation strategy.

Normative approaches in socio-technical systems have also been applied to govern free-riding behaviour in a decentralised manner [19]. Distinctly from mechanism design, normative approaches specify how sanctions or rewards are applied to agent behaviours in different contexts. Rewards are usually applied as incentives for agents to behave according to the norms specified (usually as deontic logic representations and axiomatisations [33]). On the other hand, sanctions are applied when agents deviate from what is specified as norm-adhering behaviour. Agents are then able to reason about the consequences of their actions. Moreover, some agents of the multi-agent system have designated sanctioning/punishment roles which they have to execute when they observe other agents that deviate from the norm. One example of such a system is Ghorbani and Bravo’s ADICO model for governing the commons [10, 12]. Implementing such a norm-based governing system can also be used to explore how institutions for governing the commons emerge [11, 33]. In particular, they can be used to study the management of knowledge in self-organising systems [22].

We must also make the observation that our current MB perspective does not go into explaining the normative aspects of socio-technical systems like hybrid societies. MB is agnostic w.r.t. an agent’s capability of reasoning about deciding how to behave given some norms present in the multi-agent system. The mechanism-design perspective is a more appropriate conceptual perspective to take in MB. However, that does not mean that MB and normative approaches are incompatible. Norms can very well be defined such that agents know what to expect if they decide to pursue a certain strategy. The most obvious mechanism where norms can be applied is in the social learning process. Instead of assuming that agents are self interested and aim to maximise their payoff when they adopt another agent’s strategy, a normative reasoning mechanism can be used to increase the complexity of this decision. For instance, we can make an agent aware of the presence of a sanctioning mechanisms. These mechanisms can take the form of agents which only play the roles of punishers and which are triggered if they deviate from the norms. The agent would then be inclined to maintain cooperation and not select defection. But then again, our evolutionary system allows agents to achieve similar outcomes by selecting, which roles they want to adopt. The combination of available strategies acts itself as a form of mechanism design where agents change roles to keep each other in check. The Peer-Hybrid Interrogation strategy, for instance, can be viewed as a decentralised process for implementing pro-social norms [27], i.e., norms that incentivise pro-social behaviour. If we are to interpret Peer-Hybrid Interrogation from Nardin et al.’s conceptual sanctioning model [29], the strategy is a combination of Detector, Evaluator, and Executor. The Peer-Hybrid Interrogators detect deception, they evaluate it after it happens, and then apply a sanction. These actions are associated with the costs and rewards of the Peer-Hybrid Interrogators. In this context, the pro-social behaviour would be the sharing of truthful information between agents of a hybrid society.

MB has so far been proposed as a discipline to study the behaviour of machines as part of complex environments like societies that are either too difficult or impossible to formalize analytically. Perhaps future work should focus on combining the mechanism design with normative approaches within an MB discipline in order to explore what kind of explicit normative rules can be used to design socio-technical ecosystems in which agents will be self-interested to regulate deceptive behaviour. This might also be helpful to study the behaviour of machines [37, 41] as well as of humans when they interact with machines, and of potentially the overall deceptive ecosystems themselves [20, 50].

In this study, we designed a new agent-based model of PGGs to show (i) how cooperation can be re-established in self-organising societies where decentralised regulatory mechanisms exist and where the social learning skills of agents are bounded, i.e., not infinite; and (ii) how these systems show resilience against large-scale deceptive attacks under certain conditions.

We have shown that even under the assumption of weak social learning agents can indeed re-establish cooperation in the face of deception when they are given the possibility to adopt a Peer-Hybrid Interrogation strategy. By weak social learning, we mean that agents follow a stochastic process when adopting another agents strategy, rather than a deterministic one such as strong social learning. These populations also show that they are resilient and able to re-establish cooperation if large-scale deception attacks happen. However, there are still limitations regarding re-establishing cooperation, especially when deception attacks are frequent and when populations are very large.

As future work, we plan to explore how the effects of this limitation w.r.t. population size can be reduced or eliminated. This is crucial in finding regulatory mechanisms that implement deliberative democracy successfully in larger hybrid societies.

Another potential avenue for future work is to study mechanisms that consider various reasons for performing deception from an ethical perspective [48]. Deception can be malicious, but it can be done for good reasons too, which do bring some benefits with it. For instance, both References [18] and [17] argue that deception in the context of human-AI interactions can improve both efficiency and smoothness of communication, while [42] show that humans do not perceive AI agents any different compared with humans that use deception as part of their job roles in future-of-work scenarios. Perhaps it is better then to ask ourselves sooner rather than later what tradeoffs emerge in terms of efficiency and smoothness of communication, and what costs are we willing to pay in order for machines to be truthful and transparent about their non-human nature and communicative acts.

I would like to give special thanks to the Reviewers for taking the time and effort necessary to review this manuscript.

For values of social learning that are lower than 100 (\(s\lt 100\)), agents stop distinguishing between the payoffs of the strategy. In Figure 7(a) and 7(b), we can see how for small values for social learning such as \(s=0.1\), the selection of strategies becomes volatile, and no single strategy or combination of strategies becomes stable or dominant. This effect when reducing social learning \(s \rightarrow 0\) is also shown by Sigmund et al. in Figure 3 of their article labelled ‘The competition between peer- and pool-punishers in voluntary PGGs’, where we can see their sensitivity analysis that varies \(s\) from \(10^{-4}\) to \(10^4\) [46].