Papers by Brett Borghetti
Lecture Notes in Computer Science, 2016
Lecture Notes in Computer Science, 2015
2015 IEEE 14th International Conference on Machine Learning and Applications (ICMLA), 2015
Advances in Intelligent Systems and Computing, 2016
One of the most important elements of agent performance in multi-agent systems is the ability for... more One of the most important elements of agent performance in multi-agent systems is the ability for an agent to predict how other agents will behave. In many domains there are often different modeling systems already available that one could use to make behavior predictions, but the choice of the best one for a particular domain and a specific set of agents is often unclear. To find the best available prediction, we would like to know which model would perform best in each possible world state of the domain. However, when we have limited resources and each prediction query has a cost we may need to decide which queries to pursue using only estimates of their benefit and cost: metareasoning. To estimate the benefit of the computation, a metareasoner needs a robust measurement of performance quality. In this work we present a metareasoning system that relies on a prediction performance measurement, and we propose a novel model performance measurement that fulfils this need: Weighted Prediction Divergence.
This paper proposes a novel method to characterize the performance of autonomous agents in the Tr... more This paper proposes a novel method to characterize the performance of autonomous agents in the Trading Agent Competition for Supply Chain Management (TAC-SCM). We create benchmarking tools that manipulate market environments to control the conditions and provide guidelines to test trading agents. Using these tools, we show how developers can inspect their agents and unveil behaviors that might otherwise have gone undiscovered. * We would like to thank Professor Maria Gini for all her help and support with this effort. Eric Sodomka is an undergraduate student.
Lecture Notes in Computer Science, 2007
This paper proposes a novel method to characterize the performance of autonomous agents in the Tr... more This paper proposes a novel method to characterize the performance of autonomous agents in the Trading Agent Competition for Supply Chain Management (TAC-SCM). We create benchmarking tools that manipulate market environments to control the conditions under which we test trading agents. Using these tools, we show how developers can inspect their agents and unveil behaviors that might otherwise have gone undiscovered.
2015 48th Hawaii International Conference on System Sciences, 2015
This paper proposes a novel method to characterize the per- formance of autonomous agents in the ... more This paper proposes a novel method to characterize the per- formance of autonomous agents in the Trading Agent Compe- tition for Supply Chain Management (TAC-SCM). We create benchmarking tools that manipulate market environments to control the conditions and provide guidelines to test trading agents. Using these tools, we show how developers can in- spect their agents and unveil behaviors that might otherwise have gone undiscovered.
We advance the field of research involving modeling opponents in interesting adversarial environm... more We advance the field of research involving modeling opponents in interesting adversarial environments: environments in which equilibrium strategies are intractable to calculate or undesirable to use. We motivate the need for opponent models in such environments by showing how successful opponent modeling agents can exploit nonequilibrium strategies and strategies using equilibrium approximations. We develop a new, flexible measurement which can be used to quantify how well our model can predict the opponent's behavior independently from the performance of the agent in which it resides. We show how this metric can be used to find areas of model improvement that would otherwise have remained undiscovered and demonstrate the technique for evaluating opponent model quality in the poker domain. We introduce the idea of performance bounds for classes of opponent models, present a method for calculating them, and show how these bounds are a function of only the environment and thus inv...
This research focuses on answering the question: How do we val-idate the decisions of autonomous ... more This research focuses on answering the question: How do we val-idate the decisions of autonomous trading agents? We start with a holistic approach, exploring evaluation techniques for measur-ing the performance of competitive autonomous agents during con-trolled competitive simulations. We focus on cases where the num-ber of opponents is small such as in the Trading Agent Compe-tition for Supply Chain Management (TAC-SCM). In these situa-tions each agent has a large influence on the outcome of the game. We create a methodology, provide benchmarking tools and metrics for developers to reveal and resolve undesirable decision-making behavior in their agents. This framework provides a basis from which to scale the research to the larger context of real world trad-ing agents.
We develop a new method for handling incomplete knowledge when inferencing over Bayesian Knowledg... more We develop a new method for handling incomplete knowledge when inferencing over Bayesian Knowledge Bases(BKB) using genetic algorithms(GA) . The fitness function for a genetic algorithm requires that we give a score to each solution it generates. When a solution is complete, we can use the joint probability of our generated solution, but when incompleteness occurs, the joint probability is undefined. In this paper, we characterize incompleteness in BKBs and present an easily computable method for scoring a solution which contains incompleteness. 1 Introduction A Bayesian Knowledge Base (BKB) is a knowledge representation for information containing uncertainty. BKBs use random variables(RVs) and probabilities to represent knowledge. Unlike other approaches such as Bayesian Networks[4], BKBs are designed to handle incompleteness in our knowledge 2 . One of the inferencing tasks performed on BKBs is belief revision. In belief revision, we attempt to find the most likely state for every...
IEEE Communications Surveys & Tutorials, 2015
2010 IEEE Symposium on New Frontiers in Dynamic Spectrum (DySPAN), 2010
Dynamic spectrum access has proposed tiering radios into two groups: Primary Users (PUs) and Seco... more Dynamic spectrum access has proposed tiering radios into two groups: Primary Users (PUs) and Secondary Users (SUs). PUs are assumed to have reserved spectrum available to them, while SUs (operating in overlay mode) must share whatever spectrum is available. The threat of Emulation Attacks, in which users pretend to be of a type they are not (either PU or SU) in order to gain unauthorized access to spectrum, has the potential to severely degrade the expected performance of the system. We analyze this problem within a Bayesian game framework, in which users are unsure of the legitimacy of the claimed type of other users. We show that depending on radios' beliefs about the fraction of PUs in the system, a policy maker can control the occurrence of emulation attacks by adjusting the gains and costs associated with performing or checking for emulation attacks.
Proceedings of the 2010 Winter Simulation Conference, 2010
Although mobile wireless communication provides connectivity where hardwired links are difficult ... more Although mobile wireless communication provides connectivity where hardwired links are difficult or impractical, environmental conditions can still hinder communications. Increasing transmission power reduces battery life and increases susceptibility to eavesdropping. Adding stationary repeater nodes is impractical for highly mobile users in dangerous environments. Using remotely-controlled mobile relay nodes requires centralized control schemes which and adds network traffic overhead and introduces a single point of failure at the controller. An alternative is to create a Mobile Agent Relay Network (MARN). Each autonomous node in the MARN is an agent that decides where to move to maintain the network connectivity using only locally available information from onboard sensors and communication with in-range neighbor nodes. MARN agents form and maintain a communication network that provides connectivity for users while reducing the overall radio frequency footprint, minimizing the likelihood of detection and eavesdropping. We characterize the footprint reduction both theoretically and in simulation.
2009 IEEE/RSJ International Conference on Intelligent Robots and Systems, 2009
Coalition formation algorithms are generally not applicable to real-world robotic collectives sin... more Coalition formation algorithms are generally not applicable to real-world robotic collectives since they lack mechanisms to handle uncertainty. Those mechanisms that do address uncertainty either deflect it by soliciting information from others or apply reinforcement learning to select an agent type from within a set. This paper presents a coalition formation mechanism that directly addresses uncertainty while allowing the agent
Thinking about Thinking, 2011
One of the most important elements of agent performance in multi-agent systems is the ability for... more One of the most important elements of agent performance in multi-agent systems is the ability for an agent to predict how other agents will behave. In many domains there are often different modeling systems already available that one could use to make behavior predictions, but the choice of the best one for a particular domain and a specific set of agents is often unclear. To find the best available prediction, we would like to know which model would perform best in each possible world state of the domain. However, when we have limited resources and each prediction query has a cost we may need to decide which queries to pursue using only estimates of their benefit and cost: metareasoning. To estimate the benefit of the computation, a metareasoner needs a robust measurement of performance quality. In this work we present a metareasoning system that relies on a prediction performance measurement, and we propose a novel model performance measurement that fulfils this need: Weighted Prediction Divergence.
IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews), 2000
As surveillance becomes ubiquitous, the amount of data to be processed grows along with the deman... more As surveillance becomes ubiquitous, the amount of data to be processed grows along with the demand for manpower to interpret the data. A key goal of surveillance is to detect behaviors that can be considered anomalous. As a result, an extensive body of research in automated surveillance has been developed, often with the goal of automatic detection of anomalies. Research into anomaly detection in automated surveillance covers a wide range of domains, employing a vast array of techniques. This review presents an overview of recent research approaches on the topic of anomaly detection in automated surveillance. The reviewed studies are analyzed across five aspects: surveillance target, anomaly definitions and assumptions, types of sensors used and the feature extraction processes, learning methods, and modeling algorithms.
IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics), 2000
We develop an upper bound for the potential performance improvement of an agent using a best resp... more We develop an upper bound for the potential performance improvement of an agent using a best response to a model of an opponent instead of an uninformed game-theoretic equilibrium strategy. We show that the bound is a function of only the domain structure of an adversarial environment and does not depend on the actual actors in the environment. This bounds-finding technique will enable system designers to determine if and what type of opponent models would be profitable in a given adversarial environment. It also gives them a baseline value with which to compare performance of instantiated opponent models. We study this method in two domains: selecting intelligence collection priorities for convoy defense and determining the value of predicting enemy decisions in a simplified war game.
Uploads
Papers by Brett Borghetti