Regardless of formalization used, one on-going challenge for AI systems that model legal proceedings is accounting for contextual issues, particularly where judicial decisions are made in criminal cases. The law assumes a rational approach to rule application in deciding a defendant’s guilt; however, judges and juries can behave irrationally. What should a model prize: efficiency, accuracy, or fairness? Exactly whether and how to incorporate the psychology of courtroom interactions into formal models or expert systems has only just begun to be (...) examined in a serious fashion. Here, I outline data from the United States which suggest that trying to incorporate psychological biases into formal models of legal decision-making will be challenging. I focus on the use of neuroscience data in criminal trials, homing in on so-called group-to-individual inferences. I argue that data which should be the most effective at swaying judicial decisions are in fact those most likely not to make a difference in the disposition of the case. I conclude that judges often assign culpability by ignoring what our best science regarding how human decision-making occurs. (shrink)