player-evaluation.qmd

# Player Evaluation {#sec-playerEvaluation}

## Getting Started {#sec-playerEvaluationGettingStarted}

### Load Packages {#sec-playerEvaluationLoadPackages}

```{r}
library("tidyverse")
```

## Overview

Evaluating players for fantasy football could be thought of as similar to the process of evaluating companies when picking stocks to buy.
You want to evaluate and compare various assets so that you get the assets with the best value.

There are various domains of criteria we can consider when evaluating a football player's fantasy prospects.
Potential domains to consider include:

- athletic profile
- historical performance
- health
- age and career stage
- situational factors
- matchups
- cognitive and motivational factors
- fantasy value

The discussion that follows is based on my and others' *impressions* of some of the characteristics that may be valuable to consider when evaluating players.
However, the extent to which any factor is actually relevant for predicting future performance is an empirical question and should be evaluated empirically.

## Athletic Profile {#sec-evalAthleticProfile}

Factors related to a player's athletic profile include factors such as:

- body shape
    - height
    - weight
    - hand size
    - wing span (arm length)
- body function
    - agility
    - strength
    - speed
    - acceleration/explosiveness
    - jumping ability

In terms of body shape, we might consider a player's height, weight, hand size, and wing span (arm length).
Height allows players to see over opponents and to reach balls higher in the air.
Thus, greater height is particularly valuable for Quarterbacks and Wide Receivers.
Heavier players are tougher to budge and to tackle.
Greater weight is particularly valuable for Linemen, Fullbacks, and Tight Ends, but it can also be valuable—to a deree—for Quarterbacks, Running Backs, and Wide Receivers.
Hand size and wing span is particularly valuable for people catching the ball; thus, a larger hand size and longer wing span are particularly valuable for Wide Receivers and Tight Ends.

In terms of body function, we can consider a player's agility, strength, speed, acceleration/explosiveness, and jumping ability.
For Wide Receivers, speed, explosiveness, and jumping ability are particularly valuable.
For Running Backs, agility, strength, speed, and explosiveness are particularly valuable.

Many aspects of a player's athletic profile, including tests of speed (40-yard dash), strength (bench press), agility (20-yard shuttle run; three cone drill), and jumping ability (vertical jump; broad jump) are available from the National Football League (NFL) Combine, which is especially relevant for evaluating rookies.
We demonstrate how to import data from the NFL Combine in @sec-combine.
There are also calculators that integrate information about body shape and information from the NFL Combine to determine a player's relative athletic score (RAS) for their position: <https://ras.football/ras-calculator/>

## Skill {#sec-skill}

When scouting players, scouts consider not only the player's [athletic profile](#sec-evalAthleticProfile), but also their position-relevant skill.
For instance, how good are they are reading the defense, passing the ball, running routes, catching balls, making defenders miss tackles, taking care of the ball, consistency, etc.
Scouting and evaluating skill is a complicated endeavor, and even the professional scouts frequently make mistakes in their evaluations and predictions.
You can certainly read skill evaluations about various players; however, unlike metrics of athletic profile, we do not have direct access to the player's underlying skill.
Some may say, "You know it when you see it."
But, this is not particularly useful when trying to identify players who are undervalued or overvalued—because the skill evaluations are likely already "baked into" a player's projections.
Because we do not have direct access to a player's skill, we tend to rely on indirect metrics of their ability, such as [historical performance](#sec-evalHistoricalPerformance).

## Historical Performance {#sec-evalHistoricalPerformance}

### Overview {#sec-evalHistoricalPerformanceOverview}

> "The best predictor of future behavior is past behavior." – Unknown

> "Past performance does not guarantee future results." – A common disclaimer about investments.

Factors relating to historical performance to consider could include:

- performance in college
    - draft position
- performance in the NFL
- efficiency
- consistency

It is important to consider a player's past performance.
However, the extent to which historical performance may predict future performance may depend on many factors such as (a) the similarity of the prior situation to the current situation, (b) how long ago the prior situation was, and (c) the extent to which the player (or situation) has changed in the interim.
For rookies, the player does not have prior seasons of performance in the NFL to draw upon.
Thus, when evaluating rookies, it can be helpful to consider their performance in college or in their prior leagues.
However, there are large differences between the situation in college and the situation in the NFL, so prior success in college may not portend future success in the NFL.
An indicator that intends to be prognostic of future performance, and that accounts for past performance, is a player's draft position—that is, how early (or late) was a player selected in the NFL Draft.
The earlier a player was selected in the NFL Draft, the greater likelihood that the player will perform well; however, this is somewhat countered by the fact that the teams with the highest draft picks tend to be the worst based on the prior season's record.

For players who have played in the NFL, past performance becomes more relevant because, presumably, the prior situation is more similar (than was their situation in college) to their current situation.
Nevertheless, lots of things change from game to game and season to season: injuries, coaches, coaching strategies, teammates, etc.
So just because a player performed well or poorly in a given game or season does not necessarily mean that they will perform similarly in subsequent games/seasons.
Nevertheless, historical performance is one of the best indicators we have.

We demonstrate how to import historical player statistics in @sec-downloadActualStats.
We demonstrate how to calculate historical player statistics in @sec-calculateActualStats.
We demonstrate how to calculate historical fantasy points in @sec-calculateActualPoints.

### Efficiency {#sec-evalHistoricalEfficiency}

In addition to how many fantasy points a player scores in terms of historical performance, we also care about efficiency and [consistency](#sec-evalHistoricalConsistency).
How efficient were they given the number of opportunities they had?
This is important to consider because different players have different opportunities.
For example, some Running Backs had more opportunity than others (e.g., more carries), so comparing players merely on rushing yards would be misleading.
If they were relatively more efficient, they will likely score more points than many of their peers when given more opportunities.
If they were relativelly inefficient, their capacity to score fantasy points may be more dependent on touches/opportunities.
Efficiency might be operationalized by indicators such as yards per passing attempt, yards per rushing attempt, yards per target, yards per reception, etc.

### Consistency {#sec-evalHistoricalConsistency}

In terms of consistency, how consistent was the player they from game to game and from season to season?
For instance, we could examine the standard deviations of players' fantasy points across games in a given season.
However, the standard deviation tends to be upwardly biased as the mean increases.
So, we can account for the player's mean fantasy points per game by dividing their game-to-game standard deviation of fantasy points ($\sigma$) by their mean fantasy points across games ($\mu$).
This is known as the coefficient of variation (CV), which is provided in @eq-coefficientOfVariation.

$$
CV = \frac{\sigma}{\mu}
$$ {#eq-coefficientOfVariation}

Players with a lower standard deviation and a lower coefficient of variation (of fantasy points across games) are more consistent.
In the example below, Player 2 might be preferable to Player 1 because Player 2 is more consistent; Player 1 is more "boom-or-bust."
Despite showing a similar mean of fantasy points across weeks, Player 2 shows a smaller week-to-week standard deviation and coefficient of variation.

```{r}
#| code-fold: true

set.seed(1)

playerScoresByWeek <- data.frame(
  player1_scores = rnorm(17, mean = 20, sd = 7),
  player2_scores = rnorm(17, mean = 20, sd = 4),
  player3_scores = rnorm(17, mean = 10, sd = 4),
  player4_scores = rnorm(17, mean = 10, sd = 1)
)

consistencyData <- data.frame(t(playerScoresByWeek))

weekNames <- paste("week", 1:17, sep = "")

names(consistencyData) <- weekNames
row.names(consistencyData) <- NULL

consistencyData$mean <- rowMeans(consistencyData[,weekNames])
consistencyData$sd <- apply(consistencyData, 1, sd)
consistencyData$cv <- consistencyData$sd / consistencyData$mean

consistencyData$player <- c(1, 2, 3, 4)

consistencyData <- consistencyData %>% 
  select(player, mean, sd, cv, week1:week17)

round(consistencyData, 2)
```

## Health {#sec-evalHealth}

Health-related factors to consider include:

- current injury status
- injury history

It is also important to consider a player's past and current health status.
In terms of a player's current health status, it is important to consider whether they are injured or are playing at less than 100% of their typical health.
In terms of a player's prior health status, one can consider their injury history, including the frequency and severity of injuries and their prognosis.

We demonstrate how to import injury reports in @sec-injuries.

## Availability {#sec-evalAvailability}

In addition to [injuries](#sec-evalHealth), other factors can affect a player's availability, including contract hold-outs or league suspensions, for instance, due to violating the league's conduct policy.
Some players may be "holding out", which means that they may refuse to play until they sign a more desirable contract with the team (e.g., for more money or for a longer-term contract).

## Age and Career Stage {#sec-evalAgeCareerStage}

Age and career stage-related factors include:

- age
- experience
- touches

A player's age is relevant because of important age-related changes in a player's speed, ability to recover from injury, etc.
A player's experience is relevant because players develop knowledge and skills with greater experience.
A player's prior touches/usage is also relevant, because it speaks to how many hits a player may have taken.
For players who take more hits, it may be more likely that their bodies "break down" sooner.

## Situational Factors {#sec-evalSituation}

Situational factors one could consider include:

- team quality
- role on team
- teammates
- opportunity and usage
    - snap count
    - touches/targets
    - red zone usage

Football is a team sport.
A player is embedded within a broader team context; it is important to consider the strength of their team context insofar as it may support— or detract from—a player's performance.
For instance, for a Quarterback, it is important to consider how strong the pass blocking is from the Offensive Line.
Will they have enough time to throw the ball, or will they be constantly under pressure to be sacked?
It is also important to consider the strength of the pass catchers—the Wide Receivers and Tight Ends.
For a Running Back, it is important to consider how strong the run blocking is from the Offensive Line.
For a Wide Receiver, it is important to consider how strong the pass blocking is, and how strong the Quarterback is.

It is also important to consider a player's role on the team.
Is the player a starter or a backup?
Related to this, it is important to consider the strength of one's teammates.
For a given Running Back, if a teammate is better at running the ball, this may take away from how much the player sees the field.
For a given Wide Receiver, if a teammate is better at catching the ball, this may take some targets away from the player.
However, the team's top defensive back is often matched up against the team's top Wide Receiver.
So, if the team's top Wide Receiver is matched up against a particularly strong Defensive Back, the second- and third-best Wide Receivers may more targets than usual.

It is also important to consider a player's opportunity and usage, which are influenced by many factors, including the skill of the player, the skill of their teammates, the role of the player on the team, the coaching style, the strategy of the opposing team, game scripts, etc.
In terms of the player's opportunity and usage, how many snaps do they get?
How many touches and/or targets do they receive?
Being on the field for more snaps and receiving more touches and/or targets means that the player has more opportunities to score fantasy points.
Are they targeted in the red zone?
Red zone targets are more likely to lead to touchdown scoring opportunities, which are particularly valuable in fantasy football.

## Matchups {#sec-evalMatchups}

Matchup-related factors to consider include:

- strength of schedule
- weekly matchup

Another aspect to consider is how challenging their matchup(s) and strength of schedule is.
For a Quarterback, it is valuable to consider how strong the oppenent's passing defense is.
For a Running Back, how strong is the running defense?
For a Wide Receiver, how strong is the passing defense and the Defensive Back that is likely to be assigned to guard them?

## Cognitive and Motivational Factors {#sec-evalCogMotivational}

Other factors to consider include cognitive and motivational factors.
Some coaches refer to these as the "X Factor" or "the intangibles."
However, just as any other construct in psychology, we can devise ways to operationalize them.
Insofar as they are observable, they are measurable.

Cognitive and motivational factors one could consider include:

- reaction time
- knowledge and intelligence
- work ethic and mental toughness
- incentives
    - contract performance incentives
    - whether they are in a contract year

A player's knowledge, intelligence, and reaction time can help them gain an upper-hand even when they may not be the fastest or strongest.
A player's work ethic and mental toughness may help them be resilient and persevere in the face of challenges.
Contact-related incentives may lead a player to put forth greater effort.
For instance, a contract may have a performance incentive that provides a player greater compensation if they achieve a particular performance milestone (e.g., receiving yards).
Another potential incentive is if a player is in what is called their "contract year" (i.e., the last year of their current contract).
If a player is in the last year of their current contract, they have an incentive to perform well so they can get re-signed to a new contract.

## Fantasy Value {#sec-fantasyValue}

### Sources From Which to Evaluate Fantasy Value {#sec-fantasyValueSources}

There are several sources that one can draw upon to evaluate a player's fantasy value:

- expert or aggregated rankings
- layperson rankings
    - players' Average Draft Position (ADP) in other league [snake drafts](#sec-draftStrategySnake)
    - players' Average Auction Value (AAV) in other league [auction drafts](#sec-draftStrategyAuction)
- expert or aggregated projections

#### Expert Fantasy Rankings {#sec-fantasyValueRankings}

Fantasy rankings (by so-called "experts") are provided by many sources.
To reduce some of the bias due to a given source, some services aggregate projections across sources, consistent with a "wisdom of the crowd" approach.
[FantasyPros](https://www.fantasypros.com/nfl/rankings/consensus-cheatsheets.php) aggregates fantasy rankings across sources.
[Fantasy Football Analytics](https://fantasyfootballanalytics.net) creates fantasy rankings from projections that are aggregated across sources (see the webapp here: <https://apps.fantasyfootballanalytics.net>).

#### Layperson Fantasy Rankings: ADP and AAV {#sec-fantasyValueADPAAV}

Average Draft Position (ADP) and Average Auction Value (AAV), are based on league drafts, mostly composed of everyday people.
ADP is based on [snake drafts](#sec-draftStrategySnake), whereas AAV is based on [auction drafts](#sec-draftStrategyAuction).
Thus, ADP and AAV are consistent with a "wisdom of the crowd" approach, and I refer to them as forms of rankings by laypeople.
ADP data are provided by [FantasyPros](https://www.fantasypros.com/nfl/adp/overall.php).
AAV data are also provided by [FantasyPros](https://www.fantasypros.com/nfl/auction-values/calculator.php).

#### Projections {#sec-fantasyValueProjections}

Projections are provided by various sources.
Projections (and rankings, for that matter) are a bit of a black box.
It is often unclear how they were derived by a particular source.
That is, it is unclear how much of the projection was based on statistical analysis versus conjecture.

To reduce some of the bias due to a given source, some services aggregate projections across sources, consistent with a "wisdom of the crowd" approach.
Projections that are aggregated across sources are provided by [Fantasy Football Analytics](https://fantasyfootballanalytics.net) (see the webapp here: <https://apps.fantasyfootballanalytics.net>) and by [FantasyPros](https://www.fantasypros.com/nfl/auction-values/calculator.php).
Moreover, sites also provide season-long player futures for the over/under betting lines, which are based on Vegas lines and how people bet: <https://www.rotowire.com/betting/nfl/player-futures.php>.
In addition, sites also provide weekly projections based on Vegas lines: <https://vegasprojections.com>; <https://www.actionnetwork.com/nfl/props>.

#### Benefits of Using Projections Rather than Rankings {#sec-fantasyValueProjectionsVsRankings}

It is important to keep in mind that rankings, ADP, and AAV are specific to roster and scoring settings of a particular league.
For instance, in point-per-reception (PPR) leagues, players who catch lots of passes (Wide Receivers, Tight Ends, and some Running Backs) are valued more highly.
As another example, Quarterbacks are valued more highly in 2-Quarterback leagues.
Thus, if using rankings, ADP, or AAV, it is important to find ones from leagues that mirror—as closely as possible—your league settings.

Projected statistics (e.g., projected passing touchdowns) are agnostic to league settings and can thus be used to generate league-specific fantasy projections and rankings.
Thus, projected statisitics may be more useful than rankings because they can be used to generate rankings for your particular league settings.
For instance, if you know how many touchdowns, yards, and interceptions a Quarterback is a projected to throw (in addition to any other relevant categories for the player, e.g., rushing yards and touchdowns), you can calculate how many fantasy points the Quarterback is expected to gain in your league (or in any league).
Thus, you can calculate ranking from projections, but you cannot reverse engineer projections from rankings.

### Indices to Evaluate Fantasy Value {#sec-fantasyValueIndices}

Based on the sources above (rankings, ADP, AAV, and projections), we can derive multiple indices to evaluate fantasy value.
There are many potential indices that can be worthwhile to consider, including a player's:

- dropoff
- value over replacement player (VORP)
- uncertainty

#### Dropoff {#sec-fantasyValueDropoff}

A player's *dropoff* is the difference between (a) the player's projected points and (b) the projected points of the next-best player at that position.

#### Value Over Replacement Player {#sec-fantasyValueVORP}

Because players from some positions (e.g., Quarterbacks) tend to score more points than players from other positions (e.g., Wide Receivers), it would be inadvisable to compare players across different positions based on projected points.
In order to more fairly compare players across positions, we can consider a player's value over a typical replacement player at that position (shortened to "value over replacement player").
A player's *value over a replacement player* (VORP) is the difference between (a) a player's projected fantasy points and (b) the fantasy points that you would be expected to get from a typical bench player at that position.
Thus, VORP provides an index of how much added value a player provides.

#### Uncertainty {#sec-fantasyValueUncertainty}

A player's *uncertainty* is how much variability there is in projections or rankings for a given player across sources.
For instance, consider a scenario where three experts provide ratings about two players, Player A and Player B.
Player A is projected to score 300, 310, and 290 points by experts 1, 2, and 3, respectively.
Player B is projected to score 400, 300, and 200 points by experts 1, 2, and 3, respectively.
In this case, both players are (on average) projected to score the same number of points (300).

```{r}
#| code-fold: true

exampleData <- data.frame(
  player = c(rep("A", 3), rep("B", 3)),
  expert = c(1:3, 1:3),
  projectedPoints = c(300, 310, 290, 400, 300, 200)
)

playerA_mean <- mean(exampleData$projectedPoints[which(exampleData$player == "A")])
playerB_mean <- mean(exampleData$projectedPoints[which(exampleData$player == "B")])

playerA_sd <- sd(exampleData$projectedPoints[which(exampleData$player == "A")])
playerB_sd <- sd(exampleData$projectedPoints[which(exampleData$player == "B")])

playerA_cv <- playerA_mean / playerA_sd
playerB_cv <- playerB_mean / playerB_sd
```

```{r}
playerA_mean
playerB_mean
```

However, the players differ considerably in their uncertainty (i.e., the source-to-source variability in their projections), as operationalized with the standard deviation and coefficient variation of projected points across sources for a given player.

```{r}
playerA_sd
playerB_sd

playerA_cv
playerB_cv
```

Here is a depiction of a density plot of projected points for a player with a low, medium, and high uncertainty:

```{r}
#| label: fig-playerUncertainty
#| fig-cap: "Density Plot of Projected Points for Three Players"
#| code-fold: true

playerA <- rnorm(1000000, mean = 150, sd = 5)
playerB <- rnorm(1000000, mean = 150, sd = 15)
playerC <- rnorm(1000000, mean = 150, sd = 30)

mydata <- data.frame(playerA, playerB, playerC)

mydata_long <- mydata %>% 
  pivot_longer(
    cols = everything(),
    names_to = "player",
    values_to = "points"
  ) %>% 
  mutate(
    name = case_match(
      player,
      "playerA" ~ "Player A",
      "playerB" ~ "Player B",
      "playerC" ~ "Player C",
    )
  )

ggplot2::ggplot(
  data = mydata_long,
  ggplot2::aes(
    x = points,
    fill = name
  )
) +
  ggplot2::geom_density(alpha = .3) + 
  ggplot2::labs(
    x = "Players' Projected Points",
    title = "Density Plot of Projected Points for Three Players"
  ) +
  ggplot2::theme_classic() +
  ggplot2::theme(legend.title = element_blank())
```

Uncertainty is not necessarily a bad characteristic of a player's projected points.
It just means we have less confidence about how the player may be expected to perform.
Thus, players with greater uncertainty are risky and tend to have a higher upside (or ceiling) and a lower downside (or floor).

## Signs Versus Samples {#sec-signsSamples}

When thinking about the most important domains to assess for evaluating players and predicting their performance, it is important to distinguish between signs and samples [@DenHartigh2018].
*Signs* are indicators of underlying states.
The signs approach to assessment involves assessing processes that may predict performance, where the emphasis is on what the sign indicates about the underlying attribute, rather than on the specific behavior itself.
In football, the signs approach might involve measuring skills using separate tests where players' skills are tested in isolation.
For instance, the NFL Combine takes a signs approach to assessment, in which players have a separate test to assess for speed (e.g., 40-yard dash) and jumping ability (e.g., vertical jump).

*Samples* reflect behaviors that are close to the behavior of interest.
The samples approach to asesssment tries to assess the person's performance in a behavior, situation, and context that are as reflective as possible of the kinds of behaviors, situations, and contexts that the person would have to perform in for the position or occupation.
In the samples approach, the behavior itself is the main emphasis, rather than an underlying attribute.
In football, the samples approach might involve measuring the player's performance during games or game-like situations.
For instance, to assess the skills of a Wide Receiver, you might observe—during games or game-like situations—how well they are able to catch passes when closely guarded or double-teamed, or how well they are able to catch poorly thrown passes.

In general, samples tend to be stronger than signs for predicting player performance in sports [@DenHartigh2018].
For instance, compared to tests of speed, power, and agility at the NFL Combine, collegiate performance is a stronger predictor of performance in the NFL [@Lyons2011].
That is, previous sports performance is the best predictor of future performance [for a review, see @DenHartigh2018].

According to the theoretical perspective to sports performance known as the ecological dynamics approach, successful performance in sports involves the coordination of multiple, intertwined skills that are contextually embedded [@DenHartigh2018].
For instance, a Wide Receiver needs to coordinate skills in speed, route running, jumping, and good hands to create space from defenders and to catch poorly thrown passes in a game.
The ecological dynamics approach considers the important interaction of the player, task, and environment.
Thus, to assess players in a way that is likely to be most predictive of their future performance, it is important for the assessment to retain the player–task–environment interaction [@DenHartigh2018].
For example, it would be valuable to use assessments from games or game-like situations in which players must perform tasks that leverage multiple, intertwined skills and that are similar to the tasks you want to predict.

## Putting it Altogether {#sec-evalIntegration}

After performing an evaluation of the relevant domain(s) for a given player, one must integrate the evaluation information across domains to make a judgment about a player's overall value.
When considering how much weight to give to each of various factors, it is important to evaluate how much [predictive validity](#sec-predictiveValidity) each factor has for predicting a player's successful performance in the NFL.
Then, one can weight each variable according to its [predictive validity](#sec-predictiveValidity) using an [actuarial approach](#sec-actuarialPrediction).
[Actuarial approaches](#sec-actuarialPrediction) are described in a later chapter, in @sec-actuarialPrediction.
For now, suffice it to say that [actuarial approaches](#sec-actuarialPrediction) leverage statistical formulas, as opposed to using [judgment](#sec-humanJudgment) alone.
As described in @sec-cognitiveBias, people's judgment—including judgments by experts (e.g., professional scouts and coaches)—tends to be riddled with biases [@DenHartigh2018].
Moreover, experts tend to disagree in terms of their judgments/predictions about players [@DenHartigh2018].
In addition, there are benefits to leveraging multiple perpectives, consistent with the "[wisdom of the crowd](#sec-wisdomOfCrowd)", as described in @sec-wisdomOfCrowd.
In general, as noted in @sec-signsSamples, give more weight to [samples](#sec-signsSamples) of relevant behavior—such as past performance in games—than to [signs](#sec-signsSamples) such as NFL Combine metrics (e.g., bench press, 40-yard dash, etc.).
Where you can, use assessments from games or game-like situations in which players must perform tasks that leverage multiple, intertwined skills and that are similar to the tasks you want to predict.

When thinking about a player's value, it can be worth thinking of a player's upside and a player's downside.
Players that are more consistent may show higher downside but a lower upside.
Younger, less experienced players may show a higher upside but a lower downside.

The extent to which you prioritize a higher upside versus a higher downside may depend on many factors.
For instance, when drafting players, you may prioritize drafting players with the highest downside (i.e., the safest players), whereas you may draft sleepers (i.e., players with higher upside) for your bench.
When choosing which players to start in a given week, if you are predicted to beat a team handily, it may make sense to start the players with the highest downside.
By contrast, if you are predicted to lose to a team by a good margin, it may make sense to start the players with the highest upside.

::: {.content-visible when-format="html"}

## Session Info {#sec-playerEvaluationSessionInfo}

```{r}
sessionInfo()
```

:::