08-appendix-b.Rmd

# Validation of the Operating Model in Chapter \@ref(ch3) {#appendix-b}

\noindent
For any simulation model used in the context of management strategy evaluation, the reliability of inferences drawn is conditional on the ability of the model components to capture the important behavioral properties of the real system. Here, a brief validation is provided that the fishery component of the operating model did in fact provide a reasonable representation of the real system for the case where the fishery is unrestricted.

First, it is important that the model be able to replicate the relationship between total Chinook salmon run size and total Chinook salmon subsistence harvest. Capturing this pattern was important to ensure that the fishery would not inadvertently harvest an unrealistically large or small amount of fish in different run sizes than would typically occur, which would confound the inference regarding strategy performance. As shown in Figure \@ref(fig:HvN), this historical relationship has been quite noisy for the observed historical time series, though an increasing pattern has emerged: in general, more fish have been harvested in years with large runs than years with small runs. It was found that by tuning the catchability ($q$) and effort response coefficients, this pattern could be reproduced quite well. Additionally, the scale and variability of modeled chum/sockeye harvests were also similar to the historically observed distribution (Figure \@ref(fig:chsk-harvest)) -- this was not critical given chum/sockeye harvests did not inform any objectives, but the agreement contributes more evidence that the effort response model was adequately calibrated.

The next behavior of interest was the spatiotemporal distribution of harvest. Because in-river salmon fisheries are sequential, fish harvested in one area are invulnerable to harvest (and escapement) in upriver areas. It also means that communities in downriver communities may finish fishing earlier in the season because they are the first to experience favorable fishing conditions (_i_._e_., high in-river abundance and resulting catch rates; in the Kuskokwim River drying weather also plays an important role). If the timing of harvest was not captured adequately, this would be an indication that the effort response coefficients were improperly tuned and could result in unrealistic conclusions. The patterns and variability in the day of the year at which various percentiles of Chinook salmon harvest were attained by reach compared between observed data and the modeled outcomes are shown in Figure \@ref(fig:temporal-harvest). It seems that the patterns and variability in harvest timing were reasonably well-captured, particularly for downriver reaches. Reaches 14, 15, 16 and 22 seemed to have had the largest deviations between observed and modeled patterns, but given communities in these reaches harvest a negligible amount of Chinook salmon in comparison to the downriver villages (Figure \@ref(fig:spatial-harvest)), this finding is not concerning. 

The final important characteristic was the spatial distribution of end-of-season harvest. Accurately representing this component of the system would further indicate model adequacy. Figure \@ref(fig:spatial-harvest) shows a comparison of the proportion of total drainage-wide Chinook salmon subsistence harvest attributable to communities in each reach between observed and modeled outcomes. While the overall pattern was fully captured, there were moderate deviations between the model and observations in reaches 2, 3, and 4.

\clearpage

\begin{figure}
  \centering
  \includegraphics{img/Ch3/Figure B1.jpg}
  \caption{Observed and modeled Chinook salmon subsistence harvest as a function of total Chinook salmon run size. Individual black numbers are historical realizations in years with no harvest restrictions on the subsistence salmon fishery. Individual grey dots are modeled outcomes, each representing a hypothetical salmon run with different random subpopulation compositions, run timing, and species ratios. Fitted models display close agreement between the average simulated and observed harvest outcomes across the range of run sizes. Vertical dotted lines show the boundaries of important run size strata used in this analysis.}
  \label{fig:HvN}
\end{figure}

\begin{figure}
  \centering
  \includegraphics{img/Ch3/Figure B2.jpg}
  \caption{Comparison of the inter-annual distribution of observed and modeled chum/sockeye salmon harvests by all villages located in the Kuskokwim River.}
  \label{fig:chsk-harvest}
\end{figure}

\clearpage

\begin{figure}
  \centering
  \includegraphics{img/Ch3/Figure B3.jpg}
  \caption{Comparison of the day of the year at which various percentiles of Chinook salmon harvest were attained by reach between observed and modeled outcomes. Variability in the observed boxplots is due to inter-annual variability in run size and timing and represents between-simulation variability for the modeled outcomes. Reach numbers are ordered from downriver to upriver. Note that not all reaches contain communities that harvest salmon.}
  \label{fig:temporal-harvest}
\end{figure}

\clearpage

\begin{figure}
  \centering
  \includegraphics{img/Ch3/Figure B4.jpg}
  \caption{Comparison of the proportion of total drainage-wide Chinook salmon subsistence harvest attributable to communities in each reach between observed and modeled outcomes. Variability in the observed boxplots is due to inter-annual variability, and represents between-simulation variability for the modeled outcomes. Reach numbers are ordered from downriver to upriver. Note that not all reaches contain communities that harvest salmon.}
  \label{fig:spatial-harvest}
\end{figure}