Are you winning or losing? Why? Performance measurement will answer the first question and can help to answer the second. A sophisticated performance analysis system can provide valuable feedback for an active manager. The manager can tie his or her decisions to outcomes and identify success, failure, and possible improvements.
Performance analysis has evolved from the days when the performance goals were vague, if not primitive:
“Don’t steal any money!” “Don’t lose any money!” “Do a good job!”
“Do as well as I could at the bank!”
“Beat the market!”
“Justify your active management fee!”
The goal of performance analysis is to distinguish skilled from unskilled investment managers. Simple cross-sectional comparisons of returns can distinguish winners from losers. Time series analysis of the returns can start to separate skill from luck, by measuring return and risk. Time series analysis of returns and portfolio holdings can go the farthest toward analyzing where the manager has skill: what bets have paid off and what bets haven’t. The manager’s skill ex post should lie along dimensions promised ex ante.
The drive for sophisticated performance analysis systems has come from the owners of funds. Investment managers have, on the whole, fought an unsuccessful rear-guard action against the advance of performance analysis. This is understandable: The truly poor managers are afraid, the unlucky managers will be unjustly condemned, and the new managers have no track record. Only the skilled (or lucky) managers are enthusiastic.
Of course, these owners of funds make several key assumptions in using performance analysis: that skillful active management is possible; that skill is an inherent quality that persists over time; that statistically abnormal returns are a measure of skill; and that skillful managers identified in one period will show up as skillful in the next period. The evidence here is mixed, as we will discuss in this chapter and Chap. 20.
Performance analysis is useful not only for fund owners, but also for investment managers, who can use performance analysis to monitor and improve the investment process. The manager can make sure that the active positions in the portfolio are compensated, and that there have been no unnecessary risks in the portfolio.
Performance analysis can, ex post, help the manager avoid two major pitfalls in implementing an active strategy. The first is incidental risk: Managers may like growth stocks, for example, without being aware that growth stocks are concentrated in certain industry groups and concentrated in the group of stocks with higher volatility. The second pitfall is incremental decision making. A portfolio based on a sequence of individual asset decisions, each of them wise on the surface, can soon become much more risky than the portfolio manager intended. Risk analysis can diagnose these problems ex ante. Performance analysis can identify them ex post.
The lessons of this chapter are:
■ The goal of performance analysis is to separate skill from luck. Cross-sectional comparisons are not up to this job.
■ Returns-based performance analysis is the simplest method for analyzing both return and risk, and distinguishing skill from luck.
■ Portfolio-based performance analysis is the most sophisticated approach to distinguishing skill and luck along many different dimensions.
■ Performance analysis is most valuable to the sponsor (client) when there is an ex ante agreement on the manager’s goals and an indication of how the manager intends to meet those goals.
■ Performance analysis is valuable to the manager in that it lets the manager see which active management decisions are compensated and which are not.
The fundamental goal of performance analysis is to separate skill from luck. But, how do you tell them apart? In a population of 1000 investment managers, about 5 percent, or 50, should have exceptional performance by chance alone. None of the successful managers will admit to being lucky; all of the unsuccessful managers will cite bad luck.
We present a facetious analysis of the market in Fig. 17.1. We have divided the managers along the dimensions of skill and luck. Those with both skill and luck are blessed. They deserve to thrive, and they will. Those with neither skill nor luck are doomed. Natural selection is cruel but just. But what about the two other categories? Those managers with skill but no luck are forlorn. Their historical performance will not reflect their true skill. And, finally, there is the fourth category. These managers have luck without skill. We call them the insufferable. Most managers can easily think of someone else they believe is in this category.
Fortunately or unfortunately, we observe only the combination of skill and luck. Both the blessed and the insufferable will show up with positive return histories. The challenge is to separate the two groups.
The simple existence of positive returns does not prove skill. Almost half of all roulette players achieve positive returns each spin of the wheel, but over time they all lose. The existence of very large positive returns also does not prove skill. How much risk was taken on in generating that return? Performance analysis will involve comparing ex post returns to ex ante risk in a statistically rigorous way.
Chapter 12 included brief mentions of the standard error of the information ratio. The approximate result is
where Y measures the number of years of observation.1 The number of years enters because we define the information ratio as an annualized statistic. Equation (17.1) implies that to determine with 95 percent confidence (t statistic = 2) that a manager belongs in the top quartile (IR = 0.5) will require 16 years of observations.2 It is a fact of investment management life that proof of investment prowess will remain elusive.
We can view the basic predicament from another angle. What if you are truly a top quartile manager, with an information ratio of 0.5? What is the probability that your monthly, quarterly, annual returns are positive? Figure 17.2 shows the result as the horizon varies. Over one month, you have only a 56% chance of positive realized alpha. Over a 5-year horizon, this rises to 87%. This implies that over the standard 5-year horizon, 13% of skilled managers will have negative realized alphas. Given the horizons for careers, and for ideas in the investment business, luck will always play a role.
The efficient markets hypothesis suggests that active managers have no skill. In its strong form, the hypothesis states that all currently known information is already reflected in security prices. Since all information is already in the prices, no additional information is available to active managers to use in generating exceptional returns. Active returns are completely random. The semistrong version states that all publicly available information is already reflected in security prices. Active management skill is really insider trading! The weak form of the hypothesis claims only that all previous price-based information is contained in current prices. This rules out technical analysis as skilled active management, but would allow for skillful active management based on fundamental and economic analysis.
There have also been many academic studies of active managers’ performance. These studies have focused on three related questions:
■ Does the average active manager outperform the benchmark?
■ Do the top managers have skill?
■ Does positive performance persist from period to period?
Chapter 20 will review these questions in more detail. The initial studies of mutual funds showed that the average manager underperformed the benchmark, in proportion to fund expenses, and that performance did not persist from period to period. Some recent studies have shown that the average manager matches the benchmark net of fees, that top managers do have statistically significant skill, and that positive performance may persist. Other studies have found no evidence for persistence of performance. The conclusion of all these conflicting studies is that even if performance does
Figure 17.2 Top quartile performance (IR = 0.5).
persist, it certainly doesn’t persist at any impressively high rate. Do 52 percent or 57 percent of winners repeat, and is this statistically significant?
We begin our in-depth discussion of performance analysis by defining returns—this may seem obvious, but there are several definitions. Should we use compound returns or average returns, arithmetic returns or logarithmic returns? Compound returns have the benefit of providing an accurate measure of the value of the ending portfolio.3 Arithmetic returns provide the benefit of using a linear model of returns across periods. We can see these points with an example. Let RP(t) be the portfolio’s total return in period t, and let RB(t) and RF(t) be the total return on the benchmark and the risk-free asset. The compound total return on portfolio P over periods 1 through T, RP(1,T), is the product
The geometric average return for portfolio P, gP is the rate of return per period that would give the same cumulative return:
The average log return zP is
or
The geometric average return is compounded annually, while the average log return is compounded continuously. Finally, the (arithmetic) average return aP is
It is always4 true that zP ≤ gP ≤ aP This does not necessarily say that one measure is better to use than the others. It does indicate that consistency is important, to make sure we are not comparing apples and oranges.
These issues become even more important when we attribute each period’s return to different sources, and then aggregate all the sources over time. To cumulate returns, we need to account for cross products. We discuss one approach to this in the technical appendix.
The simplest type of performance analysis is a table that ranks active managers by the total performance of their fund over some period. Table 17.1 illustrates a typical table, showing median performance, key percentiles, and the performance of a diversified and widely followed index (the S&P 500), for a universe of institutional equity portfolios covered by the Plan Sponsor Network (PSN) over the period January 1988 through December 1992. These crosssectional comparisons can provide a useful feel for the range of performance numbers over a period; however, they have four drawbacks:
■ They typically do not represent the complete population of institutional investment managers. Table 17.1 includes only those institutional equity portfolios that began no later than 1983, still existed in 1993, and are covered in the PSN database.
■ These cross-sectional comparisons usually contain survivorship bias, which is increasingly severe the longer the horizon. Table 17.1 does not include firms that went out of business between 1983 and 1993.
■ These cross-sectional comparisons ignore the fact that some of the reporting managers are managing $150 million portfolios, while others are managing $15 billion portfolios. The rule is one man, one vote—not one dollar, one vote.
■ Cross-sectional comparisons do not adjust for risk. The top performer may have taken large risks and been lucky. We cannot untangle luck and skill in this comparison.
Figure 17.3 shows the impact of using a cross-sectional snapshot. Compare two managers, A and B. Over a 5-year period, Manager A has achieved a cumulative return 16 percent above the benchmark, while Manager B has outperformed by almost 20 percent. Based on this rather limited set of information, most people would prefer B to A, since B has clearly done better over the 5-year period.
Figure 17.3 Cumulative return comparison.
Figure 17.3, however, shows the time paths that A and B followed over the 5-year period. After seeing Fig. 17.3, most observers prefer A to B, since A obviously incurred much less risk than B in getting to the current position.5 If you had stopped the clock at most earlier times in the 5-year period, A would have been ahead.
Performance analysis must account for both return and risk.
The development of the CAPM and the notion of market efficiency in the 1960s encouraged academics to consider the problems of performance analysis. The CAPM maintained that consistent exceptional returns by one manager were unlikely. The academics devised tests to see if their theories were true, and the by-products of these tests are performance analysis techniques. These techniques analyze time series of returns. One approach, first proposed by Jensen (1968), separates returns into systematic and residual components, and then analyzes the statistical significance of the residual component. According to the CAPM, the residual return should be zero, and positive deviations from zero signify positive performance.
The CAPM also states that the market portfolio has the highest Sharpe ratio (ratio of excess return to risk), and Sharpe (1970) proposed performance analysis based on comparing Sharpe ratios. We will discuss the Jensen approach first, and then the Sharpe approach.
Basic returns-based performance analysis according to Jensen involves regressing the time series of portfolio excess returns against benchmark excess returns, as discussed in Chap. 12.
Figure 17.4 Returns regression.
Figure 17.4 shows the scatter diagram of excess returns to the Major Market Index portfolio and the S&P 500, together with a regression line, over the period from January 1988 through December 1992. The estimated coefficients in the regression are the portfolio’s realized alpha and beta:
Alpha appears in the diagram as the intercept of the regression line with the vertical axis. Beta is the slope of the regression line. For the above example, αP = 0.03 percent per month and βP = 0.92. The regression divides the portfolio’s excess return into the benchmark component, βP · rB(t), and the residual component, θP(t) = αP + εP(t). Note that in this example, the residual return is quite different from the active return, because the active beta is −0.08. While the alpha is 3 basis points per month, the average active return is −4 basis points per month.
The CAPM suggests that alpha should be zero. The regression analysis provides us with confidence intervals for our estimates of alpha and beta. The t statistic for the alpha provides a rough test of the alpha’s statistical significance. A rule of thumb is that a t statistic of 2 or more indicates that the performance of the portfolio is due to skill rather than luck. The probability of observing such a large alpha by chance is only 5 percent, assuming normal distributions.
The t statistic for the alpha is approximately
where αP and ωP are not annualized and T is the number of observations (periods). The t statistic measures whether αP differs significantly from zero, and a significant t statistic requires a large αP relative to its standard deviation, as well as many observations. For the example above, the t statistic for the estimated αP is only 0.36, not statistically distinct from zero.
Chapter 12 has already discussed t statistics and their relation to information ratios and information coefficients. The t statistic measures the statistical significance of the return. The information ratio measures the ratio of annual return to risk, and is related to investment value added. Though closely related mathematically, they are fundamentally different quantities. The t statistic measures statistical significance and skill. The information ratio measures realized value added, whether it is statistically significant or not. While Jensen focused on alphas and t statistics, information ratios, given their relationship to value added, are also important for performance analysis.
The basic alternative to the Jensen approach is to compare Sharpe ratios for the portfolio and the benchmark. A portfolio with
where 
denotes mean excess return over the period, has demonstrated positive performance. Once again, we can analyze the statistical significance of this relationship. Assuming that the standard errors in our estimates of the mean returns 
and 
dominate the errors in our estimates of σP and σB, the standard error of each Sharpe ratio is approximately 
, where N is the number of observations. Hence a statistically significant (95 percent confidence level) demonstration of skill occurs when6
Dybvig and Ross (1985) have shown7 that superior performance according to Sharpe implies positive Jensen alphas, but that positive Jensen alphas do not imply positive performance according to Sharpe.
There are several refinements of the returns-only regression-based performance analysis. Some are statistical in nature. They refine the statistical tests. Examples of statistical refinements include Bayesian corrections and adjustments for heteroskedasticity and autocorrelations. Other refinements stem from financial theory. They attempt to extract additional information from the time series of returns. Examples of financial refinements include analyzing benchmark timing, using a priori betas, analyzing value added, controlling for public information, style analysis, and controlling for size and value. The last three refinements are controversial, in that they all argue that managers should receive credit only for returns beyond those available through various levels of public information. These proposals raise the bar on an already difficult enterprise.
The first statistical refinement is a Bayesian correction. The Bayesian correction allows us to use our prior knowledge about the distribution of alphas and betas across managers. For example, imagine that we know that the prior distribution of monthly alphas has mean 0 and standard deviation of 12.5 basis points per month. We then expect an alpha of 0, and would be “surprised” (a two-standard-deviation event) if the alpha were more than ±3.00 percent per year (25 basis points per month). We can apply similar logic to the observed betas. The Bayesian analysis allows one to take this prior information into consideration in making judgments about the “true” values of αP and βP For more information about this topic, see Vasicek (1973).
One of the assumptions underlying the regression model is that the error terms εP(t) have the same standard deviation for each t. We can employ various schemes to guard against failure of that assumption. We call this heteroskedasticity in the regression game.
A third statistical problem is autocorrelation. We assume that the error terms εP(t) are uncorrelated. If there is significant autocorrelation, then we can make an adjustment. This arises, for example, if we examine returns on overlapping periods.
One financially based refinement to the regression model is a benchmark timing component. The expanded model is
We include the variable yP to determine whether the manager has any benchmark timing skill. The model includes a “downmarket” beta, βP and an “up-market” beta, βP + γP. If γP is significantly positive, then we say that there is evidence of timing skill; benchmark exposure is significantly different in up and down cases. Figure 17.5 indicates how βP, αp and γP relate to performance.
In our example of the Major Market Index portfolio versus the S&P 500 portfolio, not surprisingly, there is no evidence of benchmark timing ability. Over the period from January 1988
through December 1992, βP = 0.95 and γP =-0.05. The coefficient γP is not statistically distinct from zero, with a t statistic of only −0.41.
There is a longer discussion of the performance measurement aspects of benchmark timing in Chap. 19. See also the paper by Henriksson and Merton (1981).
Another embellishment of returns-based analysis is improved estimation of the beta. This can take the form of using a beta that is estimated before the fact. As we will discuss in Chap. 19, this can help in avoiding spurious correlations between the portfolio returns and benchmark returns. In the example of the Major Market Index portfolio versus the S&P 500 from 1988 through 1992, this can make a difference. While the realized beta was 0.92, the monthly forecast beta over the period ranged from 0.98 to 1.03. Changing from realized to forecast beta changes the portfolio’s alpha from 3 basis points per month to −4 basis points per month.
A different approach to analyzing the pattern of returns is to use the concept of value added and ideas from the theory of valuation (Chap. 8). The idea is to look at the pattern of portfolio excess returns and market excess returns. Suppose we have T = 60 months of returns, {rP(t), rB(t), rF(t)} for t = 1, 2,..., T. We can think of a deal that says, “In the future the returns will equal {rP(t), rB(t), rF(t)} with probability 1/T.” How much would you pay for the opportunity to get the portfolio return under those conditions? You would pay one unit to get the risk-free or market returns; i.e., they are priced fairly. If the portfolio performs very well, you might be willing to pay 1.027 to get the portfolio returns. In that case, we say that the value added is 2.7 percent. If you were willing to pay only 0.974, then there would be a loss in value of 2.6 percent. The appendix shows how this analysis might be carried out.
Ferson and Schadt (1996) and Ferson and Warther (1996) have argued that the standard regression [Eq. (17.7)] doesn’t properly condition for different market environments. They claim two things: first, that public information on dividend yields and interest rates can usefully predict market conditions, and second, that managers earn their living through nonpublic information. As a result, they adjust the basic CAPM regression to condition for public information. For example, they suggest the regression
Equation (17.12) basically allows for beta varying with economic conditions, as modeled linearly through the market dividend yield y(t) and the risk-free rate iF(t). Many managers would argue, with some justification, that Eq. (17.12) penalizes them by including ex post insight into the relationship between yields, interest rates, and market conditions.
So far, all the advances discussed in returns-based performance analysis still rely on a prespecified benchmark, typically a standard index like the S&P 500. Sharpe (1992) proposed style analysis to customize a benchmark for each manager’s returns, in order to measure the manager’s contribution more exactly.
Style analysis attempts to extract as much information as possible from the time series of portfolio returns without requiring the portfolio holdings. Like the factor model approach, style analysis assumes that portfolio returns have the form
where the {rj(t)} are returns to J styles, the hPj measure the portfolio’s holdings of those styles, and uP(t) is the selection return, the portion of the return which style cannot explain. Here the styles typically allocate portfolio returns along the dimensions of value versus growth, large versus small capitalization, domestic versus international, and equities versus bonds. In addition to the returns to the portfolio of interest, the estimation approach also requires returns to portfolios that capture those styles.
We estimate holdings hPj via a quadratic program:
subject to
and
This differs from regression in two key ways. First, the holdings must be nonnegative and sum to 1. Second, the procedure minimizes the variance of the selection returns, not 
. The objective does not penalize large mean selection returns—as regression would do—but only variance about that mean.
Style analysis requires only the time series of portfolio returns and the returns to a set of style indices. The result is a top-down attribution of the portfolio returns to style and selection. According to style analysis, the style holdings define the type of manager, and the selection returns distinguish among managers. Managers can demonstrate skill by producing large selection returns. We can calculate manager information ratios using the mean and standard deviation of the managers’ selection returns.
In general, we can use style analysis to (1) identify manager style, (2) analyze performance, and (3) analyze risk. The first application, identifying manager style, is controversial. Several researchers [e.g., Lobosco and DiBartolomeo (1997) and Christopherson and Sabin (1998)] have pointed out the large standard errors associated with the estimated weights, driven in part by the significant correlation between the style indices. But this application, by itself, is of limited use. Identifying manager style usually requires no fancy machinery. Managers publicize their styles, and a peek at their portfolios can usually verify the claim.
Style-based performance analysis may also be inaccurate, although it is usually an improvement over the basic returns-based methodologies. It is an excellent tool for large studies of manager performance. Inaccuracies tend to cancel out from one manager to another in the large sample, and accurate and timely information on portfolio holdings is unavailable.
Risk analysis could use style analysis to identify portfolio exposures to style indices. Risk prediction would follow from these exposures, a style index covariance matrix, and an estimate of selection risk (based on historical selection returns). We could assume selection returns uncorrelated across managers. Once again, this would improve on risk prediction based only on beta, but would fall far short of the structural models we discussed in Chap. 3.
Fama and French (1993) have proposed a performance analysis methodology very similar in spirit to Sharpe’s style analysis. Their approach to performance uses the regression
This looks like a standard CAPM regression with two additional terms. The return SMB(t)(“small minus big”) is the return to a portfolio long small-capitalization stocks and short large-capitalization stocks. The return HML(t)(“high minus low”) is the return to a portfolio long high-book-to-price stocks and short low-book-to-price stocks. So Sharpe uses a quadratic programming approach and indices split along size and value (book-to-price) dimensions. Fama and French control along the same dimensions and use standard regression.
How do they build their two portfolio return series? First, each June, they identify the median capitalization for New York Stock Exchange (NYSE) stocks. They use that median to classify all stocks (including AMEX and NASDAQ stocks) as S (for small) or B (for big).
Second, using end-of-year data, they sort all stocks by book-to-price ratios. They classify the bottom 30 percent as L (for low), the middle 40 percent as M (for medium), and the top 30 percent as H (for high). These two splits lead to six portfolios: S/L, S/M, S/H, B/L, B/M, and B/H.
They then calculate capitalization-weighted returns to each of the six portfolios.
Finally, they define SMB(t) as the difference between the simple average of S/L, S/M, and S/H and the simple average of B/L, B/M, and B/H. Effectively, SMB(t) is the return on a net zero investment portfolio that is long small-capitalization stocks and short large-capitalization stocks, with long and short sides having roughly equal book-to-price ratios.
Similarly, they define HML(t) as the difference between the average of S/H and B/H and the average of S/L and B/L. Once again, this is the return on a net zero investment portfolio that is long high-book-to-price stocks and short low-book-to-price stocks, with long and short sides having roughly equal market capitalizations.
Carhart (1997) has extended this approach by also controlling for past 1-year momentum.
Returns-based analysis is a top-down approach to attributing returns to components, expost, and statistically analyzing the manager’s added value. At its simplest, the attribution is between systematic and residual returns, with managers given credit only for achieved residual returns. Style analysis is similar in approach, attributing returns to several style classes and giving managers credit only for the remaining selection returns. Returns-based performance analysis schemes typically allocate part of the returns to systematic or style components and give managers credit only for the remainder.
Portfolio-based performance analysis is a bottom-up approach, attributing returns to many components based on the ex ante portfolio holdings and then giving managers credit for returns along many of these components. This allows the analysis not only of whether the manager has added value, but of whether he or she has added value along dimensions agreed upon ex ante. Is he a skillful value manager? Does her value added arise from stock selection, beyond any bets on factors? Portfolio-based performance analysis can reveal this. In contrast to returns-based performance analysis, performance-based analysis schemes can attribute returns to several components of possible manager skill.
The returns-only analysis works without the full information available for performance analysis. We can say much more if we look at the actual portfolios held by the managers. In fact, two additional items of information can help in the analysis of performance:
■ The portfolio holdings over time
■ The goals and strategy of the manager
The analysis proceeds in two steps: performance attribution and performance analysis. Performance attribution focuses on a single period, attributing the return to several components. Performance analysis then focuses on the time series of returns attributed to each component. Based on statistical analysis, where (if anywhere) does the manager exhibit skill and add value?
Performance attribution looks at portfolio returns over a single period and attributes them to factors. The underlying principle is the multiple-factor model, first discussed in Chap. 3:
Examining returns ex post, we know the portfolio’s exposures xpj(t) at the beginning of the period, as well as the portfolio’s realized return rp(t) and the estimated factor returns over the period. The return attributed to factor j is
The portfolio’s specific return is up(t).
We are free to choose factors as described in Chap. 3, and in fact we typically run performance attribution using the same riskmodel factors. However, we are not in principle limited to the same factors as are in our risk model. In general, just as in the returns-based analysis, we want to choose some factors for risk control and others as sources of return. The risk control factors are typically industry or market factors, although later we can analyze skill in picking industries.
The return factors can include typical investment themes such as value or momentum. In building risk models, we always use ex ante factors: that is, those based on information known at the beginning of the period. For return attribution, we could also consider ex post factors: that is, those based on information known only at the end of the period. For example, we could use a factor based on IBES earnings forecasts available at the end of the period. We could interpret returns attributed to this factor as evidence of the manager’s skill in forecasting IBES earnings projections.
Beyond the manager’s returns attributed to factors will remain the specific return to the portfolio. A manager’s ability to pick individual stocks, after controlling for the factors, will appear in this term. We call this term specific asset selection.
We typically think of the specific return as the component of return which cross-sectional factors cannot explain. That view suggests that we simply lump the portfolio’s specific return all together. But for an individual strategy, some attributions of specific return may also make sense. If our strategy depends on analyst information, we may want to group specific returns by analyst. We think our auto industry analyst adds value. If this is true, we should see a positive contribution from auto-stock specific asset selection. Similarly, the specific returns can tell us if our strategy works better in some sectors than in others. This term doesn’t tell us whether we have successfully picked one sector over another, it tells us whether we can pick stocks more accurately in one sector than in another.
Note that we have many choices as to how to attribute returns. We can choose the factors for attribution. We can attribute specific returns. We can even attribute part of our returns to the constraints in our portfolio construction process (e.g., we lost 32 basis points of performance last year as a result of our optimizer constraints).8 Performance attribution is not a uniquely defined process. Commercially available performance analysis products choose widely applicable attribution schemes. Customized systems have no such limitations.
We can apply performance attribution to total returns, active returns, and even active residual returns. For active returns, the analysis is exactly the same, but we work with active portfolio holdings and returns:
To break down active returns into systematic and residual, remember that we can define residual exposures as
where we simply subtract the active beta times the benchmark’s exposure from the active exposure, and residual holdings similarly as
Substituting these into Eq. (17.20), and remembering that 
, we find
Equation (17.23) will allow a very detailed analysis of the sources of active returns relative to the benchmark.
As an example of performance attribution, consider the analysis of the Major Market Index portfolio versus an S&P 500 benchmark over the period January 1988 through December 1992. For now, focus on the returns over January 1988. Using the BARRA U.S. Equity model (version 2), the factor exposures are shown in Table 17.2.
Table 17.2 illustrates the attributed active return. Table 17.3 summarizes the attribution between systematic and residual this month. The active beta of the Major Market Index versus the S&P 500 is only 0.02, and so the active residual component is very
close to the active component. Comparing Tables 17.2 and 17.3, the active common-factor component is −0.84 percent and the active residual common-factor component is −0.75 percent.
Performance analysis begins with the attributed returns each period, and looks at the statistical significance and value added of the attributed return series. As before, this analysis will rely on t statistics and information ratios to determine statistical significance and value added.
For concreteness, consider the attribution defined in Eq. (17.23), with active returns separated into systematic and residual, and active residual returns further attributed to common factors and specific returns.
Start with the time series of active systematic returns. Most straightforward is a simple analysis of the mean return and its t statistic. However, according to the CAPM, we expect a positive return here if the active beta is positive on average. Hence, we will go one additional step and separate this time series into three components: one arising from the average active beta and the expected benchmark return, one arising from the average active beta and the deviation of realized benchmark return from its expectation, and the third from benchmark timing—deviations of the active beta from its mean. The first component, based on the average active beta and the expected benchmark return, is not a component of active management.
The total active systematic return over time is
In Eqs. (17.24) through (17.27), 
is the average active beta, 
is the average benchmark excess return over the period, and μB is the long-run expected benchmark excess return.
The analysis of the time series of attributed factor returns and specific returns is more straightforward.9 We can examine each series for its mean, t statistic, and information ratio. For these, we need not only the mean returns, but also the risk for each factor. We can base risk on the realized standard deviation of the time series or on the ex ante forecast risk. The technical appendix describes an innovative approach which combines the two risk estimates, weighting realized risk more heavily the more observations there are in the analysis period.
Performance analysis, just like performance attribution, is not uniquely defined. The scheme outlined here is simply a reasonable approach to distinguishing the various sources of typical strategy returns. It will sometimes prove useful to customize a performance analysis scheme to a particular strategy, in order to isolate more precisely its sources of value added.
Table 17.4 summarizes this analysis for the example of the Major Market Index portfolio versus the S&P 500 benchmark.10 Not surprisingly, given this example, Table 17.4 exhibits no strong demonstrations of skill or value added.
Now that we have analyzed each source of risk in turn, we can identify the best and worst policies followed by the manager: those time series which have achieved the highest and lowest returns on average. Here is where the manager’s predefined goals and strategies should shine through. Stock pickers should see specific asset selection as one of their best strategies. Value managers should see value factors as their best strategies. Ex ante strategies that are inconsistent with best policy analysis can signal to the owner of the funds that the active manager has deviated in strategy and can signal to the manager that the strategy isn’t doing what he or she expects it to do.
Table 17.5 displays the best and worst policies for the example Major Market Index portfolio versus the S&P 500 benchmark. Previous analysis showed that the example included no demonstration of skill or value added, and comparing Table 17.5 to Table 17.2, we can see that the best and worst policies simply correspond to the largest-magnitude active exposures.
The goal of performance analysis is to separate skill from luck. The more information available, the better the analysis. Using a simple cross section of returns to differentiate managers is insufficient. A time series of returns to managers and benchmarks can separate skill from luck. The most accurate performance analysis utilizes information on portfolio holdings and returns over time, to not only separate skill from luck, but also identify where the manager has skill.
The science of performance analysis began in the 1960s with the seminal academic work of Sharpe (1966, 1970), Jensen (1968), and Treynor (1965). They used the CAPM as a starting point for developing the returns-based methodology described in this chapter. Their goal was to test market efficiency and analyze manager performance, a topic we will cover in Chap. 20.
Since then, many other academics have developed performance analysis methodologies, often motivated by the desire to further test market efficiency and manager performance. Some advances have come from application of clever statistical insights to the CAPM framework. Other refinements have followed new developments in finance theory. For example, Fama and French (1993) have proposed a new scheme which explicitly controls for size and book-to-market effects.
Most often, the academic treatments have focused on returns-based analysis, although Daniel, Grinblatt, Titman, and Wermers (1997) control for size, book-to-price, and momentum at the asset level (using quintile portfolios) and then aggregate specific returns up to the portfolio level.
Most of these new academic developments are contained within the practitioner-developed portfolio-based analysis methodology described in this chapter.
Beckers, Stan. “Manager Skill and Investment Performance: How Strong Is the Link?” Journal of Portfolio Management, vol. 23, no. 4, 1997, pp. 9-23.
Carhart, Mark M. “On Persistence in Mutual Fund Performance.” Journal of Finance, vol. 52, no. 1, 1997, pp. 57-82.
Christopherson, Jon A., and Frank C. Sabin. “How Effective Is the Effective Mix?” Journal of Investment Consulting, vol. 1, no. 1, 1998, pp. 39-50.
Daniel, Kent, Mark Grinblatt, Sheridan Titman, and Russ Wermers. “Measuring Mutual Fund Performance with Characteristic-based Benchmarks.” Journal of Finance, vol. 52, no. 3, 1997, pp. 1035-1058.
DeBartolomeo, Dan, and Erik Witkowski. “Mutual Fund Misclassification: Evidence Based on Style Analysis.” Financial Analysts Journal, vol. 53, no. 5, 1997, pp. 32-43.
Dybvig, Philip H., and Stephen A. Ross. “The Analytics of Performance Measurement Using a Security Market Line.” Journal of Finance, vol 40, no. 2, 1985, pp. 401-416.
Fama, Eugene F., and Kenneth R. French. “Common Risk Factors in the Returns on Stocks and Bonds.” Journal of Financial Economics, vol. 33, no. 1, 1993, pp. 3-56.
Ferson, Wayne E., and Rudi W. Schadt. “Measuring Fund Strategy and Performance in Changing Economic Conditions.” Journal of Finance, vol. 51, no. 2, 1996, pp. 425-461.
Ferson, Wayne E., and Vincent A. Warther. “Evaluating Fund Performance in a Dynamic Market.” Financial Analysts Journal, vol. 52, no. 6, 1996, pp. 20-28.
Grinold, Richard C., and Kelly K. Easton. “Attribution of Performance and Holdings.” In Worldwide Asset and Liability Modeling, edited by William T. Ziemba and John M. Mulvey (Cambridge, England: Cambridge University Press, 1998), pp. 87-113.
Henriksson, Roy D., and Robert C. Merton. “On Market Timing and Investment Performance II. Statistical Procedures for Evaluating Forecasting Skills.” Journal of Business, vol 54, no. 4, 1981, pp. 513-533.
Ippolito, Richard A. “On Studies of Mutual Fund Performance 1962-1991,” Financial Analysts Journal, vol. 49, no. 1, 1993, pp. 42-50.
Jensen, Michael C. “The Performance of Mutual Funds in the Period 1945-1964.” Journal of Finance, vol. 23, no. 2, 1968, pp. 389-416.
Jones, Frank J., and Ronald N. Kahn. “Stock Portfolio Attribution Analysis.” In The Handbook of Portfolio Management, edited by Frank J. Fabozzi (New Hope, PA: Frank J. Fabozzi Associates, 1998), pp. 695-707.
Lehmann, B., and D. Modest. “Mutual Fund Performance Evaluation: A Comparison of Benchmarks and Benchmark Comparisons.” Journal of Finance, vol. 42, no. 2, 1987, pp. 233-265.
Lobosco, Angelo, and Dan DiBartolomeo. “Approximating the Confidence Intervals for Sharpe Style Weights.” Financial Analysts Journal, vol. 53, no. 4,1997, pp. 80-85.
Modigliani, Franco, and Leah Modigliani. “Risk-Adjusted Performance.” Journal of Portfolio Management, vol. 23, no. 2, 1997, pp. 45-54.
Rudd, Andrew, and Henry K. Clasing, Jr. Modern Portfolio Theory, 2nd ed. (Orinda, Calif.: Andrew Rudd, 1988).
Sharpe, William F. “Mutual Fund Performance.” Journal of Business, vol 39, no. 1, 1966, pp. 119-138.
____. Portfolio Theory and Capital Markets (New York: McGraw-Hill, 1970).
____. “Asset Allocation: Management Style and Performance Measurement.” Journal of Portfolio Management, vol. 18, no. 2, 1992, pp. 7-19.
Treynor, Jack L. “How to Rate Management of Investment Funds.” Harvard Business Review, vol. 43, no. 1, January-February 1965, pp. 63-75.
Treynor, Jack L., and Fischer Black. “Portfolio Selection Using Spectal Information under the Assumptions of the Diagonal Model with Mean Variance Portfolio Objectives and Without Constraints.” In Mathematical Models in Investment and Finance edited by G. P. Szego and K. Shell (Amsterdam: North-Holland 1972).
Vasicek, Oldrich A. “A Note on Using Cross-Sectional Information in Bayesian Estimation of Security Betas.” Journal of Finance, vol. 28, no. 5, 1973, pp. 1233-1239.
1. Joe has been managing a portfolio over the past year. Performance analysis shows that he has realized an information ratio of 1 and a t statistic of 1 over this period. He argues that information ratios are what matter for value added, and so who cares about t statistics? Is he correct? What can you say about Joe’s performance?
2. Jane has managed a portfolio for the past 25 years, realizing a t statistic of 2 and an information ratio of 0.4. She argues that her t statistic proves her skill. Compare her skill and added value to Joe’s.
3. Prove the more exact result for the standard error of the information ratio,
Assume that errors in the mean and standard deviation of the residual returns are uncorrelated, and use the normal distribution result:
for a sample standard deviation from N observations.
4. Show that changing the information ratio from an annualized to a monthly statistic does not improve our ability to measure investment performance. It will still require a 16-year track record to demonstrate top-quartile performance with 95 percent confidence. First calculate the standard error of a monthly IR. Second, convert a top-quartile IR of 0.5 to its monthly equivalent. Finally, calculate the required time period to achieve a statistic of 2.
5. Using Table 17.2, identify the largest active risk index and industry exposures and the largest risk index and industry attributed returns for the Major Market Index versus the S&P 500 from January 1988 to December 1992. Must the largest attributed returns always correspond to the largest active exposures?
6. Given portfolio returns of {5 percent, 10 percent, −10 percent} and benchmark returns of {1 percent, 5 percent, 10 percent}, what is the cumulative active return over this period? What are the cumulative returns to the portfolio and benchmark?
7. Why should portfolio-based performance analysis be more accurate than returns-based performance analysis?
8. How much statistical confidence would you have in an information ratio of 1 measured over 1 year? How many years of performance data would you need in order to measure an information ratio of 1 with 95 percent confidence?
9. Show that a portfolio Sharpe ratio above the benchmark Sharpe ratio implies a positive alpha for the portfolio, but that a positive alpha does not necessarily imply a Sharpe ratio above the benchmark Sharpe ratio.
We will discuss three technical topics in this appendix: how to cumulate attributed returns, how to combine forecast and realized risk numbers for performance analysis, and a valuation-based approach to performance analysis.
We will investigate two issues here: cumulating active returns and cumulating more generally attributed returns. Let RP(t) be the portfolio’s total return in period t, and let RB(t) and RF(t) be the total return on the benchmark and the risk-free asset. The compound total return on portfolio P over periods 1 through T, RP(1,T), is the product
Similarly, we calculate the cumulative benchmark return as
Hence the active cumulative return must be
Note that we do not calculate active cumulative returns by somehow cumulating the period-by-period active returns. For example,
Now consider the more general problem of cumulating attributed returns, and just focus on the problem of cumulating the portfolio returns (not active returns). For each period t,
and hence
Equation (17A.6) contains many cross-product terms. We would like to write this as
attributing the cumulative return linearly to factors plus a cross-product correction δCP. There are two straightforward approaches to defining the cumulative attributed returns one based on a bottom-up view and the other based on a top-down view. The bottom-up view cumulates each attributed return in isolation:
The top-down view attributes cumulative returns by deleting each factor in turn from the cumulative total return and observing the effect:
We recommend the top-down approach [Equation (17A.9)], which often leads to smaller cross-product correction terms δCP in Eq. (17A.7). Given that the cross-product term is usually small, and that intuition for it is limited, we often attribute the cross-product term back to the factors, proportional to either the factor risk or the factor return.
We observe returns over T periods t = 1,..., T and wish to analyze performance. Prior to the period, the estimated risk of these returns was σprior(0). The realized risk for the returns is σreal. Both risk numbers are sample estimates of the “true” risk. What is the best overall estimate of risk, given these two estimates?
According to Bayes, if we have two estimates, x1 with standard error σ1 and x2 with standard error σ2, and the estimation errors are uncorrelated, then the best linear unbiased estimate, given these two estimates, is
Equation (17A.10) provides the overall estimate with minimum standard error σ.
We also know that the standard error of a sampled variance is approximately
where T is the number of observations in the sample and we assume that the distribution of the underlying variable is normal.
Combining Eqs. (17A.10) and (17A.11), our best risk estimate is
where T0 measures the number of observations11 used for the estimate of σprior(0).
The theory of valuation (Chap. 8) defined valuation multiples υ such that
where p(0) is the current value of the asset based on its possible future values cf(T) at time T. Defining total returns as
we see that
One aspect of the valuation multiples, which is shown by Eq. (17A.15), is that they fairly price all assets. Within the context of the CAPM and APT, all returns are fairly priced with respect to portfolio Q. A manifestation of this is that under this adjusted measure, they all have the same value. Equation (17A.15) states that the set of possible returns R should be worth $1.00 using the valuation multiples u.
In the valuation-based approach to performance analysis, the benchmark plays the role of portfolio Q. We determine the valuation multiples by the requirement that they fairly price the benchmark and the risk-free asset. The observed set of benchmark returns and the observed set of risk-free returns will each be priced at $1.00. How much will the observed portfolio returns be worth then?
How do we choose the valuation measure? We could use the results from Chap. 8, that
This has certain problems, as discussed before; for instance, it isn’t guaranteed to be positive. Alternatively, we can use a result from continuous time option theory, that
where we use δ here as a proportionality constant. Given Eq. (17A.17), the valuation multiples are guaranteed to be positive, and we can choose δ and σ by the requirement that they fairly price the observed set of benchmark returns and risk-free returns:
Once we have used Eqs. (17A.18) and (17A.19) to determine δ and σ, we can calculate the value added of the portfolio as
We can apply Eq. (17A.20) to attributed returns, as well as to the total portfolio returns, to calculate value added factor by factor:
Now, using Eq. (17A.19) and switching the summation order leads to
1. Over a 60-month period, the forecast market variance was (17 percent)2, with a standard error of (5.1 percent)2, and the realized (sample) variance was (12 percent)2. What is the best estimate of market variance over this period?
1. Compare the Major Market Portfolio to the S&P 500 over the past 60 months. What were the best and worst policies of this active portfolio?
2. What are the largest and smallest attributed returns in the most recent month?
