CFA Society Chicago Book Club:

How to Measure Anything: Finding the Value of Intangibles in Business by Douglas W. Hubbard

how-to-measure-anythingPath dependence is the phenomenon often used to explain why people sometimes persist with practices that are no longer optimal or economically rational. Statistics is another area where path dependence has struck. The statistical techniques that students learn in school, the ones that practitioners apply in industry, and the ones researchers use in journal publications often aren’t the best or the most appropriate ones but rather the ones that continue to be used because they’ve always been used.  Douglas Hubbard’s How to Measure Anything (2010) attempts to update some of those techniques for the 21st century.  In addition, he offers some refreshing perspectives on behavioral finance and the biases that adversely affect decision makers, even the so-called professional decision makers at the executive level in industry and government. Finally, he offers a unifying framework for decomposing complex problems into individual variables, assessing the value of reducing the uncertainty for each of those variables, measuring those variables, and finally determining probabilities through Monte Carlo simulations and Bayesian statistics.

Starting with antiquated statistical techniques, every former stats 101 student probably remembers going through some type of hypothesis testing exercise such as testing if a coin is fair, a drug works, voters prefer a candidate, etc. Those tests take a null hypothesis, such as assuming that a coin is fair, flipping it multiple times, and then determining the probability of observing a series of outcomes if the coin were fair. If the probability of observing a series of outcomes on a supposedly fair coin is less than some arbitrary threshold, usually five percent, the experimenter rejects the null hypothesis and concludes that the coin is not fair.  For example, the probability of observing five heads out of five flips on a fair coin is 3.13%, which would cause an experimenter using the five percent threshold to reject the null hypothesis that the coin is fair. The five percent shibboleth comes from the statistician Sir Ronald Fisher’s 1925 paper “Statistical Methods for Research Workers.” He wrote a year later in “The Arrangement of Field Experiments” (1926) that the threshold was arbitrary and that other thresholds may be used; however, the damage had been done and the five percent threshold remains as a venerated relic.

The whole process is a convoluted way to approximate the more useful question: What’s the probability of getting a heads on a given coin? The Bayesian approach to statistics, in contrast to the frequentist approach previously described, seeks to do just that. Mr. Hubbard notes that the term “Bayesian” was first used by Fischer himself as a derogatory reference to adherents of the approach named after Rev. Thomas Bayes. Rev. Bayes is credited with developing the first formulation of how new evidence can be used to update prior beliefs.  In the case of Bayesian statistics, new evidence is used to update prior assumptions about probabilities.

Once the distribution of the relevant variables or drivers is better known, Mr. Hubbard postulates a relationship between the variables and generates a hypothetical distribution of the phenomenon that one is trying to predict using Monte Carlo simulations. First developed to solve intractable problems in nuclear physics, modern computing power has made the technique accessible to anyone with a personal computer and an Excel spreadsheet. Instead of trying to compute the probability of a phenomenon such as rolling a two with a pair of dice (“snake eyes”), Monte Carlo simulations flip the problem by simulating thousand or perhaps millions of rolls and then determining what percentage of the rolls were twos. With an Excel spreadsheet, Mr. Hubbard shows how to calculate distributions and expected values for complex phenomenon after estimating the distribution of the underlying variables and their relationships. Monte Carlo simulations are seldom taught in introductory statistics courses. The topic is usually reserved for advanced classes and special topics classes even though the basics of the technique are no more complicated than regression modeling and several other topics that are covered in introductory classes.

With new statistical tools in tow, Mr. Hubbard then sets forth on finding what to measure.  Here Mr. Hubbard again notes a pernicious tendency among decision makers to either measure what’s easy to measure or what they’re already familiar with. The solution, Mr. Hubbard argues, is to triage variable before trying to reduce uncertainty about them by introducing metrics to quantify the costs and benefits of acquiring additional information about each variable. He starts with the Expected Value of Perfect Information (EVPI): What would it be worth to know a presently unknown quantity with complete certainty? He then works backwards to determine the incremental Expected Cost of Information (ECI) and the incremental Expected Value of Information (EVI). Finally, he adds a time component, noting that for some decisions the value of information is perishable. Mr. Hubbard notes that adding the time component can prevent what pioneering decision theorist Howard Raiffa called, “Solving the right problem too late.”

In addition to the tendency to measure the wrong things and measure in the wrong amounts, Mr. Hubbard notes several other behavioral and cognitive biases, such as expectancy bias and overconfidence. Instead of just rehashing problems that already have been noted extensively in the behavioral finance literature, Mr. Hubbard goes further and offers solutions, especially to the problem of overconfidence and quantifying uncertainty. When asked to calculate a 90% confidence interval for an unknown quantity, such as the wingspan of a Boeing 747 aircraft, most people choose too narrow a range. Mr. Hubbard shows that with training the average person can estimate ranges for unknown quantities such that on average the true value falls within their estimated range 90% of the time. The training, called “calibration training,” is simple to conduct and has a tremendous success rate.  Organizations should probably spend more time training their executives to become better decision makers given how much time and money as they spend sending them to conferences, hiring executive coaches, and giving them physical and psychological assessments.

When the CFA Society Chicago’s Book Club met to discuss Mr. Hubbard’s book in April 2017, most of the participants welcomed his fresh approach to quantitative and empirical problem solving. If there were any misgivings about the book, they were that it didn’t fully live up to its title: “Finding the Value of Intangibles in Business.” The participants would have welcomed more examples of how the techniques described could be used to value business units or firms that make intensive use of intangibles such as brand identity, intellectual property, or perhaps others.

Hopefully, this won’t be the last time that Mr. Hubbard crosses paths with the Society and we’ll get to fulfill that promise.


Building Investor Trust Through GIPS

On May 9th, CFA Society Chicago members gathered to hear a panel of experts address the merits of adopting the Global Investment Performance Standards (GIPS) in the Vault Room at 33 N. LaSalle. The eminent panel comprised a service provider, a regulator, and an asset manager user and included:

  • Daniel Brinks, compliance examiner with the Securities and Exchange Commission (SEC) with a focus on investment advisors,
  • Richard Kemmling, CPA, CIPM, CGMA, President of Ashland Partners & Company LLC, a specialty CPA firm that was a pioneer in the GIPS verification business, and serves over 700 client firms in that area.
  • Matthew Lyberg, CFA, CIPM, Senior Vice President and Director of Performance Attribution with Acadian Asset Management.

DSC_3715Anju Grover, CIPM, senior GIPS analyst with the Investment Performance Standards Policy Group of the CFA Institute (CFAI) served as moderator. In her opening remarks she pointed out that 2017 marks the 30th anniversary of GIPS which she described as one of the CFA Institute’s most successful products. Despite the fact that adopting GIPS is completely voluntary, they are widely recognized as a best practice for reporting investment performance by asset managers, asset owners, and consultants all around the world.

The first question Ms. Grove put to the panel was why GIPS would be important to retail investors. Brinks responded that retail investors are just as demanding of a performance standard as are institutional investors, and GIPS fills the bill. Lyberg noted that the line separating retail and institutional investors is blurring. The decline in the popularity of pension plans in favor of defined contribution plans is a primary example. Retail investors are the end users of DC plans and are responsible for investment choices, but the plans are designed, managed, and overseen by investment professionals. So they serve both retail and institutional masters. GIPS also adds a layer of due diligence to a plan, a theme the panelists repeated throughout the event. Kemmling pointed out that GIPS compliance is a common requirement for listing products on the investment platforms that advisors (he specifically mentioned Morgan Stanley and Merrill Lynch) use for their retail clients.

As to challenges firms encounter in adopting GIPS, the panelists listed:

  • Lack of adequate data, or records; difficulty in handling unique accounts,
  • Changes in operating systems that occur during implementation,
  • Incomplete buy-in from all parts of a firm (marketing, accounting, compliance, etc.), and
  • Full support from senior management. The latter point is particularly critical to assure firms commit adequate resources to attain compliance.

Why should firms bear the cost of GIPS compliance? Kemmling answered that they provide a “best practices” process for client reporting and that the verification process provides insight into industry practices. Brinks stated that while GIPS compliance is not required by law or regulation, he considers it in the category of “nice to see” when he examines an asset manager. The verification process is a second pair of eyes –outside eyes–on results reporting. He added that he observes fewer serious problems in general when he examines firms that follow GIPS. The CFA Institute has been training SEC examiners on GIPS so they can understand what the standards mean to adopting firms and apply that knowledge during examinations.

In response to questions from the audience regarding difficulties in complying with GIPS, the panel noted challenges in applying them to more complicated strategies such as currency overlays and alternatives. They suggested that this be a focus of the next revision to the standards which is already underway and targeted for 2020. This revision should also make the standards easier to apply to fund vehicles and for internal reporting to management. The current standards are most easily applied to reporting composite returns to clients, which was their original intent.

Regarding the breadth of acceptance of GIPS, Grover said the CFAI is still gathering data but counts 1,600 firms around the world that claim compliance for at least a portion of their assets. This includes 85 of the 100 largest asset managers who account for 60% of total industry assets under management. Lyberg noted that investment consultants are expanding the adoption of GIPS by using compliance as a screen for including firms in management searches.

When asked how a firm should begin to adopt GIPS, Lyberg suggested starting out modestly by writing high level policies and procedures and making them more detailed over time with experience. He recommended attending the CFAI’s annual GIPS conference to build knowledge and to make contact with other firms that have already adopted the standards. Challenges a firm may encounter include clients who demand using a different performance benchmark than what the firms uses for a strategy, tension between various stakeholders at a firm (e.g., between marketing and compliance), and resistance from legal counsel which often advises against bold statements of compliance that might seem to be guarantees.

As to the benefits to the public from using GIPS, Brinks stated that increased comparability leads to better informed investment decisions and more efficient markets. He noted the decline in fraud tied to inflated claims about performance since the introduction of GIPS thirty years ago. Kemmling noted that measuring the positive impact of GIPS is difficult but they were created for the benefit of investors and are an indication of asset managers’ commitment of resources in support of investors. Grover stated that adopting GIPS for greater transparency and comparability was simply “the right thing to do”.

For final takeaways the panelists offered the following:

  • Lyberg said GIPS levels the playing field among managers, adding that compliant managers couldn’t compete with fraudulent firms such as Bernie Madoff’s.
  • Kemmling, acknowledged that while compliance is not easy, it isn’t expensive and is certainly achievable. Most of the 700 firms his company verifies have less than $1 billion in AUM, indicating the success of small firms at complying with GIPS.
  • Brinks recommended that adopting firms think very carefully about how to apply the standards, looking to the future when writing their policies and procedures to avoid any potential conflicts between them and their capabilities.

Volunteer of the Month: May 2017


Brad Adams, CFA

Longtime member (22 years to be exact!), Brad Adams, CFA, has served on five advisory groups since 2012. In 2014 he also chaired what was previously the Finance Advisory Group. Currently, he is a member of both the Communications and Professional Development Advisory Groups.

Today Brad is being recognized for his efforts on the Communications Advisory Group. Regularly attending meetings and volunteering to be a reporter at the Society’s events, Brad has made a strong contribution to the Communications Committee.

Brad has published seven CFA Society Chicago Blog articles since the launch of the online publication and is scheduled for more. Help us in thanking Brad for attending Society events and keeping members informed!

Job well done! We appreciate all you do for the Society.