Analyzing the Due Diligence Process to Produce Better Outcomes

Despite its importance throughout the investment ecosystem, due diligence receives relatively little attention and study compared to other core functions.

There isn’t much industry-produced research about the “how” of due diligence.  One of the goals of our Advanced Due Diligence and Manager Selection course is to provide a collection of sources on such topics.  (The main goal is advancing the state of the art in the qualitative analysis of investment organizations.)

There is also a paucity of academic research on the subject.  In the first line of their paper, “Due Diligence and the Allocation of Venture Capital,” Jack Xiaoyong Fu and Lucian Taylor write, “Due diligence is widespread in practice but largely absent from academic research.”

The paper includes a number of interesting conclusions which we’ll cover before considering broader issues regarding due diligence.

A novel approach

In the abstract, Fu and Taylor summarize their methodology:  “Using cell phone signal data, we measure the duration of pre-investment meetings between venture capitalists (VCs) and startup employees.”

What that means in practice:

To construct our proxy for the amount of due diligence on a VC deal, we analyze cell phone signals near VC and startup offices.  This fully anonymous dataset includes cell phone location information with timestamps, recorded whenever an app is active or running in the background.  While this technology fully complies with legal standards and contains no individually identifiable information, it still allows us to measure potential meetings between VCs and startup employees.

Note the word potential.  The authors thoroughly cover problems with the data.  Most importantly, the analysis only covers in-person meetings at VC or startup offices, leaving out phone calls, virtual meetings, and meetings at other locations.  A variety of database and technology issues mean that the dataset is not as complete and accurate as would be preferred.  Also, in-person meetings were disrupted by the pandemic, so trends in visits were altered during that time.

The phone information was matched to PitchBook data on 22,000 deals, enabling the series of analyses about the frequency and length of in-person meetings (and their relationship to investment outcomes) that are covered in the paper.

Conclusions in the paper

The authors had hypothesized that there would be less due diligence done during “hot” periods in the markets for deals.  The figure below shows the relationship between quarterly changes in deal volume (shifted one quarter forward) and the median due diligence minutes over the same two quarters; the results match the hypothesis:  “In short, busier investors perform less due diligence.”

In one sense that seems obvious, since with a static staff there is less time for each deal when there are more deals to be researched.  But that simple calculation raises a question about whether due diligence standards should change when the demand for venture capital does.

Other findings (with “VCs” referring to the broad range of investors in venture capital, not just VC funds):

~ There is “a strong negative relation between diligence levels and the number of investments the VC makes within 18 months of the focal deal.”

~ “Some VCs ‘spray and pray,’ making many investments with little due diligence.  Other VCs follow a more selective strategy, making fewer investments but performing more diligence.”

~ The authors looked at different kinds of investors, concluding that “diligence levels are lower for ‘nontraditionals’ such as [corporate venture capital buyers], growth/expansion investors, hedge funds, and other asset managers.”  Despite their higher level of assets under management, asset managers generally do less due diligence.  (As covered in an issue of the Fortnightly, Morningstar concluded that “mutual fund managers do not appear to be skilled private-company stock-pickers.”)

~ Geography plays into the choice as to whether to do in-person due diligence.  When the VC and the startup are further away from each other it happens less frequently.  One theme of the paper is that due diligence is costly.  The time and expense of travel is one of the more obvious costs.

~ “On average, diligence levels are higher for investors who manage fewer assets and make fewer other deals in the surrounding months.”

~ And, regarding the bottom line, “less due diligence is associated with more dispersed investment outcomes” and “an investor’s full-sample exit rate — a proxy for investment success — has a strong, positive correlation with its due diligence intensity.”

The bigger picture (part one)

Some investment practitioners are quick to dismiss conclusions coming out of academic research in their area of expertise — after all, they live in the real world, not in an ivory tower.  (That certainly isn’t the case with quantitative investors, who may argue about the methods employed in an analysis but are always thirsting for new ideas to incorporate into their strategies.)  A good rule for all is to consider the quality of the work and then match the evidence presented versus their experience and prior beliefs.  Sometimes a paper provides a piece to the ongoing puzzle of market behavior and opportunity.  Avoiding academic research by inclination prevents capitalizing on that possibility.

In this case, Fu and Taylor have been clear that their approach only addresses one aspect of due diligence — in-person meetings — and for a variety of reasons is an incomplete assessment of what they are trying to judge.  (“Our measure captures only a portion of VC due diligence, and the proxy we use is noisy.”)

So, how should we think about their conclusions?

~ Their methods echo those of practitioners these days, who use alternative datasets as inputs in investment decision making.  Similar concerns regarding collection practices, data shortcomings, and analytical assumptions can exist in those applications.  Nonetheless, there still can be value in incorporating the data into a strategy.  The same principle can be applied here.

~ Since the conclusions fit with other indications of capital allocation practices, they serve as a reminder that such examinations can prompt important discussions about the value of due diligence and how it is conducted.

The bigger picture (part two)

From the paper:

We model due diligence as producing a signal about the quality of the startup-VC match.  The investor chooses how precise of a signal to obtain, analogous to how much due diligence to perform.  This choice involves a tradeoff:  learning is costly but allows a more profitable investment choice.

How do organizations go about choosing “how precise a signal to obtain” and “how much due diligence to perform”?  The reasons behind those important decisions are often murky, with workloads and budgets seeming to determine strategy and tactics.  Instead there should be clearly-defined arguments for dealing with the trade-offs involved to optimize the quality of due diligence.  (Doing that usually results in deeper work on fewer opportunities.)

An obvious place to start is by considering the value of in-person meetings versus virtual ones.  (A previous posting dealt with the topic of onsite visits.)  Fu and Taylor made a choice to have “meetings lasting longer than five hours flagged as false positives since they likely indicate other activities.”  That filter could be questioned, since those doing exhaustive due diligence are likely to exceed that limit.

(Granted, since startups are usually small and less complex than developed organizations, the filter probably makes more sense in this case than it would if the study concerned the due diligence of other kinds of firms.)

Whether to do in-person meetings, where they should be held, how to conduct them, how long they should be, etc. constitute just one category of decision choices among many that come into play in the due diligence process.  Each of them deserves to be thoughtfully examined when setting standards and preferences — and should be revisited with some regularity.  New areas of research and special situations can often lead to needed reassessments of existing practices.

Some kind of tracking of methods is helpful in thinking about the impact of the choices.  Just as the authors of the paper tried to compare the “intensity” of the due diligence effort (in one respect) to outcomes, connecting the successes and failures of investments to the types of due diligence employed might reveal areas for improvement.  Usually the assessment of an investment is all about the strategy and entity involved; it should also include a review of the methods of diligence and selection employed.

One important step to take in that direction is elevating discussions about those methods throughout the decision making process.  Including sections in an investment memo about the “how” of the due diligence — and having it be an always-reviewed part of oversight by supervisors and governing bodies before approving an investment — results in an ongoing dialogue that actively considers allocation decisions in light of the quality of the investigatory effort behind them.

That increased focus on the nature of the process naturally leads to an ongoing effort to improve due diligence methods and better decision making overall.