Why great business leaders are rare

Considering the huge amounts corporates are willing to pay even on pale approximations of greatness¹, it is surprising no one has been able to increase the supply of this more-precious-than-gold talent. In a population the size of India’s, surely there should be sufficient raw talent with the vision, social intelligence, personality, deprivation^2, and desire that all go to make great leaders. What are the systemic sins of selection that eliminate the best leaders in the vast pool of talent surrounding us, from making it to the top?

A great leader is a person who successfully overcomes environmental, competitive, technological, or organizational challenges that are stiff enough to overwhelm above-average leaders. In the absence of such demands, mediocrity topped with oratory and flash can pass off for greatness. To rise to extraordinary challenges, great leaders must be able to meld together disparate internal elements and make ordinary people rise to extraordinary heights (even under extreme stress) for attaining the leader’s vision, purpose, and transformational goals for the organization. The leader of a large organization is obviously not an expert in everything and one who cannot judge which expert to trust for what is doomed. S/he also needs to be unfailingly fair both for retaining the trust of employees at large and to sustain external collaborations and partnerships. Lastly, great leaders should leave their organizations strong and invigorated when they pass on (rather than overstretched or exhausted) and in hands that are well able to meet prospective opportunities and difficulties. 

For organizations with developed talent management programs, seeking top leadership from outside must be the exception rather than the rule

There are primarily three sources modern business enterprises use for CEO successors. These are:

• Internal talent
• Owner families
• External stars

How do we miss great talent from each of these channels?

Infirmities in internal identification

All things considered, internal talent should be the most promising place for finding leaders and the least likely to yield unpleasant surprises. Yet, in the absence of some important precautions, which many organizations ignore, we can land up overlooking or eliminating the treasure under our noses and leave the dross to float to the top.

There were several ways traditionally managed organizations choked off the flow of the best leaders from reaching the top, and some still follow these. The hoariest method was to cling to seniority as the sole basis for progression. While this may have some merits when the choice is for a predominantly value-guardian role or where the goals of the selectors may diverge from those of the institution the person is to lead, this is so rarely the case for corporate CEO roles that we can dismiss this anachronistic practice with the derision it deserves. More insidious and, therefore, more difficult to counter is the bureaucratic solution to eliminating contenders in an ever-narrowing pyramid by converting pre-requisites into the ultimate differentiators for selection. For example, while above-average performance must be an eligibility criterion for progression, making a mechanistic ordering of contenders based on the number of A+ performance ratings in the previous so many years may well favour the fortunate and leave truly brilliant leaders by the wayside. Moreover, such an approach encourages what could be the most pernicious misuse of traditional evaluation systems i.e. gaming them. An earlier column has dealt with our predilection for and expertise in manipulating systems.³ Advancing the careers of favorites and sabotaging those who are brilliant but outspoken (or do not belong to the evaluator’s 'group') brings out our greatest gaming skills and our worst instincts. Several of my friends who have had brilliant careers but didn’t make the final cut to the top, share one characteristic: they speak their minds and can’t suffer fools or knaves regardless of the latters’ levels. It needs just one rater who feels an uppity executive needs to be put in place for the rating to turn lukewarm. In a highly competitive situation, lukewarm is sufficient to drive a competitor out of the reckoning. Some brilliant contenders do make it to the top even in these circumstances but they need to add upward servility to their genius and competence if they don’t wish to be 'lukewarmed' out.

Most modern processes for judging leadership potential are helpful in narrowing the field and identifying people with above-average promise for filling future leadership roles after suitable development

Assessment centers and other newer methods for identifying high potential talent are a clear improvement on the purely subjective screens of the past. While these methods (when well implemented) can certainly improve the pool of internal talent from which top leaders are picked, their impact on decisions about the actual choices of top leaders have been marginal. The reasons are not far to seek. Most modern processes for judging leadership potential are helpful in narrowing the field and identifying people with above average promise for filling future leadership roles after suitable development. However, they cannot identify individual CEOs with pinpoint accuracy. As such, they are best used to build pools of fast-track talent rather than for making individual picks.⁴ However, the modern techniques we use cannot generate talent where none exists. As such, they need to be integrated into full-featured manpower planning systems that inject diverse high-quality sources at critical inflow points throughout the organization. Even the best raw material can be wasted through the vagaries of the internal promotion system where biases eliminate some of the best before (and, tragically, sometimes after) they are put through relatively impartial potential assessing systems. The use of these conventional progression processes is inevitable since assessment centres cannot be used for each level change. Moreover, unless HR is particularly active to provide cutting-edge learning and structured experiences to Hi-Pos as well as to save them from the jealousies of their peers and seniors (and from their own conceit and arrogance) during the first few years after repositioning, the organization can end up with a lot of Icarus crashes dotting the horizon and very few undamaged executives available for entering the final CEO/CXO sweepstakes. One final reason the new methodologies have limited use for CEO-level leader selection is that they attempt to evaluate (in an artificial setting⁵) a generic set of leadership competencies which usually have been chosen simply based on the face validity of the constructs and their congruence with CEO / CHRO preferences. In very few cases, attempts are made to validate these with actual performance, but there are too few data points available in the past for such inferences to hold. Moreover, new demands emerge just too fast for anyone to be confident that whatever non-motherhood-and-apple-pie competencies are identified will still be linchpins for future success. 

A great leader is a person who successfully overcomes environmental, competitive, technological, or organizational challenges that are stiff enough to overwhelm above-average leaders

The most frequently used method of internal CEO selection is choice exercised by the incumbent CEO, with varying degrees of Board involvement and influence. Research published by the Graduate School of Business, Stanford, summarized the pitfalls of CEOs selecting successors: "First, most CEOs have minimal experience in evaluating CEO talent. The decision to promote or reassign a senior executive to a new functional area is very different from a decision to give that individual primary responsibility for the entire organization… Second, despite their interest in the long-term success of the organization, CEOs are also concerned with their personal legacy. A retiring CEO might want to ensure continuity of the strategy that he or she has put in place when instead the company requires change. Other CEOs might actually want their successor to fail – or at least perform worse than they did… Third, strong and successful CEOs … are more likely to steer the choice of a successor toward one who has characteristics similar to themselves. However, these individuals, because of their long association with the company, might lack perspective on how the organization needs to change going forward."⁶

Further muddying the waters is the effort of internal aspirants who play on the pet preferences and predilections of incumbent CEOs making the choices. Some CEOs think they can get unbiased views of internal candidate capabilities by having informer-confidantes implanted at strategic points in the organization. These precautions are usually sabotaged by the smarter aspirants who either 'buy out' the informants or pump favorable information through them. As a result, it is the most politically savvy rather than the greatest leaders who edge the others out. Clearly, there is room for a Chanakya-style advisor (with discriminating judgment of people but no stake in the succession) to guide the CEO. Such advisors are rare – and so are great successors. 

Dynastic dangers

Risks of getting mediocre leaders are maximal when the selection-pool is limited to members of the promoter’s family. Even that wisest of Roman Emperors, Marcus Aurelius, left the empire to the soon-to-be mad Commodus when he limited his search to his progeny. 

The findings of recent research by Francisco Pérez-González are revealing and unequivocal. "The evidence shows that family heirs hurt firm performance. Firms in which the succeeding CEO is related by blood or marriage to their predecessors consistently underperform… The costs of heir-underperformance are large … It is not surprising that the share of firms that are replacing their retiring family CEOs with professional managers is on the rise… The findings may also imply that managerial ability, like other individual physical or performance characteristics, such as height or earnings, tends to mean-revert… Finding a competent heir from the offspring of the current head is often disastrous."7

Force-fitting owner-scions into leadership roles increases the probability of a sub-optimal choice

Force-fitting owner-scions into leadership roles increases the probability of a sub-optimal choice by several orders of magnitude. The problem of circumscribed choice is considerably aggravated because dynastic successors evade the winnowing process for eliminating the unfit that the normal climb up the organizational ladder imposes on others. I have yet to see an owner-family member chucked out for incompetence or misbehavior (unless it is with another from the anointed family). Consequently, when they are catapulted into leadership positions, they are like wide-eyed innocents, open to the blandishments of the smartest operators in the senior eco-system of the organizations. Not having competed in their way to the top, they lack many survival and leadership skills, among the most critical of which are:

• Judging the true character and worth of people
• Making tactical compromises and diplomatic collaborations with people whom one may not like
• Facing setbacks and the consequences of one’s own poor decisions without looking elsewhere for parking the blame or oneself

Empirically, there are certainly many cases where scions have led their inherited empires perfectly well if not brilliantly. Such favorable circumstances occur particularly when networks, net worth, and stakeholder belief in dynastic magic reign supreme. These benign scenarios are further bolstered when barriers to entry and relatively stable industries keep competition in check. Conversely, when organizations have to operate under tremendously competitive or environmental challenges and just being above average is not enough, this source of leadership shows its greatest limitations. At such times of stress, the best available talent cannot be substituted by the best available person within a family without great long-term damage to the organization. As Warren Buffett put it, this limitation is like ''choosing the 2020 Olympic team by picking the eldest sons of the gold-medal winners in the 2000 Olympics.''⁸

Riskier roulette

For organizations with developed talent management programs, seeking top leadership from outside must be the exception rather than the rule. Certain situations, however, may leave the organization with no other option. Even when they have not been remiss in pulling brilliant talent to the second rung, the best of these aspirants may seek options elsewhere if the incumbent CEO is too long in the role (this is one of the reasons corporate Titans tend to be followed by pygmies). Another consequence of charismatic CEOs who stay on and on is that potential successors at the next level tend to clone the CEOs in terms of style and mannerisms even if they can’t match the incumbents’ sheer genius. All of these become severe impediments to making internal choices, especially when organizations need to be triaged, turned around or transformed. Imported leaders are then the only choice. 

While the chances of finding great leaders internally are limited primarily by the richness of the talent pools enterprises start with (and the validity of their progression processes) external selections are additionally constricted by the representativeness of the behavior that can be extracted for observation during selection (and the validity of the inferences drawn from it). Leaving such selections solely to the outgoing CEO brings us again to the problems we identified at the end of the section on 'Infirmities in Internal Identification'. About a year back, I wrote a column on what can go wrong in CHRO selection.⁹ Most of those warnings apply to choosing CEOs as well (though the cost of errors can be considerably higher for CEO selections) and are summarized here:

• Over-specifying the job results in a needlessly restricted shortlist of mediocre jacks-of-all-trades or CV inflators. 
• Relying primarily on unstructured CEO interviews almost guarantees sub-optimal choices.¹⁰
• Reference checks that are inadequate hurried or left to the search firm (instead of marshaling all available senior contacts and Board resources to weave a virtual 360º picture) provide false comfort while actually being useless.
• Discovery of wrong choices is delayed because people are as reluctant to tell an emperor he chose a lousy successor as they are to tell him he is wearing no clothes.

Perfectly adequate CEOs can emerge from selections vitiated with the shortcomings listed here. However, the chances of finding a truly great business leader in this fashion are about the same as stumbling across the Kohinoor while struggling through the metro-dug streets of Mumbai.

For organizations with developed talent management programs, seeking top leadership from outside must be the exception rather than the rule

Marginal cures

Broadly speaking, unless a major culture change is exigent, we are probably best off finding leaders from within. This, of course, assumes there is a robust process for identifying (without eliminating the hidden gems), fast-tracking, and providing accelerated and structured development opportunities to a few and having discriminating and disinterested advisors guiding the final choices. Equally fundamental is filling the funnel mouth with relevant diversities. Apart from the diversity of gender and sexual orientation we must have a multiplicity of intelligence¹¹, educational backgrounds¹², castes¹³, ethnicities and communities. If we do not make extraordinary, and unconventional efforts in sourcing (rather than limiting ourselves to elite educational sources which do much of the pre-selection and homogenization for us) and progression planning, those who could have become leaders of true genius may never enter the portals of business organizations or grow in them. They may simply wither unknown in menial or mundane jobs without providing the outstanding leadership they could have:

Full many a gem of purest ray serene,

The dark unfathom'd caves of ocean bear:

Full many a flow'r is born to blush unseen,

And waste its sweetness on the desert air. ¹⁴ 

References: 

1. Herman Aguinis, Geoffrey P Martin, Luis R Gomez- Mejia, Ernest H. O’Boyle, Jr. and Harry Joo, The two sides of CEO pay injustice: A power law conceptualization of CEO over and underpayment, Management Research: Journal of the Iberoamerican Academy of Management, Vol. 16 No. 1, 2018.
2. "One could speculate that the common inclusive childhood experience for all these monumental figures was intense positive attachment to one parent coupled with some intensely negative attachment to the other or an intensely traumatic and negative youthful experience… Tension, conflict, and insecurity have marked the early lives of other leaders as well." 
3. James M Burns, Leadership, Harper Perennial Political Classics, 2010.
4. Visty Banaji, Gaming goals can kill businesses, People Matters, 25th November 2019.
5. Has the assessment centre had its day?, Municipal Journal, March 2017.
6. Charles E Lance, Why Assessment Centers Do Not Work the Way They Are Supposed To, Industrial and Organizational Psychology, 1, 2008,
7. David F Larcker, Stephen A Miles and Brian Tayan, The Handpicked CEO Successor, The Stanford Closer Look Series, November  2014.
8. Francisco Pérez-González, Does inherited control hurt firm performance?
9. David Cay Johnston, Dozens of Rich Americans Join In Fight to Retain the Estate Tax, The New York Times, 14 February 2001.
10. Visty Banaji, Help! The CHRO I picked is a lemon - How CEOs can choose better HR heads, People Matters, 14th March 2019.
11. Jason Dana, Robyn Dawes and Nathanial Peterson, Belief in the unstructured interview: The persistence of an illusion, Judgment and Decision Making, Vol. 8, No. 5, September 2013.
12. Howard E Gardner, Multiple Intelligences: New Horizons in Theory and Practice, Perseus Books, 2006.
13. Visty Banaji, Stretch them: A simple philosophy of development, People Matters, 19th February 2018, ().
14. Visty Banaji, There is an Elephant in the Room- And the Blind Men of Indostan Can’t See it, People Matters, 26th September 2018.
15. Thomas Gray, Elegy Written in a Country Churchyard, 1751.