What follows is the first draft of the chapter overview section for a book proposal. The book I’m proposing to write is called How Often? The Untapped Science Behind A Misunderstood Question About Work, Business, and Life. A chapter overview describes what will happen in each part and chapter of the book.
The book aims to make event history analysis a “household name” among business leaders. Event history analysis is the beautiful, extensible, flexible, and sadly underused science of how often stuff happens. It’s also called duration modeling, or failure analysis, and parts of it are known as survival analysis, competing risk analysis, and multi-state modeling. I want the book to do for event history analysis what Prediction Machines did for machine learning.
The book will explain intuitively what event history analysis is, how to do it, mistakes to avoid, and how to use it to guide business strategy, decision-making, and product design.
Most of its examples will come from workforce management, which is both my area of expertise and an issue common across businesses.
Some sections of the draft are redacted because they refer to a partnership that is not yet set in stone.
I encourage feedback via LinkedIn or Twitter. Slide into my DMs!
Here goes…
Introduction
I introduce the book’s central claim: A fundamental question about work, business, and life itself is simply, “How often does stuff happen?” I let that question hang in readers’ minds as I embark on a brief but entertaining tangent about the etymology of the word “often” and its antonym “seldom.” This paired etymology, I argue, is a metaphor for both the question, “How often?”, and the misunderstandings that many people have about it.
Returning to my initial claim, I argue that this deceptively basic question lies at the heart of so many others in business. It lies also at the heart of the social sciences, as well as many of the “prediction machines” that our society increasingly uses to automate decision-making. The question is so pervasive, I argue, because it is fundamental to our understanding of reality itself.
For how foundational the question is, few businesses realize they need the answer, or even know whether they’re asking it. Those aware of the question tend to address it in ways that guarantee the wrong answer. Worse still, the methods they use limit their opportunity to address other fruitful inquiries that spring from the question’s roots.
I explain how the book will help readers avoid those mistakes and missed opportunities by teaching them the principles of event history analysis, the largely untapped science of how often stuff happens. I promise readers that by reading this book they will learn how to use this science to devise better strategies, make better decisions, and build better products. They’ll also understand why there has never been a better time to introduce event history analysis into their business practices. I go on to briefly run through the history of the field, describing its cross-disciplinary success stories, the quirky figures who developed its foundations, and some of the brilliant scientists who continue to refine it today.
I move on to briefly introduce myself, why I wrote the book, and why readers should trust my judgment. I demonstrate my credentials by describing projects where I’ve used event history analysis to help businesses solve tough problems and expose egregious analytical errors.
After introducing the first two characters in the book (event history analysis and me), I move on to the third and fourth. The third character in the book is a survey that will ask people in business operations and analytics occupations about how their employers use the type of data that event history analysis was made for. The book will reference the survey’s results to measure how often businesses make the mistakes outlined by the book, and how often they avoid those mistakes by using event history analysis instead.
The fourth character in the book is the set of data sources I use to demonstrate the decision-making tools and product features made possible by event history analysis. [REDACTED.] I explain that most examples come from workforce management. Not only is this field my area of expertise. Because the robots and algorithms have yet to take over, most businesses still employ people, making workforce management a common issue.
Finally, I outline the book’s three parts.
PART I – How often? The fundamental question
This part introduces the key concepts of event history analysis in a way that is accessible and relevant to business leaders. Going in, many readers will think they’re in for a boring set of lecture notes. I acknowledge their apprehension in a brief introduction to Part I, but explain why they’ll be in awe of event history analysis by the end of the third chapter. I assure them that any pain they experience along the way is worth it, because it gives them the tools they will need to understand how to correct the mistakes outlined in Part II, and how to build the strategies, tactics, and products described in Part III.
Chapter 1: Decent exposure, at any rate
This chapter teaches the four foundations of event history analysis: state, event, risk period (also called exposure time), and censoring. To do this, I combine examples from real-world business problems, fresh analysis of real data, intuitive explanations, and engaging (sometimes humorous) graphics. It’s not lost on me that colloquial usage of terms like “exposure” and “censoring” may call funny images into some readers’ minds. I’ll (tastefully) exploit that comedy to keep the reader amused.
Chapter 2: Guessing the hazard
This chapter builds on the foundations of Chapter 1 to introduce the concept of an exposure rate, which is literally a measure of how often something happens (such as a worker leaving an employer, or a gamer buying downloadable content). I explain that an exposure rate is also called a hazard rate, a term coming from its use in mortality and machine failure analysis.
Building from this basic concept, I teach readers about the hazard function, which describes how a hazard rate changes (or doesn’t) over the time you spend waiting for stuff to happen (that is, the exposure time). I explain how you can decompose a hazard function into parts that represent meaningful life-cycle processes. Then I use examples from biology, sports science, and workforce analytics to demonstrate the ubiquity of one such decomposition, which breaks everything from death rates to batting averages down into parts representing development, maturity, and aging processes. I inspire readers to think about the life-cycle processes that govern the hazard functions relevant to their own lines of business.
Lastly, I ask readers to imagine that hazard functions are unobservable characteristics that everything – every employee, every child, every basketball player, every toaster oven – carries within it. These invisible truths represent how often stuff might happen to us (me, you, your toaster oven) in the future, and how uncertain we are about those fates. Our task as event history analysts is to learn the shape of these functions through careful analysis. But I caution readers that this is only the first step. We also need to know how to use those functions to answer questions beyond just, “How often?” That brings us to the next chapter.
Chapter 3: The rate of all evals
This chapter begins by describing the differences between rates, which I described in Chapter 2, to closely related quantities called probabilities. I explain how this relationship is just one example of how exposure rates – answers to the question “How often does stuff happen?” – are the root of the answer to so many other questions. From there, I demonstrate the broad range of such questions through worked examples, intuitive explanations, and engaging graphics.
PART II – Often wrong: Fundamental misunderstandings
With the fundamentals of event history analysis laid out, we’re ready to cover the biggest mistakes that businesses and even savvy analysts make when it comes to answering the question, “How often?” For each of these unhealthy practices, I show how event history analysis provides both the diagnosis and the treatment. I illustrate each mistake (along with its correction) through a combination of comedy, toy examples, studies I’ve seen shared on LinkedIn, my career experience, fresh analysis of real datasets, and engaging graphics. In addition, I use my survey of business operations and analytics professionals to estimate how common these mistakes and their corrections are across industries.
Chapter 4: Never do this
This chapter covers the mistakes made most often by people wholly unfamiliar with event history analysis. The key issues are: failing to properly define states and events; measuring risk periods incorrectly; ignoring censored data; and confusing rates with probabilities.
Chapter 5: Selden do that
This chapter covers the mistakes made by both novices and experienced practitioners of event history analysis. These mistakes include: assuming that the events of interest will eventually happen in all cases; ignoring competing risks; improper handling of recurring events; using the wrong event history analysis output to address causal as opposed to descriptive or predictive questions; and using the wrong methods to evaluate event history model predictions.
Chapter 6: These rates are killing us
This chapter dives deeper into the consequences faced by businesses who make the mistakes described in chapters four and five. First, I synthesize and expand on the results of my survey of business operations and analytics workers. Second, I introduce the concept of scientific debt (an analogue of technical debt). I show how scientific debt is an increasing concern in both the private and public sectors. Then I explain how event history analysis can help companies reduce this debt and pay it back.
PART III: Often right: How to use event history analysis to drive business results
Here I shift from constructive criticism to critical construction. The last four chapters are about how to incorporate the principles and methods of event history analysis into a business’s strategy, tactics, and product design.
Chapter 7: Selden a better time
Before discussing strategy and the rest of it, I make a three-part case for why now is an especially good time to adopt event history analysis. First, companies are collecting more and more event history data every day, and it has become easier than ever to analyze that data, all thanks to cloud-based computing and data storage services. This creates opportunities that the rest of the chapters in Part III argue are best exploited using event history analysis. Second, with great volumes of event history data comes great responsibility, as the chapters in Part II demonstrated. Third, recent methodological advances unlock opportunties that didn’t exist earlier in the 21st century. For example, recent findings in causal event history analysis make it possible to measure in great detail how a company’s interventions impact business-critical event histories. Meanwhile, continuing advances in machine learning and probabilistic modeling allow us to produce more accurate and precise estimates of hazard functions, and how they vary.
Chapter 8: “[How often stuff happens] has become an important element of strategy.”
I repurpose a quote attributed to Regis McKenna and envision a future in which event history analysis drives business strategy. In this future, businesses build the foundations of their analytics in large part (though of course not completely) on the foundations of event history analysis. They outline the states and event flows that define their key business processes. Then they estimate the hazard functions that apply to these states and events. They define the key performance metrics that can be computed from these hazard rates. And they continuously expand on these estimates thanks to the flexibility and extensibility of event thistory methods. The results: more accurate answers to more important questions with less science debt.
Chapter 9: “[Event history analysis s]trategy without tactics is the slowest route to victory.”
This chapter provides specific examples of how event history analysis output can help businesses make decisions that align with their strategy. I group these examples into the three categories of data science defined by Miguel Hernán, John Hsu, and Brian Healy: description, prediction, and what-if analysis (otherwise known as counterfactual prediction).
Chapter 10: Ship early and ship often
This chapter demonstrates products that event history analysis alone makes possible. I provide business-to-customer, business-to-business, and business-internal examples. Each of these demos will be available live online.
Conclusion
Aside from tying together the ideas from Parts I through III, the conclusion asks a tough question: If event history analysis is so great and if it’s been around for so long, why is it still a mostly “untapped” science? I close with the hope that the book answered that question and, more importantly, will eventually make it irrelevant.