Building Great Data Products
Hi there!
You all may enjoy subscribing to the Managing Data Science newsletter from Harvard Business Review.
-Bernadette
________________________________
From: Harvard Business Review <noreply@a.email.hbr.org>
Sent: Wednesday, March 3, 2021 9:02 AM
To: Bernadette Aylward <baylward@cste.org>
Subject: Managing Data Science (3 of 8): Building Products
Using data isn’t just about making decisions; the real power of data science is often in creating new products. In this issue Emily Glassberg Sands of Coursera explains how the development of data products differs from traditional software. Comments or feedback about this newsletter?
[Harvard Business Review | Managing Data Science]<https://t.a.email.hbr.org/r/?id=h758f570d%2Cdcaee1e%2Cdcaee20&s=bJ0JwSARWHqx7Xoyd-GzRornOAsNgoQja6ivtw07Y3I>
3/8
Building Great Data Products
Using data isn’t just about making decisions; the real power of data science is often in creating new products. In this issue Emily Glassberg Sands of Coursera explains how the development of data products differs from traditional software. Comments or feedback about this newsletter? Email us!<mailto:newsletterteam@hbr.org?subject=Re:%20Managing%20Data%20Science%20(3%20of%208):%20Building%20Products>
[Emily Glass]
BY Emily Glassberg Sands
Emily Glassberg Sands is vice president of data science at Coursera. @emilygsands<https://t.a.email.hbr.org/r/?id=h758f570d%2Cdcaee1e%2Cdcaee21&s=2W_qhF9VMe5RexqEe6eE3yjDDdkeDEnvk8xFgUa6q1Q>
Products fueled by data and machine learning can be a powerful way to solve users’ needs and stave off competition. Classic examples include Google search and Amazon product recommendations, but the opportunity extends far beyond the tech giants. To tackle the challenge, companies should emphasize cross-functional collaboration, evaluate and prioritize data products with an eye to the long term, and start simple.
Stage 1: Identify the opportunity
To bring data science together with product and business leaders effectively, make sure data scientists are in touch with the needs of users and the business. Have them serve as evangelists, socializing data opportunities throughout the organization. Develop the data savvy of product and business groups, too. And give data science a seat at the table in product and strategy discussions.
When prioritizing projects, keep an eye on the future. The best data products get better with age, both because more data improves performance and because data products can be extended to power multiple applications. Focusing too much on near-term performance can mean promising medium- or long-term opportunities don’t get the investments they need. The criticality of high-quality data cannot be overstated; investments in collecting and storing data should be prioritized at every stage.
Stage 2: Build the product
Data products require you to test whether the algorithm works and whether users like it — creating an inherent tension between how much to invest in the R&D and how quickly to get the application out to confirm that it solves a core need.
While there’s no silver bullet for simultaneously validating the tech and the product-market fit, staged execution can help. Lightweight models are generally faster to ship, and so, while deep learning can be powerful, in most cases it’s not the place to start. External data sources, whether open source or buy/partner solutions, can accelerate development. Narrowing the domain to a subset of users or use cases can reduce the algorithmic challenge. Hand-curation — where humans do the work you hope the model will eventually do, or at least review and tweak the model’s output — can further accelerate development.
Stage 3: Evaluate and iterate
Evaluating results after a data product launch isn’t straightforward, because the product may improve substantially as you collect more data, enabling much more functionality over time. Before canning a data product that seems to be underperforming, ask your data scientists: At what rate is the product improving organically from data collection? How much low-hanging fruit is there for algorithmic improvements? What kinds of applications will this unlock in the future? Depending on the answers, a product with uninspiring metrics might deserve to be continued.
Data products often need iteration on both the algorithms and the UI. Where regular iteration on the algorithm will be necessary, consider designing the system so that data scientists can independently deploy and test new models in production.
Read the full article [https://hbr.org/resources/images/newsletters/arrow-5272f1.png] <https://t.a.email.hbr.org/r/?id=h758f570d%2Cdcaee1e%2Cdcaee22&s=LxEErr-lkvrdl8hgc0U1j0Ucd50n8b26c_etHOnGyZ4>
ADVERTISEMENT
[https://sli.hbr.org/imp?s=681719&li={LIST_ID}&e=baylward@cste.org&p=DM121467] <https://t.a.email.hbr.org/r/?id=h758f570d%2Cdcaee1e%2Cdcaee23&p1=baylward%40cste.org&p2=DM121467&s=KS7fh240t1qNgOln06-7Js9w6CmGc7dt6NGArA9KPMU>
[https://sli.hbr.org/imp?s=681720&li={LIST_ID}&e=baylward@cste.org&p=DM121467] <https://t.a.email.hbr.org/r/?id=h758f570d%2Cdcaee1e%2Cdcaee24&p1=baylward%40cste.org&p2=DM121467&s=gmJpwNxE-aE0cQcevQrdmqcRB0KoZHrZ9db5JYAQysI> [https://sli.hbr.org/imp?s=681722&li={LIST_ID}&e=baylward@cste.org&p=DM121467] <https://t.a.email.hbr.org/r/?id=h758f570d%2Cdcaee1e%2Cdcaee25&p1=baylward%40cste.org&p2=DM121467&s=h81iFbrIfHhI67Ly87f43V8JokDAhzVehYc02UVHQpc>
We also recommend:
01
Finding the Platform in Your Product<https://t.a.email.hbr.org/r/?id=h758f570d%2Cdcaee1e%2Cdcaee26&s=cQhmJuSWDh_hZTV-uz0-N_VHeLeLsncyc5TyGVIy3jg>
By Andrei Hagiu and Elizabeth J. Altman, HBR
[https://hbr.org/resources/images/article_assets/2017/06/R1704G_HILGER-700x329.jpg]<https://t.a.email.hbr.org/r/?id=h758f570d%2Cdcaee1e%2Cdcaee27&s=3djL3vOaq4hYQv7aZD8kBLzrcfj5M6Ce9gNVTfGCbho>
02
Everything We Wish We’d Known About Building Data Products<https://t.a.email.hbr.org/r/?id=h758f570d%2Cdcaee1e%2Cdcaee2a&s=7l3BUrkNBnSUq1rl-zMwnEwvRV2467a5Q5y3IY5KhfA>
First Round Review
03
Alibaba and the Future of Business<https://t.a.email.hbr.org/r/?id=h758f570d%2Cdcaee1e%2Cdcaee2b&s=Wv1notltMgiiRZU2mXmA3eQd8wyQvTjBHT9TCFs0y8U>
By Ming Zeng, HBR
[https://hbr.org/resources/images/article_assets/2018/08/R1805F_CAMPBELL-1024x481.jpg]<https://t.a.email.hbr.org/r/?id=h758f570d%2Cdcaee1e%2Cdcaee39&s=om_xX8kmpvD4xPec-puofyr03w9aNa8C-QNrp16CSZU>
04
Embracing Agile<https://t.a.email.hbr.org/r/?id=h758f570d%2Cdcaee1e%2Cdcaee28&s=nGI8VENSj9WGEM-LH5ctB5I4QYBAFzfegs4WQoiyOr8>
By Darrell K. Rigby, Jeff Sutherland, and Hirotaka Takeuchi, HBR
[https://hbr.org/resources/images/article_assets/2016/03/R1605B_STJOHN-1900x893.jpg]<https://t.a.email.hbr.org/r/?id=h758f570d%2Cdcaee1e%2Cdcaee29&s=2yZaEaV4vsFh0Rpl6_8y-l_Ec1WlWKyxHSVhRTbnvoo>
05
Models Will Run the World<https://t.a.email.hbr.org/r/?id=h758f570d%2Cdcaee1e%2Cdcaee2c&s=AsUopaVQCGE_U3mk4dAgimKDGWg52SaK6ZgDcavJSlA>
By Steven A. Cohen and Matthew W. Granade, WSJ
plus
Lots of data products make recommendations: Netflix suggests what to watch, Amazon what to buy, Spotify what to listen to, and Stitch Fix what to wear. But just how good are they at predicting our individual tastes? In a recent study<https://t.a.email.hbr.org/r/?id=h758f570d%2Cdcaee1e%2Cdcaee2d&s=1SZcpzA-GJ00i3T-3UIJZqGf-y8VK_rz563KCmMSaBQ,> researchers from Harvard, the University of Chicago, and Cornell compared human and algorithmic recommendations about which jokes people would like. Study participants were recruited in pairs; most pairs knew each other well. Each participant was shown 12 jokes and asked to rate how funny they were on a numerical scale. Then they were shown how their partner had rated four of the 12 jokes, and asked to predict how that person had rated the remaining eight. Their guesses were compared with those made by a simple collaborative filtering algorithm, which based its predictions<https://t.a.email.hbr.org/r/?id=h758f570d%2Cdcaee1e%2Cdcaee2e&s=hvMBc3rjJBv_uef5NivprdM9VrjWxce4N7KLEvQCuIw> on how similar jokes had been rated by other participants. The algorithm was more accurate, on average, indicating that very simple algorithms can outperform humans “even for a highly subjective domain that might be uniquely human,” as the authors put it.
In this 8-part series
1/8 Getting Started with Data Science
By Hilary Mason, general manager of machine learning at Cloudera and founder of Fast Forward Labs <https://t.a.email.hbr.org/r/?id=h758f570d%2Cdcaee1e%2Cdcaee33&s=ptI2PLpLc66gsbdKERSVHpgC2r9fEPPwpShRMfHuuDs>
2/8 Managing Data Scientists
By Angela Bassa, head of analytics, data science, and machine learning at iRobot <https://t.a.email.hbr.org/r/?id=h758f570d%2Cdcaee1e%2Cdcaee34&s=8p7EAAUYJJCeyrnkhvMTP-VXVNQcSbFPMXzozr0ad6E>
3/8 Building Great Data Products
By Emily Glassberg Sands, vice president of data science at Coursera <https://t.a.email.hbr.org/r/?id=h758f570d%2Cdcaee1e%2Cdcaee35&s=45kg4jN-oxdamhTbYrWhXpxJDrOJU98qj7v_-vB5FTk>
4/8 Coming next
The Kinds of Data Scientist
By Yael Garten, director of Siri Data Science and Engineering at Apple
5/8 Adopting AI
By Andrew Ng, general partner at AI Fund and CEO of Landing AI
6/8 Setting Up an AI Lab
By Foteini Agrafioti, chief science officer at the Royal Bank of Canada and head of Borealis AI
7/8 Curiosity-Driven Data Science
By Eric Colson, chief algorithms officer emeritus at Stitch Fix
8/8 What Analysts Do
By Cassie Kozyrkov, chief decision scientist at Google
Feedback or questions? <mailto:newsletterteam@hbr.org?subject=Re:%20Managing%20Data%20Science%20(3%20of%208):%20Building%20Products>
[tw]<https://t.a.email.hbr.org/r/?id=h758f570d%2Cdcaee1e%2Cdcaee2f&s=lx4of7wgOfS-aKFiiYi7dg-VeKBMzA9qAeHS_gA5MKA> [in] <https://t.a.email.hbr.org/r/?id=h758f570d%2Cdcaee1e%2Cdcaee30&s=wv1BHYaRXRaCAfsW23SJcU_RzL0Hn289rRyuq-D7VGI> [fb] <https://t.a.email.hbr.org/r/?id=h758f570d%2Cdcaee1e%2Cdcaee31&s=JzKnb4QMnkSMCn-9sVcdISBV0QFxu8vzyldWohEhkgY>
Read online<https://t.a.email.hbr.org/r/?id=h758f570d%2Cdcaee1e%2Cdcaee32&p1=%40deniGLAi1v8mEzJlZD-1MhA68lRNMHZb2yfdCMkbCGnBXhYjOMozJSnXje1bqWLv6wHxPJl63fVHAlTuljn2fBzmdd7JYtstUh8qgbXsvUbs9DhX&s=Ivz9Y9fgsBfnGISaP0AJtmORtXm_-Z6sl8sg2ske89U>
Privacy policy<https://t.a.email.hbr.org/r/?id=h758f570d%2Cdcaee1e%2Cdcaee38&s=4D4nVTR7YE7jHa2HzEmE5DuTo_2qrXmW8j6oSC1PS4g>
Manage newsletter preferences<https://t.a.email.hbr.org/r/?id=h758f570d%2Cdcaee1e%2Cdcaee36&s=6fcUCWo5Bm1FiWSA4nKROZnH-LVaiNl_9pmG1cdsl3E>
Unsubscribe<https://t.a.email.hbr.org/r/?id=h758f570d%2Cdcaee1e%2Cdcaee37&p1=%40ey4HAeFCJLcbVs6h8B4GrvEXn0t-zCN6sj0mq9uaiP1QpWmhcEa1Ti_s_6ZRRE0WAE0wVb-dVgskEK-p9iVPJqht4xo&s=vSSCch44rMSfp-eDl20ZF1T8Kw_O_8Ci-4xvVBLOvmQ>
To ensure email delivery, add noreply@a.email.hbr.org to your address book, contacts, or safe sender list.
Copyright © 2021 Harvard Business School Publishing,
an affiliate of Harvard Business School. All rights reserved.
Harvard Business Publishing
20 Guest St, Suite 700, Brighton, MA 02135
You all may enjoy subscribing to the Managing Data Science newsletter from Harvard Business Review.
-Bernadette
________________________________
From: Harvard Business Review <noreply@a.email.hbr.org>
Sent: Wednesday, March 3, 2021 9:02 AM
To: Bernadette Aylward <baylward@cste.org>
Subject: Managing Data Science (3 of 8): Building Products
Using data isn’t just about making decisions; the real power of data science is often in creating new products. In this issue Emily Glassberg Sands of Coursera explains how the development of data products differs from traditional software. Comments or feedback about this newsletter?
[Harvard Business Review | Managing Data Science]<https://t.a.email.hbr.org/r/?id=h758f570d%2Cdcaee1e%2Cdcaee20&s=bJ0JwSARWHqx7Xoyd-GzRornOAsNgoQja6ivtw07Y3I>
3/8
Building Great Data Products
Using data isn’t just about making decisions; the real power of data science is often in creating new products. In this issue Emily Glassberg Sands of Coursera explains how the development of data products differs from traditional software. Comments or feedback about this newsletter? Email us!<mailto:newsletterteam@hbr.org?subject=Re:%20Managing%20Data%20Science%20(3%20of%208):%20Building%20Products>
[Emily Glass]
BY Emily Glassberg Sands
Emily Glassberg Sands is vice president of data science at Coursera. @emilygsands<https://t.a.email.hbr.org/r/?id=h758f570d%2Cdcaee1e%2Cdcaee21&s=2W_qhF9VMe5RexqEe6eE3yjDDdkeDEnvk8xFgUa6q1Q>
Products fueled by data and machine learning can be a powerful way to solve users’ needs and stave off competition. Classic examples include Google search and Amazon product recommendations, but the opportunity extends far beyond the tech giants. To tackle the challenge, companies should emphasize cross-functional collaboration, evaluate and prioritize data products with an eye to the long term, and start simple.
Stage 1: Identify the opportunity
To bring data science together with product and business leaders effectively, make sure data scientists are in touch with the needs of users and the business. Have them serve as evangelists, socializing data opportunities throughout the organization. Develop the data savvy of product and business groups, too. And give data science a seat at the table in product and strategy discussions.
When prioritizing projects, keep an eye on the future. The best data products get better with age, both because more data improves performance and because data products can be extended to power multiple applications. Focusing too much on near-term performance can mean promising medium- or long-term opportunities don’t get the investments they need. The criticality of high-quality data cannot be overstated; investments in collecting and storing data should be prioritized at every stage.
Stage 2: Build the product
Data products require you to test whether the algorithm works and whether users like it — creating an inherent tension between how much to invest in the R&D and how quickly to get the application out to confirm that it solves a core need.
While there’s no silver bullet for simultaneously validating the tech and the product-market fit, staged execution can help. Lightweight models are generally faster to ship, and so, while deep learning can be powerful, in most cases it’s not the place to start. External data sources, whether open source or buy/partner solutions, can accelerate development. Narrowing the domain to a subset of users or use cases can reduce the algorithmic challenge. Hand-curation — where humans do the work you hope the model will eventually do, or at least review and tweak the model’s output — can further accelerate development.
Stage 3: Evaluate and iterate
Evaluating results after a data product launch isn’t straightforward, because the product may improve substantially as you collect more data, enabling much more functionality over time. Before canning a data product that seems to be underperforming, ask your data scientists: At what rate is the product improving organically from data collection? How much low-hanging fruit is there for algorithmic improvements? What kinds of applications will this unlock in the future? Depending on the answers, a product with uninspiring metrics might deserve to be continued.
Data products often need iteration on both the algorithms and the UI. Where regular iteration on the algorithm will be necessary, consider designing the system so that data scientists can independently deploy and test new models in production.
Read the full article [https://hbr.org/resources/images/newsletters/arrow-5272f1.png] <https://t.a.email.hbr.org/r/?id=h758f570d%2Cdcaee1e%2Cdcaee22&s=LxEErr-lkvrdl8hgc0U1j0Ucd50n8b26c_etHOnGyZ4>
ADVERTISEMENT
[https://sli.hbr.org/imp?s=681719&li={LIST_ID}&e=baylward@cste.org&p=DM121467] <https://t.a.email.hbr.org/r/?id=h758f570d%2Cdcaee1e%2Cdcaee23&p1=baylward%40cste.org&p2=DM121467&s=KS7fh240t1qNgOln06-7Js9w6CmGc7dt6NGArA9KPMU>
[https://sli.hbr.org/imp?s=681720&li={LIST_ID}&e=baylward@cste.org&p=DM121467] <https://t.a.email.hbr.org/r/?id=h758f570d%2Cdcaee1e%2Cdcaee24&p1=baylward%40cste.org&p2=DM121467&s=gmJpwNxE-aE0cQcevQrdmqcRB0KoZHrZ9db5JYAQysI> [https://sli.hbr.org/imp?s=681722&li={LIST_ID}&e=baylward@cste.org&p=DM121467] <https://t.a.email.hbr.org/r/?id=h758f570d%2Cdcaee1e%2Cdcaee25&p1=baylward%40cste.org&p2=DM121467&s=h81iFbrIfHhI67Ly87f43V8JokDAhzVehYc02UVHQpc>
We also recommend:
01
Finding the Platform in Your Product<https://t.a.email.hbr.org/r/?id=h758f570d%2Cdcaee1e%2Cdcaee26&s=cQhmJuSWDh_hZTV-uz0-N_VHeLeLsncyc5TyGVIy3jg>
By Andrei Hagiu and Elizabeth J. Altman, HBR
[https://hbr.org/resources/images/article_assets/2017/06/R1704G_HILGER-700x329.jpg]<https://t.a.email.hbr.org/r/?id=h758f570d%2Cdcaee1e%2Cdcaee27&s=3djL3vOaq4hYQv7aZD8kBLzrcfj5M6Ce9gNVTfGCbho>
02
Everything We Wish We’d Known About Building Data Products<https://t.a.email.hbr.org/r/?id=h758f570d%2Cdcaee1e%2Cdcaee2a&s=7l3BUrkNBnSUq1rl-zMwnEwvRV2467a5Q5y3IY5KhfA>
First Round Review
03
Alibaba and the Future of Business<https://t.a.email.hbr.org/r/?id=h758f570d%2Cdcaee1e%2Cdcaee2b&s=Wv1notltMgiiRZU2mXmA3eQd8wyQvTjBHT9TCFs0y8U>
By Ming Zeng, HBR
[https://hbr.org/resources/images/article_assets/2018/08/R1805F_CAMPBELL-1024x481.jpg]<https://t.a.email.hbr.org/r/?id=h758f570d%2Cdcaee1e%2Cdcaee39&s=om_xX8kmpvD4xPec-puofyr03w9aNa8C-QNrp16CSZU>
04
Embracing Agile<https://t.a.email.hbr.org/r/?id=h758f570d%2Cdcaee1e%2Cdcaee28&s=nGI8VENSj9WGEM-LH5ctB5I4QYBAFzfegs4WQoiyOr8>
By Darrell K. Rigby, Jeff Sutherland, and Hirotaka Takeuchi, HBR
[https://hbr.org/resources/images/article_assets/2016/03/R1605B_STJOHN-1900x893.jpg]<https://t.a.email.hbr.org/r/?id=h758f570d%2Cdcaee1e%2Cdcaee29&s=2yZaEaV4vsFh0Rpl6_8y-l_Ec1WlWKyxHSVhRTbnvoo>
05
Models Will Run the World<https://t.a.email.hbr.org/r/?id=h758f570d%2Cdcaee1e%2Cdcaee2c&s=AsUopaVQCGE_U3mk4dAgimKDGWg52SaK6ZgDcavJSlA>
By Steven A. Cohen and Matthew W. Granade, WSJ
plus
Lots of data products make recommendations: Netflix suggests what to watch, Amazon what to buy, Spotify what to listen to, and Stitch Fix what to wear. But just how good are they at predicting our individual tastes? In a recent study<https://t.a.email.hbr.org/r/?id=h758f570d%2Cdcaee1e%2Cdcaee2d&s=1SZcpzA-GJ00i3T-3UIJZqGf-y8VK_rz563KCmMSaBQ,> researchers from Harvard, the University of Chicago, and Cornell compared human and algorithmic recommendations about which jokes people would like. Study participants were recruited in pairs; most pairs knew each other well. Each participant was shown 12 jokes and asked to rate how funny they were on a numerical scale. Then they were shown how their partner had rated four of the 12 jokes, and asked to predict how that person had rated the remaining eight. Their guesses were compared with those made by a simple collaborative filtering algorithm, which based its predictions<https://t.a.email.hbr.org/r/?id=h758f570d%2Cdcaee1e%2Cdcaee2e&s=hvMBc3rjJBv_uef5NivprdM9VrjWxce4N7KLEvQCuIw> on how similar jokes had been rated by other participants. The algorithm was more accurate, on average, indicating that very simple algorithms can outperform humans “even for a highly subjective domain that might be uniquely human,” as the authors put it.
In this 8-part series
1/8 Getting Started with Data Science
By Hilary Mason, general manager of machine learning at Cloudera and founder of Fast Forward Labs <https://t.a.email.hbr.org/r/?id=h758f570d%2Cdcaee1e%2Cdcaee33&s=ptI2PLpLc66gsbdKERSVHpgC2r9fEPPwpShRMfHuuDs>
2/8 Managing Data Scientists
By Angela Bassa, head of analytics, data science, and machine learning at iRobot <https://t.a.email.hbr.org/r/?id=h758f570d%2Cdcaee1e%2Cdcaee34&s=8p7EAAUYJJCeyrnkhvMTP-VXVNQcSbFPMXzozr0ad6E>
3/8 Building Great Data Products
By Emily Glassberg Sands, vice president of data science at Coursera <https://t.a.email.hbr.org/r/?id=h758f570d%2Cdcaee1e%2Cdcaee35&s=45kg4jN-oxdamhTbYrWhXpxJDrOJU98qj7v_-vB5FTk>
4/8 Coming next
The Kinds of Data Scientist
By Yael Garten, director of Siri Data Science and Engineering at Apple
5/8 Adopting AI
By Andrew Ng, general partner at AI Fund and CEO of Landing AI
6/8 Setting Up an AI Lab
By Foteini Agrafioti, chief science officer at the Royal Bank of Canada and head of Borealis AI
7/8 Curiosity-Driven Data Science
By Eric Colson, chief algorithms officer emeritus at Stitch Fix
8/8 What Analysts Do
By Cassie Kozyrkov, chief decision scientist at Google
Feedback or questions? <mailto:newsletterteam@hbr.org?subject=Re:%20Managing%20Data%20Science%20(3%20of%208):%20Building%20Products>
[tw]<https://t.a.email.hbr.org/r/?id=h758f570d%2Cdcaee1e%2Cdcaee2f&s=lx4of7wgOfS-aKFiiYi7dg-VeKBMzA9qAeHS_gA5MKA> [in] <https://t.a.email.hbr.org/r/?id=h758f570d%2Cdcaee1e%2Cdcaee30&s=wv1BHYaRXRaCAfsW23SJcU_RzL0Hn289rRyuq-D7VGI> [fb] <https://t.a.email.hbr.org/r/?id=h758f570d%2Cdcaee1e%2Cdcaee31&s=JzKnb4QMnkSMCn-9sVcdISBV0QFxu8vzyldWohEhkgY>
Read online<https://t.a.email.hbr.org/r/?id=h758f570d%2Cdcaee1e%2Cdcaee32&p1=%40deniGLAi1v8mEzJlZD-1MhA68lRNMHZb2yfdCMkbCGnBXhYjOMozJSnXje1bqWLv6wHxPJl63fVHAlTuljn2fBzmdd7JYtstUh8qgbXsvUbs9DhX&s=Ivz9Y9fgsBfnGISaP0AJtmORtXm_-Z6sl8sg2ske89U>
Privacy policy<https://t.a.email.hbr.org/r/?id=h758f570d%2Cdcaee1e%2Cdcaee38&s=4D4nVTR7YE7jHa2HzEmE5DuTo_2qrXmW8j6oSC1PS4g>
Manage newsletter preferences<https://t.a.email.hbr.org/r/?id=h758f570d%2Cdcaee1e%2Cdcaee36&s=6fcUCWo5Bm1FiWSA4nKROZnH-LVaiNl_9pmG1cdsl3E>
Unsubscribe<https://t.a.email.hbr.org/r/?id=h758f570d%2Cdcaee1e%2Cdcaee37&p1=%40ey4HAeFCJLcbVs6h8B4GrvEXn0t-zCN6sj0mq9uaiP1QpWmhcEa1Ti_s_6ZRRE0WAE0wVb-dVgskEK-p9iVPJqht4xo&s=vSSCch44rMSfp-eDl20ZF1T8Kw_O_8Ci-4xvVBLOvmQ>
To ensure email delivery, add noreply@a.email.hbr.org to your address book, contacts, or safe sender list.
Copyright © 2021 Harvard Business School Publishing,
an affiliate of Harvard Business School. All rights reserved.
Harvard Business Publishing
20 Guest St, Suite 700, Brighton, MA 02135