Beyond Growth Hacking: A Recap. #GHC15

One of the compelling sessions I attended at GHC15 was the Beyond Growth Hacking Session.

panelists slide

Panelists were: Miri Pruzan(Facebook),Winnie Wang(Airbnb),  Erica Lockheimer(Linkedin), Molly Long(Uber), Vivian Qu(Pinterest). 

(Notice that I have used the colors of the panelists’ shirts in the photo to highlight their names accordingly. I’ll use the same colors in further mentions of their names).

Hosted by the Five Panelists above, the session started on quite an agile note!

I think presenting this in dialogue format will make it more interesting for you. So, here goes…

In the last few years there has been an increase in people googling the phrase “Growth Hacking” – Miri Pruzan

Q:  What does a growth engineer do?

Winnie: – Increasing a company’s top line metric. A growth engineer uses A/B testing to find the best user experience and subsequently helps increase top line metric.

Erica: Strategic areas in Growth Hacking include:  Registration on boarding. 

  • Onboard Users, help them complete their profile.
  • Help them create a network 
  • Talk to your users.

 Q: Why do we even need a growth team?

Miri – At Facebook, One of the issues the growth team is focused on is what users actually do on the site.

They track users’ behavior, e.g drop off pages. 

Molly – Product team focuses on making the product (pie) more delicious  e.g at Uber(reducing ETA-Estimated Time of Arrival ) while the growth concentrates on making this product(pie) bigger.

Vivian – Three years ago at Pinterest we saw our growth metrics go down and thought we should create a growth team. More than about seeing your metrics go down, a growth team owns the services and frameworks that allows you to sequence when the product education goes out. 

Also, We run a ton of A/B experiments, they can have a central repo where people who take over can go to catch up.

Growth teams are necessary for things like trying out projects e.g the invite-only Pinterest  we tried in the Philippines- you need a growth team to own that and monitor the numbers and see if the project is a success.

 Q: What metrics are we trying to move? 

Miri :  At Facebook we’re very interested in user retention. 

Vivian:  We originally tracked monthly active users. about 1 yr ago, we stepped back to make sure we were monitoring the right metric to track.

We chose a metric that most aligned with the product experience. You have to make sure to track the right metric which is aligned with the product experience because this metric will be optimized and you do not want to end up tracking the wrong thing.

Erica: – It’s really about sign up, but we started changing it with something we call quality sign up.  Quality profile information, Reachable info, Incremental connections are things used to measure this. Also get them engaged within 28 days. We also measured invites from friends and engagement.

Winnie – At Airbnb , the most important metric was the number of nights booked.

 We shifted our focus to intention to book. We also shifted to using registered users as top line metric.

Of course, metric used will depend on the context.

Molly– From supply side, number of driver signups per week. Another key metric we are starting to look at is viral factors.(Exponential growth)

Erica– The thing that is so important to growth teams is monitoring. It is very important to have the right monitoring in place.

Miri– It is not just about the metrics going down, it should be possible to explain why metrics go up.

Miri– Sometimes you take a hit at metrics for long term effect.  E.g Forcing Facebook messenger on people. made people mad, but we were trying to provide a better messaging experience. A year later we see it as a success.

Q: What is the difference between growth hacking & growth engineering?

Molly– I see hacking as more about experimentation & iteration. Doing many things and finding out which works.

while growth engineering is all about owning these growth practices.

Vivian– I disagree with you(Molly) on growth hacking. It has a negative connotation . It sounds like being deceptive and growing users by selling a product in a way that it is not. The second negative connotation is like writing bad code to get your numbers up, but I find that the technical debt incurred is a lot and not worth it.

Molly– I think there are little tricks you can do to get your numbers up e.g at Uber we changed a “share with friends” text to “free rides” on the share panel and it increased click through rate by 100%

Winnie: growth in general is all about empathizing with the user.

Miri– (We should rename Growth hacking to growth experimentation) *Laughter*

Q: What are different approaches taken to grow products?

Molly: In paid marketing for instance you don’t want to be too dependent on cash to grow or to be in the negative. At uber we are banking on viral factor. e.g Referrals. which will increase exponential growth. and reduce your dependence on cash when this is optimized.

Erica: Focus on loops, making smart suggestions to users which leads to smart networks. Focus on nurturing. This helps the viral factor.

A Q&A for the audience followed after the main discussion and here are some questions covered.

Audience: How do you best communicate with teams that are important to growth?

Erica:We have tools that track experiments and how they impact growth.

Vivian: Data engineering team works closely with growth team to tackle these things.

Molly: We work heavily with city and operations team.

Audience: How do you use push notifications to improve user engagement?

Erica:Build intelligence around it to make it relevant to users, and timing.

Miri: Having a cross functional product helps. Then on some platforms for instance, you do not push all notifications.

Audience:How has rebranding & redesigning (e.g of logos) impacted your metrics?

Winnie – It shot up metrics quite a bit. But you have to be really careful as the growth in metrics is usually short term and after a while the numbers drop.

Erica: It’s a scary thing because sometimes you can’t predict the outcome but you have to monitor and iterate accordingly

Audience: How long do you run experiment to determine success or failure?

Erica: It depends. You have to understand your product to determine that.

Winnie: It depends also on how your experiment is set up, some experiments take longer to converge results.

Vivian also added that one has to be very careful which experiment is run.

Break Up With Your Codebase? Or Make it Work?

This session (at GHC15) was of immense interest to me because I have been in this situations so many times at work.

You know what I mean?

When the code base is borderline unbearable and some hard decisions have to be made that would weigh on cost and developer resources.

Anyways, the session was more of a discussion though than an expose and I will try to reproduce it here as well as I can.

One question was: How do you minimize risk when migrating to a new code version?

A great way to do that is to maintain multiple versions, and run parallel versions before migrating. This basically means running the new and old platform at the same time for a while for proper tests and checks before actually migrating your data and users to the new.

Another way to minimize risk when deciding to go with refactor or rewrite  is testing. Unit testing. Documentation of every little change/step is also very helpful.

A third helpful tip is for you to measure as much as you can. While migrating measure every thing that you can  to see that all is going well, if not, adjust. Also incremental changes help for easy roll backs if things aren’t going well.

Backups too shouldn’t be forgotten. Absolutely necessary. 

 Trisha, one of the speakers, emphasized a little more on testing. She said that it is important to make sure your current users are not adversely affected by any changes to data, and a way to ensure this is testing with production data to make sure everything is fine.

One should also keep in mind the fact that some techniques used to minimize risks can increase risks in other areas.

Then the discussion moved to how the speakers had dealt with a wrong decision to rewrite code. 

Some dealt with it by scratching the whole product itself when it turned out that the rewrite made the product more complicated and difficult for users to use, despite being a great product, that is. 

Also, If you are going to onboard customers to a new version you have to do a lot of hand holding, Support them every step of the way to help them get used to the new flow. 

On resource allocation, it was agreed that recreating a system -especially a working system- is always a huge engineering investment. It normally takes more time than was anticipated.

Sometimes recreating a platform will not even be of interest to customers, so a balance has to be stricken and things weighed appropriately before making these decisions.

On prevention, there are ways to minimize legacy Code that is not maintainable.

One way to prevent this is  by doing code reviews

Another is Pair programming, and that makes so much sense.

Also, a constant aim should be to improve your existing code,constantly ask question about things- like if is the most efficient way- and constantly assess things to know if they are optimized.Even if it’s documentation.

Sara Tansey said that she tried as much as possible not to start over if she could help it in any way. 

buc3

Lastly, Be willing to change Code. Have everyone on the team on all of the code. Let go of Code Ownership- that thing where each developer owns a section of the code and doesn’t want anyone else to work on it.

All in all, I would say I concluded that rewriting code was to be avoided in favor of refactoring when working with Legacy code that is hard to maintain. If it could be helped, that is. 🙂

buc5

Developing Market-Leading Products: Strategies and Techniques for Growth Engineering

I attended a growth engineering session featuring speakers: Helena Tan (Fitbit)  Anna Majkowska (Pinterest) June Wang (Instagram) and Karen Kim (Airbnb)

It was anchored by June Wang from Instagram and was very Interactive. The four panelists  did a marvelous expose on developing market leading products and gave us insights based on data from the companies they work in.

market2 market3

The discussion  was structured around the four stages of the growth cycle.

On User acquistion, one thing that worked well is SEO, according to Anna from Pinterest. Especially important while trying to grow internationally, where there is a lot less brand awareness/ word of mouth. Pinterest is largely image based, she said and since it is pretty challenging for search engines to interpret images, their strategy was to make more content.

Also, they paid close attention to quality and filtered content so that less than quality content didn’t get through. Another pretty good strategy employed was use of site maps. They also limited amount of content accessible by a user that was not signed up; and prompted them to sign up if they wanted to see more. They basically gave a sneak peek of value proposition.

I learned that about 70% of things you try  on growth will probably not work. You have to try a lot of things.And when you find that which works, optimize and improve it.

June Wang talked about making the sign up process seamless as a way to acquire users. She also pointed out some thing that were not so obvious that had to be done to secure users. One of which was reducing errors on sign up forms.(For example using double checks for passwords). The other was mitigating abandonment by adding a confirmation of sign out for users.

 Helena Tan talked on Personalized ads. She emphasized presenting relevant content to customers based on their behavior, information  of which is and should be tracked.

On User activation, Karen from Airbnb mentioned that a challenge is scaling. At Airbnb they try to minimize friction on web flow. They minimize difficulties in using the platform  so that they can win users’ trust. She also talked of payments. An important trend is mobile payments. They also had a lot of success with presenting users with methods of payment that was familiar to them.

On the third stage of growth which is User Retention (Keeping the romance with users alive), Helena Tam advocated personalized retargeting ads as an effective technique  It is important for an advertiser to have full picture of a customer’s behavior across different devices. That of course means that this behavior has to be tracked.

There are some tricks to bringing back users who have stopped engaging. In this, timing is really important. The sooner you can bring people back the better.According to June, Study has showed that bringing them back sooner was more effective.

The session ended with a Q & A section which was even more illuminating. I thoroughly enjoyed it and you can totally contact me for the notes if you want.

Oh and PS: I got to take a selfie with Helena. 🙂

Photo 15-10-2015, 11 37 25

Yes! …Only a few hours left till GHC15 starts.

Many hours ago, I arrived in Houston for the Grace Hopper Celebration (GHC) of women in computing 2015.

Right now, I can almost feel GHC15 on my fingertips.

In a few hours the biggest convention of women in computing will be taking place and I am going to be witnessing it live.  I feel super excited and wide awake even though I should be asleep in order to be up early.

Well, because of the overwhelming nature of activities tomorrow, I decided to make a finite prioritized list of companies’ booths to visit. When I work through my list, I will have direction -I hope- and not end up walking around the George Brown convention center confused-as I am prone to be when I don’t prioritize.

Marking out sessions to attend today is also on my to-do list before I go to bed.

I.Can’t.Wait.

See you there if you’re going to be there!

#OurTimeToLead

#GHC15

 

1 2