Delivering Change

When you think of great products, what do you think of? Products that have altered the way office jobs are done (excel, outlook, DropBox), or changed the way we interface with computers and technology (iPhone). Hearing from the VP of Product at Change.org recently was an opportunity to hear from the head of one of the most impactful products present in the market today, as evidenced by the impact on politics around the

BlogPostYellowShirt

French president responding to Change.org petition

globe. This presentation concerned the organizational tools that the team at Change.org used which led to the successful creation of an entirely new product line, circa 2011, to support a major shift in Change.org‘s business model.

The problem facing Change.org was that the revenue source had been a pure B2B model, which was no longer viable. Meaning that the grass roots petition platform that exists today did not exist. It became clear to the leadership of the organization that the business model was no longer feasible and had to be changed due to external market factors. Nick (VP of Product) led this effort from the product side and by using a few different concepts and methodologies successfully led the team to create the product line which made Change.org what it is today.

Problems Faced

The main internal problems that the company faced in developing an entirely new product line were the following:

  1. Speed of Delivery: Slow progress towards product completion
  2. Alignment: Getting the organization to work in the same direction
  3. Focus: Inability to make meaningful progress towards the goals that have been set

In order to confront this problem, there were a few different tools that were utilized by the team at Change.org that stood out to me. The team  utilized some of the obvious efficiency levers, like automating tasks, but the strategic/organizational decisions are the most interesting.

Avalanche

As a previous product owner, one of the main frustrations regularly encountered was the inability to make forward progress due to all of the work needed to “keep the lights on” as it’s commonly phrased. Change.org made the conscious decision to focus solely on creating new features for a limited and focused period of time. That means no fixing of features that break (I assume other than P1’s). No devops working performance tuning. The whole organization fully focused on features.

One of the anecdotes shared that stuck with me was when it was mentioned that two college hires ended up working on a feature with the CTO. When you have an entire organization aligned behind a single goal, the ability to have people naturally move across, up, and down traditional power structures to achieve that goal seems to be a huge gain in creating innovative solutions. In addition to having more deliveries towards that single goal, the fact of the matter is that also comes the ability to see the pain points and what other teams are dealing with/issues they have.

Once the avalanche was done, Nick mentioned how devops had a much better understanding of the issues facing the application development team due to the fact that these devops developers had been doing feature work on the application. With this understanding they were then able to take these learnings and implement solutions that were able to make big impacts within the organization.

Shots on Goal

Anyone who has worked in an agile environment should be aware of this, but the affect that this had in the organization is worth mentioning. Specifically how deliberately this discussion was had with leadership. What Nick realized is that they were entering an entirely new market. Knowing this, he knew that it was unlikely that the organization would be able to accurately predict what features the new consumers would adopt in their product line without releasing many features and finding out what worked.

To that point, Nick organized his team to focus on flexing their speed muscle, as opposed to quality. Why deliver 10 perfect features when you could deliver 100 working features and have a much higher chance that the organization will strike gold? That strategy led to multiple success in the new product line, and an ability to conduct and test experiments faster than they thought possible under the previous paradigm which tried to achieve both speed and quality. When a team is focused on an immense effort, making demonstrable progress is usually one of the hardest/most difficult things. By focusing on speed, much faster feedback loops were enabled, with some of these features contributing to more than 20% revenue growth from the new product line.

OKR

Objective and Key Results methodology. Google used it. Intel used it, and countless other organizations. It’s not reasonable to think that an organization can deliver an entirely new product line and business model without complete organizational focus. Using OKRs Change.org was able to create a clear focus. As a result the teams were able to deliver on what is meaningful and impacts the bottom line, as opposed to working towards multiple objectives and goals that cause people to tug and pull against one another.

Blog2-measurewhatmatters

Great book which provides a history and overview of OKRs

Working in multiple organizations, I’ve noticed that people don’t know what to do when their are too many goals. Having the ability to walk into a room and point to a single goal means that the organization is more easily able to coordinate and implement. When there is no clear goal unnecessary road blocks, and fiefdoms tend to spring up and lead to the organization pulling in different directions getting in front of the accomplishment of organizational goals.

Operating teams using the above three strategies worked for Change.org. Using some of these strategies has worked for other large and small organizations, and from what I’ve seen thus far in my career, could be used in many other organizations to transform the way products are delivered and impact the bottom line.

Using Redash for Visualization

Over the winter break I was having a conversation with my cousin concerning the awesomeness of Tableau and all it offers. While Tableau is a best in class product, there are a couple points that he raised which are valid points against the effectiveness of Tableau.

  1. Tableau uses it’s own proprietary language and functions for a lot of aggregations/advanced functionality that could be done in SQL. SQL based tools are better, he referred to Metabase explicitly among others, due to the fact that most analysts know the language and therefore will easily be able to pick it up.
  2. Tableau isn’t open source, so as a user, if something doesn’t exist in an open source tool and I know the language, a custom feature to the tool can be added easily (depending upon your ability to code in the respective language).

With that in mind, that got me down the path of looking at open source reporting and dash boarding tools that are heavily SQL based. When looking, I cracked upon both a Metabase and Redash instance with which to play around with. Metabase had a good number of features available, but an extremely limited amount of rows which could be ingested by the tool on the free tier.

herokutiers

Heroku – Different pricing tiers

So, not wanting to spend any money upfront, I went over to Redash and started using it to build the first dashboards in the tool using the free trial. Needless to say, I fell in love with the tool instantly then my love was instantly tempered by other limitations present in in Redash. Below is the dashboard that was created, which can no longer be accessed unfortunately due to the free trial period ending, but appeared as pictured below.

redash-dashboard

Redash dashboard for Steam and Sony Marketplace price changes

Running through the charts, you can see the game and pricing data that has been collected by my scrapers. Scrolling through the different charts you’ll see the following:

  1. Largest Price Drops and Increases: Shows daily what the maximum price increase has been, and largest decrease has been, along with the average price change for all items which had a price change that day.
  2. Average Price Change: The average price for items which changed price, along with the average previous price for all items in the Steam Marketplace and Sony Play Store which had a price change for the day. In green is average price change between the old price and new price.
  3. Most Recent Price Decreases: The last 20 items which have decreased in price, and associated data.
  4. Most Recent Price Increases: The last 20 items which have increased in price, along with associated data.

At the bottom of each of these panels you’ll see when the data last was refreshed from the database.

Now onto the original reason for writing this article. The pros and cons of using and building dashboards in this tool.

Pros

  1. No row limit encountered on the free trial! What a great feature. The first requirement I had was that I wanted to be able to ingest a large amount of data and do aggregations using that data. Not having a hard row limit that I ran into with my small 40,000 record data-set originally sold me on this tool.
  2. Easy to get up and running. Setting up with tool, and making use of the connectors already present in the tool was extremely easy.
  3. SQL interface is extremely intuitive to anyone who has used SQL Server Management Studio/PgAdmin, or any other database querying GUI tool.
  4. Refreshes are extremely easy to schedule and reliable.

Cons

  1. Free trial, and free software if self hosted. If hosted using Redash.io like I did, the price is $49 a month on the lowest tier. Metabase hosting on Heroku is just as easy, and cheaper to use in the long term for small side projects.
  2. No aggregations can be done within visualizations, which in my view is a must-have with any dash boarding/reporting tool. Redash forces you to push that logic into SQL code, which results in redundant/complex queries. It also forces you to pre-aggregate, so the feature of “No row limit” in the pro’s section no longer applies.
  3. Visualization features are basic/limited. Other dash boarding tools allow you do stacked bars, generally more customization option, tool-tips, choosing color schemes, etc. which are not easily available or are limited compared to a tool like Tableau.
redash-queryeditor

Redash query editor. Combine multiple query visualizations to create a dashboard.

Redash was a tool meant for another use case. Perhaps one where you need a basic tool for monitoring ETL, or some other system…Using Redash was a good experience, but using something like Metabase is the tool for me at the moment. Due to the fact that in Redash all calculations have to be pushed down to the SQL and cannot be done with aggregations in the tool, along with the pricing, it doesn’t seem to suit my use case.