Author: Brad Ledford

Categories
Uncategorized

Go Forth and Deploy

For many of us in software of a certain age, making changes to and releasing software has traditionally been a perilous activity. Customers had to be hand-held, release sequences carefully orchestrated, and contingencies planned in case of failure. And so, the best releases were done by the best release engineers (remember that title?)–typically release engineers who were most averse to risk. Thankfully, we now know there’s a better way.

Software today is now ideally deployed into cloud-based infrastructures, with little or no customer involvement, with no risk of downtime, and–most critically–completely automated. The risk of breakage is minimized as the duration of Agile iterations dwindles to zero, ensuring that when breakages occur, they can be mitigated in minutes. What was once a massively delicate and stress-filled activity, can be automated into oblivion and reduced to an afterthought. This is the realm of continuous deployment and delivery and it is outstanding.

As teams I’ve worked with over the years have taught me, there are some key considerations in ensuring that continuous delivery is achievable, while maintaining everyone’s sanity.

Careful Code Review

Changes made to the code have always needed peer review. But in an environment where, once a changed has the approval of peers and quality assurance, it is immediately shipped off to the customer, the importance of thorough code review increases dramatically. This is not to say that perfection is the goal (an all too common aim of naive, less experienced programmers), but rather that every change is evaluated for it’s quality and thoroughness relative to it’s potential for negative impact on the application.

Release in the Middle of the Day

If you are waiting until off-hours to release your code, it’s probably a sign that your deployment process is fragile and not correctly mitigating the risk of downtime. Fear that you will break the application at peak usage breeds an overall fear of any change, much less a complex and innovative one. This, of course, has exceptions and is subject to the idiosyncrasies of your industry, product, and customer needs. But as a general rule, the best time to release software is when everyone is ready to monitor and turn around any fixes.

Of course, there are ways to avoid the impact of breakages during peak usage altogether, such as rolling updates, Green/Blue deployments, canary releases, and even very new approaches like Houston.

This is not an uncontroversial subject so flexibility on execution here is necessary. Go with what the team is comfortable with, but also make decisions that build confidence in automated releases at any time of the day.

How Fast Can You Fix Your Mistakes

When continuously releasing software to production, there is an increased need to be alerted, automatically, when an issue has occurred. The human rigor typically applied to big-bang releases–such as manual canary testing, full regression test, and so on–may not be possible given the frequency of releases. As such, when a breakage does occur, it’s critical to have a systematic approach for detecting and redeploying changes (or rolling back, as last resort).

A common model for managing security incidents applies to continuous delivery, as well. This graphic illustrates a few key KPIs that should be kept in mind.

IMG_4021.PNG

While this graphic is intended to describe responding to security incidents, I like to imagine the same metrics apply when the incident in question is a planned release of your software that goes wrong. By managing the time to detect a problem independently from the time to recovery, a team ensures the most effective reaction times in the case of a failed release.

Stop Penalizing Failure

It is not a matter of if, but when, a software release will go wrong. How the team reacts to this failure is a good indication of whether or not they are ready for continuous deployment. In my professional past, I have been very negative and punitive in my feedback when projects are not successfully deployed. Healthier cultures and mentorship have shown me that a team that can calmly and quickly correct problems on their own is much more desirable.

“If you want something new, you have to stop doing something old”
― Peter F. Drucker

Infrastructure as Code == 🔥

When your entire application can be reasoned about, there is a sense that the risk of any given code change can be more easily reasoned about as well. For instance, if the configuration of an application clearly denotes that a production Web server cluster is set to deploy as a “rolling update” with a minimum of six instances, then the deployment team knows what to expect as the rollout proceeds.

Continuously integrating, testing, and releasing your application to customers is exhilarating and liberating, as long as you manage the risks of something going wrong. If you can identify and eliminate or minimize these risks, the benefits to your developers and ops people are massive.

So go forth and deploy things! Or, more accurately, build robots to deploy things.

Categories
Productivity

Working Alone

When was the last time you worked by yourself for any serious length of time? As one of the perks at my current employer, engineering and product teams are encouraged to work from home on Tuesdays. For me it’s the best way to re-energize after the “collaboration fatigue” of a typical week.

Yet, alone, there is nothing directing your tasks or conversation (and thus your mind). Nothing which is not in your direct control. If you started the day with a goal to achieve (congrats to you!), then you are only reason it didn’t get done. This is both empowering and daunting, as many of us feel the weight of distractions throughout the day as a minute-by-minute threat to our productivity. But to work from home, alone, is to say to the world: “On this day, it’s on me to be effective and, damn it, at least give myself a chance!”

Taking the time to work alone brings clarity to things easily deferred in the maelstrom of the modern day workplace. Haven’t cleaned up your product roadmap? Need to file some expenses ahead of a deadline? These are the non-critical tasks that pile up throughout the course of a week, so easily abandoned as the invasive “open office” brings new issues to light at a lightning pace.

It’s not that we don’t love our coworkers or care about their issues. We even enjoy the office dogs (Captain Crunch and Amelia)! Each interaction at the office cements our commitment to each other, the mission of the company, and the idea that we are a work family. But just as spending too much time with one’s family can create anxiety, so too can an extended period of time with your work brethren. It becomes impossible, at times, to relax enough to just bare down and do the work.

“Your ability to generate power is directly proportional to your ability to relax.”
– David Allen

So, here are a few things I make sure to do when working alone…

Take Regular Breaks

I find it’s critical to stop working at regular intervals and get my head out of the work. I’ll take a stroll around the block, pace around the house, or stretch my back for a few minutes. Anything that relieves the mental pressure of staying focused for too long without coming up for air (as would happen naturally in an office setting).

Do Your Best Work

Save the work of highest value for your time alone. Since at the office there is such a high likelihood that you will be disrupted, the critical work requiring high levels of focus should be diverted to alone time. Focused time to work is a gift and should only be spent on things worth the dedicated effort.

Plan the Day

Don’t let a quiet, distraction-free work environment go to waste by not starting the day with a plan (or making the plan the first part of your day). This is something I absolutely struggle with, as I tend to just dig in first thing. But on a day of prescribed enhanced effectiveness, I believe it’s best to have a plan of attack.

Enjoy It While it Lasts

Focus is fleeting. The common reaction to having a long period of uninterrupted time, I’ve noticed, is to actually fritter it away doing low value work. Knowledge workers, I believe, have become imprinted with the behavior of the busy workplace, letting their minds become distracted and then later hating themselves for it! We should relieve ourselves of the idea of the “perfect” work day and just enjoy the productive time we have.

So, take a break. Go home, to a coffee shop, or to a quiet park and get comfortable with silence as it pushes you to begin closing loops in your mind. Be mindful of the fact that you are in charge of the day, that there is nothing you should allow to distract the day’s effort.

And most of all, relax, and get to work.

Categories
Uncategorized

Breaking Down Git Activity of the Swift Programming Language

A Review of Activity on the Swift Github Repo

The following is a review of the Swift Programming Language recently open sourced by Apple using the gitstats utility and the freely available Github Activity charts.

The Vitals

Generated: 2015–12–14 21:34:08 (in 225 seconds)
Report Period: 2010–07–17 16:50:59 to 2015–12–14 21:12:49
Age: 1,978 days, 1,438 active days (72.70%)
Total Files: 9,744
Total Lines of Code: 1,045,060 (2,768,677 added, 1,723,617 removed)
Total Commits: 29,925 (average 20.8 commits per active day, 15.1 per all days)
Authors: 259 (average 115.5 commits per author)

Weekly Activity

Commits by Week for Past 32 Weeks, GitStats

Commits by Week for Past 52 Weeks, Github

Clearly, this is a very active project for Apple and it’s 259 contributors. With the exception of the typical lulls during July 4th, Thanksgiving, and Christmas holidays, the activity is pretty consistent week-to-week. In the past three weeks, the spike in commits is amazing to see. Apple must be thrilled with this activity, though a non-trivial amount of activity appears to be external contributors catching typos. Here’s a word cloud from titles of the past 7 days of merged commits:


Hour of the Day

Commits by Hour of the Day, Gitstats

Activity by Hour, Github

The daily pattern here is pretty typical of modern, distributed development teams, if maybe a little later in the afternoon than a typical company-supported project. It might be interesting to plot commits by hour to this to see if key hours of the day shifted over time.

Commits by Day of the Week


Whoa! Are there that many commits happening on the weekend? Of course this is influenced by late Friday night work trickling into Saturday morning, but I was surprised to see the Sunday activity being roughly 30% of the peak day (Thursday).

Month of the Year


As mentioned above, US holidays had an obvious productivity impact on the contributions to the Swift repository, with June and November being popular months to take vacations. That’s my theory, anyway. One wonders if the productivity of a given software project of this scale could be improved by hiring programmers in countries with inverse holiday patterns to US. 😉

Commits by Year

Commits over Time for Entire Report Period, Gitstats

Commits over Time since July 2010, Github

This is clearly a healthy and active project within Apple. What’s interesting to consider is how the language activity since mid-2014 flattened to ~175–200 commits per week. This might be an indicator that “there’s only so much to be done” or that there was a specific constraint being invoked by Apple to control the amount of change, which would be a curious hypothesis considering that they have more talented and capable C++ and Swift programmers than any organization in the world.

Commits by Timezone

Commits by Timezone of Contributor

Safe to assume here that most of the contributors to Swift do so from Cupertino. This will now likely shift over time as contributors not based in California begin to contribute to the platform.

Top Authors


Chris Lattner is clearly “Neo” to the Swift Language’s “Matrix”. Other names in this list are all key Apple engineers including Doug Gregor, Joe Groff, Dmitri Hrybenko, Jordan Rose, Michael Gottesman, Dave Abrahams, John McCall, Nadav Rotem, and Mark Lacey.

Lines of Code by Author


Commits by Author

The best visualizations for contributors comes on the Github Contributors summary page.


Commits by Domain


Nothing surprising here with 97% of commits coming from Apple employees.

Lines of Code Over Time


Just your typical 1M+ line of code project growing smoothly over time. What will be really interesting will be to see whether open sourcing the application will result in a commensurate increase in the number of lines of code or if the size code will be kept in check.

Number of Files Over Time


Again, a fairly smooth growth curve, save for the bump in December 2014 which appears to be massive activity from Doug Gregor relating to Clang Importer.

Language Mix


The Swift Programming Language consists almost entirely of C++ and Swift, which consists primarily of code in the test folder.

Summary

That’s it! I love reviewing code bases using gitstats and the charts in Github (which are getting better and better, BTW) and was curious to break down Swift. I hope you thought this was interesting…

Categories
Uncategorized

Evolution of the Zappos Product Page

We’ve come a long way in how we present products to our customers since the early days of the Internet. Let’s dust off WayBackMachine.org and see what’s changed…

Then

Zappos Product Page — Circa 2003

We’ll focus on Zappos.com circa March 2003 (or thereabouts). One of the things that stands out to me is how, arguably, e-commerce product page design has failed to evolve significantly from the basic template of 2003. Of particular note: the layout of the product images relative to the “buy box” (above and to the left), the next and previous image carousel metaphor, the Add to Cart button placement, and position of product descriptions relative to other elements on the page. Many e-commerce sites remain stuck in this paradigm.

Nostalgically, the screen from 2003 reminds me of the struggles web designers (myself included, albeit in B2B space) had in those days, including:

  • “Maybe a back button on the page will keep people from hitting Back in the browser too easily and leaving the site!”
  • “What if they scroll down and don’t see the Back to Browsing button we added at the top??? Let’s put another one at the bottom!”
  • “Internet Explorer 6.0 only supports standard <select> elements, so we have to design the variant pickers using that!”
  • “Oh, and the Size picker needs to filter the Color picker to ensure availability, so yes the page has to refresh”
  • <table><tr><td>We’re in table-based layout hell</td></tr></table>

Remember thinking this way? I sure do.

Now

Twelve years later the 2015 Zappos.com is aesthetically quite different, though similar in a number of ways.

Zappos.com Product Page in 2015
  • We no longer feel the need to cram everything “above the fold”. While Nielsen’s original usability studies were pivotal in guiding our desktop-centric designs in the past, the user experience community (including Nielsen) has acknowledged that scrolling is here to stay due as much to the shift to mobile as anything else.
  • Calls to Action, in particular the Add to Cart button, are extremely easy to find and target.
  • The new “Tell a Friend” feature is the obligatory social network button bar…and it’s arguably no more effective than the old feature was
  • Brand imagery (see “paul green” logo in each example) is less prominent than the more important elements of Price, Customer Reviews/Rating, and Product Recommendations.
  • Whereas the 2003 example provided tools (e.g. “Search for a Brand”, “Browse More Styles…”) for finding more products, the 2015 is more proactive and uses the now table-stakes recommendation engine approach to proactively suggest similar items.
  • While we have a better handle on the use of white space, we seemingly remain constricted to table- or grid-based layouts.

Summary

While Zappos is just a single entry in the vast library of historical product pages that can be referenced, I think it represents an interesting contrast between the past and present that is common in many e-commerce designs.

What, one wonders, are some other comparisons with the past worth making?

Categories
Uncategorized

The Power of Periscope and a Startup Lesson by Dan Martell

This morning, I did not plan to attend a presentation by @DanMartell, but he just taught me (and 40 or so other Periscope followers) a micro-lesson in scaling a startup.

This is the power of Periscope…

…the ability to engage an audience, immediately, when a thought or feeling strikes you as it did today for Mr. Martell. Large-scale business/technical conference organizers take note: your industry is being disrupted.

Mr. Martell dropping knowledge

While on the way to pick up his children, Dan took the time to engage with his follower community on the topic of founder productivity. Having just had a lengthy conversation, apparently, with some fellow entrepreneurs on a boating excursion, his thoughts can be summarized as:

  1. Hire for day-to-day tasks (including cleaning, laundry, menial tasks, etc.)
  2. Hire for operational tasks (payroll, email, proposal writing, errands, etc.)
  3. Hire for sales and marketing (leverage revenue > spend activity…an example being those marketing efforts that return $3 for every $1 spent)

If you can figure out a business model to generate enough income to be able to hire these people…this is how you become a business owner rather than a business operator.

Rather than repeat and continue to distill Dan’s thoughts here. I will just link to the Periscope recording here. There are additional insights into determining to your EHR (effective hourly rate) — which should be a minimum of $100/hr), 80/20 cost analysis, and some recommended reading…


Periscope is phenomenal for micro-lessons of this nature, assuming you are following the right people. Just thought I’d share.

Categories
Uncategorized

The Willy Wonka Factor

Running Your Product Strategy like a Madman in a Chocolate Factory

As I type, my kids are watching Willy Wonka and the Chocolate Factory—not the proper original film starring Gene Wilder, mind you, but the rather goofy one with Jack Sparrow. While suppressing my misgivings for Tim Burton’s adaptation of one of my childhood favorites, I observed something interesting about the plot that is equally inspiring in both the 1971 and 2005 versions. Put simply, there is a highly desirable network effect surrounding the Wonka Bar; one that I believe can be emulated in any product-centric company if one is willing to wear the felt top hat.


In the story, the narrator tells of a chocolate factory—the chocolate factory—which is under a massive competitive attack. The trade secrets of the company are leaked and spied upon relentlessly. So much so, in fact, that Wonka chooses to shutter the factory over suffering the ongoing morass of spying and stolen intellectual property. This brings us to point number one of the Wonka Factor…

1. Be the indisputable leader in providing at least one platform-level feature which has either a secret ingredient or high barrier to being copied

Willy Wonka has Everlasting Gobstoppers, chewing gum that never loses flavor, and all the special ingredients that make up the brand. These core features make up the foundation upon which the rest of the company is built. There is no more obvious example in modern-day business of this point than Apple and the iPhone, whose industrial design has an entire army of Slugworths wondering what that secret ingredient is. Some other Wonkas with secret factories we all want Golden Tickets to:

  • Google and it’s Search Algorithm
  • Amazon and Two-day Prime Shipping
  • Tesla and the Supercharger Network
  • Pinterest and the Inspiration Graph

These are examples of companies that consistently deliver on a Wonka-like promise of unique, whimsical and habit-forming offerings in their respective markets.

Still, simply having a product or service that is highly desirable is not enough; you must have a history of successfully delivering and supporting this value to leverage the Wonka Factor. Further, that value must not be so easily replicated as to allow 100 other competitors to come into the market and lick your product’s secrets right off the walls. The snozzberries taste like snozzberries!

I know of a gas station locally that has the most expensive gasoline in town, but also a good car wash, which is always free with a fill-up. Their commercials highlight excited customers coming back for the car wash, seemingly uncaring that they spend $6 more per tank. While this gas station’s choice to offer a car wash is obvious and reproducible, it is an expensive addition to the overall value offering and thus represents a high barrier feature.

But the competition never rests. Before Slugworth and his ilk chipped away at Wonka’s market share, he was the best in town. Everyone wanted his stuff. So, when the competition looms on the horizon it becomes increasingly important to keep existing customers loyal. But that loyalty must be rewarded…

2. Reward loyalty with a truly timeless experience, something that is worthy of resurrecting should it ever go away

The Golden Ticket. While it ultimately represented the mechanism for Wonka to find his successor, it was also a supremely effective marketing ploy to relaunch the Wonka Bar brand. A chance for anyone, of any walk of life, to experience something magical…if only for a day. The entire world was enthralled with the chance at winning, even if only 5 tickets were ever produced.

There are many products and experiences that, were they to go away there would be an innate yearning for it to make a return. Some examples:

  • Chevrolet Corvette
  • Herman Miller Aeron Chair
  • Amazon’s 1-Click Ordering
  • Subzero Refrigerator
  • Oreo Cookie

These products have become part of our culture, like landmarks for us to contextualize our daily interactions with the world around us. Take away these landmarks and one is left wandering around trying to find new versions of them, often peddled by the Slugworths of the world who never seem to get it right.

The Golden Ticket was a chance to resurrect the household-name-normalcy of the Willy Wonka brand. A chance to be part of that resurrection for a fan of the brand is something very visceral, calling one back to something profound. If you’ve achieved this level of Wonkafication, your product or service is likely indistinguishable from your brand; your customers indistinguishable from your advocates, your employees indistinguishable from your Marketing team.

And finally…

3. Actualize someone’s dream, so that you might go on dreaming new things.

As I watch Gene Wilder casually flying through the air in a flying elevator, explaining to Charlie that the chocolate factory is now his, it’s a moment of uplifting clarity about the purpose of the entire film. This is the moment that the dreams of one man passes on to another, so that the vision does not die with the man.

We should strive to build products and services that are worth passing on to the world. As cliché as it may seem, the makers of the world should seek to execute ideas worth remembering…the Everlasting Gobstopper of our lives.


Categories
Uncategorized

Google+ and Hangouts Need Just One Thing

How Google Could Own All Communication

There is one thing preventing Google Plus and Hangouts from becoming massive. Simply put, it’s the fact that my personal and work Google accounts (and all the other accounts I have) cannot be rolled into one, socially-connected and all encompassing “communication” profile.

Imagine if LinkedIn or Facebook operated this way, giving you a new account for every email you managed on the platform. Each identity within those systems would become a weak facsimile, a ghost, of the other profile. This is exactly what happens with Google+.

Here is my work Google+ profile:


And the Google+ profile page for my personal Gmail:


Which one would you follow? Why would you pick one over the other?

The fact that one has to ask “which one of these is the ‘real Brad’” manifests itself throughout the Google ecosystem. If you’re using Hangouts at work, for instance, look what happens when you search for your coworker, Brad Ledford:


Both personal and work accounts show in the results, along with the myriad other Brad Ledfords in the Interwebs. This is confusing enough that my wife consistently uses my work account to chat with me (which I find exceedingly annoying, honey!) when the more appropriate account would be my personal one. I also consistently get emails from people who found my page on Google+, likely from ranking highly in search results, but are ultimately communicating with the wrong Brad. Sidebar: Hey Brad in North Carolina…you’re a popular dude but you need better SEO!

So what’s the big deal? Is this even a real problem to solve?

Yes. It’s huge because there are already too many ways to communicate in the modern suite of daily-use applications. Facebook, Gmail, LinkedIn, SMS, and their ilk all try to maintain open systems of communication, bending us into mental pretzels try and manage it all. Having multiple channels of communication open within the Google ecosystem is like adding salt to the, uh, pretzel.

If done well, I think Google+ and Hangouts could be at the center of it all, binding our systems and their activity feeds into a single realtime communication profile. I imagine this as a simple Venn diagram with g+ in the middle.


Is it possible or has Facebook messaging already beat Google to the punch?

Categories
Uncategorized

A Kink in the Fire Hose

How I Unfriended 160 People, Denied Skill Endorsements and Found Some Sanity

A few weeks ago, I found myself staring at a Facebook post that probably resonates with many of you. A “friend” had just indicated that they liked a particular link from The Oatmeal. It was a minor distraction, but for me it was the last straw on the proverbial camel’s back. I neither needed nor wanted this and the thousands of distractions that preceded it no matter how funny, ironic, or inspiring. I took the next 3 hours and unfriended this person and 159 others.

Immediately, my use of Facebook dropped to almost nothing, as I was not getting pinged with alerts to check my feed. Beautiful.

Yet, it leaves one to wonder: is there is any natural reprieve from this onslaught of social activity? How do we stay connected but still not suffer the rising tide of distracting content, direct ads, and subtle promotional marketing? This is not seemingly an easy answer, as previous attempts at using the more granular and one-thing-at-a-time filtering mechanisms in Facebook were getting me no where…I had to purge.

The purge is likely to be the answer for many for the foreseeable future. With so many channels of communication open, receiving too much social chatter is like a Web server backed up with too many requests…sometimes it’s easier to just bounce the server than kill threads one a time. But social networks, shopping engines, and media sites don’t appreciate this “screw it, I’m done” mentality and rather want to avoid it due to the vanity metrics that would be lost so quickly.

And that’s unfortunate, since I believe that shopping, socializing, and consuming content is an experience that the user should control, entirely. Imagine a big button in Facebook labeled “Shut off Ads for…days” or an Instagram option to “Go Private” with your photo stream. Or perhaps a mode in Amazon that extolled a link to “Browse like someone who loves books” even though Amazonian big data says I actually like to browse for UHD TVs…shhhh, I’m cultured I swear! I would pay for these options and I believe others would too.

In the past few days, I took my social decoupling a bit further and removed a lot of attributes about me on LinkedIn, including skills which I had been endorsed for that I really have no expertise in, and other services that track interests and demographics. I’ve noticed a much more dynamic and diverse set of content on at least my LinkedIn feed, but the fire hose of “stuff” in my feed continues to swell. I’m now left to wonder what a “purge” in LinkedIn would look and feel like, to myself and others.

Alas, though, the purge has given me back some sanity and I will likely do it again, probably no later than 3 months from now. I may even resort to digital hermit-ism if it comes to it, given how advanced search marketing is becoming…a dichotomy for someone in e-commerce. Will those of us in B2C product development and design ever learn to be more respectful of the fire hose we all have planted to our lips? Can I share content such as this article without contributing to others’ woes?

Hmm…