A Tale of Two Decades

Appreciating the engineering divide between the noughties and the twenties

The expectations we have today of a software engineer are very different from what we used to have a couple of decades ago, when I took my first steps into this industry. Engineers are expected to be fully rounded and multi-talented - a jack of all trades, if you will. This can be demonstrated when we compare these two eras from two different lenses -

1. The influence of the Internet and associated technologies, primarily the virtualization of everything

2. The updated ways of working that are now possible due to these technologies

This post is mainly a trip down memory lane, to establish the stark differences in the landscape when viewed across a span of two decades. You could also say, it is intended to help indulge myself in nostalgia, and remind you of how old I really am!

I also invite you to click through the many links in the article below to additionally enjoy a visual glimpse into this era that used to be.

The Context

Let’s calibrate ourselves by sampling something from the everyday life of a fledgling, young engineer in the early 2000s, emerging from the dot-com bust, and sleepwalking straight into the Great Recession.

The first cellphone I bought (Nokia 6610) with my early paychecks stood apart on the weight of a color display, FM radio and a WAP browser! Data plans however, were unheard of, and the phone didn’t have WiFi built-in.

My bachelor pad had internet served over the Cable TV connection, but it would take over an hour to download an episode or buffer it, so we would instead watch our favorite shows at work on weekends.

You had to wave down an auto rickshaw for a ride, and haggle over prices. Oh wait, we still do that…

At Work

I was using a powerful, 512MB RAM portable laptop (Dell XPS) as a developer, but it was so heavy, I developed back spasms at a very young age lugging it to work every day.

We used to huddle in a meeting room to dial in to an AT&T conference number at 8pm to meet with our colleagues state-side. If you were lucky, or holding an important meeting, you got the room with the Polycom installed.

I had a powerful server in our server lab assigned to me as my dev environment, that I had to often walk up to for a physical reboot or troubleshooting when it stopped responding over the LAN.

(Not) Me hard at work fixing my dev server

The internet was mainly for Stack Overflow, MSN messenger (chat communication with the team) and working with open source, plus browsing Twitter when it used to be a lot more healthy. Everything else we used was on the intranet - bug tracker, email, dev environments, build systems, corporate wiki, network drives for file sharing.

Team Structure

We used to have a flat hierarchy led by a CTO, and

1. A dedicated QA team with no automation,

2. a dedicated build engineer,

3. an IT engineer specializing in hardware,

4. a database and Lucene expert with 20+yrs of experience,

5. a few system engineers that built the distributed system for data processing (this is where I belonged),

6. a few application engineers that built the MC for our MVC architecture,

7. and a few front-end engineers to build the V into MVC.

Shipping Software

We used to ship our software to enterprise customers as DVDs, alongside a thick, printed, admin manual on how to set it up. More often though, we would install our software on a rack of servers and ship that along with a solution engineer to help set it up at the customer site and get them going.

We were considered revolutionary for shipping every 3 months - unheard of in the enterprise software business where annual releases were the norm. When we started shipping Virtual Machine images as opposed to those racks full of servers, we were blazing a trail. Builds took more than an hour, deployments took longer, but a large part was automated - again, considered cutting edge back in the day.

Fast Forward

In more recent times, I’ve been on teams that have built powerful developer platforms, a marketplace, and SaaS products with a team that essentially looks like this -

1. A few QA engineers relying on automation

2. A few front-end developers

3. A few back-end developers

All our environments - for developers or customers - were on AWS, and no one on my team ever saw a server rack, except in pictures.

Collaboration is built into every tool we use, and works as long as you are connected to the internet, which in turn is ubiquitous.

Most of our work happens on the browser, through a plethora of SaaS applications - source control, builds, deployments, infrastructure management, issue tracking, documentation, messaging, meetings. Coding continues to stem the tide, but how long will that last?

Where work happens

We deployed to production very frequently, often multiple times a day, with only big launches (“Marketplace 2.0!”) being planned ahead of time.

Most of our tests, as well as the builds and deployments are fully automated, requiring only human oversight.

---

Quite a chasm, eh? How do you begin to explain this great divide between my early days and where we find ourselves today? The Internet was always around, but what evolved is its ubiquity and the technologies it fostered riding on this wave. We will look closer at the role of the Internet and virtualization in our next post as one way to reason about this leap forward.