"Lost and Found" : Part 3 - I do not think it means what you think it means

In the post-college world I recounted in Part 2, things are much different than I imagined. At this point in my career, I was working on increasingly more side software projects. Software Engineering, in practice, was still a bit of a black box to me. Since I was working on the support side of the company, I only saw the end product¹. Occasionally, our team would have a discussion on the new features coming out and we would conduct various levels of training. However, I still had very little insight into how these ideas or features came to exist. Who defined them? How were they designed? In the end, I was just a consumer of the software and subjected to the end-user experience. Once in awhile, bugs that had existed for years would finally get fixed or features we had clamored for would be released. In reality, though, we weren't the designated end-users of the system; our customers were. Many of the changes were directed at customer reported issues, which is generally good for business. Much of my frustration from working in support was dealing with this dilemma of not seeing the changes we, as a support team, wanted to see. My desire to switch over to the engineering side of the company was to see if I could help effect change in that area.

To my surprise, it wasn't as easy as I had imagined. The first few weeks on the engineering team were very enlightening. My expectation coming in was to start digging into the code-base, learn the basic development cycle practices such as code-review, team meetings, design discussions, etc. That expectation was not met, and was instead turned on its head. There was a lot more disorganization than I imagined possible, which was bad for the company overall, but a great opportunity in waiting. You see, before I switched departments, I had been promoted over the years up to a support leadership position managing our most skilled technicians. I had a knack for process control and here was a great chance to institute a fresh process from the ground up. Instead of diving in as a software developer, I steadily fleshed out my role as a Scrum Master. I was even sent off to get certified. There was a lot of work to do. My arrival on the team happened to coincide with a total system re-implementation of our operational and business support systems. We adopted scrum and agile practices with a fervour and we had the management and team support to make it a success.

Scrum didn't solve all of our problems though. While we delivered the project with much greater success than previous projects, we still missed deadlines and budgets. Despite those downfalls, the thought of trying to meet those guidelines in the pre-scrum era were unfathomable; both to the whole team and to me. It was a very satisfying experience overall. However, as we neared the remaining few months of the project, my desire for technical and intellectual problem solving started to gnaw at me. I had a great sense of proper process control, but it wasn't fulfilling enough to keep me content. I eventually stepped down from the Scrum Master position and into a dedicated software developer role. I felt I could affect a much greater influence on the development practices by rising to a leadership role within the team. That goal has certainly driven my style and design decisions over the last few years and continues to be a motivating factor for doing things as close to the right way as possible.

Once we released our two years of effort to production, everything changed. Scrum became hard to maintain within the tightly controlled QA and release processes we had developed over the years and continued to use. The priority of features, changes, and bug fixes didn't fit nicely into our two week sprints. Various processes were created by various members of the team in order to address issues that came up. The end result worked, for the most part, and got software releases out the door, but ideal of scrum and it's processes became mostly obsolete. We weren't developing in a vacuum anymore. We had stakeholders to answer to that we didn't have before. All of the conflicting priorities made maintaining our agile processes difficult. Early on in the Scrum adoption process, it seemed easy and natural to strictly adhere to the ideal workflow. As reality keeps reminding us, though, ideologies should be applied within reason. Each company, each team, each project all have their own little quirks. There's never a silver bullet that will solve all problems for all cases. Even today, we're trying to adapt and see what works and what doesn't.

Overall, developing production software has been much different than I expected. Long gone are my dreams of creating the myriad of fantastic tools and features that I yearned for while in support. The demands of business, customers, and product strategy have out shadowed those dreams. That is not to say that many of those ideas won't ever come to fruition, but it will likely be by sheer necessity to support our customers and not nearly to the extent with which I had hoped. Resource limitations are much more restrictive than I had imagined and it's through process control and implementation that we can get the most bang for the buck. In Part 4 I'll start getting into the nitty-gritty of the trials and tribulations of the software development lifecycle like QA, unit testing, code reviews, and more.

photo credit: Princess Bride [Motion picture]. (1987).